This document discusses concurrency in operating systems and different programming languages. It explains how concurrency works at the OS level using schedulers and threads/processes. It then compares implementations of concurrency in Python, Java, and Go. Python uses threads but is constrained by the GIL, Java uses native threads, and Go uses lightweight goroutines scheduled across OS threads. The document cautions that while goroutines make concurrency easy, there are still costs to consider. It concludes by noting languages evolve over time and no approach is inherently unable to handle high concurrency applications.

1. Understanding How Concurrency Work in OS
Learning Conurrency in Python, Java and Golang
G7
genchi.lu@cymetrics.io

2. Concurrency in OS

3. Concurrency in OS - Scheduler

4. Concurrency in OS - Thread and Process
code data file
register stack
code data file
register register
register
stack
stack
stack
thread
thread
single thread process multi-thread process

5. Concurrency in CPython

6. Concurrency in CPython - How
● multi thread (OS native thread)
● multi process
● async (it’s more about async io)

7. Concurrency in CPython - Thread (1)

8. Concurrency in CPython - Thread (2)
number of python’s threads is equal
to number of OS’s thread

9. Concurrency in CPython - Thread And GIL (1)
Constrain of Python thread: GIL
There always only one thread is running per python process.

10. Concurrency in CPython - Thread And GIL (2)
Total CPU usage is always about 100%, no matter how many cores your machine
is.

11. Concurrency in CPython - Process (1)

12. All CPU is full of usage
Concurrency in CPython - Process (2)

13. Empirically gauging the difference in
overhead can be difficult, but in
general it is much more time
consuming to create and manage
processes than threads.
Concurrency in CPython - Process (3)

14. Concurrency in Java

15. Concurrency in Java - How
multi thread (OS native thread)

16. Concurrency in Java - Thread(1)

17. Concurrency in Java - Thread (2)
number of Java’s threads is great
then number of OS’s thread
(Some threads for the purpose of
maintaining JVM, like GC)

18. Concurrency in Java - memory cost of thread (1)

19. Concurrency in Java - memory cost of thread (1)
● java -cp java-1.0-SNAPSHOT.jar ThreadNum
● java -cp java-1.0-SNAPSHOT.jar -Xss200k ThreadNum
● java -cp java-1.0-SNAPSHOT.jar -Xss1g ThreadNum

20. Concurrency in Java - Overhead of Thread (1)
● Create Thread
● Destroy Thread
● Context Switch

21. Concurrency in Java - Overhead of Thread (2)

22. Concurrency in Java - Overhead of Thread (3)

23. Concurrency in Java - Profile Overhead(1)
number of threads = 10

24. Concurrency in Java - Profile Overhead (2)
number of threads = 5000

25. Concurrency in Golang

26. Concurrency in Golang - Goroutine on Thread (1)

27. Concurrency in Golang - Goroutine on Thread (2)
● Pick Strategy
○ Pick up one goroutine from thread’s local queue
○ Pick up one goroutine from global queue
○ Network Poller
○ Pick up one goroutine from other thread’s local queue (work stealing)
● Every thread would execute one goroutine about 10ms, then switch.
● Golang uses GOMAXPROCS parameter to number of threads to use.
○ default: number of cores.

28. Concurrency in Golang - Goroutine on Thread (3)

29. Concurrency in Golang - Goroutine on Thread (4)
number of goroutines is much less
then number of OS’s thread

30. Concurrency in Golang - Goroutine on Thread (5)

31. Concurrency in Golang - Blocking System Call (1)

32. Concurrency in Golang - Blocking System Call (2)

33. Concurrency in Golang - Blocking System Call (3)

34. Concurrency in Golang - Blocking System Call (4)

35. Concurrency in Golang - Blocking System Call (5)

36. Concurrency in Golang - Blocking System Call (6)
the number of OS’s threads is
growing to hundreds.

37. Recap

38. Recap
● How concurrency work in OS level
● How CPython implement concurrency on top of OS
○ multi thread
○ multi process
● How Java implement concurrency on top of OS
○ Thread
● The challenge of thread model
● How golang implement concurrency on top of OS
○ goroutine

39. Final Though

40. Final Though - Python not suitable in High Concurrency?
● Instagram use Python to build their product
● “It’s hard to do” does not mean “It CAN NOT”

41. Final Though - Evolution of Java’s Concurrency
● Java had use green thread before Java 1.2
● As green threads have some limitations compared to native threads,
subsequent Java versions dropped them in favor of native threads
● Quasar Project
● A library that provides high-performance lightweight threads.
● Loom Project
● Introduce fibers as lightweight, efficient threads managed by the Java
Virtual Machine

42. Final Though - Goroutine is not Silver Bullet
● Goroutine make is easy to handle concurrency
● Does not mean that you can use goroutine in whatever scenario without cost
○ ex. high concurrency with blocking system call

43. QA

44. All Sample Code

45. Reference
● Understanding the Python GIL
● Go scheduler: Implementing language with lightweight concurrency
● The Go netpoller
● The Go scheduler
● Going inside Java’s Project Loom and virtual threads
● Green threads
● Quasar
● Loom Project
● Adopting Python Asyncio in Large Scale Project
● Scaling Instagram Infrastructure

46. Thank You

47. About Cymetrics
● Tech Blog