Understanding Coroutine

1. 深⼊入理解协程
⼩小拿@果壳
2016.04.07

2. 协程 == yield ?

3. ⺫⽬目录
• 定义与背景 （为什么产⽣生）
• 理解协程的概念 （是什么）
• 协程的没落与复兴 （历史）
• 原型实现* in C （落实到程序）
• 效率、Python与Go （为什么重要）

4. 协程 = coroutine
co- 英语的派⽣生词缀，能跟动词、名词、形容词相缀合，表⽰示
together, joint 等意思，也就是『⼀一起』

5. 定义
• Coroutines are computer program components
that generalize subroutines for nonpreemptive
multitasking, by allowing multiple entry points for
suspending and resuming execution at certain
locations.
• Coroutines are well-suited for implementing
more familiar program components such as
cooperative tasks, exceptions, event loop,
iterators, infinite lists and pipes.

6. 背景
• According to Donald Knuth, the term coroutine
was coined by Melvin Conway in 1958, after he
applied it to construction of an assembly
program.

7. 编译器

8. 理解协程的概念

9. 调⽤用⽅方式
• subroutine: …a subroutine merely was an extension
of the computer hardware, introduced to save lines of
coding. (调⽤用/被调⽤用)
• coroutine: …as a team of programs, each member of
the team having a certain job to do. (协同/相互调⽤用)
• WIKI: Coroutines are computer program components
that generalize subroutines for nonpreemptive
multitasking, by allowing multiple entry points for
suspending and resuming execution at certain
locations. from TAOCP

10. 调度⽅方式
• 协作式调度
• 抢占式调度
• WIKI: generalize subroutines for nonpreemptive
multitasking

11. 执⾏行流程
多边扫描算法 multipass

12. • psychological difference
• time difference
• space difference
协程与多遍扫描的转换

13. 协程与多遍扫描的转换
• ⽣生产者/消费者模式
• 后⾯面需要前⾯面扫描的统计信息

14. compare with multiple-pass
algorithm
Little old lady, riding a bus. "Little boy, can you tell
me how to get off at Pasadena Street?"
Little boy. "Just watch me, and get off two stops
before I do."
(The joke is that the little boy gives a two-pass algorithm.)
from TAOCP

15. 协程的没落与复兴

16. ⾃自顶向下程序设计⽅方法
• top-down design ⾃自顶向下，逐步求精
• structured programming 结构化程序设计
• 过程调⽤用的⽅方式进⾏行流程控制
• 循序/选择/重复
• goto statement

17. ⾃自顶向下程序设计⽅方法

18. Python
• 迭代器 iteratior
• ⽣生成器 generator
• …
• ⽣生产者/消费者
• 多线程/竟态同步/缓冲区
• 任务轻量/线程机制过于复杂

19. tornado
• C10K
• high concurrency
• select/poll/epoll
• event-driven
• callback
• async IO

20. async IO

21. callback hell

22. coroutine/yield

23. 原型实现* in C

24. 实现
让出yield / 恢复resume

25. 实现

26. 实现

27. 再看定义
• Coroutines are computer program components
that generalize subroutines for nonpreemptive
multitasking, by allowing multiple entry points for
suspending and resuming execution at certain
locations.
• Coroutines are well-suited for implementing
more familiar program components such as
cooperative tasks, exceptions, event loop,
iterators, infinite lists and pipes.

28. 效率、Python与Go
从效率的⾓角度看协程

29. Parallelism vs Concurrency
• 多线程
• 并发 Concurrency
• 并⾏行 Parallelism
Concurrency is about dealing with lots of things at once.
Parallelism is about doing lots of things at once.
— Rob Pike

30. Python
• 单线程
• GIL (Global Interpreter Lock)
All tasks working
within Python must
acquire the singular
GIL to legally
process, effectively
limiting all Python to
a single thread of
execution.

31. Go与并⾏行编程
⼀一个关键字 go

32. goroutine的调度
• M: Machine (OS thread)
• P: logical Processor
• G: Goroutine
并⾏行度 runtime.GOMAXPROCS(runtime.NumCPU())

33. Go与异步⾮非阻塞
在Go语⾔言中，如何让⼀一个⺴⽹网络连接使⽤用⾮非阻塞I/O？怎么对⺴⽹网络连接进⾏行异步读
写？
这⼏几个问题回答是：Go的⺴⽹网络程序不使⽤用异步操作，⼀一切操作都是同步的，每
个goroutine处理⼀一个连接。
在Go代码层⾯面，开发者看到的是使⽤用goroutine来进⾏行阻塞式读写，⽽而在Go的
内部实现中，则是利⽤用异步操作，通过对goroutine的调度来完成对事件的处
理。作为最⼩小的调度单位，goroutine之间是并发（可能⾮非并⾏行）执⾏行的。
『以同步的形式表达了异步的⾏行为』
写程序 = ⽤用程序语⾔言表达意图

34. 协程带来了什么
Go通过协程统⼀一了异步、并发与并⾏行的表达
Python通过协程利⽤用了异步带来的效率提升
Go通过协程利⽤用了异步、并⾏行带来的效率提升

35. goroutine vs coroutine
在其他语⾔言中，⽐比如 Python、Lua或者C#中都有协程的概念。
goroutine 可以看做 coroutine 在 Go 中的实现，不过与通常意义的 coroutine
还是有两点不同：
• goroutine 意味着并⾏行；coroutine ⼀一般不是
• goroutine 通过 channel 来通信；coroutine 通过让出
yield 和恢复 resume 操作来通信

36. 参考
Reference:
1 https://en.wikipedia.org/wiki/Coroutine https://zh.wikipedia.org/wiki/%E5%8D%8F%E7%A8%8B↩
2 Donald Knuth, The Art of Computer Programming, Volume 1. Addison-Wesley, ISBN 0-201-89683-4. Section
1.4.2 describes coroutines in the “pure” form.↩
3 http://geek.csdn.net/news/detail/49827 计算机语⾔言协程的历史、现在和未来↩
4 http://blog.chinaunix.net/uid-17299695-id-3059110.html epoll的⾼高效实现原理, https://www.zhihu.com/
question/20122137↩
5 http://yanyiwu.com/work/2014/12/20/c-coroutine.html 谈谈并发编程中的协程↩
6 http://demo.pythoner.com/itt2zh/ch5.html 异步Web请求↩
7 http://www.chiark.greenend.org.uk/~sgtatham/coroutines.html Simon Tatham, Coroutines in C↩
8 http://searchsoftwarequality.techtarget.com/definition/structured-programming-modular-programming
structured programming(modular programming)↩
9 https://segmentfault.com/a/1190000003063859 Linux IO模式及 select、poll、epoll详解↩
10 http://www.goinggo.net/2014/01/concurrency-goroutines-and-gomaxprocs.html Concurrency, Goroutines
and GOMAXPROCS↩
11 http://www.cnblogs.com/yjf512/archive/2012/07/19/2599304.html Golang runtime 浅析↩
12 https://www.zhihu.com/question/20862617 golang的goroutine是如何实现的↩
13 https://talks.golang.org/2012/waza.slide Concurrency is not Parallelism↩
14 http://ifeve.com/cpp-concurrency-vs-parallel/ C++11并发编程指南↩
15 http://wiki.jikexueyuan.com/project/the-way-to-go/14.1.html 并发、并⾏行和协程↩

37. END
justinli.ljt@gmail.com