TaskFlow is a Python library for OpenStack (and other projects) that helps make task execution easy, consistent, scalable and reliable.

1. Openstack taskflow
果凍

2. 簡介
● 任職於迎廣科技
○ python
○ openstack
● c++, java, scala
● http://about.me/ya790206
● http://blog.blackwhite.
tw/
● https://github.
com/ya790206/call_seq

3. 聯絡資料
● 個人信箱： http://www.blackwhite.
tw/#/contact
● 公司信箱：jam.kao at in-win.com.tw

4. 關注 openstack 以下技術
● cinder
● glance
● ceilometer
● horizon
● log
● monitor and alert

5. What is it?
● TaskFlow is a Python library for OpenStack
(and other projects) that helps make task
execution easy, consistent, scalable and
reliable.

6. What it help us to do?
1. Retry
2. Revert
3. Each independent task (or independent
subgraph) in the graph could be ran in
parallel when its dependencies have been
satisfied.
4. Concurrent / Parallel

7. 前情提要
A 要找 B 和 C 去餐廳 X 進行祕密會談。只要有人
無法到場，則取消該次會談
1. 聯絡 A 和 B
2. 聯絡 X(餐廳)

8. def call(name, phone, msg):
if phone in ['123']:
print 'name: %s, phone: %s, msg: %s, status: failed' % (name, phone,
msg)
return False
print 'name: %s, phone: %s, msg: %s, status: success' % (name, phone, msg)
return True

9. call('B', '123', 'invite B')
call('C', '124', 'invite C')
call('X', '124', 'make a reservation in X')

10. def try_call(name, phone_list, msg):
ret = False
for phone in phone_list:
ret = call(name, phone, msg)
if ret:
break
return ret

11. try_call('B', ['123'], 'invite B')
try_call('C', ['124', '126'], 'invite C')
try_call('X', ['224'], 'make a reservation in X')

12. try:
revert_list = []
successed = try_call('B', ['123'], 'invite B')
if not successed:
raise Exception()
revert_list.append(lambda: try_call('B', ['123'], 'cancel B'))
successed = try_call('C', ['124', '126'], 'invite C')
if not successed:
raise Exception()
revert_list.append(lambda: try_call('B', ['123'], 'cancel B'))
successed = try_call('X', ['224'], 'make a reservation in X')
if not successed:
raise Exception()
revert_list.append(lambda: try_call('B', ['123'], 'cancel B'))
except:
for action in revert_list:
action()

13. Q: Do we have a better way?
A: Yes, we have openstack taskflow

14. Tasks
A task (derived from an atom) is the smallest
possible unit of work that can have a execute
& rollback sequence associated with it.

15. Flows
A flow is a structure that links one or more
tasks together in an ordered sequence
pattern. When a flow rolls back, it executes
the rollback code for each of it's child tasks
using whatever reverting mechanism the task
has defined as applicable to reverting the
logic it applied.

16. Patterns
1. Linear
2. Unordered
3. Graph

17. Engine
1. Engines are what really runs your atoms
(tasks, flows).
2. Engines decide which atom to run and
when.
3. Engine type:
a. serial(default)
b. parallel
c. worker-based

18. store
(dict)
get
dependence
provides
Task
get
dependence
provides
Task

20. provides: Any outputs this task produces.

21. empty store
(dict)
After executing
Provider task Dmsg
->
XXX
Dphone_l
ist ->
[XXX,
YYY]
Dnam
e ->
ZZZ

22. Task
dependence

23. rebind: An immutable input resource mapping
dictionary that can be used to alter the inputs
given to this task.
provides: Any outputs this task produces.

24. Dphone_l
ist ->
[XXX,
Dmsg
->
XXX
Dnam YYY]
e ->
ZZZ
get
dependence
from store for
CallTask
(Dname,
Dphone,
Dmsg)
Run
CallTask
Provide
`Ddone`
Dphone_l
ist ->
[XXX,
YYY]
Dmsg
->
XXX
Dnam
e ->
ZZZ
Ddone
->
True
Provide -> CallStack
Q: Why it is Dname istead of name
A: because of rebind
Q: where is the
Dphone?

26. flow
provide_flow call_flow

28. revert rules:
● 每次 task 執行失敗，則會 revert 。
● 只要 flow 中有個 task 執行失敗且無法
retry，則已經執行過的 task 則會 revert。

29. Demo
https://gist.github.
com/ya790206/14229922355077713
ad9
http://goo.gl/gyd3K6

30. Another example
x2 = y3+y4
x1 = y1+y2
x5 = x1+x3
x3 = x1+x2
x4 = x2+y5
x6 = x5+x4
x7 = x6+x6
x7
x6
x5 x4
x4
x2 y5
y3 y4
x1
y1 y2
x3
x1 x2

31. Graph pattern
A tasks dependents are guaranteed to be
satisfied before the task will run. Each
independent task (or independent subgraph) in
the graph could be ran in parallel when its
dependencies have been satisfied.

32. Code
https://github.
com/openstack/taskflow/blob/master/taskflo
w/examples/graph_flow.py

33. Why Openstack need taskflow?
OpenStack code has grown organically, and
does not have a standard and consistent way
to perform sequences of code in a way that
can be safely resumed or rolled back if the
calling process is unexpectedly terminated
while the code is busy doing something

34. Results
With widespread use of TaskFlow, OpenStack
can become very predictable and reliable,
even in situations where it's not deployed in
high availability configurations.

35. Summary
TaskFlow is a Python library for OpenStack
(and other projects) that helps make task
execution easy, consistent, scalable and
reliable.

36. 迎廣雲端研發中心介紹
● 提供 openstack 相關服務
○ 教育訓練
○ 客製化
○ 佈署

37. 徵才
● web 前端工程師(angularjs, html5, css)
● web 後端工程師(python)
● 熟悉 linux 網路與 SDN
● 熟悉 linux 分散式儲存與 SDS

38. Reference
https://wiki.openstack.org/wiki/TaskFlow

39. 建議學習順序
1. simple_linear.py
2. calculate_linear.py
3. reverting_linear.py
4. retry_flow.py
5. graph_flow.py
6. simple_linear_listening.py
7. calculate_in_parallel.py
8. pseudo_scoping.py