With developers, an awesome product isn’t everything, or at least we found that out. More than a product you need a platform. But what is a platform? Learn tips and tricks about building a public API using the latest and greatest tools: OpenAPI, Python 3 and asyncio. This presentation was given by Optimizely engineers Nick DiRienzo & Vinay Tota at PyBay 2017.

1. How Optimizely scaled
its REST API with asyncio

2. Nick DiRienzo
@nickdirienzo
Software Engineer
Optimizely
@Optimizely
Vinay Tota
@vinaytota
Senior Software Engineer
Optimizely

3. We’ll talk about:
• Optimizely’s API story
• What makes an API “good”
• Optimizely’s API gateway
• Improving it with asyncio
• Lessons learned

4. Where we started
Where we started

5. Where we started

6. Optimizely Classic was an
A/B Testing Solution

7. Optimizely X is an experimentation platform

8. Optimizely X Web
Visual Editor Based Experimentation

9. Optimizely X Full Stack
Code Based Experimentation
Optimizely X Web
Visual Editor Based Experimentation

10. Optimizely X Full Stack
Code Based Experimentation
Optimizely X Web
Visual Editor Based Experimentation
Experimentation Platform

13. What is happening?

17. How can we turn a good web UI into
a good API?

18. What makes an API “good”?

19. Easy to use APIs
What makes an API “good”?

20. More than mapping a data model to an
HTTP endpoint

21. API is the UI for developers

22. TODO: Add results slide.

23. Getting Optimizely Results
1. Get Experiment data

24. Getting Optimizely Results
1. Get Experiment data
2. Get Layer data

25. Getting Optimizely Results
1. Get Experiment data
2. Get Layer data (whatever that is)

26. Getting Optimizely Results
1. Get Experiment data
2. Get Layer data (whatever that is)
3. Get Project Settings

27. Getting Optimizely Results
1. Get Experiment data
2. Get Layer data (whatever that is)
3. Get Project Settings
4. Get results data

28. 1. Get Experiment data
2. Get Layer data (whatever that is)
3. Get Project Settings
4. Get results data
GET /v2/experiments/:id/results
Getting Optimizely Results

29. Easy to use APIs
What makes an API “good”?
Ensured Trust

30. Build a business on our APIs

31. How do we ensure trust?
No surprises

32. How do we ensure trust?
No surprises
Reliable platform

33. How do we ensure trust?
No surprises
Reliable platform
Performant platform

34. Trust can be made with:
Interface Stability

36. Interface stability means...
Beta & Stable APIs

37. Interface stability means...
Beta & Stable APIs
No breaking
changes

38. Interface stability means...
Beta & Stable APIs
No breaking
changes
Versioning

39. Trust can be made with:
Interface Stability
Reliability

40. 99.999%

42. Trust can be made with:
Interface Stability
Reliability
Performance

43. Performance is mission critical

44. Performance can be a
dealbreaker

45. Trust can be made with:
Interface Stability
Reliability
Performance

46. Easy to use APIs
What makes an API “good”?
Ensured Trust Documentation

47. Write documentation

48. Write documentation

49. Generate documentation

50. Generate documentation with
swagger-ui to ensure docs
are...

51. Generate documentation with
swagger-ui to ensure docs
are...
Up to Date

52. Generate documentation with
swagger-ui to ensure docs
are...
Up to Date Comprehensive

53. Generate documentation with
swagger-ui to ensure docs
are...
Up to Date Comprehensive Accurate

55. Easy to use APIs
What makes an API “good”?
Ensured Trust Documentation

56. How can we provide
“good” APIs?
How can we provide
“good” APIs?

57. Getting Optimizely Results
1. Get Experiment data
2. Get Layer data (whatever that is)
3. Get Project Settings
4. Get results data

58. 1. Get Experiment data
2. Get Layer data (whatever that is)
3. Get Project Settings
4. Get results data
/v2/experiments/:id/results
Getting Optimizely Results

59. We’ve got data all over

60. We’ve got data all over
Experiments

61. We’ve got data all over
Experiments Projects

62. We’ve got data all over
Experiments Projects Results

63. We’ve got data all over
Experiments Projects Results
API
Gateway

64. Benefits of an API gateway
Operations through the API look a lot like
their web UI counterparts

65. Benefits of an API gateway
Operations through the API look a lot like
their web UI counterparts
Allows us to orchestrate APIs instead of
data models

66. Benefits of an API gateway
Operations through the API look a lot like
their web UI counterparts
Allows us to orchestrate APIs instead of
data models
Pulls problems away from business logic
into a higher-level

67. Benefits of an API gateway
Operations through the API look a lot like
their web UI counterparts
Allows us to orchestrate APIs instead of
data models
Pulls problems away from business logic
into a higher-level
Internal and external consumers go
through the same application

68. api.optimizely.com What we’ve got

69. Python 3 + Pyramid + pyramid_swagger

70. We need to do a bunch of things in parallel...

71. Pyramid and Multiprocessing
with multiprocessing.Pool() as pool, multiprocessing.Manager() as manager:
response_queue = manager.Queue()
try:
# send our requests get back responses from the response_queue
finally:
pool.join()
pool.terminate()

72. Pyramid and Multiprocessing
with multiprocessing.Pool() as pool, multiprocessing.Manager() as manager:
response_queue = manager.Queue()
try:
# send our requests get back responses from the response_queue
finally:
pool.join()
pool.terminate()

73. Pyramid and Multiprocessing
with multiprocessing.Pool() as pool, multiprocessing.Manager() as manager:
response_queue = manager.Queue()
try:
# send our requests get back responses from the response_queue
finally:
pool.join()
pool.terminate()

74. Pyramid and Multiprocessing
with multiprocessing.Pool() as pool, multiprocessing.Manager() as manager:
response_queue = manager.Queue()
try:
# send our requests get back responses from the response_queue
finally:
pool.join()
pool.terminate()

75. An Unhealthy Situation
AWS Elastic Load Balancer 5XX

76. An Unhealthy Situation
Textbook case for non blocking IO
AWS Elastic Load Balancer 5XX

77. Not the Experience We Want

78. Blocking IO: “Normal” PythonApplicationKernel
Start

79. Blocking IO: “Normal” PythonApplicationKernel
Start read()

80. Blocking IO: “Normal” PythonApplicationKernel
Start read()
Initiate read

81. Blocking IO: “Normal” PythonApplicationKernel
Start read()
Initiate read
Receive
response

82. Blocking IO: “Normal” PythonApplicationKernel
Start read()
Initiate read
read()
returns
Receive
response

83. Blocking IO: “Normal” PythonApplicationKernel
Start read()
Initiate read
read()
returns
Receive
response

84. Nonblocking IOApplicationKernel
Start

85. Nonblocking IOApplicationKernel
Start read()

86. Nonblocking IOApplicationKernel
Start read()
Initiate read,
returns
immediately

87. Nonblocking IOApplicationKernel
Start read()
Initiate read,
returns
immediately
poll()until
data has arrived

88. Nonblocking IOApplicationKernel
Start read()
Initiate read,
returns
immediately
Receive
response
poll()until
data has arrived

89. Nonblocking IOApplicationKernel
Start read()
Initiate read,
returns
immediately
read()
Receive
response
poll()until
data has arrived

90. Nonblocking IOApplicationKernel
Start read()
Initiate read,
returns
immediately
read()
Receive
response
poll()until
data has arrived

91. Nonblocking IOApplicationKernel
poll(5)

92. Nonblocking IOApplicationKernel
poll(5) poll(5)

93. Nonblocking IOApplicationKernel
poll(5) poll(5)poll(5)

94. Nonblocking IOApplicationKernel
Receive
response
poll(5) poll(5)poll(5)

95. Nonblocking IOApplicationKernel
read()
Receive
response
poll(5) poll(5)poll(5)

96. Nonblocking IOApplicationKernel
read()
Receive
response
poll(5) poll(5)poll(5)

97. Non Blocking IO
Tw
isted
N
odeJS
Tornado
2002
N
etty2004 20092006
EventM
achine
2008
Eventlet
2015
Asyncio
2012
G
o/G
orutines

98. What’s it good for?
Proxy

99. What’s it good for?
Proxy

100. What’s it good for?
Push Updates/Chat

101. What’s it good for?
Push Updates/Chat

102. How do you schedule this work?

103. What framework?

104. Enter Asyncio
Python 3.4

105. async def i_take_a_while():
print('slow func start')
await asyncio.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_take_a_while()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_take_a_whilei_run_fast

106. async def i_take_a_while():
print('slow func start')
await asyncio.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_take_a_while()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_take_a_whilei_run_fast

107. async def i_take_a_while():
print('slow func start')
await asyncio.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_take_a_while()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_take_a_whilei_run_fast
fast func
start

108. async def i_take_a_while():
print('slow func start')
await asyncio.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_take_a_while()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_take_a_whilei_run_fast
fast func
start

109. async def i_take_a_while():
print('slow func start')
await asyncio.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_take_a_while()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_take_a_whilei_run_fast
slow func
start
fast func
start

110. async def i_take_a_while():
print('slow func start')
await asyncio.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_take_a_while()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_take_a_whilei_run_fast
slow func
start
fast func
start

111. async def i_take_a_while():
print('slow func start')
await asyncio.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_take_a_while()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_take_a_whilei_run_fast
slow func
start
fast func
start

112. async def i_take_a_while():
print('slow func start')
await asyncio.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_take_a_while()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_take_a_whilei_run_fast
slow func
start
fast func
start
1 sec after start
fast func
done

113. async def i_take_a_while():
print('slow func start')
await asyncio.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_take_a_while()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_take_a_whilei_run_fast
slow func
start
fast func
start
1 sec after start
fast func
done

114. async def i_take_a_while():
print('slow func start')
await asyncio.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_take_a_while()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_take_a_whilei_run_fast
slow func
start
fast func
start
1 sec after start
fast func
done

115. async def i_take_a_while():
print('slow func start')
await asyncio.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_take_a_while()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_take_a_whilei_run_fast
slow func
start
fast func
start
1 sec after start
fast func
done
3 sec after start
slow func
done

116. Blocking -> Asyncio
Gunicorn Pyramid Pyramid View

117. Blocking -> Asyncio
Gunicorn Pyramid Pyramid View
Switch to asyncio worker class http://
docs.gunicorn.org/en/stable/design.html#asyncio-
workers

118. Blocking -> Asyncio
Gunicorn Pyramid Pyramid View
Switch to aiopyramid https://github.com/housleyjk/aiopyramid

119. Blocking -> Asyncio
Gunicorn Pyramid Pyramid View
We can rewrite the views incrementally? Right?

120. async def i_block():
print('slow func start')
time.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_block()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_blocki_run_fast

121. async def i_block():
print('slow func start')
time.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_block()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_blocki_run_fast
fast func
start

122. async def i_block():
print('slow func start')
time.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_block()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_blocki_run_fast
fast func
start

123. async def i_block():
print('slow func start')
time.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_block()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_blocki_run_fast
block func
start
fast func
start

124. async def i_block():
print('slow func start')
time.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_block()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_blocki_run_fast
block func
start
fast func
start

125. async def i_block():
print('slow func start')
time.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_block()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_blocki_run_fast
block func
start
fast func
start
3 sec later
block func
done

126. async def i_block():
print('slow func start')
time.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_block()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_blocki_run_fast
block func
start
fast func
start
3 sec later
block func
done

127. async def i_block():
print('slow func start')
time.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_block()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_blocki_run_fast
block func
start
fast func
start
3 sec later
block func
done

128. async def i_block():
print('slow func start')
time.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_block()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_blocki_run_fast
block func
start
fast func
start
3 sec later
block func
done

129. async def i_block():
print('slow func start')
time.sleep(3)
print('slow func done')
async def i_run_fast():
print('fast func start')
await asyncio.sleep(1)
print('fast func done')
async def run_me():
await asyncio.gather(
i_run_fast(),
i_block()
)
loop = asyncio.get_event_loop()
loop.run_until_complete(run_me())
loop.close()
Event loop i_blocki_run_fast
block func
start
fast func
start
3 sec later
block func
done
fast func
done

130. Blocking and Nonblocking IO Don’t Mix

131. Blocking and Nonblocking IO Don’t Mix

132. asyncio all the things

133. asyncio all the things

134. How Do You Make Big Changes?

135. How Do You Make Big Changes?

136. How Do You Make Big Changes?

137. Code is Social

138. Code is Social
• Better make sure it actually works!

139. Code is Social
• Better make sure it actually works!
• Pull Request Open 3+ weeks

140. Code is Social
• Better make sure it actually works!
• Pull Request Open 3+ weeks
• Everyone on team added as a reviewer

141. Code is Social
• Better make sure it actually works!
• Pull Request Open 3+ weeks
• Everyone on team added as a reviewer
• 63 comments on Pull Request

142. Code is Social
• Better make sure it actually works!
• Pull Request Open 3+ weeks
• Everyone on team added as a reviewer
• 63 comments on Pull Request
• Meetings for questions/concerns

143. Happily ever after
AWS Elastic Load Balancer 5XX

144. Happily ever after
AWS Elastic Load Balancer 5XX

145. Happily ever after
99th percentile latency

146. Happily ever after
99th percentile latency

147. asyncio

148. • asyncio from the start is ideal
asyncio

149. • asyncio from the start is ideal
• But can add it later if you have to
asyncio

150. • asyncio from the start is ideal
• But can add it later if you have to
• Still early, but support will only improve
asyncio

151. Good APIs are Hard

152. Good APIs are Hard
• Lots of details to get right

153. Good APIs are Hard
• Lots of details to get right
• Using the right technologies helps

154. Good APIs are Hard
• Lots of details to get right
• Using the right technologies helps
• Always keep iterating and improving

155. Come visit us at our booth!
@Optimizely
Try out Optimizely:
optimizely.com/get-started

156. Q&A