PyWren is a serverless framework that allows data scientists to easily scale Python code across AWS Lambda. It uses Lambda to parallelize work by mapping Python functions to a large dataset. The functions and data are serialized and uploaded to S3, which then triggers Lambda. Results are stored in S3. This allows data science problems that take minutes or hours to be solved to complete in seconds by parallelizing across thousands of Lambda instances. PyWren aims to abstract away the complexity of serverless infrastructure so data scientists can focus on their code instead of operations.

1. MICROSERVICES &TERAFLOPS
Effortlessly scaling data science with PyWren
#thecloudistoodamnhard
Shivaram Venkataraman
PhD Candidate, UC Berkeley
Eric
Jonas
Ben
Recht
Ion
Stoica
Qifan
Pu
Berkeley
Center for
Computational
Imaging

2. WHO AM I ?
PhD candidate.AMPLab, RISELab @ UC Berkeley
Dissertation: System design for large scale machine learning
Apache Spark PMC Member
Contributions to Spark core, MLlib, SparkR

4. “BIG” DATA

5. “BIG” DATA
(near-by) stars

6. “BIG” DATA
(near-by) stars neurons

7. “BIG” DATA
(near-by) stars neurons

8. “BIG” DATA
(near-by) stars neurons
size
number
data size

9. “BIG” DATA
(near-by) stars neurons
size
number
data size
10^9 m
1
2 PB

10. “BIG” DATA
(near-by) stars neurons
size
number
data size
10^9 m
1
2 PB
10^-5m
10^11
12 TB/sec

11. images courtesy NASA SOHO
you are here

12. images courtesy NASA SOHO
Sun in UV (304 Å)you are here

13. Solar Flare Prediction Using Photospheric and Coronal Image Data.
Jonas, Bobra, Shankar, Recht. American Geophysical Union, 2016

14. Solar Flare Prediction Using Photospheric and Coronal Image Data.
Jonas, Bobra, Shankar, Recht. American Geophysical Union, 2016

15. NEUROSCIENCE AT ALL SCALES

16. NEUROSCIENCE AT ALL SCALES

17. NEUROSCIENCE AT ALL SCALES

18. NEUROSCIENCE AT ALL SCALES

20. Adaptive Optics

21. Superresolution
Adaptive Optics

22. Superresolution
Tomography
Adaptive Optics

23. Superresolution
Phase contrastTomography
Adaptive Optics

27. How do you get
busy physicists and
electrical engineers
to give up Matlab?
How do we get
busy astronomers
to give up IDL?

28. PREVIOUSLY, ON

29. PREVIOUSLY, ON

30. Why is there no
“cloud button”?
PREVIOUSLY, ON

31. #THECLOUDISTOODAMNHARD

32. #THECLOUDISTOODAMNHARD
• What type? what
instance? What base
image?

33. #THECLOUDISTOODAMNHARD
• What type? what
instance? What base
image?
• How many to spin up?
What price? spot?

34. #THECLOUDISTOODAMNHARD
• What type? what
instance? What base
image?
• How many to spin up?
What price? spot?
• wait,Wait,WAIT oh god

35. #THECLOUDISTOODAMNHARD
• What type? what
instance? What base
image?
• How many to spin up?
What price? spot?
• wait,Wait,WAIT oh god
• now what? DEVOPS

36. WHAT DO WE WANT?
1. Very little overhead for setup
once someone has an AWS account. In particular,
no persistent overhead -- you don't have to keep
a large (expensive) cluster up and you don't have
to wait 10+ min for a cluster to come up

37. WHAT DO WE WANT?
2. As close to zero overhead for users as possible
In particular, anyone who can write python
should be able to invoke it through a reasonable
interface. It should support all legacy code

38. WHAT DO WE WANT?
3. Target jobs that run in the
minutes-or-more regime.

39. “Most wrens are small and rather inconspicuous, except
for their loud and often complex songs.”
PyWrenpywren.io

40. pip install pywren

41. PYWREN:THE API

42. USING
“SERVERLESS INFRASTRUCTURE”

43. Powered by Continuum Analytics
+

44. • 300 seconds
single-core (AVX2)
• 512 MB in /tmp
• 1.5GB RAM
• Python, Java, Node
AWS LAMBDA

45. • 300 seconds
single-core (AVX2)
• 512 MB in /tmp
• 1.5GB RAM
• Python, Java, Node
AWS LAMBDA

46. • 300 seconds
single-core (AVX2)
• 512 MB in /tmp
• 1.5GB RAM
• Python, Java, Node
AWS LAMBDA

47. LAMBDA SCALABILITY

48. LAMBDA SCALABILITY

49. YOU CAN DO A LOT OF
WORK WITH MAP!
ETL

50. YOU CAN DO A LOT OF
WORK WITH MAP!
ETL
parameter
tuning

51. IMAGENET EXAMPLE
Preprocess 1.4M images from
IMAGENET
Compute GIST
image descriptor
(some random
python code off
the internet)

57. 80%+ of time
is spent doing
actual work

58. HOW IT WORKS
your laptop the cloud

59. HOW IT WORKS
your laptop the cloud
future = runner.map(fn, data)

60. HOW IT WORKS
your laptop the cloud
future = runner.map(fn, data)
Serialize func and data

61. HOW IT WORKS
your laptop the cloud
future = runner.map(fn, data)
Serialize func and data
Put on S3
func datadatadata

62. HOW IT WORKS
your laptop the cloud
future = runner.map(fn, data)
Serialize func and data
Put on S3
Invoke Lambda
func datadatadata

63. HOW IT WORKS
your laptop the cloud
future = runner.map(fn, data)
Serialize func and data
Put on S3
Invoke Lambda
func datadatadata

64. HOW IT WORKS
pull job from s3
your laptop the cloud
future = runner.map(fn, data)
Serialize func and data
Put on S3
Invoke Lambda
func datadatadata

65. HOW IT WORKS
pull job from s3
download anaconda runtime
your laptop the cloud
future = runner.map(fn, data)
Serialize func and data
Put on S3
Invoke Lambda
func datadatadata

66. HOW IT WORKS
pull job from s3
download anaconda runtime
python to run code
your laptop the cloud
future = runner.map(fn, data)
Serialize func and data
Put on S3
Invoke Lambda
func datadatadata

67. HOW IT WORKS
pull job from s3
download anaconda runtime
python to run code
your laptop the cloud
future = runner.map(fn, data)
Serialize func and data
Put on S3
Invoke Lambda
func datadatadata
future.result()

68. HOW IT WORKS
pull job from s3
download anaconda runtime
python to run code
your laptop the cloud
future = runner.map(fn, data)
Serialize func and data
Put on S3
Invoke Lambda
func datadatadata
future.result()
poll S3

69. HOW IT WORKS
pull job from s3
download anaconda runtime
python to run code
pickle result
your laptop the cloud
future = runner.map(fn, data)
Serialize func and data
Put on S3
Invoke Lambda
func datadatadata
future.result()
poll S3

70. HOW IT WORKS
pull job from s3
download anaconda runtime
python to run code
pickle result
stick in S3
your laptop the cloud
future = runner.map(fn, data)
Serialize func and data
Put on S3
Invoke Lambda
func datadatadata
future.result()
poll S3
result

71. HOW IT WORKS
pull job from s3
download anaconda runtime
python to run code
pickle result
stick in S3
your laptop the cloud
future = runner.map(fn, data)
Serialize func and data
Put on S3
Invoke Lambda
func datadatadata
future.result()
poll S3
unpickle and return
result

72. THE ROLE OF SERIALIZATION

73. THE ROLE OF SERIALIZATION
• Python’s dynamism is a help!

74. THE ROLE OF SERIALIZATION
• Python’s dynamism is a help!
• default serialization method,“pickle”, doesn’t
support everything, leading to cloudpickle

75. THE ROLE OF SERIALIZATION
• Python’s dynamism is a help!
• default serialization method,“pickle”, doesn’t
support everything, leading to cloudpickle
• How to handle modules? recursively walk and
package

76. THE ROLE OF SERIALIZATION
• Python’s dynamism is a help!
• default serialization method,“pickle”, doesn’t
support everything, leading to cloudpickle
• How to handle modules? recursively walk and
package
• C-dependencies don’t work, so must build runtimes

77. (Leptotyphlops carlae)

78. (Leptotyphlops carlae)
Want our runtime to include

79. (Leptotyphlops carlae)
Start 1205MB
Want our runtime to include

80. (Leptotyphlops carlae)
Start
conda clean
1205MB
Want our runtime to include

81. (Leptotyphlops carlae)
Start
conda clean
977 MB
1205MB
Want our runtime to include

82. (Leptotyphlops carlae)
Start
conda clean
eliminate pkg
977 MB
1205MB
Want our runtime to include

83. (Leptotyphlops carlae)
Start
conda clean
eliminate pkg
977 MB
1205MB
946 MB
Want our runtime to include

84. (Leptotyphlops carlae)
Start
Delete non-AVX2 MKL
conda clean
eliminate pkg
977 MB
1205MB
946 MB
Want our runtime to include

85. (Leptotyphlops carlae)
Start
Delete non-AVX2 MKL
conda clean
eliminate pkg
977 MB
1205MB
946 MB
670 MB
Want our runtime to include

86. (Leptotyphlops carlae)
Start
Delete non-AVX2 MKL
strip shared libs
conda clean
eliminate pkg
977 MB
1205MB
946 MB
670 MB
Want our runtime to include

87. (Leptotyphlops carlae)
Start
Delete non-AVX2 MKL
strip shared libs
conda clean
eliminate pkg
977 MB
1205MB
946 MB
670 MB
510MB
Want our runtime to include

88. (Leptotyphlops carlae)
Start
Delete non-AVX2 MKL
strip shared libs
conda clean
eliminate pkg
delete pyc
977 MB
1205MB
946 MB
670 MB
510MB
Want our runtime to include

89. (Leptotyphlops carlae)
Start
Delete non-AVX2 MKL
strip shared libs
conda clean
eliminate pkg
delete pyc
977 MB
1205MB
441MB
946 MB
670 MB
510MB
Want our runtime to include

90. LIMITATIONS
(AND IDEAS FOR SOLUTIONS)
•Lambda limits
•S3 Limits
•richer operations

91. LIMITATIONS
(AND IDEAS FOR SOLUTIONS)
•Lambda limits
•S3 Limits
• richer operations

92. MAP IS NOT ENOUGH

93. MAP IS NOT ENOUGH?
A lot of data analytics looks like:

94. MAP IS NOT ENOUGH?
A lot of data analytics looks like:
Data

95. MAP IS NOT ENOUGH?
A lot of data analytics looks like:
ETL /
preprocessingData

96. MAP IS NOT ENOUGH?
A lot of data analytics looks like:
ETL /
preprocessing featurizationData

97. MAP IS NOT ENOUGH?
A lot of data analytics looks like:
ETL /
preprocessing featurizationData machine learning

98. MAP IS NOT ENOUGH?
A lot of data analytics looks like:
ETL /
preprocessing featurizationData machine learning
Great PyWren Fit

99. MAP IS NOT ENOUGH?
A lot of data analytics looks like:
ETL /
preprocessing featurizationData machine learning
Distributed!
Scale!TensorFlow
Deep MLBaseGreat PyWren Fit

100. MAP IS NOT ENOUGH?
A lot of data analytics looks like:
ETL /
preprocessing featurizationData machine learning
Distributed!
Scale!TensorFlow
Deep MLBaseGreat PyWren Fit

101. SINGLE-MACHINE REDUCE
What if we use one big
server !?

102. SINGLE-MACHINE REDUCE
What if we use one big
server !?
cores RAM COST
x1.32xlarge 64 2TB $14/hr
x1.16xlarge 32 1TB $7/hr
p2.16xlarge 32 +
16 GPUs
750 GB $14/hr
r4.16xlarge 32 500 GB $4/hr

103. SINGLE-MACHINE REDUCE
What if we use one big
server !?
futures = exec.map(function, data)
answer = exec.reduce(reduce_func, futures)
cores RAM COST
x1.32xlarge 64 2TB $14/hr
x1.16xlarge 32 1TB $7/hr
p2.16xlarge 32 +
16 GPUs
750 GB $14/hr
r4.16xlarge 32 500 GB $4/hr

104. WHAT ABOUT SHUFFLES ?

107. S3
TERASORT IN PYWREN

108. S3 map(partition_keys)
TERASORT IN PYWREN

109. S3 map(partition_keys)
Elastic Cache
(Redis)
TERASORT IN PYWREN

110. S3 map(partition_keys)
Elastic Cache
(Redis)
map(sort_partition)
TERASORT IN PYWREN

111. S3 map(partition_keys)
Elastic Cache
(Redis)
map(sort_partition) S3
TERASORT IN PYWREN

112. 204
369
431
1,471
0 300 600 900 1200 1500 1800
(1000, 30)
(1000, 10)
(500, 10)
(100, 10)
time (seconds)
(workers,Redisshards)
invocation
setup
S3 read/write
compute
Redis read/write
1TB SORT

113. WIP: SCHEDULING ENGINE

114. WIP: SCHEDULING ENGINE
• Rate-limiting of invocation
• intelligent retry behavior (user-specified)
• straggler detection and mitigation

115. THE SERVERLESS FUTURE

116. THE SERVERLESS FUTURE
Where we are going:

117. THE SERVERLESS FUTURE
• Users Users Users (You !!!)
Where we are going:

118. THE SERVERLESS FUTURE
• Users Users Users (You !!!)
• Circumventing limitations
Where we are going:

119. THE SERVERLESS FUTURE
• Users Users Users (You !!!)
• Circumventing limitations
• Make serialization rock-solid
Where we are going:

120. THE SERVERLESS FUTURE
• Users Users Users (You !!!)
• Circumventing limitations
• Make serialization rock-solid
• Richer primitives!
Where we are going:

121. Effortlessly scaling data science
with PyWren
Eric Jonas Ben Recht Ion StoicaQifan Pu

122. http://pywren.io
https://github.com/pywren/pywren
Effortlessly scaling data science
with PyWren
Eric Jonas Ben Recht Ion StoicaQifan Pu