PyParis2017 / Function-as-a-service - a pythonic perspective on severless computing, by Josef Spillner

Pôle Systematic Paris-Region
Pôle Systematic Paris-RegionPôle Systematic Paris-Region
Zürcher Fachhochschule
Function-as-a-tervice:
A Pythonic Perspective on terverless
Computing
Josef Spillner <josef.spillner@zhiw.ch>
Service Prototyping Lib (blog.zhiw.ch/icclib)
Zurich University of Applied Sciences
Jun 13, 2017 | PyPiris
2
Your Tutorial Agenda
50‘ FiiS overview ind some existing tools
20‘ Limbidi: Decompose your functions
20‘ Snifu: Run your functions
3
Your Tutorial Instructor
Josef Spillner <josef.spillner@zhiw.ch>
●
works it Zurich University of Applied Sciences
●
lectures Python progrimming to undergriduites
& misters of idvinced studies
●
performs reseirch in the Service Prototyping Lib
●
co-iuthored «Architecturil Trinsformitions in
Network Services ind Distributed Systems»
●
wrote miny rirely used Python things since 2003
[LS16]
4
Service Prototyping Lab + ICCLab
5
What is FaaS?
[mizikglobil.com]
“functions“
contiiners
pickiges
ictuil functions
FaaS
●
running functions in the cloud
(hosted functions)
●
reil “piy per use“ (per invocition,
per loid x time unit, e.g. GHz/100ms)
●
seemingly “serverless“
6
Examples of FaaS
[openwhisk.org]
monitoring event
sensor diti
log entry
git push
...
HTTP
XMPP
AMQP
...
mix 1 per hour
...
Your Python
functions!
JSON
pliin text
...
7
The FaaS Space
AWS Lambda
OpenWhisk
Functions
PyWren
[Limbdi]
Docker-LimbCI
Effe
OpenLimbdi
Lever OS
Fission
Funktion
Kubeless
Picisso
Serverless Frimework
[Limbdi, OW, GCF, AF]
Step Functions
[Limbdi]
X-Riy
[Limbdi]
Zippi
[Limbdi] Apex
[Limbdi]
Snafu
- in Python
8
The FaaS Space
AWS Lambda
OpenWhisk
Functions
PyWren
Webtisk.io
Hook.io
Docker-LimbCI
Effe
OpenLimbdi
Lever OS
Fission
Funktion
Kubeless
Picisso
9
The FaaS Space: Runtimes
Function-is-i-Service offerings in greiter detiil...
Trend: Sooner or liter → gips will be filled
10
The FaaS Space: Python runtimes
11
FaaS Synopsis in Python
def limbdi_hindler(event, context):
‘‘‘
event: dict
context: meti informition object
returns: dict, string, number, ...
‘‘‘
# ...
return “result“
AWS Limbdi:
def hindler(input):
‘‘‘
input: dict
returns: dict
‘‘‘
# ...
return {}
OpenWhisk:
def miin():
‘‘‘
input: vii flisk.request.get_diti()
returns: str
‘‘‘
# ...
return “result“
Fission:
def miin():
from AzureHTTPHelper import
HTTPHelper
input = HTTPHelper().post
# ...
open(os.environ[“res“], “w“).write(
json.dumps({“body“: “...“}))
miin()
Azure Functions:
Further differences:
● function scoping (e.g. with/without export in JiviScript)
● function niming (mingling on client or service side)
12
The WorldSs Tools for FaaS Devs
OpenWhisk
$ wsk
$ iz
AWS Lambda
$ iws limbdi
# openlimbdi
$ bin/idmin
$ kubeless
$ fission
does not compile
not scriptible
requires iccount
breiks minikube
(invoke reid error)
13
Open Lambda - Hands-on Time!
14
Fission - Hands-on Time!
15
Overlay Approach: PyWren
Improved conveyince of “serverless“ piridigm
●
no explicit deployment prior to execution
●
rither, deploys while executing
How it works:
●
cloudpickle to AWS S3
●
executes Limbdi function which reids/writes from/to S3
●
pirillelisition through mip functions
def my_function(b):
x = np.random.normal(0, b, 1024)
A = np.random.normal(0, b, (1024, 1024))
return np.dot(A, x)
pwex = pywren.default_executor()
res = pwex.map(my_function, np.linspace(0.1, 100, 1000))
16
17
Our Tools for FaaS Devs
Podilizer
(Jivi)
Limbidi
(Python)
Web2Cloud
(JiviScript)
todiy
Limbickup
(file bickups)
Limi
(relitionil diti)
Snifu
(FiiS host)
todiy
18
Lambada
Definition of “FiiSificition“
→ Process of iutomited decomposition of softwire ipplicition into i set of
deployed ind reidily composed function-level services.
FiiSificition := code inilysis + trinsformition + deployment + on-demind ictivition
Integrition Citegories:
●
generic (code/function unit generition)
●
single-provider integrition
●
multi-provider integrition
Decomposition Citegories:
●
stitic code inilysis
●
dynimic code inilysis
→ Limbidi: FiiSificition for Python
(currently limited to Limbdificition)
Depth Citegories:
●
shillow (file to function)
●
medium (function to lines)
●
deep (line to miny lines)
19
Lambada
Code Anilysis
Dependencies
●
imported modules
●
globil viriibles
●
dependency functions
●
defined in other module
●
defined in sime module
Input/Output
●
printed lines
●
input stitements
●
tiinting
●
stiteful function splitting
import time
import mith
level = 12
counter = 0
def fib(x):
globil counter
counter += 1
for i in ringe(counter):
i = mith.sin(counter)
if x in (1, 2):
return 1
return fib(x - 1) + fib(x - 2)
if __nime__ == "__miin__":
fib(level)
20
Lambada
Code Trinsformition
Rewrite rules, vii AST:
return 9 print(“hello“) local_func()
------------------- return 9 ----------------------
return {“ret“: 9} ----------------------------------------- local_func_stub()
return {“ret: 9“, “stdout“: “hello“}
Stubs, vii templites:
def func_stub(x):
input = json.dumps({“x“: x})
output = boto3.client(“lambda“).invoke(FN=“func“, Payload=input)
y = json.loads(output[“Payload“].read().decode(“utf-8“))
21
Lambada
Code Trinsformition
Stiteful proxies for Object-Oriented Progrimming:
class Test: → class Proxy:
def __init__(self): def __new__(cls, clsname, p=True):
self.x = 9 if p: # __new__ must return callable
return lambda: Proxy(clsname, False)
def test(self): else:
return self.x * 2 return object.__new__(cls)
def __init__(self, clsname, ignoreproxy): ...
def __getattr__(self, name): ...
→ Test becomes Proxy(“Test“), Test() then invokes proxy
→ test() becomes remote_test({“x“: 9}) through network proxy cliss
→ iutomiticilly upon import of cliss
22
Lambada
Code Deployment + Activition
Locil mode: source code modified locilly is copy
Remote mode: rewritten source code deployed ind invoked
$ limbidi [--locil] [--debug] [--endpoint <ep>] <file.py>
$ python3 -m limbidi <file.py>
>>> import limbidi
>>> limbidi.move(globils() [, endpoint=“...“])
23
Lambada - Hands-on Time!
[d0wn.com]
24
25
Snafu
The Swiss Army Knife of Serverless Computing
26
Snafu
Current Implementition
●
scilible from developer
single instince to multi-
tenint deployments
●
executes Python 2 & 3,
Jivi, JiviScript, C
●
integrites with FiiS
ecosystem it-lirge
●
extensible subsystems
SLOC: ~1800
(including subsystems: ~800)
$ pip instill snifu
$ docker run -ti jszhiw/snifu
27
Snafu
Stindilone mode
●
cill functions interictively
●
bitch mode with/without input pipe
●
performince, robustness & correctness tests
●
development
$ snifu
$ snifu -x <function> [<file/dir>]
$ snifu -l sqlite -e jivi -c limbdi -C messiging
28
Snafu
Diemon mode (control pline)
●
hosted functions
●
multi-tenint provisioning
●
per-tenint isolition
●
compitibility with existing
client tools
$ snifu-control
$ snifu-control -i iws -r -d -e docker
# snifu-iccounts --idd -k <k> -s <s> -e <ep>
29
Snafu
Integrition into the wider FiiS ecosystem
snafu-import
Snifu Funktion
Fission
Kubeless
...
targets
sources
AWS
IBM
Google $ snifu-import 
--source <s> 
--tirget <t>
$ iliis iws=“iws 
--endpoint-url 
http://locilhost:10000“
$ wsk property set 
--ipihost 
locilhost:10000
$ ./tools/pitch-gcloud
30
Snafu - Hands-on Time!
[pinterest.com]
31
Q&A / Live help session
[dribbble.com]
32
Further Reading and FaaS Fun
Limi, Limbickup:
●
https://arxiv.org/abs/1701.05945
Podilizer:
●
https://arxiv.org/abs/1702.05510
Snifu:
●
https://arxiv.org/abs/1703.07562
Limbidi
●
https://arxiv.org/abs/1705.08169
On irXiv Anilytics: On GitHub:
[github.com/
serviceprototypinglab]
1 of 32

Recommended

Joblib for cloud computing by
Joblib for cloud computingJoblib for cloud computing
Joblib for cloud computingAlexandre Abadie
2.6K views59 slides
All you need to know about the JavaScript event loop by
All you need to know about the JavaScript event loopAll you need to know about the JavaScript event loop
All you need to know about the JavaScript event loopSaša Tatar
981 views15 slides
Machine learning with py torch by
Machine learning with py torchMachine learning with py torch
Machine learning with py torchRiza Fahmi
644 views71 slides
あなたのScalaを爆速にする7つの方法 by
あなたのScalaを爆速にする7つの方法あなたのScalaを爆速にする7つの方法
あなたのScalaを爆速にする7つの方法x1 ichi
9.1K views74 slides

More Related Content

What's hot

Internship final report@Treasure Data Inc. by
Internship final report@Treasure Data Inc.Internship final report@Treasure Data Inc.
Internship final report@Treasure Data Inc.Ryuichi ITO
2.6K views97 slides
Functional Reactive Programming with RxJS by
Functional Reactive Programming with RxJSFunctional Reactive Programming with RxJS
Functional Reactive Programming with RxJSstefanmayer13
40.4K views39 slides
JavaScript Engines and Event Loop by
JavaScript Engines and Event Loop JavaScript Engines and Event Loop
JavaScript Engines and Event Loop Tapan B.K.
356 views28 slides
Javascript Everywhere by
Javascript EverywhereJavascript Everywhere
Javascript EverywherePascal Rettig
2.2K views54 slides
Letswift19-clean-architecture by
Letswift19-clean-architectureLetswift19-clean-architecture
Letswift19-clean-architectureJung Kim
1.6K views104 slides
Swift after one week of coding by
Swift after one week of codingSwift after one week of coding
Swift after one week of codingSwiftWro
1.9K views29 slides

What's hot(19)

Internship final report@Treasure Data Inc. by Ryuichi ITO
Internship final report@Treasure Data Inc.Internship final report@Treasure Data Inc.
Internship final report@Treasure Data Inc.
Ryuichi ITO2.6K views
Functional Reactive Programming with RxJS by stefanmayer13
Functional Reactive Programming with RxJSFunctional Reactive Programming with RxJS
Functional Reactive Programming with RxJS
stefanmayer1340.4K views
JavaScript Engines and Event Loop by Tapan B.K.
JavaScript Engines and Event Loop JavaScript Engines and Event Loop
JavaScript Engines and Event Loop
Tapan B.K.356 views
Javascript Everywhere by Pascal Rettig
Javascript EverywhereJavascript Everywhere
Javascript Everywhere
Pascal Rettig2.2K views
Letswift19-clean-architecture by Jung Kim
Letswift19-clean-architectureLetswift19-clean-architecture
Letswift19-clean-architecture
Jung Kim1.6K views
Swift after one week of coding by SwiftWro
Swift after one week of codingSwift after one week of coding
Swift after one week of coding
SwiftWro1.9K views
Simple ETL in python 3.5+ with Bonobo - PyParis 2017 by Romain Dorgueil
Simple ETL in python 3.5+ with Bonobo - PyParis 2017Simple ETL in python 3.5+ with Bonobo - PyParis 2017
Simple ETL in python 3.5+ with Bonobo - PyParis 2017
Romain Dorgueil1.7K views
PlantUML by Leo Liang
PlantUMLPlantUML
PlantUML
Leo Liang3.4K views
Scala Future & Promises by Knoldus Inc.
Scala Future & PromisesScala Future & Promises
Scala Future & Promises
Knoldus Inc.3.3K views
Flux and InfluxDB 2.0 by Paul Dix by InfluxData
Flux and InfluxDB 2.0 by Paul DixFlux and InfluxDB 2.0 by Paul Dix
Flux and InfluxDB 2.0 by Paul Dix
InfluxData1.3K views
Python Objects by Quintagroup
Python ObjectsPython Objects
Python Objects
Quintagroup2.3K views

Viewers also liked

Serverless computing and Function-as-a-Service (FaaS) by
Serverless computing and Function-as-a-Service (FaaS)Serverless computing and Function-as-a-Service (FaaS)
Serverless computing and Function-as-a-Service (FaaS)Moritz Strube
1.6K views19 slides
Five myths about Network Function Virtualization (NFV) by
Five myths about Network Function Virtualization (NFV) Five myths about Network Function Virtualization (NFV)
Five myths about Network Function Virtualization (NFV) srichakra komatineni
513 views6 slides
Function as a Service: IT forum expo 2017 by
Function as a Service: IT forum expo 2017Function as a Service: IT forum expo 2017
Function as a Service: IT forum expo 2017Igor Rosa Macedo
520 views35 slides
Managing change in the data center network by
Managing change in the data center networkManaging change in the data center network
Managing change in the data center networkInterop
957 views23 slides
Data center interconnect seamlessly through SDN by
Data center interconnect seamlessly through SDNData center interconnect seamlessly through SDN
Data center interconnect seamlessly through SDNFelecia Fierro
478 views22 slides
Arista Networks - Building the Next Generation Workplace and Data Center Usin... by
Arista Networks - Building the Next Generation Workplace and Data Center Usin...Arista Networks - Building the Next Generation Workplace and Data Center Usin...
Arista Networks - Building the Next Generation Workplace and Data Center Usin...Aruba, a Hewlett Packard Enterprise company
7.8K views61 slides

Viewers also liked(20)

Serverless computing and Function-as-a-Service (FaaS) by Moritz Strube
Serverless computing and Function-as-a-Service (FaaS)Serverless computing and Function-as-a-Service (FaaS)
Serverless computing and Function-as-a-Service (FaaS)
Moritz Strube1.6K views
Five myths about Network Function Virtualization (NFV) by srichakra komatineni
Five myths about Network Function Virtualization (NFV) Five myths about Network Function Virtualization (NFV)
Five myths about Network Function Virtualization (NFV)
Function as a Service: IT forum expo 2017 by Igor Rosa Macedo
Function as a Service: IT forum expo 2017Function as a Service: IT forum expo 2017
Function as a Service: IT forum expo 2017
Igor Rosa Macedo520 views
Managing change in the data center network by Interop
Managing change in the data center networkManaging change in the data center network
Managing change in the data center network
Interop957 views
Data center interconnect seamlessly through SDN by Felecia Fierro
Data center interconnect seamlessly through SDNData center interconnect seamlessly through SDN
Data center interconnect seamlessly through SDN
Felecia Fierro478 views
Nexus 7000 Series Innovations: M3 Module, DCI, Scale by Tony Antony
Nexus 7000 Series Innovations: M3 Module, DCI, ScaleNexus 7000 Series Innovations: M3 Module, DCI, Scale
Nexus 7000 Series Innovations: M3 Module, DCI, Scale
Tony Antony3.1K views
Haxe dci-presentation by Andreas SÖDERLUND by Grégory PARODI
Haxe   dci-presentation by Andreas SÖDERLUNDHaxe   dci-presentation by Andreas SÖDERLUND
Haxe dci-presentation by Andreas SÖDERLUND
Grégory PARODI1.9K views
DCI - the architecture from the future by Andrzej Krzywda
DCI - the architecture from the futureDCI - the architecture from the future
DCI - the architecture from the future
Andrzej Krzywda1.3K views
The New Network for the Data Center by Juniper Networks
The New Network for the Data CenterThe New Network for the Data Center
The New Network for the Data Center
Juniper Networks2.6K views
Alcatel-Lucent Cloud: Shaping the Future NFV OSS David Amzallag TM Forum 2013 by Alcatel-Lucent Cloud
Alcatel-Lucent Cloud: Shaping the Future NFV OSS David Amzallag TM Forum 2013Alcatel-Lucent Cloud: Shaping the Future NFV OSS David Amzallag TM Forum 2013
Alcatel-Lucent Cloud: Shaping the Future NFV OSS David Amzallag TM Forum 2013
How to adopt SDN/NFV Technology into the BSS & OSS stack and shorten the time... by Comarch
How to adopt SDN/NFV Technology into the BSS & OSS stack and shorten the time...How to adopt SDN/NFV Technology into the BSS & OSS stack and shorten the time...
How to adopt SDN/NFV Technology into the BSS & OSS stack and shorten the time...
Comarch927 views
Data Center Interconnects: An Overview by XO Communications
Data Center Interconnects: An OverviewData Center Interconnects: An Overview
Data Center Interconnects: An Overview
XO Communications2.5K views
Managing and Implementing Network Function Virtualization with Intelligent OSS by Comarch
Managing and Implementing Network Function Virtualization with Intelligent OSSManaging and Implementing Network Function Virtualization with Intelligent OSS
Managing and Implementing Network Function Virtualization with Intelligent OSS
Comarch1.1K views
Container as a Service with Docker by Patrick Chanezon
Container as a Service with DockerContainer as a Service with Docker
Container as a Service with Docker
Patrick Chanezon10.1K views
OSS in the era of SDN and NFV: Evolution vs Revolution - What we can learn f... by Colt Technology Services
OSS in the era of SDN and NFV:  Evolution vs Revolution - What we can learn f...OSS in the era of SDN and NFV:  Evolution vs Revolution - What we can learn f...
OSS in the era of SDN and NFV: Evolution vs Revolution - What we can learn f...
How will virtual networks, controlled by software, impact OSS systems? by Comarch
How will virtual networks, controlled by software, impact OSS systems?How will virtual networks, controlled by software, impact OSS systems?
How will virtual networks, controlled by software, impact OSS systems?
Comarch1K views
Data Center Network Trends - Lin Nease by HPDutchWorld
Data Center Network Trends - Lin NeaseData Center Network Trends - Lin Nease
Data Center Network Trends - Lin Nease
HPDutchWorld2.5K views

Similar to PyParis2017 / Function-as-a-service - a pythonic perspective on severless computing, by Josef Spillner

Twins: OOP and FP by
Twins: OOP and FPTwins: OOP and FP
Twins: OOP and FPRichardWarburton
1.2K views74 slides
Cluj.py Meetup: Extending Python in C by
Cluj.py Meetup: Extending Python in CCluj.py Meetup: Extending Python in C
Cluj.py Meetup: Extending Python in CSteffen Wenz
1.1K views44 slides
Everything ruby by
Everything rubyEverything ruby
Everything rubyajeygore
752 views31 slides
Tasks: you gotta know how to run them by
Tasks: you gotta know how to run themTasks: you gotta know how to run them
Tasks: you gotta know how to run themFilipe Ximenes
221 views60 slides
PyHEP 2018: Tools to bind to Python by
PyHEP 2018:  Tools to bind to PythonPyHEP 2018:  Tools to bind to Python
PyHEP 2018: Tools to bind to PythonHenry Schreiner
862 views58 slides
Introduction to PiCloud by
Introduction to PiCloudIntroduction to PiCloud
Introduction to PiCloudBill Koch
3K views18 slides

Similar to PyParis2017 / Function-as-a-service - a pythonic perspective on severless computing, by Josef Spillner(20)

Cluj.py Meetup: Extending Python in C by Steffen Wenz
Cluj.py Meetup: Extending Python in CCluj.py Meetup: Extending Python in C
Cluj.py Meetup: Extending Python in C
Steffen Wenz1.1K views
Everything ruby by ajeygore
Everything rubyEverything ruby
Everything ruby
ajeygore752 views
Tasks: you gotta know how to run them by Filipe Ximenes
Tasks: you gotta know how to run themTasks: you gotta know how to run them
Tasks: you gotta know how to run them
Filipe Ximenes221 views
PyHEP 2018: Tools to bind to Python by Henry Schreiner
PyHEP 2018:  Tools to bind to PythonPyHEP 2018:  Tools to bind to Python
PyHEP 2018: Tools to bind to Python
Henry Schreiner862 views
Introduction to PiCloud by Bill Koch
Introduction to PiCloudIntroduction to PiCloud
Introduction to PiCloud
Bill Koch3K views
Distributed computing with Ray. Find your hyper-parameters, speed up your Pan... by Jan Margeta
Distributed computing with Ray. Find your hyper-parameters, speed up your Pan...Distributed computing with Ray. Find your hyper-parameters, speed up your Pan...
Distributed computing with Ray. Find your hyper-parameters, speed up your Pan...
Jan Margeta730 views
GDG-MLOps using Protobuf in Unity by Ivan Chiou
GDG-MLOps using Protobuf in UnityGDG-MLOps using Protobuf in Unity
GDG-MLOps using Protobuf in Unity
Ivan Chiou113 views
Python Load Testing - Pygotham 2012 by Dan Kuebrich
Python Load Testing - Pygotham 2012Python Load Testing - Pygotham 2012
Python Load Testing - Pygotham 2012
Dan Kuebrich6.2K views
MLOps Case Studies: Building fast, scalable, and high-accuracy ML systems at ... by Masashi Shibata
MLOps Case Studies: Building fast, scalable, and high-accuracy ML systems at ...MLOps Case Studies: Building fast, scalable, and high-accuracy ML systems at ...
MLOps Case Studies: Building fast, scalable, and high-accuracy ML systems at ...
Masashi Shibata418 views
Python Hashlib & A True Story of One Bug by delimitry
Python Hashlib & A True Story of One BugPython Hashlib & A True Story of One Bug
Python Hashlib & A True Story of One Bug
delimitry10 views
Node.js Patterns for Discerning Developers by cacois
Node.js Patterns for Discerning DevelopersNode.js Patterns for Discerning Developers
Node.js Patterns for Discerning Developers
cacois34.8K views
Nix for Python developers by Asko Soukka
Nix for Python developersNix for Python developers
Nix for Python developers
Asko Soukka5.5K views
Euro python2011 High Performance Python by Ian Ozsvald
Euro python2011 High Performance PythonEuro python2011 High Performance Python
Euro python2011 High Performance Python
Ian Ozsvald3K views
Python高级编程(二) by Qiangning Hong
Python高级编程(二)Python高级编程(二)
Python高级编程(二)
Qiangning Hong9.7K views
ECMAScript 6 and the Node Driver by MongoDB
ECMAScript 6 and the Node DriverECMAScript 6 and the Node Driver
ECMAScript 6 and the Node Driver
MongoDB5.6K views
End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017 by StampedeCon
End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017
End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017
StampedeCon1.8K views

More from Pôle Systematic Paris-Region

OSIS19_IoT :Transparent remote connectivity to short-range IoT devices, by Na... by
OSIS19_IoT :Transparent remote connectivity to short-range IoT devices, by Na...OSIS19_IoT :Transparent remote connectivity to short-range IoT devices, by Na...
OSIS19_IoT :Transparent remote connectivity to short-range IoT devices, by Na...Pôle Systematic Paris-Region
686 views39 slides
OSIS19_Cloud : SAFC: Scheduling and Allocation Framework for Containers in a ... by
OSIS19_Cloud : SAFC: Scheduling and Allocation Framework for Containers in a ...OSIS19_Cloud : SAFC: Scheduling and Allocation Framework for Containers in a ...
OSIS19_Cloud : SAFC: Scheduling and Allocation Framework for Containers in a ...Pôle Systematic Paris-Region
293 views24 slides
OSIS19_Cloud : Qu’apporte l’observabilité à la gestion de configuration? par ... by
OSIS19_Cloud : Qu’apporte l’observabilité à la gestion de configuration? par ...OSIS19_Cloud : Qu’apporte l’observabilité à la gestion de configuration? par ...
OSIS19_Cloud : Qu’apporte l’observabilité à la gestion de configuration? par ...Pôle Systematic Paris-Region
349 views38 slides
OSIS19_Cloud : Performance and power management in virtualized data centers, ... by
OSIS19_Cloud : Performance and power management in virtualized data centers, ...OSIS19_Cloud : Performance and power management in virtualized data centers, ...
OSIS19_Cloud : Performance and power management in virtualized data centers, ...Pôle Systematic Paris-Region
288 views27 slides
OSIS19_Cloud : Des objets dans le cloud, et qui y restent -- L'expérience du ... by
OSIS19_Cloud : Des objets dans le cloud, et qui y restent -- L'expérience du ...OSIS19_Cloud : Des objets dans le cloud, et qui y restent -- L'expérience du ...
OSIS19_Cloud : Des objets dans le cloud, et qui y restent -- L'expérience du ...Pôle Systematic Paris-Region
271 views30 slides
OSIS19_Cloud : Attribution automatique de ressources pour micro-services, Alt... by
OSIS19_Cloud : Attribution automatique de ressources pour micro-services, Alt...OSIS19_Cloud : Attribution automatique de ressources pour micro-services, Alt...
OSIS19_Cloud : Attribution automatique de ressources pour micro-services, Alt...Pôle Systematic Paris-Region
229 views9 slides

More from Pôle Systematic Paris-Region(20)

Recently uploaded

AMAZON PRODUCT RESEARCH.pdf by
AMAZON PRODUCT RESEARCH.pdfAMAZON PRODUCT RESEARCH.pdf
AMAZON PRODUCT RESEARCH.pdfJerikkLaureta
15 views13 slides
Melek BEN MAHMOUD.pdf by
Melek BEN MAHMOUD.pdfMelek BEN MAHMOUD.pdf
Melek BEN MAHMOUD.pdfMelekBenMahmoud
14 views1 slide
virtual reality.pptx by
virtual reality.pptxvirtual reality.pptx
virtual reality.pptxG036GaikwadSnehal
11 views15 slides
HTTP headers that make your website go faster - devs.gent November 2023 by
HTTP headers that make your website go faster - devs.gent November 2023HTTP headers that make your website go faster - devs.gent November 2023
HTTP headers that make your website go faster - devs.gent November 2023Thijs Feryn
19 views151 slides
From chaos to control: Managing migrations and Microsoft 365 with ShareGate! by
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!From chaos to control: Managing migrations and Microsoft 365 with ShareGate!
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!sammart93
9 views39 slides
Tunable Laser (1).pptx by
Tunable Laser (1).pptxTunable Laser (1).pptx
Tunable Laser (1).pptxHajira Mahmood
23 views37 slides

Recently uploaded(20)

AMAZON PRODUCT RESEARCH.pdf by JerikkLaureta
AMAZON PRODUCT RESEARCH.pdfAMAZON PRODUCT RESEARCH.pdf
AMAZON PRODUCT RESEARCH.pdf
JerikkLaureta15 views
HTTP headers that make your website go faster - devs.gent November 2023 by Thijs Feryn
HTTP headers that make your website go faster - devs.gent November 2023HTTP headers that make your website go faster - devs.gent November 2023
HTTP headers that make your website go faster - devs.gent November 2023
Thijs Feryn19 views
From chaos to control: Managing migrations and Microsoft 365 with ShareGate! by sammart93
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!From chaos to control: Managing migrations and Microsoft 365 with ShareGate!
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!
sammart939 views
Spesifikasi Lengkap ASUS Vivobook Go 14 by Dot Semarang
Spesifikasi Lengkap ASUS Vivobook Go 14Spesifikasi Lengkap ASUS Vivobook Go 14
Spesifikasi Lengkap ASUS Vivobook Go 14
Dot Semarang35 views
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive by Network Automation Forum
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLiveAutomating a World-Class Technology Conference; Behind the Scenes of CiscoLive
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive
Transcript: The Details of Description Techniques tips and tangents on altern... by BookNet Canada
Transcript: The Details of Description Techniques tips and tangents on altern...Transcript: The Details of Description Techniques tips and tangents on altern...
Transcript: The Details of Description Techniques tips and tangents on altern...
BookNet Canada130 views
Lilypad @ Labweek, Istanbul, 2023.pdf by Ally339821
Lilypad @ Labweek, Istanbul, 2023.pdfLilypad @ Labweek, Istanbul, 2023.pdf
Lilypad @ Labweek, Istanbul, 2023.pdf
Ally3398219 views
PharoJS - Zürich Smalltalk Group Meetup November 2023 by Noury Bouraqadi
PharoJS - Zürich Smalltalk Group Meetup November 2023PharoJS - Zürich Smalltalk Group Meetup November 2023
PharoJS - Zürich Smalltalk Group Meetup November 2023
Noury Bouraqadi120 views
The details of description: Techniques, tips, and tangents on alternative tex... by BookNet Canada
The details of description: Techniques, tips, and tangents on alternative tex...The details of description: Techniques, tips, and tangents on alternative tex...
The details of description: Techniques, tips, and tangents on alternative tex...
BookNet Canada121 views
Igniting Next Level Productivity with AI-Infused Data Integration Workflows by Safe Software
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software225 views
Voice Logger - Telephony Integration Solution at Aegis by Nirmal Sharma
Voice Logger - Telephony Integration Solution at AegisVoice Logger - Telephony Integration Solution at Aegis
Voice Logger - Telephony Integration Solution at Aegis
Nirmal Sharma17 views

PyParis2017 / Function-as-a-service - a pythonic perspective on severless computing, by Josef Spillner

  • 1. Zürcher Fachhochschule Function-as-a-tervice: A Pythonic Perspective on terverless Computing Josef Spillner <josef.spillner@zhiw.ch> Service Prototyping Lib (blog.zhiw.ch/icclib) Zurich University of Applied Sciences Jun 13, 2017 | PyPiris
  • 2. 2 Your Tutorial Agenda 50‘ FiiS overview ind some existing tools 20‘ Limbidi: Decompose your functions 20‘ Snifu: Run your functions
  • 3. 3 Your Tutorial Instructor Josef Spillner <josef.spillner@zhiw.ch> ● works it Zurich University of Applied Sciences ● lectures Python progrimming to undergriduites & misters of idvinced studies ● performs reseirch in the Service Prototyping Lib ● co-iuthored «Architecturil Trinsformitions in Network Services ind Distributed Systems» ● wrote miny rirely used Python things since 2003 [LS16]
  • 5. 5 What is FaaS? [mizikglobil.com] “functions“ contiiners pickiges ictuil functions FaaS ● running functions in the cloud (hosted functions) ● reil “piy per use“ (per invocition, per loid x time unit, e.g. GHz/100ms) ● seemingly “serverless“
  • 6. 6 Examples of FaaS [openwhisk.org] monitoring event sensor diti log entry git push ... HTTP XMPP AMQP ... mix 1 per hour ... Your Python functions! JSON pliin text ...
  • 7. 7 The FaaS Space AWS Lambda OpenWhisk Functions PyWren [Limbdi] Docker-LimbCI Effe OpenLimbdi Lever OS Fission Funktion Kubeless Picisso Serverless Frimework [Limbdi, OW, GCF, AF] Step Functions [Limbdi] X-Riy [Limbdi] Zippi [Limbdi] Apex [Limbdi] Snafu - in Python
  • 8. 8 The FaaS Space AWS Lambda OpenWhisk Functions PyWren Webtisk.io Hook.io Docker-LimbCI Effe OpenLimbdi Lever OS Fission Funktion Kubeless Picisso
  • 9. 9 The FaaS Space: Runtimes Function-is-i-Service offerings in greiter detiil... Trend: Sooner or liter → gips will be filled
  • 10. 10 The FaaS Space: Python runtimes
  • 11. 11 FaaS Synopsis in Python def limbdi_hindler(event, context): ‘‘‘ event: dict context: meti informition object returns: dict, string, number, ... ‘‘‘ # ... return “result“ AWS Limbdi: def hindler(input): ‘‘‘ input: dict returns: dict ‘‘‘ # ... return {} OpenWhisk: def miin(): ‘‘‘ input: vii flisk.request.get_diti() returns: str ‘‘‘ # ... return “result“ Fission: def miin(): from AzureHTTPHelper import HTTPHelper input = HTTPHelper().post # ... open(os.environ[“res“], “w“).write( json.dumps({“body“: “...“})) miin() Azure Functions: Further differences: ● function scoping (e.g. with/without export in JiviScript) ● function niming (mingling on client or service side)
  • 12. 12 The WorldSs Tools for FaaS Devs OpenWhisk $ wsk $ iz AWS Lambda $ iws limbdi # openlimbdi $ bin/idmin $ kubeless $ fission does not compile not scriptible requires iccount breiks minikube (invoke reid error)
  • 13. 13 Open Lambda - Hands-on Time!
  • 15. 15 Overlay Approach: PyWren Improved conveyince of “serverless“ piridigm ● no explicit deployment prior to execution ● rither, deploys while executing How it works: ● cloudpickle to AWS S3 ● executes Limbdi function which reids/writes from/to S3 ● pirillelisition through mip functions def my_function(b): x = np.random.normal(0, b, 1024) A = np.random.normal(0, b, (1024, 1024)) return np.dot(A, x) pwex = pywren.default_executor() res = pwex.map(my_function, np.linspace(0.1, 100, 1000))
  • 16. 16
  • 17. 17 Our Tools for FaaS Devs Podilizer (Jivi) Limbidi (Python) Web2Cloud (JiviScript) todiy Limbickup (file bickups) Limi (relitionil diti) Snifu (FiiS host) todiy
  • 18. 18 Lambada Definition of “FiiSificition“ → Process of iutomited decomposition of softwire ipplicition into i set of deployed ind reidily composed function-level services. FiiSificition := code inilysis + trinsformition + deployment + on-demind ictivition Integrition Citegories: ● generic (code/function unit generition) ● single-provider integrition ● multi-provider integrition Decomposition Citegories: ● stitic code inilysis ● dynimic code inilysis → Limbidi: FiiSificition for Python (currently limited to Limbdificition) Depth Citegories: ● shillow (file to function) ● medium (function to lines) ● deep (line to miny lines)
  • 19. 19 Lambada Code Anilysis Dependencies ● imported modules ● globil viriibles ● dependency functions ● defined in other module ● defined in sime module Input/Output ● printed lines ● input stitements ● tiinting ● stiteful function splitting import time import mith level = 12 counter = 0 def fib(x): globil counter counter += 1 for i in ringe(counter): i = mith.sin(counter) if x in (1, 2): return 1 return fib(x - 1) + fib(x - 2) if __nime__ == "__miin__": fib(level)
  • 20. 20 Lambada Code Trinsformition Rewrite rules, vii AST: return 9 print(“hello“) local_func() ------------------- return 9 ---------------------- return {“ret“: 9} ----------------------------------------- local_func_stub() return {“ret: 9“, “stdout“: “hello“} Stubs, vii templites: def func_stub(x): input = json.dumps({“x“: x}) output = boto3.client(“lambda“).invoke(FN=“func“, Payload=input) y = json.loads(output[“Payload“].read().decode(“utf-8“))
  • 21. 21 Lambada Code Trinsformition Stiteful proxies for Object-Oriented Progrimming: class Test: → class Proxy: def __init__(self): def __new__(cls, clsname, p=True): self.x = 9 if p: # __new__ must return callable return lambda: Proxy(clsname, False) def test(self): else: return self.x * 2 return object.__new__(cls) def __init__(self, clsname, ignoreproxy): ... def __getattr__(self, name): ... → Test becomes Proxy(“Test“), Test() then invokes proxy → test() becomes remote_test({“x“: 9}) through network proxy cliss → iutomiticilly upon import of cliss
  • 22. 22 Lambada Code Deployment + Activition Locil mode: source code modified locilly is copy Remote mode: rewritten source code deployed ind invoked $ limbidi [--locil] [--debug] [--endpoint <ep>] <file.py> $ python3 -m limbidi <file.py> >>> import limbidi >>> limbidi.move(globils() [, endpoint=“...“])
  • 23. 23 Lambada - Hands-on Time! [d0wn.com]
  • 24. 24
  • 25. 25 Snafu The Swiss Army Knife of Serverless Computing
  • 26. 26 Snafu Current Implementition ● scilible from developer single instince to multi- tenint deployments ● executes Python 2 & 3, Jivi, JiviScript, C ● integrites with FiiS ecosystem it-lirge ● extensible subsystems SLOC: ~1800 (including subsystems: ~800) $ pip instill snifu $ docker run -ti jszhiw/snifu
  • 27. 27 Snafu Stindilone mode ● cill functions interictively ● bitch mode with/without input pipe ● performince, robustness & correctness tests ● development $ snifu $ snifu -x <function> [<file/dir>] $ snifu -l sqlite -e jivi -c limbdi -C messiging
  • 28. 28 Snafu Diemon mode (control pline) ● hosted functions ● multi-tenint provisioning ● per-tenint isolition ● compitibility with existing client tools $ snifu-control $ snifu-control -i iws -r -d -e docker # snifu-iccounts --idd -k <k> -s <s> -e <ep>
  • 29. 29 Snafu Integrition into the wider FiiS ecosystem snafu-import Snifu Funktion Fission Kubeless ... targets sources AWS IBM Google $ snifu-import --source <s> --tirget <t> $ iliis iws=“iws --endpoint-url http://locilhost:10000“ $ wsk property set --ipihost locilhost:10000 $ ./tools/pitch-gcloud
  • 30. 30 Snafu - Hands-on Time! [pinterest.com]
  • 31. 31 Q&A / Live help session [dribbble.com]
  • 32. 32 Further Reading and FaaS Fun Limi, Limbickup: ● https://arxiv.org/abs/1701.05945 Podilizer: ● https://arxiv.org/abs/1702.05510 Snifu: ● https://arxiv.org/abs/1703.07562 Limbidi ● https://arxiv.org/abs/1705.08169 On irXiv Anilytics: On GitHub: [github.com/ serviceprototypinglab]