SlideShare a Scribd company logo
Twisted
                       Présentation & Usecases




Monday, June 1, 2009
Twisted?

                       • Framework pour applis réseau
                       • 100% Python, quelques optimisations en C
                       • Projet opensource stable, bien maintenu
                       • Programmation asynchrone

Monday, June 1, 2009
Asynchrone?

                       • Synonymes : “event-driven”, “non-blocking”
                       • Toutes les fonctions doivent retourner
                         “rapidement”
                       • 1 seul thread
                       • Reactor pattern

Monday, June 1, 2009
Sync vs. Async
         def google_sync(search)
             html = urllib.urlopen(“http://google.com/?q=”+search”).read()
             return [r[1] for r in re.split(“<h3>(.*?)</h3>”,html)]

         print google_sync(“pycon fr”)
         print google_sync(“pycon us”)




         def google_async(search)
             async_call = twisted.web.client.getPage(“http://google.com/?q=”+search”)
             async_call.addCallback(gotresults)

         def gotresults(html)
             print [r[1] for r in re.split(“<h3>(.*?)</h3>”,html)]

         google_async(“pycon fr”)
         google_async(“pycon us”)

         twisted.internet.reactor.run()



Monday, June 1, 2009
“C’est plus compliqué!”
                       • Oui mais
                       • 100 requêtes en parallèle ?




Monday, June 1, 2009
“C’est plus compliqué!”
                       • Oui mais
                       • 100 requêtes en parallèle ?
         results = []

         class google_thread(Thread):
             def __init__(self,search):
                 self.search = search

                def run():
                    html = urllib.urlopen(“http://google.com/?q=”+search”).read()
                    results.append([r[1] for r in re.split(“<h3>(.*?)</h3>”,html)])

         threads = [google_thread(“pycon fr”), google_thread(“pycon us”)]
         [thread.start() for thread in threads]
         print results



Monday, June 1, 2009
Et là ?
         results = []

         class google_thread(Thread):
             def __init__(self,search):
                 self.search = search

                def run():
                    html = urllib.urlopen(“http://google.com/?q=”+search”).read()
                    results.append(“Resultats pour ‘%s’ :” % self.search)
                    results.append([r[1] for r in re.split(“<h3>(.*?)</h3>”,html)])

         threads = [google_thread(“pycon fr”), google_thread(“pycon us”)]
         [thread.start() for thread in threads]




Monday, June 1, 2009
Thread safety
         results = []

         class google_thread(Thread):
             def __init__(self,search):
                 self.search = search

                def run():
                    html = urllib.urlopen(“http://google.com/?q=”+search”).read()
                    acquire_lock(results)
                    results.append(“Resultats pour ‘%s’ :” % self.search)
                    results.append([r[1] for r in re.split(“<h3>(.*?)</h3>”,html)])
                    release_lock(results)

         threads = [google_thread(“pycon fr”), google_thread(“pycon us”)]
         [thread.start() for thread in threads]




Monday, June 1, 2009
Thread safety

                       • Locks
                       • Queues
                       • Semaphores
                       • ...

Monday, June 1, 2009
Thread safety




Monday, June 1, 2009
Version asynchrone

         def google_async(search)
             async_call = twisted.web.client.getPage(“http://google.com/?q=”+search”)
             async_call.addCallback(gotresults)

         def gotresults(html)
             print [r[1] for r in re.split(“<h3>(.*?)</h3>”)]

         google_async(“pycon fr”)
         google_async(“pycon us”)

         twisted.internet.reactor.run()




Monday, June 1, 2009
Version asynchrone
                       • 1 seul thread!
         results = []

         def google_async(search)
             async_call = twisted.web.client.getPage(“http://google.com/?q=”+search”)
             async_call.addCallback(gotresults,search)

         def gotresults(html,search)
             results.append(“Resultats pour ‘%s’ :” % search)
             results.append([r[1] for r in re.split(“<h3>(.*?)</h3>”,html)])

         google_async(“pycon fr”)
         google_async(“pycon us”)

         twisted.internet.reactor.run()




Monday, June 1, 2009
Deferreds
          def google_async(search)
              async_call = twisted.web.client.getPage(“http://google.com/?q=”+search”)
              async_call.addCallback(gotresults)




                       • Promesse d’un resultat futur
                       • addCallback
                       • addErrback
                       • Chains
                       • DeferredLists
Monday, June 1, 2009
Deferreds




Monday, June 1, 2009
Reactor

                       • “Dont call us, we’ll call you”
                       • Event Loop
                       • S’occupe d’appeller tous les callbacks
                       • “Remplace” le GIL, thread switching
                       • “Pluggable” : select/poll, epoll, GUI, ...

Monday, June 1, 2009
Autres avantages
                       • Librairie très complète : HTTP, SSH, IRC,
                         DNS, IMAP, Jabber, SMTP, Telnet, ...
                       • Ne pas réinventer la roue / patterns
                       • Déploiement rapide d’applis complexes
                       • Threadpool
                       • Encapsulation/design
                       • Moins de surprises
Monday, June 1, 2009
Utilisateurs de Twisted
                       • Apple
                       • NASA
                       • Justin.tv
                       • Bittorrent / Zope / Freevo / Buildbot / ...
                       • ???
                       • Jamendo :)
Monday, June 1, 2009
Twisted chez Jamendo

                       • Upload servers
                       • Log servers
                       • Radio servers
                       • Widget servers
                       • Streaming / Download servers

Monday, June 1, 2009
Download servers
                       • 10k lignes de code
                       • HTTP “sécurisé”, download queues
                       • FTP avec virtual filesystem
                       • Génération de zips à la volée
                       • Seeds BitTorrent
                       • Logs UDP / Monitoring
                       • twisted.manhole
Monday, June 1, 2009
Merci de votre attention!

                                 Sylvain Zimmer
                         sylvain@jamendo.com
                            twitter.com/sylvinus




Monday, June 1, 2009

More Related Content

Similar to Twisted presentation & Jamendo usecases

[Golang] 以 Mobile App 工程師視角,帶你進入 Golang 的世界 (Introduction of GoLang)
[Golang] 以 Mobile App 工程師視角,帶你進入 Golang 的世界 (Introduction of GoLang) [Golang] 以 Mobile App 工程師視角,帶你進入 Golang 的世界 (Introduction of GoLang)
[Golang] 以 Mobile App 工程師視角,帶你進入 Golang 的世界 (Introduction of GoLang) Johnny Sung
 
Performance Improvements In Browsers
Performance Improvements In BrowsersPerformance Improvements In Browsers
Performance Improvements In BrowsersGoogleTecTalks
 
Performance Improvements in Browsers
Performance Improvements in BrowsersPerformance Improvements in Browsers
Performance Improvements in Browsersjeresig
 
Microblogging via XMPP
Microblogging via XMPPMicroblogging via XMPP
Microblogging via XMPPStoyan Zhekov
 
Symfony 2.0
Symfony 2.0Symfony 2.0
Symfony 2.0GrUSP
 
Generator Tricks for Systems Programmers
Generator Tricks for Systems ProgrammersGenerator Tricks for Systems Programmers
Generator Tricks for Systems ProgrammersHiroshi Ono
 
Performance, Games, and Distributed Testing in JavaScript
Performance, Games, and Distributed Testing in JavaScriptPerformance, Games, and Distributed Testing in JavaScript
Performance, Games, and Distributed Testing in JavaScriptjeresig
 
JDD 2017: Performance tests with Gatling (Andrzej Ludwikowski)
JDD 2017: Performance tests with Gatling (Andrzej Ludwikowski)JDD 2017: Performance tests with Gatling (Andrzej Ludwikowski)
JDD 2017: Performance tests with Gatling (Andrzej Ludwikowski)PROIDEA
 
An Introduction to Go
An Introduction to GoAn Introduction to Go
An Introduction to GoCloudflare
 
Performance tests with Gatling (extended)
Performance tests with Gatling (extended)Performance tests with Gatling (extended)
Performance tests with Gatling (extended)Andrzej Ludwikowski
 
Voicecon - Mashups with Tropo.com
Voicecon - Mashups with Tropo.comVoicecon - Mashups with Tropo.com
Voicecon - Mashups with Tropo.comVoxeo Corp
 
2015-GopherCon-Talk-Uptime.pdf
2015-GopherCon-Talk-Uptime.pdf2015-GopherCon-Talk-Uptime.pdf
2015-GopherCon-Talk-Uptime.pdfUtabeUtabe
 
Geeks Anonymes - Le langage Go
Geeks Anonymes - Le langage GoGeeks Anonymes - Le langage Go
Geeks Anonymes - Le langage GoGeeks Anonymes
 
10 reasons to be excited about go
10 reasons to be excited about go10 reasons to be excited about go
10 reasons to be excited about goDvir Volk
 
How go makes us faster (May 2015)
How go makes us faster (May 2015)How go makes us faster (May 2015)
How go makes us faster (May 2015)Wilfried Schobeiri
 
JavaScript in 2015
JavaScript in 2015JavaScript in 2015
JavaScript in 2015Igor Laborie
 
第1回PHP拡張勉強会
第1回PHP拡張勉強会第1回PHP拡張勉強会
第1回PHP拡張勉強会Ippei Ogiwara
 
Introduction to Programming in Go
Introduction to Programming in GoIntroduction to Programming in Go
Introduction to Programming in GoAmr Hassan
 
Going Live! with Comet
Going Live! with CometGoing Live! with Comet
Going Live! with CometSimon Willison
 

Similar to Twisted presentation & Jamendo usecases (20)

[Golang] 以 Mobile App 工程師視角,帶你進入 Golang 的世界 (Introduction of GoLang)
[Golang] 以 Mobile App 工程師視角,帶你進入 Golang 的世界 (Introduction of GoLang) [Golang] 以 Mobile App 工程師視角,帶你進入 Golang 的世界 (Introduction of GoLang)
[Golang] 以 Mobile App 工程師視角,帶你進入 Golang 的世界 (Introduction of GoLang)
 
Performance Improvements In Browsers
Performance Improvements In BrowsersPerformance Improvements In Browsers
Performance Improvements In Browsers
 
Performance Improvements in Browsers
Performance Improvements in BrowsersPerformance Improvements in Browsers
Performance Improvements in Browsers
 
Microblogging via XMPP
Microblogging via XMPPMicroblogging via XMPP
Microblogging via XMPP
 
Symfony 2.0
Symfony 2.0Symfony 2.0
Symfony 2.0
 
Generator Tricks for Systems Programmers
Generator Tricks for Systems ProgrammersGenerator Tricks for Systems Programmers
Generator Tricks for Systems Programmers
 
Performance, Games, and Distributed Testing in JavaScript
Performance, Games, and Distributed Testing in JavaScriptPerformance, Games, and Distributed Testing in JavaScript
Performance, Games, and Distributed Testing in JavaScript
 
JDD 2017: Performance tests with Gatling (Andrzej Ludwikowski)
JDD 2017: Performance tests with Gatling (Andrzej Ludwikowski)JDD 2017: Performance tests with Gatling (Andrzej Ludwikowski)
JDD 2017: Performance tests with Gatling (Andrzej Ludwikowski)
 
An Introduction to Go
An Introduction to GoAn Introduction to Go
An Introduction to Go
 
Performance tests with Gatling (extended)
Performance tests with Gatling (extended)Performance tests with Gatling (extended)
Performance tests with Gatling (extended)
 
Voicecon - Mashups with Tropo.com
Voicecon - Mashups with Tropo.comVoicecon - Mashups with Tropo.com
Voicecon - Mashups with Tropo.com
 
2015-GopherCon-Talk-Uptime.pdf
2015-GopherCon-Talk-Uptime.pdf2015-GopherCon-Talk-Uptime.pdf
2015-GopherCon-Talk-Uptime.pdf
 
Geeks Anonymes - Le langage Go
Geeks Anonymes - Le langage GoGeeks Anonymes - Le langage Go
Geeks Anonymes - Le langage Go
 
10 reasons to be excited about go
10 reasons to be excited about go10 reasons to be excited about go
10 reasons to be excited about go
 
How go makes us faster (May 2015)
How go makes us faster (May 2015)How go makes us faster (May 2015)
How go makes us faster (May 2015)
 
JavaScript in 2015
JavaScript in 2015JavaScript in 2015
JavaScript in 2015
 
第1回PHP拡張勉強会
第1回PHP拡張勉強会第1回PHP拡張勉強会
第1回PHP拡張勉強会
 
Introduction to Programming in Go
Introduction to Programming in GoIntroduction to Programming in Go
Introduction to Programming in Go
 
Whats New In Groovy 1.6?
Whats New In Groovy 1.6?Whats New In Groovy 1.6?
Whats New In Groovy 1.6?
 
Going Live! with Comet
Going Live! with CometGoing Live! with Comet
Going Live! with Comet
 

More from Sylvain Zimmer

Developer-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing oneDeveloper-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing oneSylvain Zimmer
 
Ranking the Web with Spark
Ranking the Web with SparkRanking the Web with Spark
Ranking the Web with SparkSylvain Zimmer
 
[fr] Introduction et Live-code Backbone.js à DevoxxFR 2013
[fr] Introduction et Live-code Backbone.js à DevoxxFR 2013[fr] Introduction et Live-code Backbone.js à DevoxxFR 2013
[fr] Introduction et Live-code Backbone.js à DevoxxFR 2013Sylvain Zimmer
 
140byt.es - The Dark Side of Javascript
140byt.es - The Dark Side of Javascript140byt.es - The Dark Side of Javascript
140byt.es - The Dark Side of JavascriptSylvain Zimmer
 
Joshfire Framework 0.9 Technical Overview
Joshfire Framework 0.9 Technical OverviewJoshfire Framework 0.9 Technical Overview
Joshfire Framework 0.9 Technical OverviewSylvain Zimmer
 
Javascript Views, Client-side or Server-side with NodeJS
Javascript Views, Client-side or Server-side with NodeJSJavascript Views, Client-side or Server-side with NodeJS
Javascript Views, Client-side or Server-side with NodeJSSylvain Zimmer
 
no.de quick presentation at #ParisJS 4
no.de quick presentation at #ParisJS 4no.de quick presentation at #ParisJS 4
no.de quick presentation at #ParisJS 4Sylvain Zimmer
 
Web Crawling with NodeJS
Web Crawling with NodeJSWeb Crawling with NodeJS
Web Crawling with NodeJSSylvain Zimmer
 
Archicamp présentation
Archicamp présentationArchicamp présentation
Archicamp présentationSylvain Zimmer
 

More from Sylvain Zimmer (9)

Developer-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing oneDeveloper-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing one
 
Ranking the Web with Spark
Ranking the Web with SparkRanking the Web with Spark
Ranking the Web with Spark
 
[fr] Introduction et Live-code Backbone.js à DevoxxFR 2013
[fr] Introduction et Live-code Backbone.js à DevoxxFR 2013[fr] Introduction et Live-code Backbone.js à DevoxxFR 2013
[fr] Introduction et Live-code Backbone.js à DevoxxFR 2013
 
140byt.es - The Dark Side of Javascript
140byt.es - The Dark Side of Javascript140byt.es - The Dark Side of Javascript
140byt.es - The Dark Side of Javascript
 
Joshfire Framework 0.9 Technical Overview
Joshfire Framework 0.9 Technical OverviewJoshfire Framework 0.9 Technical Overview
Joshfire Framework 0.9 Technical Overview
 
Javascript Views, Client-side or Server-side with NodeJS
Javascript Views, Client-side or Server-side with NodeJSJavascript Views, Client-side or Server-side with NodeJS
Javascript Views, Client-side or Server-side with NodeJS
 
no.de quick presentation at #ParisJS 4
no.de quick presentation at #ParisJS 4no.de quick presentation at #ParisJS 4
no.de quick presentation at #ParisJS 4
 
Web Crawling with NodeJS
Web Crawling with NodeJSWeb Crawling with NodeJS
Web Crawling with NodeJS
 
Archicamp présentation
Archicamp présentationArchicamp présentation
Archicamp présentation
 

Recently uploaded

Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomSalesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomCzechDreamin
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfFIDO Alliance
 
Strategic AI Integration in Engineering Teams
Strategic AI Integration in Engineering TeamsStrategic AI Integration in Engineering Teams
Strategic AI Integration in Engineering TeamsUXDXConf
 
Connecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKConnecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKUXDXConf
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...FIDO Alliance
 
AI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekAI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekCzechDreamin
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...FIDO Alliance
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessUXDXConf
 
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfHow Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfFIDO Alliance
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...CzechDreamin
 
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka DoktorováCzechDreamin
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityScyllaDB
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIES VE
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераMark Opanasiuk
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxJennifer Lim
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastUXDXConf
 
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxUnpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxDavid Michel
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfFIDO Alliance
 
Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsStefano
 

Recently uploaded (20)

Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomSalesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
 
Strategic AI Integration in Engineering Teams
Strategic AI Integration in Engineering TeamsStrategic AI Integration in Engineering Teams
Strategic AI Integration in Engineering Teams
 
Connecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKConnecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAK
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
AI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekAI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří Karpíšek
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfHow Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
 
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxUnpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 

Twisted presentation & Jamendo usecases

  • 1. Twisted Présentation & Usecases Monday, June 1, 2009
  • 2. Twisted? • Framework pour applis réseau • 100% Python, quelques optimisations en C • Projet opensource stable, bien maintenu • Programmation asynchrone Monday, June 1, 2009
  • 3. Asynchrone? • Synonymes : “event-driven”, “non-blocking” • Toutes les fonctions doivent retourner “rapidement” • 1 seul thread • Reactor pattern Monday, June 1, 2009
  • 4. Sync vs. Async def google_sync(search) html = urllib.urlopen(“http://google.com/?q=”+search”).read() return [r[1] for r in re.split(“<h3>(.*?)</h3>”,html)] print google_sync(“pycon fr”) print google_sync(“pycon us”) def google_async(search) async_call = twisted.web.client.getPage(“http://google.com/?q=”+search”) async_call.addCallback(gotresults) def gotresults(html) print [r[1] for r in re.split(“<h3>(.*?)</h3>”,html)] google_async(“pycon fr”) google_async(“pycon us”) twisted.internet.reactor.run() Monday, June 1, 2009
  • 5. “C’est plus compliqué!” • Oui mais • 100 requêtes en parallèle ? Monday, June 1, 2009
  • 6. “C’est plus compliqué!” • Oui mais • 100 requêtes en parallèle ? results = [] class google_thread(Thread): def __init__(self,search): self.search = search def run(): html = urllib.urlopen(“http://google.com/?q=”+search”).read() results.append([r[1] for r in re.split(“<h3>(.*?)</h3>”,html)]) threads = [google_thread(“pycon fr”), google_thread(“pycon us”)] [thread.start() for thread in threads] print results Monday, June 1, 2009
  • 7. Et là ? results = [] class google_thread(Thread): def __init__(self,search): self.search = search def run(): html = urllib.urlopen(“http://google.com/?q=”+search”).read() results.append(“Resultats pour ‘%s’ :” % self.search) results.append([r[1] for r in re.split(“<h3>(.*?)</h3>”,html)]) threads = [google_thread(“pycon fr”), google_thread(“pycon us”)] [thread.start() for thread in threads] Monday, June 1, 2009
  • 8. Thread safety results = [] class google_thread(Thread): def __init__(self,search): self.search = search def run(): html = urllib.urlopen(“http://google.com/?q=”+search”).read() acquire_lock(results) results.append(“Resultats pour ‘%s’ :” % self.search) results.append([r[1] for r in re.split(“<h3>(.*?)</h3>”,html)]) release_lock(results) threads = [google_thread(“pycon fr”), google_thread(“pycon us”)] [thread.start() for thread in threads] Monday, June 1, 2009
  • 9. Thread safety • Locks • Queues • Semaphores • ... Monday, June 1, 2009
  • 11. Version asynchrone def google_async(search) async_call = twisted.web.client.getPage(“http://google.com/?q=”+search”) async_call.addCallback(gotresults) def gotresults(html) print [r[1] for r in re.split(“<h3>(.*?)</h3>”)] google_async(“pycon fr”) google_async(“pycon us”) twisted.internet.reactor.run() Monday, June 1, 2009
  • 12. Version asynchrone • 1 seul thread! results = [] def google_async(search) async_call = twisted.web.client.getPage(“http://google.com/?q=”+search”) async_call.addCallback(gotresults,search) def gotresults(html,search) results.append(“Resultats pour ‘%s’ :” % search) results.append([r[1] for r in re.split(“<h3>(.*?)</h3>”,html)]) google_async(“pycon fr”) google_async(“pycon us”) twisted.internet.reactor.run() Monday, June 1, 2009
  • 13. Deferreds def google_async(search) async_call = twisted.web.client.getPage(“http://google.com/?q=”+search”) async_call.addCallback(gotresults) • Promesse d’un resultat futur • addCallback • addErrback • Chains • DeferredLists Monday, June 1, 2009
  • 15. Reactor • “Dont call us, we’ll call you” • Event Loop • S’occupe d’appeller tous les callbacks • “Remplace” le GIL, thread switching • “Pluggable” : select/poll, epoll, GUI, ... Monday, June 1, 2009
  • 16. Autres avantages • Librairie très complète : HTTP, SSH, IRC, DNS, IMAP, Jabber, SMTP, Telnet, ... • Ne pas réinventer la roue / patterns • Déploiement rapide d’applis complexes • Threadpool • Encapsulation/design • Moins de surprises Monday, June 1, 2009
  • 17. Utilisateurs de Twisted • Apple • NASA • Justin.tv • Bittorrent / Zope / Freevo / Buildbot / ... • ??? • Jamendo :) Monday, June 1, 2009
  • 18. Twisted chez Jamendo • Upload servers • Log servers • Radio servers • Widget servers • Streaming / Download servers Monday, June 1, 2009
  • 19. Download servers • 10k lignes de code • HTTP “sécurisé”, download queues • FTP avec virtual filesystem • Génération de zips à la volée • Seeds BitTorrent • Logs UDP / Monitoring • twisted.manhole Monday, June 1, 2009
  • 20. Merci de votre attention! Sylvain Zimmer sylvain@jamendo.com twitter.com/sylvinus Monday, June 1, 2009