TLPG November 2012 - Eventlet

Eventletfor Fun and Profit @colinhowe

Me, me, me... MeeeeeeeeeVP Engineering at ConversocialActive on github: http://github.com/colinhowe● Monner● Django SamplerDrivel on Twitter: @colinhowe

Software as a serviceHelp companies deliver customer service onsocial mediaHundreds of millions of remote API calls daily

APIs are slowCan take seconds to respondBlocking doesnt workLots of processes meanslots of memory usedThreading... eugh.

Eventlet to the Rescue!Eventlet is broadly similar to geventgevent was inspired by Eventletgevent has a cleaner APIgevent is possibly fastergevent can mostly do what eventlet does

Eventlet - Usageurls = [ ... ]import eventletfrom eventlet.green import urllib2def fetch(url): print "opening", url body = urllib2.urlopen(url).read() print "done with", url return url, bodypool = eventlet.GreenPool(200)for url, body in pool.imap(fetch, urls): print "got body from", url, "of length", len(body)

Why Eventlet?

Eventlet - Monkey Patchingimport eventletrequests = eventlet.import_patched(requests)# Only requests is patched and only in this module# Exceptions are a nightmare

Eventlet - Monkey Patchingimport eventletimport some_modulesome_module2 = eventlet.import_patched(some_module)try: raise some_module2.BoringException()except some_module.BoringException: print Caught exceptionexcept: print Oh dear

Stuff weve learnedand probably applies to gevent too :)

Open Sockets

Async + DatabasesBe careful when using databases in async codeLots of connections = database deathUse a real thread to manage connections andread from a queue of requestsUse eventlet to wait on these

Rate Limits

DebuggingIs hard at the best of times.Its harder when your stack has been killed...

Debuggingdef blowup(): raise Exception(Broken :()def some_method(): some_other_method()def some_other_method(): eventlet.spawn(blowup)some_method()

DebuggingTraceback (most recent call last): File ".../eventlet/hubs/hub.py", line 336, in fire_timers timer() File ".../eventlet/hubs/timer.py", line 56, in __call__ cb(*args, **kw) File ".../eventlet/greenthread.py", line 192, in main result = function(*args, **kwargs) File "<ipython-input-1-bbd458ecf6a0>", line 2, in blowup raise Exception(Broken :()Exception: Broken :(

DebuggingMake well-defined, concise, APIsPass through any context that might helpThink about failureThink about logging

Thanks for ListeningSlides will appear on Slideshare Tweet me @colinhowe Any questions?

TLPG November 2012 - Eventlet

by colinthehowe on Nov 21, 2012

Accessibility

Categories

Upload Details

Usage Rights

Statistics

TLPG November 2012 - Eventlet Presentation Transcript