Talk we gave Benoit and I at the OSDC/OWF in Paris.

1. High Availability
Server apps
Tarek Ziade & Benoît Chesneau
Monday, October 3, 2011 1

2. • The C10K problem
• “It's time for web servers to handle ten
thousand clients simultaneously, don't you
think? After all, the web is a big place now. “
• http://www.kegel.com/c10k.html
Monday, October 3, 2011 2

3. More & more hits
Monday, October 3, 2011 3

4. More & more media
Monday, October 3, 2011 4

5. • 1993 - 1998 - CGI (and java applets...)
• Apache & Prefork - simple
• Multiprocessing - threads vs events
A bit of history
Monday, October 3, 2011 5

6. Monday, October 3, 2011 6

7. • 1993 - 1998 - CGI (and java applets...)
• Apache & Prefork - simple
• Multiprocessing - threads vs events
A bit of history
Monday, October 3, 2011 7

8. Clients
Socket Port 80
Worker
Application
Minitel
Monday, October 3, 2011 8

9. Waiting...
Monday, October 3, 2011 9

10. Moar
Monday, October 3, 2011 10

11. Fork
Monday, October 3, 2011 11

12. Fork
Monday, October 3, 2011 12

13. Fork
Monday, October 3, 2011 13

14. Fork
Monday, October 3, 2011 14

15. Multiprocessing
Monday, October 3, 2011 15

16. multiprocessing
Monday, October 3, 2011 16

17. • Threads or Events ?
• Apache, Nginx, Node.js, .... (Erlang)
Monday, October 3, 2011 17

18. • MPM Prefork
$ ab -n 1000000 -c 250 http://127.0.0.1/
$ ps ax|grep apache2|wc -l
152
• MPM Worker (hybride prefork + threads
par requêtes)
$ ps ax|grep apache2|wc -l
5
Monday, October 3, 2011 18

19. • 1 master process, several workers
• Prefork
• Accepts several connections per worker
• Events
Monday, October 3, 2011 19

20. • 1 single thread
• 1 event loop (READ/WRITE)
• Context switching
Event Loop
Monday, October 3, 2011 20

21. • Nginx VS Apache
• http://blog.webfaction.com/a-little-holiday-
present
Monday, October 3, 2011 21

22. Nginx versus Apache (with the worker-MPM)
for serving a small static file:
Monday, October 3, 2011 22

23. Memory usage
Monday, October 3, 2011 23

24. http://www.ostinelli.net/a-comparison-between-misultin-mochiweb-cowboy-nodejs-and-tornadoweb/
Choice
Monday, October 3, 2011 24

25. • WSGI (PEP 3333)
• Prefork - shared socket
• async & sync workers (gevent, eventlet,
tornado, ...)
• Easy to use in Django, Paster, ...
Monday, October 3, 2011 25

26. Fork
Monday, October 3, 2011 26

27. Fork + async
Monday, October 3, 2011 27

28. • HTTP Stream
• Restartless upgrades ala Nginx
• Simple CLI:
$ gunicorn -w 3 test:app
• greins (https://github.com/meebo/greins)
Monday, October 3, 2011 28

29. Monday, October 3, 2011 29

30. Monday, October 3, 2011 30

31. asynchronous
or
synchronous
Monday, October 3, 2011 31

32. • bandwidth-bound & CPU-bound apps
• Fast operations required
• Needs a way to hold incoming connections
(cf nginx + gunicorn sync)
synchronous
Monday, October 3, 2011 32

33. • websocket, longpolling
• apps with long, blocking calls
• streaming
• slow clients
• slowloris & other DoS
asynchronous
Monday, October 3, 2011 33

34. gevent & eventlet
• Network libraries
• synchronous API
• monkey patch
• co-routines
• Code is always running async
gevent & eventlet
Monday, October 3, 2011 34

35. twisted
• asynchronous library
• Events & callbacks
• Twisted server == single event loop
twisted
Monday, October 3, 2011 35

36. Monday, October 3, 2011 36

37. • message passing
• greenlets
• py.py --withmod-_stackless
http://www.grant-olson.net/python/intro-to-stackless-python
stackless
Monday, October 3, 2011 37

38. • ulimit is your friend
• somaxcon
• ...
Tuning
Monday, October 3, 2011 38

39. Monday, October 3, 2011 39

40. Monday, October 3, 2011 40

41. Firefox Sync
Monday, October 3, 2011 41

42. Monday, October 3, 2011 42

43. • 10 frontends web
• +123 mysql (percona)
• 6 Open Ldap servers (2 masters, 4 slaves)
• ~1.5 M active users
• ~4 M registered users
• ~800 RPS for the storage
• CPU: 20% !!! ETooMuch
Monday, October 3, 2011 43

44. • silverlining (http://cloudsilverlining.org/) by
@ianb
• kraftwerk (http://kraftwerk-wsgi.org)
Deploy
Monday, October 3, 2011 44

45. • libcloud (http://libcloud.apache.org/): deploy
on the “clouds”
• juju (https://juju.ubuntu.com/) : ubuntu
cloud (openstack + ensemble or ec2)
Deploy
Monday, October 3, 2011 45

46. • Apache Bench (ab)
• httperf
• Apache Jmeter
• Grinder
• Funkload
Bench
Monday, October 3, 2011 46

47. Funkload
Monday, October 3, 2011 47

48. Funkload
Monday, October 3, 2011 48

49. Thanks !
• @benoitc
• @tarek_ziade
Moar ???
Monday, October 3, 2011 49