Dlsiecyx Pgroeammr
The cool stuff for scaling
Required Listening
Disclaimer about me
- I like both Ruby and Python, but I dislike the generator/deferred
model they implement for non-block...
Disclaimer about dyslexia
From wikipedia (http://en.wikipedia.org/wiki/Dyslexia):
"... Many students reach higher educatio...
Basics
Asynchronous I/O - Posix aio_*
Non-Blocking I/O
Select, Poll, EPoll, KQueue
Evented I/O - Abstraction over any I/O ...
Scalability, not Performance
Scalability, not Performance
1. Performance: same work with less effort
2. Scalability: more work with more resources
3. S...
Wait, Wat ?
Sequential Blocking Example
Ruby
Python
URL Fetch Benchmark
URL Fetch Benchmark
2 urls
Sequential blocking I/O using ruby: 0m3.061s
Non-blocking I/O using ruby and Eventmachine: 0m0....
Python and Twisted
Python/Twisted
Python/Twisted - Verbose and Inlined
Python/Twisted - Generators
Python/Twisted - Deferreds
Python/Twisted - Callbacks
Python/Twisted - Inline Callbacks
Python and GEvent
Python/gevent
Python/gevent - monkey patch
Python/gevent - join the greenlets
Ruby and EventMachine
Ruby/EventMachine
Ruby/EventMachine - Generators
Ruby/EventMachine - Deferreds
Ruby/EventMachine - Callbacks
Ruby/EventMachine/EM-Synchrony - Fiber based
Node.js
Node.js
Node.js - Creating and ending the Request
Node.js - Event handlers
Intermission: Erlang
Erlang
Erlang - Spawning a process to each URL
Erlang - Single URL download
Reminder of what we wanted to do
Ruby
Python
Architectural building blocks
Decoupling db writes with Message Queues
Coupled comment
Uncoupled comment - producer
Uncoupled comment - consumer
Cache
HTML processing - no cache
http://github.com/gleicon/vuvuzelr/proxy_no_cache.rb
HTML processing - Cached
http://github.com/gleicon/vuvuzelr/proxy.rb
Thanks
http://zenmachine.wordpress.com
http://github.com/gleicon
http://www.7co.cc
@gleicon
gleicon@gmail.com
Upcoming SlideShare
Loading in...5
×

Dlsecyx pgroammr (Dyslexic Programmer - cool stuff for scaling)

6,946

Published on

Presentation at Rubyconf BR 2010

Published in: Technology
1 Comment
2 Likes
Statistics
Notes
  • Sexdaily Gram Romper,

    Now to find one for dyslexic:

    Anagrams for: dyslexic

    1 found. Displaying all:
    Dyslexic

    Grrr.....
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
6,946
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
17
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide

Dlsecyx pgroammr (Dyslexic Programmer - cool stuff for scaling)

  1. 1. Dlsiecyx Pgroeammr The cool stuff for scaling
  2. 2. Required Listening
  3. 3. Disclaimer about me - I like both Ruby and Python, but I dislike the generator/deferred model they implement for non-blocking I/O - I think node.js beats them because its better wrapped, as EM synchrony does to EM and gevent for python. - I don't think this model is mandatory, not even migrate from Blocking to non-blocking I/O is needed for most cases. The best solution may not lie in code complexity. Maybe the right architecture. - I like Erlang, the actor/message passing model and pattern matching. Way more. And I like message queues. Specially this one: http://www.restmq.com
  4. 4. Disclaimer about dyslexia From wikipedia (http://en.wikipedia.org/wiki/Dyslexia): "... Many students reach higher education before they encounter the threshold at which they are no longer able to compensate for their learning weaknesses. One common misconception about dyslexia is that dyslexic readers write words backwards or move letters around when reading. In fact, this only occurs in a very small population of dyslexic readers. Dyslexic people are better identified by writing that does not seem to match their level of intelligence from prior observations. Additionally, dyslexic people often substitute similar-looking, but unrelated, words in place of the ones intended (what/want, say/saw, help/held, run/fun, fell/fall, to/too, etc.)"
  5. 5. Basics Asynchronous I/O - Posix aio_* Non-Blocking I/O Select, Poll, EPoll, KQueue Evented I/O - Abstraction over any I/O Model Reactor - Pattern for multiplexing I/O Future Programming Generator Deferred Event Emitters The c10k problem - http://www.kegel.com/c10k. html
  6. 6. Scalability, not Performance
  7. 7. Scalability, not Performance 1. Performance: same work with less effort 2. Scalability: more work with more resources 3. Scientific measures for performance: How much and How fast 4. Tricks to improve performance can hurt scalability. e.g: Map/Reduce is scalable, not fast (not suited for real time queries) 5. Non-Blocking I/O is not an optimization for Performance 6. The time spent waiting for an external resource can be used to other tasks, given a known introduced complexity in the code. 7. Most of time architectural changes can avoid deeper code changes (1,2,3,4 - Goetz et al, Java Concurrency in Practice, M/R example is mine) (5,6,7 - Lucindo, R. http://slidesha.re/aYz4Mk, wording on 6 is mine)
  8. 8. Wait, Wat ?
  9. 9. Sequential Blocking Example
  10. 10. Ruby Python
  11. 11. URL Fetch Benchmark
  12. 12. URL Fetch Benchmark 2 urls Sequential blocking I/O using ruby: 0m3.061s Non-blocking I/O using ruby and Eventmachine: 0m0.795s 28 urls Sequential blocking I/O using ruby: 0m56.014s Non-blocking I/O using ruby and Eventmachine: 0m24.519s
  13. 13. Python and Twisted
  14. 14. Python/Twisted
  15. 15. Python/Twisted - Verbose and Inlined
  16. 16. Python/Twisted - Generators
  17. 17. Python/Twisted - Deferreds
  18. 18. Python/Twisted - Callbacks
  19. 19. Python/Twisted - Inline Callbacks
  20. 20. Python and GEvent
  21. 21. Python/gevent
  22. 22. Python/gevent - monkey patch
  23. 23. Python/gevent - join the greenlets
  24. 24. Ruby and EventMachine
  25. 25. Ruby/EventMachine
  26. 26. Ruby/EventMachine - Generators
  27. 27. Ruby/EventMachine - Deferreds
  28. 28. Ruby/EventMachine - Callbacks
  29. 29. Ruby/EventMachine/EM-Synchrony - Fiber based
  30. 30. Node.js
  31. 31. Node.js
  32. 32. Node.js - Creating and ending the Request
  33. 33. Node.js - Event handlers
  34. 34. Intermission: Erlang
  35. 35. Erlang
  36. 36. Erlang - Spawning a process to each URL
  37. 37. Erlang - Single URL download
  38. 38. Reminder of what we wanted to do
  39. 39. Ruby Python
  40. 40. Architectural building blocks
  41. 41. Decoupling db writes with Message Queues
  42. 42. Coupled comment
  43. 43. Uncoupled comment - producer
  44. 44. Uncoupled comment - consumer
  45. 45. Cache
  46. 46. HTML processing - no cache http://github.com/gleicon/vuvuzelr/proxy_no_cache.rb
  47. 47. HTML processing - Cached http://github.com/gleicon/vuvuzelr/proxy.rb
  48. 48. Thanks http://zenmachine.wordpress.com http://github.com/gleicon http://www.7co.cc @gleicon gleicon@gmail.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×