Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Like this? Share it with your network

Share

Dlsecyx pgroammr (Dyslexic Programmer - cool stuff for scaling)

on

  • 6,737 views

Presentation at Rubyconf BR 2010

Presentation at Rubyconf BR 2010

Statistics

Views

Total Views
6,737
Views on SlideShare
3,862
Embed Views
2,875

Actions

Likes
2
Downloads
17
Comments
1

6 Embeds 2,875

http://zenmachine.wordpress.com 2840
https://zenmachine.wordpress.com 22
url_unknown 7
http://irr.posterous.com 4
http://www.slideshare.net 1
http://translate.googleusercontent.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Sexdaily Gram Romper,

    Now to find one for dyslexic:

    Anagrams for: dyslexic

    1 found. Displaying all:
    Dyslexic

    Grrr.....
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Dlsecyx pgroammr (Dyslexic Programmer - cool stuff for scaling) Presentation Transcript

  • 1. Dlsiecyx Pgroeammr The cool stuff for scaling
  • 2. Required Listening
  • 3. Disclaimer about me - I like both Ruby and Python, but I dislike the generator/deferred model they implement for non-blocking I/O - I think node.js beats them because its better wrapped, as EM synchrony does to EM and gevent for python. - I don't think this model is mandatory, not even migrate from Blocking to non-blocking I/O is needed for most cases. The best solution may not lie in code complexity. Maybe the right architecture. - I like Erlang, the actor/message passing model and pattern matching. Way more. And I like message queues. Specially this one: http://www.restmq.com
  • 4. Disclaimer about dyslexia From wikipedia (http://en.wikipedia.org/wiki/Dyslexia): "... Many students reach higher education before they encounter the threshold at which they are no longer able to compensate for their learning weaknesses. One common misconception about dyslexia is that dyslexic readers write words backwards or move letters around when reading. In fact, this only occurs in a very small population of dyslexic readers. Dyslexic people are better identified by writing that does not seem to match their level of intelligence from prior observations. Additionally, dyslexic people often substitute similar-looking, but unrelated, words in place of the ones intended (what/want, say/saw, help/held, run/fun, fell/fall, to/too, etc.)"
  • 5. Basics Asynchronous I/O - Posix aio_* Non-Blocking I/O Select, Poll, EPoll, KQueue Evented I/O - Abstraction over any I/O Model Reactor - Pattern for multiplexing I/O Future Programming Generator Deferred Event Emitters The c10k problem - http://www.kegel.com/c10k. html
  • 6. Scalability, not Performance
  • 7. Scalability, not Performance 1. Performance: same work with less effort 2. Scalability: more work with more resources 3. Scientific measures for performance: How much and How fast 4. Tricks to improve performance can hurt scalability. e.g: Map/Reduce is scalable, not fast (not suited for real time queries) 5. Non-Blocking I/O is not an optimization for Performance 6. The time spent waiting for an external resource can be used to other tasks, given a known introduced complexity in the code. 7. Most of time architectural changes can avoid deeper code changes (1,2,3,4 - Goetz et al, Java Concurrency in Practice, M/R example is mine) (5,6,7 - Lucindo, R. http://slidesha.re/aYz4Mk, wording on 6 is mine)
  • 8. Wait, Wat ?
  • 9. Sequential Blocking Example
  • 10. Ruby Python
  • 11. URL Fetch Benchmark
  • 12. URL Fetch Benchmark 2 urls Sequential blocking I/O using ruby: 0m3.061s Non-blocking I/O using ruby and Eventmachine: 0m0.795s 28 urls Sequential blocking I/O using ruby: 0m56.014s Non-blocking I/O using ruby and Eventmachine: 0m24.519s
  • 13. Python and Twisted
  • 14. Python/Twisted
  • 15. Python/Twisted - Verbose and Inlined
  • 16. Python/Twisted - Generators
  • 17. Python/Twisted - Deferreds
  • 18. Python/Twisted - Callbacks
  • 19. Python/Twisted - Inline Callbacks
  • 20. Python and GEvent
  • 21. Python/gevent
  • 22. Python/gevent - monkey patch
  • 23. Python/gevent - join the greenlets
  • 24. Ruby and EventMachine
  • 25. Ruby/EventMachine
  • 26. Ruby/EventMachine - Generators
  • 27. Ruby/EventMachine - Deferreds
  • 28. Ruby/EventMachine - Callbacks
  • 29. Ruby/EventMachine/EM-Synchrony - Fiber based
  • 30. Node.js
  • 31. Node.js
  • 32. Node.js - Creating and ending the Request
  • 33. Node.js - Event handlers
  • 34. Intermission: Erlang
  • 35. Erlang
  • 36. Erlang - Spawning a process to each URL
  • 37. Erlang - Single URL download
  • 38. Reminder of what we wanted to do
  • 39. Ruby Python
  • 40. Architectural building blocks
  • 41. Decoupling db writes with Message Queues
  • 42. Coupled comment
  • 43. Uncoupled comment - producer
  • 44. Uncoupled comment - consumer
  • 45. Cache
  • 46. HTML processing - no cache http://github.com/gleicon/vuvuzelr/proxy_no_cache.rb
  • 47. HTML processing - Cached http://github.com/gleicon/vuvuzelr/proxy.rb
  • 48. Thanks http://zenmachine.wordpress.com http://github.com/gleicon http://www.7co.cc @gleicon gleicon@gmail.com