Hybrid concurrency patterns

  • 39,216 views
Uploaded on

Ruby developers need to stop using EventMachine. It's the wrong direction. …

Ruby developers need to stop using EventMachine. It's the wrong direction.

Lost in the "Threads vs Event Driven vs Process Spawning" debate is that you can combine them! Learn how Celluloid is improving thread programming by abstracting them using a higher level framework called Celluloid, how you can use Celluloid::IO to throw a reactor pattern into a thread. Using this approach, you can take advantage of threading and use all CPU power on a machine with JRuby or Rubinius. I also discuss the future of distributed objects and computing, and where I think things are going.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • i really can't tell all you young kids how much of my life i've spent debugging shitty threaded code and writing libraries to make is less error prone.

    it can't be done.

    give up. move on.
    Are you sure you want to
    Your message goes here
  • @bdicasa I certainly agree that detecting bugs caused by non-threadsafe code can be very difficult, my core thesis remains strong and I stick to it. If you focus on making your code threadsafe when you design and build it initially, then you don't NEED to worry about finding bugs due to threading issues. When thread safety is written into every object from the start (which really isn't all that hard, really) then you won't have to address threading bugs, because they won't exist.
    Are you sure you want to
    Your message goes here
  • @Brendten I'm not arguing the fact that knowing how to write thread-safe code is an absolute for any software developer. I'm also not arguing the fact that learning how to write thread safe code is hard. However issues with threading can be hard to detect, so why not use concurrency patterns that help you to avoid race conditions and deadlocks?
    Are you sure you want to
    Your message goes here
  • @peterashford9 So true, Peter. I love Ruby, but I'm not a 'Rubyist'. A problem I've always had with a large group of 'rubyists' out there is that they seem to think being a Ruby dev absolves them from learning CS fundamentals in any way. You can't be a ruby developer without being a developer. Just because you started out as a Web designer/developer and found Ruby by way of Rails doesn't mean you can just ignore and/or not learn and practice the skills that are required fundamentals for software engineers using other languages. I would never even considering hiring a Java dev who couldn't write threadsafe code. Why should I consider hiring a 'rubyist' who can't, either? I hire developers who have the skills and knowledge that form the foundations of CS. If you have that, you can learn Java, or Ruby, or Python, or Objective C, or (*gack*) C# (or even more esoteric languages and frameworks we might use from time to time like Erlang, or OCaml, or Haskell, or whatever).
    Are you sure you want to
    Your message goes here
  • @bdicasa Let's be clear: threads don't introduce problems, programmers who don't know how to use them, or who use them carelessly, introduce the problems. My argument here is that as a developer living and working in a multi-processor, multi-core world, you MUST understand threads and how to write threadsafe code. This should be a core part of everything you do when you design your software. Think about this: Tony has done a really nice job with Celluloid (and I have used it myself on occasion), but what if he gets hit by a bus, or if he suddenly decides to stop open-sourcing the code? Or what if you're not using Celluloid, and developing with MRI in mind, but along comes another developer 5 years after you've moved on to greener pastures, and decides to run your code on JRuby? What will happen to all of this code? It will be broken, and it will be your fault. Why? Because you chose to ignore the 'difficult' problems of multi-threaded concurrency when you originally designed and wrote your code. So, contrary to your assertion that 'it helps to understand how threading works,' I assert that you MUST understand how threading works (and as a corollary, how to write threadsafe code) in order to be even a competent software engineer in the world today.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
39,216
On Slideshare
0
From Embeds
0
Number of Embeds
37

Actions

Shares
Downloads
146
Comments
12
Likes
76

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Transcript

  • 1. Threads and Events inHarmony with Celluloid Kyle Drake
  • 2. Hi, I’m Kyle Drake. Iwork at Geoloqi Esri.We built a geofencing Textand real-time location streaming platform.
  • 3. Last year, IText a didtalk at KRTConf.
  • 4. I was really intopure event-driven Text(reactor patternbased) architecture.
  • 5. EventMachine, Twisted, Node.js• Event-driven Text• No threads• One CPU core• Process Spawning
  • 6. I releasedsinatra-synchrony,so I could use TextEventMachinewithout callbacks.
  • 7. Since then, I’ve Textchanged my mind.
  • 8. I’m really not a fanof EventMachine Textanymore.
  • 9. EventMachine is• A frankenstein - guts the ruby internals• Not in active development Text• Makes non-blocking IO block• Requires special code from Ruby libraries• Hard to use in an OOP way• Is really difficult to work with• Poorly documented
  • 10. Ruby developersneed to stop usingEventMachine. It’s Textthe wrongdirection.
  • 11. Check thisbenchmark Textout:
  • 12. Texthttp://rhaas.blogspot.com/2012/04/did-i-say-32-cores-how-about-64.html
  • 13. Predictions:More Cores.A lot more.
  • 14. Predictions:More libraries =more memory
  • 15. Will processspawning work forever? It might.
  • 16. It might not.Thousands of cores?
  • 17. Ruby MRI:Global Interpreter Lock (single CPU core) Rubinius and JRuby: Full threading (multiple CPU cores)~20KB per thread
  • 18. We’re not finding a lot of thread safety issues in Ruby.
  • 19. “Threading is hard”
  • 20. Threading is not an intention!
  • 21. Let’s fix it by abstractingthreads into how humans think!
  • 22. http://celluloid.io• Developed by Tony Arcieri• Actor Pattern for Ruby• Lots of inspiration from Erlang
  • 23. Each actor is aconcurrent objectrunning in its own thread
  • 24. First “Killer App”: Sidekiq Mike Perham
  • 25. Don’t get mewrong. Event-driven is still awesome.
  • 26. “..seasoned engineers areusing a mix of threaded,event-based, andalternative concurrencyapproaches like Actors” - Alex Payne
  • 27. What if we couldcombine threads and reactor patterns.. and actors?!
  • 28. https://github.com/celluloid/celluloid-io• One reactor pattern per Celluloid object• Multiple reactors? No problem!• Doesn’t mess with code outside of actor• Utilizes all CPU cores (using JRuby and Rubinius)• Websockets, messaging systems, your hugelysuccessful blog
  • 29. You dont have tochoose between threaded and evented IO!
  • 30. Let’s getdistributed.
  • 31. "I thought of objects being like biological cells and/or individual computers on a network, only able to communicate with messages" - Alan Kay
  • 32. Guess who was really into distributed network objects?
  • 33. NeXT experimented with distributedobjects in the mid 90s.NeXT was way ahead of its time.
  • 34. "Objects can message objects transparentlythat live on other machines over the network, and you dont have to worry about thenetworking gunk, and you dont have to worry about finding them, and you dont have to worry about anything. Its just as if you messaged an object thats right next door." - Steve Jobs
  • 35. “Portable Distributed Objects”
  • 36. https://github.com/celluloid/dcell • Objects talk over networks! • Uses 0MQ • Aware of eachothers’ existence • Distributed gossip protocol • Web UI • In the early stages, very promising
  • 37. CODEEXAMPLES!
  • 38. Help UsBuild This!
  • 39. Thanks! @kyledrakekyledrake.net