• Save
Event Driven Architecture - MeshU - Ilya Grigorik
Upcoming SlideShare
Loading in...5
×
 

Event Driven Architecture - MeshU - Ilya Grigorik

on

  • 16,785 views

A look at the technologies and the architecture behind the emerging real-time web. We will discuss XMPP/Jabber and AMQP protocols and explore the advantages of each over the commonly used HTTP ...

A look at the technologies and the architecture behind the emerging real-time web. We will discuss XMPP/Jabber and AMQP protocols and explore the advantages of each over the commonly used HTTP request-response cycle. As part of the workshop we will look at the available tools and libraries and work through simple examples of creating an event driven, real-time service.

Statistics

Views

Total Views
16,785
Views on SlideShare
12,634
Embed Views
4,151

Actions

Likes
94
Downloads
0
Comments
3

33 Embeds 4,151

http://www.igvita.com 3812
http://www.lifeyun.com 80
http://confluence.b2e.kz 44
http://00f.net 41
http://www.slideshare.net 41
http://leonardoeloy.com 19
http://b2e.kz 18
http://www.techvibes.com 17
http://cms 15
http://bulat.wikispaces.com 14
http://mwyrobek.tumblr.com 8
http://translate.googleusercontent.com 6
http://static.slidesharecdn.com 4
http://flavors.me 4
https://00f.net 3
http://neou.com:1208 3
http://www.redcanary.ca 2
http://www.iweb34.com 2
http://webcache.googleusercontent.com 2
http://coderwall.com 2
http://www.b2e.kz 2
http://conversation.ecairn.com 1
http://feeds.feedburner.com 1
http://www.railsfire.com 1
http://www.00f.net 1
http://feeds.igvita.com 1
http://www.hanrss.com 1
http://safe.tumblr.com 1
http://127.0.0.1:8795 1
http://74.125.113.132 1
http://74.125.155.132 1
http://209.85.229.132 1
http://www.slideee.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • If you want your presentation reviewed- allow it to be downloaded. Due to lousy web performance - online viewing is impossible. Was your intent to share information or restrict access to it? Wish I had the connection your other reviewer's did.
    Are you sure you want to
    Your message goes here
    Processing…
  • Thanks for sharing Ilya!
    Are you sure you want to
    Your message goes here
    Processing…
  • Great slides & analogy, thank you for sharing! I'd be interested in the sources regarding Amazon & Google on slide 6, it's yummie data :) Thank you!
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Event Driven Architecture - MeshU - Ilya Grigorik Event Driven Architecture - MeshU - Ilya Grigorik Presentation Transcript

  • Event-Driven Architectures Ilya Grigorik CTO / AideRSS Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Apr 7th launch: Bestest Blog discovery! Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • • Request, Response Cycle • Event Driven Architectures • Cloud Computing • Real-time Web! Fully buzzword • XMPP & AMQP compliant! • Example Questions & The slides… My blog Comments Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda View slide
  • Elastic, Event-Driven, Real-time, Staged, Services Oriented Architecture (Ready, ready? Go.) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda View slide
  • GET /home GET /home GET /home Response Request, Response Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Google: +200ms = -10% searches Amazon: +100ms = -5% purchases GET /home Response Response Time is Everything Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Bottleneck Complexity and Time Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • “Rails, Django, Seaside, Grails…” cant scale. Myth: Slow Frameworks Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Application server: 1-1000+ req/s “Fast Enough” Frameworks Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • 1. 5 ms: Process request 2. 30 ms: Authenticate user 3. 90 ms: Contact database & other services 4. 30 ms: Render response Total: 155 ms, 0.5 MB / conn OR 6.5 req / s Request, Response assembly Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • The Proxy Solution Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • The “More” Proxy Solution Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • This is madness. (after a certain point) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • 1. Receive 2. Verify 3. Dispatch 4. Aggregate 5. Handle errors 6. Render Throughput != Response Time Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Buzzword: Event-Driven Architectures (aka, reinventing the wheel) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Auto Manufacturing circa 1910 2-3 men, 1-2 days / car Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • More workers! Scaling the manufacturing process Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Then Henry Ford came along… Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • And gave us the assembly line! Scaling the manufacturing process Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Queuing theory Lean manufacturing Just in time Lower response time, higher throughput! 93 seconds / car 1M + cars / year Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Event Driven Architecture (EDA) = Assembly line except, we’re not building cars here… Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • 1. 5 ms - Receive 2. 30 ms - Verify Total: 365 ms, ~ 3 req / s 3. 30 ms - Save 4. 200 ms – Generate PDF 5. 50 ms - Email 6. 50 ms - Render GET /purchase OK Monolith Order Processing (example) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Can someone else do it? Total: 115 ms, ~ 3 req / s 1. 5 ms – Receive No ~ 70% improvement in 2. 30 ms - Verify No response time! 3. 30 ms – Save No 4. 200 ms – Generate PDF Yes 5. 50 ms – Email Yes 6. 50 ms – Render Yes GET /purchase OK Assembly Line Order Processing (example) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • 1. 5 ms - Receive 2. 30 ms - Verify 3. 30 ms - Save 1. Generate PDF 4. 10 ms – Dispatch 1. Email PDF 2. Dispatch 5. 50 ms – Render Total: 125 ms GET /purchase OK Assembly Line Order Processing (example) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • 1. 5 ms - Receive 2. 30 ms - Verify 3. 30 ms - Save 1. Generate PDF 4. 10 ms – Dispatch 1. Email PDF 2. Dispatch 5. 50 ms – Render Total: 125 ms GET /purchase OK Event-Driven Staged Architecture (SEDA) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Don’t keep the client waiting! “Can I defer it?” Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • PDF service layer Email service layer GET /purchase OK Services Oriented Architecture (SOA) helps you maintain your sanity Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Scalability of service layer GET /purchase OK SOA = Simple(r) Scalability Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Elastic Computing EC2 & Virtualization Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • 3 servers (always): $216 / month $0.10 / hour 3 servers (elastic): $120 / month 0AM – 8AM 8AM – 24PM Elastic Computing (Pay by the sip) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • 100 computers * 1 hour = 1 computer * 100 hours Elastic Computing (Pay by the sip) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Era of Parallelism Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Elastic car manufacturing? High capital investment People & Unions Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Elastic Computing = Assembly Line 2.0 EC2 & Virtualization Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Elastic, Event-Driven, Staged, Services Oriented Architecture (that’s a mouthful) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Phew. Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Buzzword: Real-time web Scientific translation: really, really, really, fast Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Ouch! Really fast Millions of response users time Real-time pain Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • HTTP? DB? … HTTP? Queue? … Real-time communication Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Emerging Standards: XMPP & AMQP Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • From: A, To: B From: A, To: B Hello! Hello! Extensible Messaging and Presence Protocol (XMPP) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Event-stream protocol Persistent connections Presence Identity and XMPP Features authentication Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • User Domain Resource ilya@aiderss.com/office Jabber Software Foundation JID: Federation, Identity & Authentication Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • <message from=quot;ilya@aiderss.com/officequot; type=quot;chatquot; to=quot;ilya@igvita.comquot; id=quot;aae1aquot;> <body>hello</body> <active xmlns=quot;http://jabber.org/protocol/chatstatesquot;/> </message> Very verbose protocol (XML) Example: Message Routing with XMPP Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • XMPP in the wild: Google Talk Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • XMPP in the wild: Google Talk + Video Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Psi: cross-platform Jabber/XMPP client Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • require quot;rubygemsquot; require quot;xmpp4rquot; jid = Jabber::JID::new(quot;ilya@aiderss.comquot;) client = Jabber::Client.new(jid) client.connect(quot;talk.google.comquot;) client.auth(quot;passwordquot;) to = quot;ilya@aiderss.comquot; subject = quot;Jabber clientquot; message = quot;Hello XMPP World!quot; piclient.send Jabber::Message::new(to, message).set_subject(subject) # <message to='ilya@igvita.com'> # <body>Hello XMPP World!</body> # <subject>Jabber client</subject> XMPP4R (Ruby) Demo # </message> Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • require quot;rubygemsquot; require quot;xmpp4rquot; jid = Jabber::JID::new(quot;ilya@aiderss.comquot;) client = Jabber::Client.new(jid) client.connect(quot;talk.google.comquot;) client.auth(quot;passwordquot;) to = quot;ilya@aiderss.comquot; subject = quot;Jabber clientquot; message = quot;Hello XMPP World!quot; piclient.send Jabber::Message::new(to, message).set_subject(subject) # <message to='ilya@igvita.com'> # <body>Hello XMPP World!</body> # <subject>Jabber client</subject> XMPP4R (Ruby) Demo # </message> Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • client.send(Jabber::Presence.new.set_type(:away)) # <presence from='daniel@aiderss.com/iMac8D2CB97D' to='ilya@aiderss.com/0EDD826C' xmlns='jabber:client'> # <show>away</show> # <priority>0</priority> # <x xmlns='http://www.apple.com/xmpp/idle'> # <idle-since>2009-04-01T21:48:15Z</idle-since> Client Idle… # </x> # </presence> client.add_message_callback do |m| puts quot;#{m.from} -- #{m.body}quot; end # > daniel@aiderss.com -- Hey! XMPP4R (Ruby) Demo Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • client.send(Jabber::Presence.new.set_type(:available)) # <presence from='daniel@aiderss.com/iMac8D2CB97D' to='ilya@aiderss.com/0EDD826C' xmlns='jabber:client'> # <show>away</show> # <priority>0</priority> # <x xmlns='http://www.apple.com/xmpp/idle'> # <idle-since>2009-04-01T21:48:15Z</idle-since> Client Idle… # </x> # </presence> client.add_message_callback do |m| puts quot;#{m.from} -- #{m.body}quot; end # > daniel@aiderss.com -- Hey! XMPP4R (Ruby) Demo Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • One-to-many distribution Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • XEP-0060: Publish-Subscribe (Pubsub) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Many wasteful checks Data? No Data? No Data? Yes HTTP / SMTP Polling: painful, wasteful Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Persistent connection Subscribe New message! Publish-Subscribe Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • <iq type='set‘ from='hamlet@denmark.lit/blogbot' to='pubsub.shakespeare.lit' id='pub1'> <pubsub xmlns='http://jabber.org/protocol/pubsub'> <publish node='princely_musings'> IQ Stanza <item> <entry xmlns='http://www.w3.org/2005/Atom'> <title>Soliloquy<title> <summary> To be, or not to be: that is the question! <summary> <link rel='alternate' type='text/html' href='http://denmark.lit/2003/12/13/atom03'/> <id>tag:denmark.lit,2003:entry-32397</id> <published>2003-12-13T18:30:02Z</published> <updated>2003-12-13T18:30:02Z</updated> </entry> </item> </publish> PubSub Protocol: Client XML </pubsub> </iq> Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • <iq type='set‘ from='hamlet@denmark.lit/blogbot' to='pubsub.shakespeare.lit' id='pub1'> <pubsub xmlns='http://jabber.org/protocol/pubsub'> <publish node='princely_musings'> <item> <entry xmlns='http://www.w3.org/2005/Atom'> <title>Soliloquy<title> <summary> To be, or not to be: that is the question! <summary> <link rel='alternate' type='text/html' AtomPub href='http://denmark.lit/2003/12/13/atom03'/> <id>tag:denmark.lit,2003:entry-32397</id> <published>2003-12-13T18:30:02Z</published> <updated>2003-12-13T18:30:02Z</updated> </entry> </item> </publish> PubSub Protocol: Client XML </pubsub> </iq> Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Distribute Publish XEP-0060: Publish-Subscribe (Pubsub) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • <message from='pubsub.shakespeare.lit' to='francisco@denmark.lit' id='foo'> <event xmlns='http://jabber.org/protocol/pubsub#event'> <items node='princely_musings'> <item id='ae890ac52d0df67ed7cfdf51b644e901'> [ ... ENTRY ... ] Subscriber A </item> </items> </event> </message> <message from='pubsub.shakespeare.lit' to='bernardo@denmark.lit' id='bar'> <event xmlns='http://jabber.org/protocol/pubsub#event'> <items node='princely_musings'> <item id='ae890ac52d0df67ed7cfdf51b644e901'> Subscriber B [ ... ENTRY ... ] </item> </items> </event> </message> XEP-0060: Publish-Subscribe (Pubsub) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Seems neat, but so what? Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • XMPP XMPP Real-time communication Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • XMPP XMPP XMPP Real-time communication Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • require 'fire_hydrant' require 'yaml' hydrant = FireHydrant.new(YAML.load(File.read(quot;config.ymlquot;))) hydrant.on_location_update do |user| puts quot;#{user.token} has moved to #{user.locations.first}.quot; Push notifications end hydrant.run! Ruby + FireEagle via XMPP Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Presence: available GET /purchase OK Elastic Computing + Presence Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Ejabberd Erlang Defacto XMPP server Djabberd Perl OpenFire Java RPM, GUI, shiny Tigase Java XMPP / Jabber Servers Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Advanced Message Queuing Protocol (AMQP) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • “AMQP is an open Internet Protocol for Business Messaging” AMQP Working Group (16 companies) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Consumer AMQP Broker Publisher AMQP Architecture Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Broker Clustering AMQP Clustering Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Routing key: usd.stock.amz Message: I like AMZ! Direct Topic Fanout Exchange Exchange Exchange AMQP Broker Internals Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Routing key: usd.stock.amz Message: I like AMZ! Direct Topic Fanout Exchange Exchange Exchange Name: amazon Queue Bind: usd.stock.amz AMQP Direct Exchange Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Routing key: usd.stock.amz Message: I like AMZ! Direct Topic Fanout Exchange Exchange Exchange Queue Queue Message: I like AMZ! AMQP Direct Exchange Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Routing key: usd.stock.msft Message: I like Microsoft! Direct Topic Fanout Exchange Exchange Exchange Name: stocks Queue Bind: usd.stock.* AMQP Topic Exchange Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Routing key: usd.stock.msft Message: I like Microsoft! Direct Topic Fanout Exchange Exchange Exchange Queue Message: I like Microsoft! AMQP Topic Exchange Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Routing key: usd.stock.msft Message: I like Microsoft! Direct Topic Fanout Exchange Exchange Exchange Queue Queue Name: stocks 1 2 Bind: “” AMQP Fanout Exchange Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Routing key: usd.stock.msft Message: I like Microsoft! Direct Topic Fanout Exchange Exchange Exchange Queue Queue Message: I like Microsoft 1 2 AMQP Fanout Exchange Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • AMQP Kung-fu: Load-balancing Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Routing key: usd.stock.amz Message: I like AMZ! Direct Topic Fanout Exchange Exchange Exchange Queue Only one client gets the message! AMQP Load Balancing Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Bind: purchase.pdf GET /purchase OK Elastic AMQP Load-balancing Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • AMQP Kung-fu: Persistence & Durability Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Routing key: usd.stock.amz Message: I like AMZ! Direct Topic Fanout Exchange Exchange Exchange Durable: True Queue Persistent: True AMQP Persistence & Durability Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Almost there :) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • require 'mq' AMQP.start(:host => 'amqp-server.com') do Publisher mq = MQ.new mq.topic('stocks').publish(quot;5.95quot;, :key => quot;usd.amzquot;) end AMQP.start(:host => 'amqp-server.com') do mq = MQ.new mq.queue('amazon stock').bind(mq.topic('stocks'), :key => 'usd.amz').subscribe{ |price| print 'apple stock', price } end AMQP + Ruby Example Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • require 'mq' AMQP.start(:host => 'amqp-server.com') do mq = MQ.new mq.topic('stocks').publish(quot;5.95quot;, :key => quot;usd.amzquot;) end Consumer AMQP.start(:host => 'amqp-server.com') do mq = MQ.new mq.queue('amazon stock').bind(mq.topic('stocks'), :key => 'usd.amz').subscribe{ |price| print ‘amazon stock', price } end AMQP + Ruby Example Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Elastic, Event-Driven, <Real-time>, Staged, Services Oriented Architecture (oi…) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • XMPP / AMQP Publisher Elastic Architecture Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • XMPP / AMQP Subscribe Exposing Publish-Subscribe API Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda
  • Thanks. Questions & The slides… My blog Comments Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda