Event Driven Architecture - MeshU - Ilya Grigorik

Event-Driven Architectures Ilya Grigorik CTO / AideRSS Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Apr 7th launch: Bestest Blog discovery! Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

• Request, Response Cycle • Event Driven Architectures • Cloud Computing • Real-time Web! Fully buzzword • XMPP & AMQP compliant! • Example Questions & The slides… My blog Comments Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Elastic, Event-Driven, Real-time, Staged, Services Oriented Architecture (Ready, ready? Go.) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

GET /home GET /home GET /home Response Request, Response Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Google: +200ms = -10% searches Amazon: +100ms = -5% purchases GET /home Response Response Time is Everything Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Bottleneck Complexity and Time Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

“Rails, Django, Seaside, Grails…” cant scale. Myth: Slow Frameworks Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Application server: 1-1000+ req/s “Fast Enough” Frameworks Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

1. 5 ms: Process request 2. 30 ms: Authenticate user 3. 90 ms: Contact database & other services 4. 30 ms: Render response Total: 155 ms, 0.5 MB / conn OR 6.5 req / s Request, Response assembly Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

The Proxy Solution Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

The “More” Proxy Solution Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

This is madness. (after a certain point) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

1. Receive 2. Verify 3. Dispatch 4. Aggregate 5. Handle errors 6. Render Throughput != Response Time Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Buzzword: Event-Driven Architectures (aka, reinventing the wheel) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Auto Manufacturing circa 1910 2-3 men, 1-2 days / car Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

More workers! Scaling the manufacturing process Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Then Henry Ford came along… Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

And gave us the assembly line! Scaling the manufacturing process Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Queuing theory Lean manufacturing Just in time Lower response time, higher throughput! 93 seconds / car 1M + cars / year Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Event Driven Architecture (EDA) = Assembly line except, we’re not building cars here… Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

1. 5 ms - Receive 2. 30 ms - Verify Total: 365 ms, ~ 3 req / s 3. 30 ms - Save 4. 200 ms – Generate PDF 5. 50 ms - Email 6. 50 ms - Render GET /purchase OK Monolith Order Processing (example) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Can someone else do it? Total: 115 ms, ~ 3 req / s 1. 5 ms – Receive No ~ 70% improvement in 2. 30 ms - Verify No response time! 3. 30 ms – Save No 4. 200 ms – Generate PDF Yes 5. 50 ms – Email Yes 6. 50 ms – Render Yes GET /purchase OK Assembly Line Order Processing (example) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

1. 5 ms - Receive 2. 30 ms - Verify 3. 30 ms - Save 1. Generate PDF 4. 10 ms – Dispatch 1. Email PDF 2. Dispatch 5. 50 ms – Render Total: 125 ms GET /purchase OK Assembly Line Order Processing (example) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

1. 5 ms - Receive 2. 30 ms - Verify 3. 30 ms - Save 1. Generate PDF 4. 10 ms – Dispatch 1. Email PDF 2. Dispatch 5. 50 ms – Render Total: 125 ms GET /purchase OK Event-Driven Staged Architecture (SEDA) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Don’t keep the client waiting! “Can I defer it?” Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

PDF service layer Email service layer GET /purchase OK Services Oriented Architecture (SOA) helps you maintain your sanity Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Scalability of service layer GET /purchase OK SOA = Simple(r) Scalability Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Elastic Computing EC2 & Virtualization Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

3 servers (always): $216 / month $0.10 / hour 3 servers (elastic): $120 / month 0AM – 8AM 8AM – 24PM Elastic Computing (Pay by the sip) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

100 computers * 1 hour = 1 computer * 100 hours Elastic Computing (Pay by the sip) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Era of Parallelism Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Elastic car manufacturing? High capital investment People & Unions Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Elastic Computing = Assembly Line 2.0 EC2 & Virtualization Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Elastic, Event-Driven, Staged, Services Oriented Architecture (that’s a mouthful) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Phew. Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Buzzword: Real-time web Scientific translation: really, really, really, fast Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Ouch! Really fast Millions of response users time Real-time pain Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

HTTP? DB? … HTTP? Queue? … Real-time communication Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Emerging Standards: XMPP & AMQP Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

From: A, To: B From: A, To: B Hello! Hello! Extensible Messaging and Presence Protocol (XMPP) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Event-stream protocol Persistent connections Presence Identity and XMPP Features authentication Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

User Domain Resource ilya@aiderss.com/office Jabber Software Foundation JID: Federation, Identity & Authentication Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

<message from=quot;ilya@aiderss.com/officequot; type=quot;chatquot; to=quot;ilya@igvita.comquot; id=quot;aae1aquot;> <body>hello</body> <active xmlns=quot;http://jabber.org/protocol/chatstatesquot;/> </message> Very verbose protocol (XML) Example: Message Routing with XMPP Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

XMPP in the wild: Google Talk Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

XMPP in the wild: Google Talk + Video Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Psi: cross-platform Jabber/XMPP client Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

require quot;rubygemsquot; require quot;xmpp4rquot; jid = Jabber::JID::new(quot;ilya@aiderss.comquot;) client = Jabber::Client.new(jid) client.connect(quot;talk.google.comquot;) client.auth(quot;passwordquot;) to = quot;ilya@aiderss.comquot; subject = quot;Jabber clientquot; message = quot;Hello XMPP World!quot; piclient.send Jabber::Message::new(to, message).set_subject(subject) # <message to='ilya@igvita.com'> # <body>Hello XMPP World!</body> # <subject>Jabber client</subject> XMPP4R (Ruby) Demo # </message> Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

client.send(Jabber::Presence.new.set_type(:away)) # <presence from='daniel@aiderss.com/iMac8D2CB97D' to='ilya@aiderss.com/0EDD826C' xmlns='jabber:client'> # <show>away</show> # <priority>0</priority> # <x xmlns='http://www.apple.com/xmpp/idle'> # <idle-since>2009-04-01T21:48:15Z</idle-since> Client Idle… # </x> # </presence> client.add_message_callback do |m| puts quot;#{m.from} -- #{m.body}quot; end # > daniel@aiderss.com -- Hey! XMPP4R (Ruby) Demo Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

client.send(Jabber::Presence.new.set_type(:available)) # <presence from='daniel@aiderss.com/iMac8D2CB97D' to='ilya@aiderss.com/0EDD826C' xmlns='jabber:client'> # <show>away</show> # <priority>0</priority> # <x xmlns='http://www.apple.com/xmpp/idle'> # <idle-since>2009-04-01T21:48:15Z</idle-since> Client Idle… # </x> # </presence> client.add_message_callback do |m| puts quot;#{m.from} -- #{m.body}quot; end # > daniel@aiderss.com -- Hey! XMPP4R (Ruby) Demo Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

One-to-many distribution Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

XEP-0060: Publish-Subscribe (Pubsub) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Many wasteful checks Data? No Data? No Data? Yes HTTP / SMTP Polling: painful, wasteful Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Persistent connection Subscribe New message! Publish-Subscribe Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

<iq type='set‘ from='hamlet@denmark.lit/blogbot' to='pubsub.shakespeare.lit' id='pub1'> <pubsub xmlns='http://jabber.org/protocol/pubsub'> <publish node='princely_musings'> IQ Stanza <item> <entry xmlns='http://www.w3.org/2005/Atom'> <title>Soliloquy<title> <summary> To be, or not to be: that is the question! <summary> <link rel='alternate' type='text/html' href='http://denmark.lit/2003/12/13/atom03'/> <id>tag:denmark.lit,2003:entry-32397</id> <published>2003-12-13T18:30:02Z</published> <updated>2003-12-13T18:30:02Z</updated> </entry> </item> </publish> PubSub Protocol: Client XML </pubsub> </iq> Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

<iq type='set‘ from='hamlet@denmark.lit/blogbot' to='pubsub.shakespeare.lit' id='pub1'> <pubsub xmlns='http://jabber.org/protocol/pubsub'> <publish node='princely_musings'> <item> <entry xmlns='http://www.w3.org/2005/Atom'> <title>Soliloquy<title> <summary> To be, or not to be: that is the question! <summary> <link rel='alternate' type='text/html' AtomPub href='http://denmark.lit/2003/12/13/atom03'/> <id>tag:denmark.lit,2003:entry-32397</id> <published>2003-12-13T18:30:02Z</published> <updated>2003-12-13T18:30:02Z</updated> </entry> </item> </publish> PubSub Protocol: Client XML </pubsub> </iq> Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Distribute Publish XEP-0060: Publish-Subscribe (Pubsub) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

<message from='pubsub.shakespeare.lit' to='francisco@denmark.lit' id='foo'> <event xmlns='http://jabber.org/protocol/pubsub#event'> <items node='princely_musings'> <item id='ae890ac52d0df67ed7cfdf51b644e901'> [ ... ENTRY ... ] Subscriber A </item> </items> </event> </message> <message from='pubsub.shakespeare.lit' to='bernardo@denmark.lit' id='bar'> <event xmlns='http://jabber.org/protocol/pubsub#event'> <items node='princely_musings'> <item id='ae890ac52d0df67ed7cfdf51b644e901'> Subscriber B [ ... ENTRY ... ] </item> </items> </event> </message> XEP-0060: Publish-Subscribe (Pubsub) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Seems neat, but so what? Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

XMPP XMPP Real-time communication Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

XMPP XMPP XMPP Real-time communication Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

require 'fire_hydrant' require 'yaml' hydrant = FireHydrant.new(YAML.load(File.read(quot;config.ymlquot;))) hydrant.on_location_update do |user| puts quot;#{user.token} has moved to #{user.locations.first}.quot; Push notifications end hydrant.run! Ruby + FireEagle via XMPP Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Presence: available GET /purchase OK Elastic Computing + Presence Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Ejabberd Erlang Defacto XMPP server Djabberd Perl OpenFire Java RPM, GUI, shiny Tigase Java XMPP / Jabber Servers Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Advanced Message Queuing Protocol (AMQP) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

“AMQP is an open Internet Protocol for Business Messaging” AMQP Working Group (16 companies) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Consumer AMQP Broker Publisher AMQP Architecture Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Broker Clustering AMQP Clustering Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Routing key: usd.stock.amz Message: I like AMZ! Direct Topic Fanout Exchange Exchange Exchange AMQP Broker Internals Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Routing key: usd.stock.amz Message: I like AMZ! Direct Topic Fanout Exchange Exchange Exchange Name: amazon Queue Bind: usd.stock.amz AMQP Direct Exchange Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Routing key: usd.stock.amz Message: I like AMZ! Direct Topic Fanout Exchange Exchange Exchange Queue Queue Message: I like AMZ! AMQP Direct Exchange Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Routing key: usd.stock.msft Message: I like Microsoft! Direct Topic Fanout Exchange Exchange Exchange Name: stocks Queue Bind: usd.stock.* AMQP Topic Exchange Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Routing key: usd.stock.msft Message: I like Microsoft! Direct Topic Fanout Exchange Exchange Exchange Queue Message: I like Microsoft! AMQP Topic Exchange Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Routing key: usd.stock.msft Message: I like Microsoft! Direct Topic Fanout Exchange Exchange Exchange Queue Queue Name: stocks 1 2 Bind: “” AMQP Fanout Exchange Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Routing key: usd.stock.msft Message: I like Microsoft! Direct Topic Fanout Exchange Exchange Exchange Queue Queue Message: I like Microsoft 1 2 AMQP Fanout Exchange Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

AMQP Kung-fu: Load-balancing Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Routing key: usd.stock.amz Message: I like AMZ! Direct Topic Fanout Exchange Exchange Exchange Queue Only one client gets the message! AMQP Load Balancing Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Bind: purchase.pdf GET /purchase OK Elastic AMQP Load-balancing Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

AMQP Kung-fu: Persistence & Durability Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Routing key: usd.stock.amz Message: I like AMZ! Direct Topic Fanout Exchange Exchange Exchange Durable: True Queue Persistent: True AMQP Persistence & Durability Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Almost there :) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

require 'mq' AMQP.start(:host => 'amqp-server.com') do Publisher mq = MQ.new mq.topic('stocks').publish(quot;5.95quot;, :key => quot;usd.amzquot;) end AMQP.start(:host => 'amqp-server.com') do mq = MQ.new mq.queue('amazon stock').bind(mq.topic('stocks'), :key => 'usd.amz').subscribe{ |price| print 'apple stock', price } end AMQP + Ruby Example Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

require 'mq' AMQP.start(:host => 'amqp-server.com') do mq = MQ.new mq.topic('stocks').publish(quot;5.95quot;, :key => quot;usd.amzquot;) end Consumer AMQP.start(:host => 'amqp-server.com') do mq = MQ.new mq.queue('amazon stock').bind(mq.topic('stocks'), :key => 'usd.amz').subscribe{ |price| print ‘amazon stock', price } end AMQP + Ruby Example Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Elastic, Event-Driven, <Real-time>, Staged, Services Oriented Architecture (oi…) Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

XMPP / AMQP Publisher Elastic Architecture Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

XMPP / AMQP Subscribe Exposing Publish-Subscribe API Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

Thanks. Questions & The slides… My blog Comments Event-Driven Architectures http://bit.ly/eda-mesh @igrigorik #meshu #eda

http://www.igvita.com	3812
http://www.lifeyun.com	80
http://www.slideshare.net	41
http://00f.net	41
http://confluence.b2e.kz	36
http://leonardoeloy.com	19
http://b2e.kz	17
http://www.techvibes.com	16
http://cms	13
http://bulat.wikispaces.com	11
http://mwyrobek.tumblr.com	7
http://translate.googleusercontent.com	6
http://flavors.me	4
http://static.slidesharecdn.com	4
https://00f.net	3
http://neou.com:1208	3
http://webcache.googleusercontent.com	2
http://www.b2e.kz	2
http://coderwall.com	2
http://www.redcanary.ca	2
http://www.iweb34.com	2
http://www.hanrss.com	1
http://www.railsfire.com	1
http://www.00f.net	1
http://conversation.ecairn.com	1
http://feeds.igvita.com	1
http://feeds.feedburner.com	1
http://safe.tumblr.com	1
http://74.125.113.132	1
http://74.125.155.132	1
http://209.85.229.132	1
http://127.0.0.1:8795	1

Event Driven Architecture - MeshU - Ilya Grigorik

by Ilya Grigorik on Apr 06, 2009

Accessibility

Categories

Upload Details

Usage Rights

32 Embeds 4,134

Statistics

Event Driven Architecture - MeshU - Ilya Grigorik Presentation Transcript