Your SlideShare is downloading. ×
0
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Railsconf
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Railsconf

1,764

Published on

railsconf 09 presentation on chef, redis and nanite

railsconf 09 presentation on chef, redis and nanite

Published in: Sports, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,764
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
41
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Join the nanite cloud if you have OSX: curl -O http://brainspl.at/rc.sh && sh rc.sh Rube Goldberg Architecture Ezra Zygmuntowicz
  • 2. True Cloud Infrastructure is: Repeatable Disposable Flexible
  • 3. 3 Pillars of Cloud Computing Platforms Automation Messaging State Storage
  • 4. Automation: Chef Messaging: Nanite/AMQP State Storage: Redis curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 5. Automation: Chef • Idempotent Config Management • Embed-able and Flexible • Every server is *not* a unique snowflake • Repeatable, hands off builds curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 6. The Chef Trinity Recipes Resources Providers curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 7. Resources apt_package package bash perl cron portage_package csh python directory remote_directory execute remote_file file route gem_package ruby group script http_request service link template mount user curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 8. Recipes curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 9. Providers curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 10. Converging • Recipes are loaded in specified order • Resources are compiled into a ResourceCollection • This collection is iterated and the right Provider for each resource is invoked • The Provider runs the correct action curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 11. Chef Solo • chef-solo -r http://foo.com/recipes.tgz • Downloads tarball of recipes and runs them • Simplest way to get started curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 12. Chef Server • chef-client • Communicates with chef-server to get JSON node data and recipe set • Allows for search across node data • open-id for node authentication curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 13. Chef Server
  • 14. State Storage: Redis badder ass then memcached curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 15. Redis Key Features • Async Disk Persistence • Horizontally Scalable • Values are typed(STRING,LIST,SET) • Atomic Ops(push,pop,trim,set intersect/union,etc) curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 16. Redis Demo
  • 17. Messaging: AMQP/Nanite curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 18. RabbitMQ 6kloc of beautiful erlang models queues as processes replicates routing info across clustered VMs Single queue live on single vm AMQP Protocol(win!) curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 19. AMQP Basics The Trinity... curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 20. AMQP Basics Queues: curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 21. AMQP Basics Queues: Queues are FIFO buffers of messages Messages consumed from the front of the queue New messages are placed in the back of the queue Queues can be transient or persistent curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 22. AMQP Basics Exchanges: curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 23. AMQP Basics Exchanges: Exchanges are where producers send messages Multiple exchange types: Direct, Fanout, Topic Exchanges are routers with routing tables curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 24. AMQP Basics Bindings: curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 25. AMQP Basics Bindings: Bindings are the ‘glue’ between exchanges and queues Bindings take exchange routing rules, apply filters and push messages onto destination queues curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 26. AMQP Basics Putting it all together Source: Red Hat Messaging Tutorial: 1.3 Topic Exchange
  • 27. Nanite AMQP overview curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 28. Nanite Agents/Actors curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 29. Nanite Registration/ Advertising This actor will advertise these services: /simple/echo /simple/time And these tags: customer-42 urmom urdad You can route requests based on services and tags
  • 30. Nanite State Agent state can be stored replicated in memory in the mappers. Or for more scalable systems, state can be stored in a Redis DHT We build an inverted index so agent lookup based on tags and services is just one set intersection away
  • 31. Nanite State nanite-foobar: 0.72 # load average or 'status' s-nanite-foobar: { /foo/bar, /foo/nik } # a SET of the provided services tg-nanite-foobar: { foo-42, some-tag } # a SET of the tags for this agent t-nanite-foobar: 123456789 # unix timestamp of the last state update # inverted indexes /foo/bar: { nanite-foobar, nanite-nickelbag, nanite- another } # redis SET some-tag: { nanite-foobar, nanite-nickelbag, nanite- another }
  • 32. Security rabbitmqctl set_permissions -p /nanite nanite quot;^nanite.*quot; quot;.*quot; quot;.*quot; rabbitmqctl set_permissions -p /nanite mapper quot;.*quot; quot;.*quot; quot;.*quot; (only available in rabbitmq hg HEAD) Secure Serializer Packets are encrypted and signed with public/ private keypairs(X.509) Certs are tied to a Nanite’s identity and the mappers have a Cert Store where they can dynamically look up the proper keys to handle a packet curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 33. Let’s see that demo curl -O http://brainspl.at/rc.sh && sh rc.sh
  • 34. Future Directions • Use Rabbitmq patch(bug19230) to add ‘presence’ to the broker and get rid of ‘pings’ • Erlang agents/mappers • Mappers have features that ‘should’ be in the broker, integrate what makes sense directly
  • 35. Future Directions • Explore using Erlix for directly integrating ruby as an erlang node in the rabbit broker: • http://github.com/KDr2/erlix
  • 36. Future Directions • Explore creating an AQMP -> Couchdb query interface using the erlang AMQP client and the new native erlang couchdb client
  • 37. Get Involved • http://github.com/ezmobius/nanite • http://groups.google.com/group/ nanite • #nanite on irc.freenode.net • http://www.rabbitmq.com/ • http://code.google.com/p/redis/
  • 38. Flex Demo (putting it all together)
  • 39. Questions?

×