Rube Goldberg
  Architecture

Building a Better
Mousetrap for the
     Cloud
  Ezra Zygmuntowicz
3 Pillars of a good
 Cloud Infrastructure
    Automation
Command & Control
Scalable State Storage
Automation:
         Chef
Command & Control:
        Nanite
Scalable State Storage:
        Redis
Go big or go home...
Automation: Chef
• Idempotent configuration management
• Embed-able and flexible
• Heavy lifting behind Solo/Flex
• No more ...
Basic Concepts:

   Resources
    Recipes
   Providers
Resources:
apt_package
     bash                         package
     cron                           perl
      csh       ...
Recipes:
Providers:
Converging
• Recipes are loaded in specified order
• Resources are compiled into objects and
  stored in a ResourceCollecti...
Chef Solo

• chef-solo -r http://foo.com/recipes.tgz
• Downloads a tarball of recipes and runs
  them
• Dead simple to get...
Chef Server
• chef-client
• Communicates with the chef-server to get
  recipes and JSON data
• Allows for searching across...
Chef Server
State Storage: Redis




persistent memcached on steroids
Redis Features
• Asynchronous Persist to disk
• Horizontally scalable
• Values can have types
• STRING’s, LIST’s and SET’s...
Let’s see a demo
Nanite
Built around RabbitMQ

 • Written in erlang, cluster-able, highly
   scalable, fast as hell.
 • AMQP protocol provides man...
<3 RabbitMQ
Nanite agents
consist of multiple Actors
Nanite agents advertise
their services and status
      Feeds#crawl
       advertises:
      /feeds/crawl
Load average is ...
Nanite Mappers
Track nanites and their advertised services and status

   Can do dispatch based on a number of factors
   ...
Multiple Dispatch
      Styles
Least loaded dispatch
and the fitness function
Agents ping the mapper exchange
  every @ping_time seconds.
  Mappers track the state of all
 nanites and remove them from...
Nanite gives us:
• Presence, we know when nanites are ready
  for requests or not.
• Self assembly, nanites can come and g...
Nanite Demo
Please download and run this shell script:
https://cloud.engineyard.com/ete.sh
Questions?
Upcoming SlideShare
Loading in...5
×

Building a Better Mousetrap for the Cloud

897

Published on

Published in: Economy & Finance, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
897
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Building a Better Mousetrap for the Cloud

  1. 1. Rube Goldberg Architecture Building a Better Mousetrap for the Cloud Ezra Zygmuntowicz
  2. 2. 3 Pillars of a good Cloud Infrastructure Automation Command & Control Scalable State Storage
  3. 3. Automation: Chef Command & Control: Nanite Scalable State Storage: Redis
  4. 4. Go big or go home...
  5. 5. Automation: Chef • Idempotent configuration management • Embed-able and flexible • Heavy lifting behind Solo/Flex • No more “every server is a unique snowflake” • Badass
  6. 6. Basic Concepts: Resources Recipes Providers
  7. 7. Resources: apt_package bash package cron perl csh portage_package directory python execute remote_directory file remote_file gem_package route group ruby http_request script link service mount template user
  8. 8. Recipes:
  9. 9. Providers:
  10. 10. Converging • Recipes are loaded in specified order • Resources are compiled into objects and stored in a ResourceCollection • ResourceCollection is iterated and the right Provider for each Resource is invoked • The Providers runs the specified action on each Resource
  11. 11. Chef Solo • chef-solo -r http://foo.com/recipes.tgz • Downloads a tarball of recipes and runs them • Dead simple to get started with
  12. 12. Chef Server • chef-client • Communicates with the chef-server to get recipes and JSON data • Allows for searching across other nodes attributes • Uses open-id for node authentication
  13. 13. Chef Server
  14. 14. State Storage: Redis persistent memcached on steroids
  15. 15. Redis Features • Asynchronous Persist to disk • Horizontally scalable • Values can have types • STRING’s, LIST’s and SET’s • Atomic Operations (push, pop, incr, decr, set intersection)
  16. 16. Let’s see a demo
  17. 17. Nanite
  18. 18. Built around RabbitMQ • Written in erlang, cluster-able, highly scalable, fast as hell. • AMQP protocol provides many nice features • Transient, Persistent and Transactional semantics
  19. 19. <3 RabbitMQ
  20. 20. Nanite agents consist of multiple Actors
  21. 21. Nanite agents advertise their services and status Feeds#crawl advertises: /feeds/crawl Load average is advertised as default status
  22. 22. Nanite Mappers Track nanites and their advertised services and status Can do dispatch based on a number of factors Run inside your Merb or Rails app or as a separate service State of all nanites is replicated across all mappers in memory *or* stored in Redis
  23. 23. Multiple Dispatch Styles
  24. 24. Least loaded dispatch and the fitness function
  25. 25. Agents ping the mapper exchange every @ping_time seconds. Mappers track the state of all nanites and remove them from mapping if they haven’t reported in within a timeout
  26. 26. Nanite gives us: • Presence, we know when nanites are ready for requests or not. • Self assembly, nanites can come and go and can run anywhere with zero configuration in the mappers. • Dispatch based on load or any fitness function that suits your app • Easily take advantage of cloud
  27. 27. Nanite Demo Please download and run this shell script: https://cloud.engineyard.com/ete.sh
  28. 28. Questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×