Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Vertebra: Bringing The Cloud Down To Earth

2,433 views

Published on

Published in: Technology, Business
  • Be the first to comment

Vertebra: Bringing The Cloud Down To Earth

  1. 1. ☔ VERTEBRA Bringing the Cloud Down To Earth Jayson Vantuyl Chief Systems Architect Engine Yard, Inc.
  2. 2. Who Is This Guy? Jayson Vantuyl Chief Systems Architect / Founder Architecting Large Networks Since 1994 Engine Yard, Inc. Large Hosting Provider focused on hosting Ruby Applications Managing Over 2,000 Virtual Servers
  3. 3. The Classic Blockers My existing resources are not in the cloud. A critical component is not in the cloud. Critical components don’t fit together. I’m on a budget: small first, scale later. My boss wants it all (or some of it) in-house.
  4. 4. Fog Computing It’s a Cloud, but it’s “down to Earth”. Fog, get it? Focus on What, Not Where Clouds Without Borders Don’t Rain On My Parade A Sky Full Of Clouds
  5. 5. The Nuts and Bolts Agents Agents XMPP XMPP Discovery Security Security Workflow & Resource Engine Database Discovery
  6. 6. XMPP XML Messaging and Ordered Message Delivery, Presence Protocol Presence IETF Standard (RFC 3920, Many Implementations, 3921) Designed To Be Extensible Each Connection Is Server-to-Server Security via Essentially Two Streaming Dial-Back XML Documents Application-Level Reliability Authentication via SRV DNS Records Transport Security
  7. 7. Resources Every point of control in the system is described by a set of a “resources” (much like a URL). These resources need not be unique in the cloud. They are organized into hierarchies. They are the point of abstraction.
  8. 8. Some Possible Resources bob@mycompany.com: sally@mycompany.com: /us/ny/new_york /ca/qc/montreal Each box is an agent that /sales/copy_writer /sales/rep offers operations on the jack@mycompany.com: jill@mycompany.com: listed resources /us/dc/washington /us/fl/tampa /research/weapons /ops/helpdesk /clearance/s /clearance/c Geography /clearance/ers /clearance/rs web@mycompany.com: mail@mycompany.com: Organizational Structure /net/tcpip /net/tcpip /server/http /server/smtp /blog /service/mailer Security Clearance us_sec@mycompany.com: ca_sec@mycompany.com: /security /security /us /ca Type Of Service us_hr@mycompany.com: ca_hr@mycompany.com: /hr /hr /us /ca
  9. 9. Operations Fundamental Unit Of Work In Vertebra Shaped Like Remote Procedure Call Receives Parameters In A Namespace Discover Agents That Provide The Resources Request — Ack — Result* — Done Scope (i.e. shotgun or sniper?)
  10. 10. Example Operations instant_message(who=/sales,msg=”Goodbye.”) fire(who=/manage/sales) kick_out(who=/security/sales) Completely Unrealistic add_entry(site=/blog,message=”Fired Sales!”) Example bob@mycompany.com: sally@mycompany.com: /us/ny/new_york /ca/qc/montreal /sales/copy_writer /sales/rep Let’s Fire Some People, The us_hr@mycompany.com: ca_hr@mycompany.com: Entire Sales Dept. /manage /manage /us /ca Then Blog About It us_sec@mycompany.com: /security ca_sec@mycompany.com: /security /us /ca web@mycompany.com: /net/tcpip /server/http /blog
  11. 11. An Ominous Message instant_message(who=/sales,msg=”Goodbye.”) “who” is a resource, fire(who=/manage/sales) discovery is done on kick_out(who=/security/sales) /sales add_entry(site=/blog,message=”Fired Sales!”) We discover bob@mycompany.com: /us/ny/new_york sally@mycompany.com: /ca/qc/montreal sally@mycompany.com and /sales/copy_writer /sales/rep bob@mycompany.com us_hr@mycompany.com: ca_hr@mycompany.com: /manage /manage /us /ca Both get “instant_message” us_sec@mycompany.com: ca_sec@mycompany.com: operation, and the agent on /security /us /security /ca their computer delivers the message, “Goodbye.” web@mycompany.com: /net/tcpip /server/http /blog
  12. 12. You’re Fired! instant_message(who=/sales,msg=”Goodbye.”) fire(who=/manage/sales) “who” is a resource, discovery is done on kick_out(who=/security/sales) /manage/sales add_entry(site=/blog,message=”Fired Sales!”) bob@mycompany.com: sally@mycompany.com: We discover us_hr and /us/ny/new_york /sales/copy_writer /ca/qc/montreal /sales/rep ca_hr, which both offer us_hr@mycompany.com: ca_hr@mycompany.com: /manage /manage /manage /us /ca the “fire” operation goes to us_sec@mycompany.com: /security ca_sec@mycompany.com: /security the the HR departments for /us /ca both countries web@mycompany.com: /net/tcpip /server/http /blog
  13. 13. Kick ‘Em Out! instant_message(who=/sales,msg=”Goodbye.”) “who” is a resource, fire(who=/manage/sales) discovery is done on kick_out(who=/security/sales) /security/sales add_entry(site=/blog,message=”Fired Sales!”) We discover us_sec and bob@mycompany.com: /us/ny/new_york sally@mycompany.com: /ca/qc/montreal ca_sec, which provide /sales/copy_writer /sales/rep /security us_hr@mycompany.com: ca_hr@mycompany.com: /manage /manage /us /ca Security (in both countries) us_sec@mycompany.com: ca_sec@mycompany.com: gets the operation to kick out /security /us /security /ca everybody that matches / sales web@mycompany.com: /net/tcpip /server/http /blog
  14. 14. Gloat! instant_message(who=/sales,msg=”Goodbye.”) fire(who=/manage/sales) “site” is a resource, discovery is performed on kick_out(who=/security/sales) /blog add_entry(site=/blog,message=”Fired Sales!”) bob@mycompany.com: sally@mycompany.com: We discover /us/ny/new_york /sales/copy_writer /ca/qc/montreal /sales/rep web@mycompany.com, us_hr@mycompany.com: ca_hr@mycompany.com: which provides /blog /manage /manage /us /ca The operation to post the us_sec@mycompany.com: /security ca_sec@mycompany.com: /security blog entry goes to the web /us /ca server agent. web@mycompany.com: /net/tcpip /server/http /blog
  15. 15. Get Your Head In The Cloud Operations in many agents with one request Replace the “sales” entry with “us” to fire the whole US workforce Replace “sales” with “us/ny” to fire the New York office Can Subdivide the HR office any time and just offer the correct resources
  16. 16. Get Your Head In The Cloud The agents are abstracted. HR in the US might be done by human with some CRM. HR in Canada might be automated, or might be paper-only, with people doing the work. Same for security, and the web server.
  17. 17. Reduced To Set Notation In the abstract, a resource is a set that There are useful operations on sets of maps to a set of agents resources, such as “coverage” Resources may be singly inherited For any operation, the set of discovered agents is the union of the The set that maps to a resource is a sets mapped to by the intersection of subset of the set that maps to its the set of requested resources and set parent of actual resources
  18. 18. What Do We Get Out Of This? Computational Patterns Component Abstraction map-reduce Same Resources, Different Code traditional RPC Same Resources, Same message passing Code job queue Different Resources, Same Code observer
  19. 19. What I Didn’t Tell You About Responses from Operations Resource Advertising and How They Aggregate Federating Clouds The RBAC Security Implementation Fault-Tolerant, Distributed Core Implementation The Workflow Engine Monitoring Protocol The Resource-Keyed Database Job Management Protocol Intrinsic I18n Distributed Audit System Extensible Data Marshaling Distributed Log System
  20. 20. This innovation brought to you by Reliability, Ease of Use, Scalability

×