Computational Patterns of the Cloud - QCon NYC 2014
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Computational Patterns of the Cloud - QCon NYC 2014

on

  • 149 views

The Cloud has undoubtedly changed the way we think about computing, IT operations, innovation, and entrepreneurship. But what are the computational patterns that have emerged from the pervasiveness of ...

The Cloud has undoubtedly changed the way we think about computing, IT operations, innovation, and entrepreneurship. But what are the computational patterns that have emerged from the pervasiveness of public clouds? What can we leverage to improve our organizations? And what are the challenges that we face going forward?

In this talk, I will introduce you to cloud computing’s paradigms and discuss their applications with practical examples from Engine Yard’s customers, peers, and partners. We will also cover antipatterns and myths. If you are curious about Cloud computing or want to improve your cloud strategy this talk is for you.

NOTE: Open an issue if you want me to explain something in more detail at the accompanying github repo: https://github.com/Randommood/QConNYC2014/

Statistics

Views

Total Views
149
Views on SlideShare
106
Embed Views
43

Actions

Likes
0
Downloads
2
Comments
0

1 Embed 43

http://inessombra.tumblr.com 43

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Computational Patterns of the Cloud - QCon NYC 2014 Presentation Transcript

  • 1. Computational Patterns of the Cloud QCon NYC 2014
  • 2. Inés SombraData Engineer @Randommood ines@engineyard.com
  • 3. Continuous Everything Today’s Agenda Distributed Systems Harvesting Services Reclaimable Resources
  • 4. Reclaimable Resources Pattern 1: Where things start
  • 5. Pre-Cloud Applications We have an application ready for deployment Ticket to request resources ( budgets ) Procurement Roll to production 
 on a hand-crafted server Maintenance Uptime Burden
  • 6. Cloud Applications We have an application ready for deployment Provision servers on demand (via APIs) & 
 deploy to production Maintenance ( for used resources only) Pay Uptime Burden
  • 7. gordolo.engineyard.com Pets vs Cattle vm002.engineyard.com Both are adorbs!
  • 8. Maintain Cloud resources are Reused ConsumeProvision Release * And things are ”elastic”
  • 9. Our experience + a few stories
  • 10. Everything fails. * At the worst time, for realsies
  • 11. Excel at Process Anticipate failure and plan for it Use postmortems, checklists, retrospectives, and play-books Take them seriously Also know that state can bite
  • 12. Everything is a recipe Infrastructure is maintained as code Resources are used to increase the availability & redundancy of applications “A self service cloud makes impossible things instant” @adrianco
  • 13. Challenges
  • 14. Importance of 
 Monitoring & Benchmarking Know your baseline * Alerting & monitoring are critical Benchmarking is still misunderstood
  • 15. What does 
 healthy mean? Many elements in place to determine health These visualizations fail us. We need better ones Complexity is complex
  • 16. What does 
 healthy mean? Many elements in place to determine health These visualizations fail us. We need better ones Complexity is complex Instance provisioned? IaaS Firewalls set up? IaaS Chef succeeded? PaaSVolumes provisioned? IaaS Process running? Role Replica up? PaaS
  • 17. Resource families match use cases Some awareness is needed Cloud resources are different than hardware- based Capacity planning is tricky
  • 18. Think about cloud resources in fluid terms: compute & release compute with: the cloud as A collection of disposable resources * Awareness is not optional
  • 19. Harvesting Services Pattern 2: We have resources, now let’s leverage services
  • 20. App Design for the cloud Surrender the filesystem1 Our app becomes an aggregate of services 2 Consume services via APIs3
  • 21. Service oriented architecture NoSQL Distilled: Fowler & Sadalage My awesome e-commerce site Shopping cart & session data Completed orders Inventory and item pricing Session Storage service K/V Store Order Persistence service Document Store Inventory & Price service RDBMS Nodes and relations service Graph Store Recommendations engine
  • 22. Our experience + a few stories
  • 23. Cloud The monorail that grew UI Provisioning API Billing Partners
  • 24. Cloud The monorail that grew UI Provisioning API Billing Partners ui.engineyard.com Provisioner (Smithy) API (Core) Billing (Johnny Cash) Partners (Tres Fiestas)
  • 25. The coupling that shouldn’t be Cloud UI Provisioning API Billing Partners
  • 26. The coupling that shouldn’t be Provisioner (Smithy) Cloud UI Provisioning API Billing Partners
  • 27. Challenges
  • 28. Operational experience can become 
 siloed Teams built around services Knowledge boundaries Geographical distribution
  • 29. Service 
 dependency and failure planning Difficult to assert health in apps that consume services Each service it’s its own (smaller) failure domain
  • 30. Importance of API design,
 maintenance, & deprecation API design is a core business competency Prioritize maintenance Retirement will happen
  • 31. Think about your app as a: collection of services connected via APIs compute with: Cloud applications leveraging services
  • 32. Distributed Systems Pattern 3: Resources + services make distributed systems as the new norm
  • 33. It’s a clustered world
  • 34. with properties & constraints * Tip: Read `Distributed systems and the end of the API in references Distributed systems exhibit a set of uniformly unintuitive behaviors related to causality, consistency, and availability *
  • 35. Our experience + a few stories
  • 36. Availability & 
 Coordination Failures & latency Degradation Dependencies
  • 37. Find the failing node Stressing Failures and adding automation Serf cluster membershipHomegrown stonith
  • 38. Challenges
  • 39. What does it 
 mean to be up? Distributed systems will break in interesting and painful ways Know your dependencies
  • 40. Distributed tracing is necessary Debugging the “it’s slow” problem sucks Inherently complex to simulate & test
  • 41. Awareness of theory Matters @seancribbs
  • 42. Distributed systems are the foundation of our infrastructure & services compute with: Cloud apps forming distributed systems
  • 43. Continuous Everything Pattern 4: cloud systems enable continuity
  • 44. CD and the importance of tests Are they fast? Can we trust them? What coverage do we have? What type of tests? * Don’t ignore their maintenance Tests 
 are critical
  • 45. Our experience + a few stories
  • 46. Our continuous Integration Branches Pull Requests Test suites Merge & deploy
  • 47. Any prod deploy Kicks off a suite Our testing evolution Master PRs Kick off a suite Dredd Tests Systems, boundaries, & integration Stacks OS + Our Image Scenario Live setup + assertions Suite Collection of scenarios
  • 48. Any prod deploy Kicks off a suite Our testing evolution Master PRs Kick off a suite Dredd Tests Systems, boundaries, & integration Stacks OS + Our Image Scenario Live setup + assertions Suite Collection of scenarios Are Databas es? Inst al l ing Ba cki ng Up Re stor ing Setti ng Up Re p li c ation Pro m ot ing Re p li c as Wr ite a b le
  • 49. Challenges
  • 50. Many testing choices. And a random note. Testing is critical and frameworks should help by streamlining choices Automation helps with process & certifications (SOCS2)
  • 51. All planning tools suck Stop thinking that a tool will fix your agile problems Continuous delivery can make flaws in goals & direction obvious. Is your vision clear?
  • 52. Cloud-based apps, resources, and services enable agility compute with: Cloud (resources, apps, services) as continuous * “The bottleneck is never code or creativity; it's lack of clarity.”Scott Berkun
  • 53. Quickly provision resources & release them when they are not needed Reclaimable Resources Leverage services to simplify areas of responsibility in our app Harvesting services Theoretical foundation as a core competency for correctness Distributed Systems The holy grail. Will reshape your company & make everything awesome Continuous Delivery tl;dr this will save you 40 minutes
  • 54. Cloud computing next “…This is how cloud computing will continue to evolve. Developers will worry less about the thousands of machines needed to run their application and more about the design of the application itself.” Eric Brewer
  • 55. Thank you @randommood 
 ines@engineyard.com github.com/Randommood/Qconnyc2014