Your SlideShare is downloading. ×
0
Availability,
                            the Cloud and
                             Everything
                          ...
Me

                            • Joe Williams
                             • Infrastructure Engineer
                    ...
• Distributed database built on CouchDB
                     • Real-time Search and Analytics
                     • Sign ...
Bias


                     • Distributed Databases (CouchDB)
                     • Amazon EC2
                     • Che...
Availability




Saturday, October 2, 2010
Availability




                     • What is Availability?




Saturday, October 2, 2010
Availability




Saturday, October 2, 2010
Availability

        “System availability refers to the accessibility of
      system services to users. A system is avai...
Availability


    “System reliability refers to the property of tolerating
    constituent component failures, for the lo...
Availability



                     • Reliability * Availability = Dependability




Saturday, October 2, 2010
Availability

                     • Availability & Reliability
                            • Mean time to failures
      ...
Availability



                     • Uptime / Downtime
                            • Perceived
                         ...
Availability



                     • Probabilistic Risk Assessment
                            • Event Tree Analysis
   ...
The Cloud




Saturday, October 2, 2010
The Cloud


                      “It never gets easier, you just go faster.”
                                   - Greg Le...
The Cloud


                     • Abstraction
                     • Commoditization
                     • Homogenous
  ...
The Cloud

                     • Costs
                            • Loss of Control
                            • Single...
The Cloud


                     • Benefits
                            • API to everything
                            • F...
The Cloud



                                                                            • Bootstrapping
                 ...
The Cloud




                     • Nodes are stateless and disposable.




Saturday, October 2, 2010
The Cloud


           "Clouds are systems ... and with systems, you have to think hard and know how to deal with issues i...
What’s Next



                     • Distributed Systems
                     • Automation
                     • Data Dr...
Distributed Systems




                                Baran (http://www.rand.org/pubs/research_memoranda/RM3420/)




Sa...
Distributed Systems




                     • RAID ain’t as redundant as it used to be.




                             ...
Distributed Systems



                     • Redundancy
                            • Duplication
                       ...
Distributed Systems


                     • Alphabet Soup
                            • ACID, CAP, BASE, 2PC, MVCC
      ...
Distributed Systems




                     • CAP == Availability




Saturday, October 2, 2010
Distributed Systems


                     • Erlang
                            • Distributed
                            ...
Distributed Systems



                     • Erlang
                            • Supervision Trees




Saturday, October...
Distributed Systems



                     • Erlang
                            • Hot Code Upgrades
                     ...
Distributed Systems
                     • Future Work
                            • Erlang Supervision Trees
            ...
Automation




Saturday, October 2, 2010
Automation




                     • Optimal use of the cloud.




Saturday, October 2, 2010
Automation




                     • Frequent deployment.




Saturday, October 2, 2010
Automation

                     • Tools
                            • Chef
                            • Puppet
         ...
Automation

                    • Erlang + Chef (as of v0.8)
                            • erl_call Provider




Saturday,...
Data Driven Operations




Saturday, October 2, 2010
Data Driven Operations


                  “What gets measured, gets managed.”
                                -Peter Druc...
Data Driven Operations




                     • Instrumentation




Saturday, October 2, 2010
Data Driven Operations




                     • Logging




Saturday, October 2, 2010
Data Driven Operations




                     • Visualization




Saturday, October 2, 2010
Data Driven Operations




                     • Demo!




Saturday, October 2, 2010
Data Driven Operations


             •       Modeling

             •       Analysis

             •       Universal Law ...
Data Driven Operations




                     • Modeling isn’t just for capacity planning.




                         ...
The End




Saturday, October 2, 2010
Questions?



                            Joe Williams - @williamsjoe




Saturday, October 2, 2010
Upcoming SlideShare
Loading in...5
×

Availability, The Cloud and Everything (version 2, Surge2010)

2,291

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,291
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
53
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Availability, The Cloud and Everything (version 2, Surge2010)"

  1. 1. Availability, the Cloud and Everything Joe Williams Saturday, October 2, 2010
  2. 2. Me • Joe Williams • Infrastructure Engineer • Cloudant • @williamsjoe • joeandmotorboat.com Saturday, October 2, 2010
  3. 3. • Distributed database built on CouchDB • Real-time Search and Analytics • Sign Up! (Free to 256MB) • cloudant.com • http://github.com/cloudant/bigcouch Saturday, October 2, 2010
  4. 4. Bias • Distributed Databases (CouchDB) • Amazon EC2 • Chef • Erlang Saturday, October 2, 2010
  5. 5. Availability Saturday, October 2, 2010
  6. 6. Availability • What is Availability? Saturday, October 2, 2010
  7. 7. Availability Saturday, October 2, 2010
  8. 8. Availability “System availability refers to the accessibility of system services to users. A system is available if it is operational for an overwhelming fraction of the time. Unlike reliability, availability is instantaneous.” Saturday, October 2, 2010
  9. 9. Availability “System reliability refers to the property of tolerating constituent component failures, for the longest time. A system is perfectly reliable if it never fails.” Saturday, October 2, 2010
  10. 10. Availability • Reliability * Availability = Dependability Saturday, October 2, 2010
  11. 11. Availability • Availability & Reliability • Mean time to failures • Mean time to repair • Durability • Fault isolation • Fault tolerance Saturday, October 2, 2010
  12. 12. Availability • Uptime / Downtime • Perceived • Actual Saturday, October 2, 2010
  13. 13. Availability • Probabilistic Risk Assessment • Event Tree Analysis • Fault Tree Analysis Apthorpe (http://www.usenix.org/events/lisa01/tech/apthorpe/apthorpe.ps) Saturday, October 2, 2010
  14. 14. The Cloud Saturday, October 2, 2010
  15. 15. The Cloud “It never gets easier, you just go faster.” - Greg Lemond Saturday, October 2, 2010
  16. 16. The Cloud • Abstraction • Commoditization • Homogenous • Ephemeral Saturday, October 2, 2010
  17. 17. The Cloud • Costs • Loss of Control • Single Points of Failure • Network Partitions / Data Locality • Unreliable • Performance Saturday, October 2, 2010
  18. 18. The Cloud • Benefits • API to everything • Fast and Flexible Resource Mgmt • “Unlimited” Resources Saturday, October 2, 2010
  19. 19. The Cloud • Bootstrapping • Time and Effort Adam Jacob and Ezra Zygmuntowicz (http://blip.tv/file/2285124/) Saturday, October 2, 2010
  20. 20. The Cloud • Nodes are stateless and disposable. Saturday, October 2, 2010
  21. 21. The Cloud "Clouds are systems ... and with systems, you have to think hard and know how to deal with issues in that environment. The scale is so much bigger, and you don't have the physical control. But we think people should be optimistic about what we can do here. If we are clever about deploying cloud computing with a clear-eyed notion of what the risk models are, maybe we can actually save the economy through technology." - Security in the Ether By David Talbot - MIT Technology Review Jan/Feb 2010 Saturday, October 2, 2010
  22. 22. What’s Next • Distributed Systems • Automation • Data Driven Operations Saturday, October 2, 2010
  23. 23. Distributed Systems Baran (http://www.rand.org/pubs/research_memoranda/RM3420/) Saturday, October 2, 2010
  24. 24. Distributed Systems • RAID ain’t as redundant as it used to be. Leventhal (http://queue.acm.org/detail.cfm?id=1670144) Saturday, October 2, 2010
  25. 25. Distributed Systems • Redundancy • Duplication • Distribution Saturday, October 2, 2010
  26. 26. Distributed Systems • Alphabet Soup • ACID, CAP, BASE, 2PC, MVCC • Vector Clocks, Eventual Consistency • Dynamo, Paxos, Chandra, Byzantine Saturday, October 2, 2010
  27. 27. Distributed Systems • CAP == Availability Saturday, October 2, 2010
  28. 28. Distributed Systems • Erlang • Distributed • Concurrent • Fault Tolerant Saturday, October 2, 2010
  29. 29. Distributed Systems • Erlang • Supervision Trees Saturday, October 2, 2010
  30. 30. Distributed Systems • Erlang • Hot Code Upgrades • Distributed Upgrades are HARD Saturday, October 2, 2010
  31. 31. Distributed Systems • Future Work • Erlang Supervision Trees • PRA / FTA / ETA Apthorpe (http://www.usenix.org/events/lisa01/tech/apthorpe/apthorpe.ps) Saturday, October 2, 2010
  32. 32. Automation Saturday, October 2, 2010
  33. 33. Automation • Optimal use of the cloud. Saturday, October 2, 2010
  34. 34. Automation • Frequent deployment. Saturday, October 2, 2010
  35. 35. Automation • Tools • Chef • Puppet • Cfengine • Bcfg2 Saturday, October 2, 2010
  36. 36. Automation • Erlang + Chef (as of v0.8) • erl_call Provider Saturday, October 2, 2010
  37. 37. Data Driven Operations Saturday, October 2, 2010
  38. 38. Data Driven Operations “What gets measured, gets managed.” -Peter Drucker Saturday, October 2, 2010
  39. 39. Data Driven Operations • Instrumentation Saturday, October 2, 2010
  40. 40. Data Driven Operations • Logging Saturday, October 2, 2010
  41. 41. Data Driven Operations • Visualization Saturday, October 2, 2010
  42. 42. Data Driven Operations • Demo! Saturday, October 2, 2010
  43. 43. Data Driven Operations • Modeling • Analysis • Universal Law of Computational Scalability • Amdahl’s Law Saturday, October 2, 2010
  44. 44. Data Driven Operations • Modeling isn’t just for capacity planning. Montagne (http://queue.acm.org/detail.cfm?id=1862187) Saturday, October 2, 2010
  45. 45. The End Saturday, October 2, 2010
  46. 46. Questions? Joe Williams - @williamsjoe Saturday, October 2, 2010
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×