SlideShare a Scribd company logo
1 of 46
Download to read offline
Availability,
                            the Cloud and
                             Everything
                                Joe Williams




Saturday, October 2, 2010
Me

                            • Joe Williams
                             • Infrastructure Engineer
                             • Cloudant
                             • @williamsjoe
                             • joeandmotorboat.com



Saturday, October 2, 2010
• Distributed database built on CouchDB
                     • Real-time Search and Analytics
                     • Sign Up! (Free to 256MB)
                     • cloudant.com
                     • http://github.com/cloudant/bigcouch


Saturday, October 2, 2010
Bias


                     • Distributed Databases (CouchDB)
                     • Amazon EC2
                     • Chef
                     • Erlang



Saturday, October 2, 2010
Availability




Saturday, October 2, 2010
Availability




                     • What is Availability?




Saturday, October 2, 2010
Availability




Saturday, October 2, 2010
Availability

        “System availability refers to the accessibility of
      system services to users. A system is available if it is
     operational for an overwhelming fraction of the time.
        Unlike reliability, availability is instantaneous.”




Saturday, October 2, 2010
Availability


    “System reliability refers to the property of tolerating
    constituent component failures, for the longest time. A
          system is perfectly reliable if it never fails.”




Saturday, October 2, 2010
Availability



                     • Reliability * Availability = Dependability




Saturday, October 2, 2010
Availability

                     • Availability & Reliability
                            • Mean time to failures
                            • Mean time to repair
                            • Durability
                            • Fault isolation
                            • Fault tolerance


Saturday, October 2, 2010
Availability



                     • Uptime / Downtime
                            • Perceived
                            • Actual




Saturday, October 2, 2010
Availability



                     • Probabilistic Risk Assessment
                            • Event Tree Analysis
                            • Fault Tree Analysis



                                   Apthorpe (http://www.usenix.org/events/lisa01/tech/apthorpe/apthorpe.ps)



Saturday, October 2, 2010
The Cloud




Saturday, October 2, 2010
The Cloud


                      “It never gets easier, you just go faster.”
                                   - Greg Lemond




Saturday, October 2, 2010
The Cloud


                     • Abstraction
                     • Commoditization
                     • Homogenous
                     • Ephemeral



Saturday, October 2, 2010
The Cloud

                     • Costs
                            • Loss of Control
                            • Single Points of Failure
                            • Network Partitions / Data Locality
                            • Unreliable
                            • Performance

Saturday, October 2, 2010
The Cloud


                     • Benefits
                            • API to everything
                            • Fast and Flexible Resource Mgmt
                            • “Unlimited” Resources



Saturday, October 2, 2010
The Cloud



                                                                            • Bootstrapping
                                                                             • Time and Effort


           Adam Jacob and Ezra Zygmuntowicz (http://blip.tv/file/2285124/)




Saturday, October 2, 2010
The Cloud




                     • Nodes are stateless and disposable.




Saturday, October 2, 2010
The Cloud


           "Clouds are systems ... and with systems, you have to think hard and know how to deal with issues in that
         environment. The scale is so much bigger, and you don't have the physical control. But we think people should
           be optimistic about what we can do here. If we are clever about deploying cloud computing with a clear-eyed
                notion of what the risk models are, maybe we can actually save the economy through technology."

                            - Security in the Ether By David Talbot - MIT Technology Review Jan/Feb 2010




Saturday, October 2, 2010
What’s Next



                     • Distributed Systems
                     • Automation
                     • Data Driven Operations




Saturday, October 2, 2010
Distributed Systems




                                Baran (http://www.rand.org/pubs/research_memoranda/RM3420/)




Saturday, October 2, 2010
Distributed Systems




                     • RAID ain’t as redundant as it used to be.




                                  Leventhal (http://queue.acm.org/detail.cfm?id=1670144)




Saturday, October 2, 2010
Distributed Systems



                     • Redundancy
                            • Duplication
                            • Distribution




Saturday, October 2, 2010
Distributed Systems


                     • Alphabet Soup
                            • ACID, CAP, BASE, 2PC, MVCC
                            • Vector Clocks, Eventual Consistency
                            • Dynamo, Paxos, Chandra, Byzantine



Saturday, October 2, 2010
Distributed Systems




                     • CAP == Availability




Saturday, October 2, 2010
Distributed Systems


                     • Erlang
                            • Distributed
                            • Concurrent
                            • Fault Tolerant



Saturday, October 2, 2010
Distributed Systems



                     • Erlang
                            • Supervision Trees




Saturday, October 2, 2010
Distributed Systems



                     • Erlang
                            • Hot Code Upgrades
                            • Distributed Upgrades are HARD




Saturday, October 2, 2010
Distributed Systems
                     • Future Work
                            • Erlang Supervision Trees
                            • PRA / FTA / ETA




                                    Apthorpe (http://www.usenix.org/events/lisa01/tech/apthorpe/apthorpe.ps)

Saturday, October 2, 2010
Automation




Saturday, October 2, 2010
Automation




                     • Optimal use of the cloud.




Saturday, October 2, 2010
Automation




                     • Frequent deployment.




Saturday, October 2, 2010
Automation

                     • Tools
                            • Chef
                            • Puppet
                            • Cfengine
                            • Bcfg2


Saturday, October 2, 2010
Automation

                    • Erlang + Chef (as of v0.8)
                            • erl_call Provider




Saturday, October 2, 2010
Data Driven Operations




Saturday, October 2, 2010
Data Driven Operations


                  “What gets measured, gets managed.”
                                -Peter Drucker




Saturday, October 2, 2010
Data Driven Operations




                     • Instrumentation




Saturday, October 2, 2010
Data Driven Operations




                     • Logging




Saturday, October 2, 2010
Data Driven Operations




                     • Visualization




Saturday, October 2, 2010
Data Driven Operations




                     • Demo!




Saturday, October 2, 2010
Data Driven Operations


             •       Modeling

             •       Analysis

             •       Universal Law of Computational Scalability

             •       Amdahl’s Law




Saturday, October 2, 2010
Data Driven Operations




                     • Modeling isn’t just for capacity planning.




                                   Montagne (http://queue.acm.org/detail.cfm?id=1862187)


Saturday, October 2, 2010
The End




Saturday, October 2, 2010
Questions?



                            Joe Williams - @williamsjoe




Saturday, October 2, 2010

More Related Content

Similar to Availability, The Cloud and Everything (version 2, Surge2010)

Calculating the ROI for XML and DITA topic-based authoring
Calculating the ROI for XML and DITA topic-based authoringCalculating the ROI for XML and DITA topic-based authoring
Calculating the ROI for XML and DITA topic-based authoringScriptorium Publishing
 
A Practical Introduction to Symfony2
A Practical Introduction to Symfony2A Practical Introduction to Symfony2
A Practical Introduction to Symfony2Kris Wallsmith
 
OpenStreetMap dongpo deng
OpenStreetMap   dongpo dengOpenStreetMap   dongpo deng
OpenStreetMap dongpo dengAidIQ
 
M.Malone Simple Geo @ Social Developers Summit
M.Malone Simple Geo @ Social Developers SummitM.Malone Simple Geo @ Social Developers Summit
M.Malone Simple Geo @ Social Developers SummitMediabistro
 
Batch Indexing & Near Real Time, keeping things fast
Batch Indexing & Near Real Time, keeping things fastBatch Indexing & Near Real Time, keeping things fast
Batch Indexing & Near Real Time, keeping things fastMarc Sturlese
 
Aegir one drupal to rule them all
Aegir one drupal to rule them allAegir one drupal to rule them all
Aegir one drupal to rule them allDevelopment Seed
 
Magic broker 2 #iot2010 presentation
Magic broker 2 #iot2010 presentationMagic broker 2 #iot2010 presentation
Magic broker 2 #iot2010 presentationAdrian Friday
 
Geoloqi - Non-visual location based augmented reality with SMS and GPS - Ope...
Geoloqi -  Non-visual location based augmented reality with SMS and GPS - Ope...Geoloqi -  Non-visual location based augmented reality with SMS and GPS - Ope...
Geoloqi - Non-visual location based augmented reality with SMS and GPS - Ope...Amber Case
 
Geoloqi: Non-visual augmented reality Open Source Bridge
Geoloqi: Non-visual augmented reality Open Source BridgeGeoloqi: Non-visual augmented reality Open Source Bridge
Geoloqi: Non-visual augmented reality Open Source BridgeAaron Parecki
 
Drupal security - Configuration and process
Drupal security - Configuration and processDrupal security - Configuration and process
Drupal security - Configuration and processGábor Hojtsy
 
DNSSEC Deployment at ROOT Zone
DNSSEC Deployment at ROOT ZoneDNSSEC Deployment at ROOT Zone
DNSSEC Deployment at ROOT ZoneMehmet Akcin
 
Gaelyk - SpringOne2GX - 2010 - Guillaume Laforge
Gaelyk - SpringOne2GX - 2010 - Guillaume LaforgeGaelyk - SpringOne2GX - 2010 - Guillaume Laforge
Gaelyk - SpringOne2GX - 2010 - Guillaume LaforgeGuillaume Laforge
 
Scaling webappswithrabbitmq
Scaling webappswithrabbitmqScaling webappswithrabbitmq
Scaling webappswithrabbitmqAlvaro Videla
 
Integrating php withrabbitmq_zendcon
Integrating php withrabbitmq_zendconIntegrating php withrabbitmq_zendcon
Integrating php withrabbitmq_zendconAlvaro Videla
 

Similar to Availability, The Cloud and Everything (version 2, Surge2010) (15)

Calculating the ROI for XML and DITA topic-based authoring
Calculating the ROI for XML and DITA topic-based authoringCalculating the ROI for XML and DITA topic-based authoring
Calculating the ROI for XML and DITA topic-based authoring
 
A Practical Introduction to Symfony2
A Practical Introduction to Symfony2A Practical Introduction to Symfony2
A Practical Introduction to Symfony2
 
OpenStreetMap dongpo deng
OpenStreetMap   dongpo dengOpenStreetMap   dongpo deng
OpenStreetMap dongpo deng
 
M.Malone Simple Geo @ Social Developers Summit
M.Malone Simple Geo @ Social Developers SummitM.Malone Simple Geo @ Social Developers Summit
M.Malone Simple Geo @ Social Developers Summit
 
Batch Indexing & Near Real Time, keeping things fast
Batch Indexing & Near Real Time, keeping things fastBatch Indexing & Near Real Time, keeping things fast
Batch Indexing & Near Real Time, keeping things fast
 
Aegir one drupal to rule them all
Aegir one drupal to rule them allAegir one drupal to rule them all
Aegir one drupal to rule them all
 
Magic broker 2 #iot2010 presentation
Magic broker 2 #iot2010 presentationMagic broker 2 #iot2010 presentation
Magic broker 2 #iot2010 presentation
 
iOS & Arduino
iOS & ArduinoiOS & Arduino
iOS & Arduino
 
Geoloqi - Non-visual location based augmented reality with SMS and GPS - Ope...
Geoloqi -  Non-visual location based augmented reality with SMS and GPS - Ope...Geoloqi -  Non-visual location based augmented reality with SMS and GPS - Ope...
Geoloqi - Non-visual location based augmented reality with SMS and GPS - Ope...
 
Geoloqi: Non-visual augmented reality Open Source Bridge
Geoloqi: Non-visual augmented reality Open Source BridgeGeoloqi: Non-visual augmented reality Open Source Bridge
Geoloqi: Non-visual augmented reality Open Source Bridge
 
Drupal security - Configuration and process
Drupal security - Configuration and processDrupal security - Configuration and process
Drupal security - Configuration and process
 
DNSSEC Deployment at ROOT Zone
DNSSEC Deployment at ROOT ZoneDNSSEC Deployment at ROOT Zone
DNSSEC Deployment at ROOT Zone
 
Gaelyk - SpringOne2GX - 2010 - Guillaume Laforge
Gaelyk - SpringOne2GX - 2010 - Guillaume LaforgeGaelyk - SpringOne2GX - 2010 - Guillaume Laforge
Gaelyk - SpringOne2GX - 2010 - Guillaume Laforge
 
Scaling webappswithrabbitmq
Scaling webappswithrabbitmqScaling webappswithrabbitmq
Scaling webappswithrabbitmq
 
Integrating php withrabbitmq_zendcon
Integrating php withrabbitmq_zendconIntegrating php withrabbitmq_zendcon
Integrating php withrabbitmq_zendcon
 

Recently uploaded

Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfPrecisely
 

Recently uploaded (20)

Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
 

Availability, The Cloud and Everything (version 2, Surge2010)

  • 1. Availability, the Cloud and Everything Joe Williams Saturday, October 2, 2010
  • 2. Me • Joe Williams • Infrastructure Engineer • Cloudant • @williamsjoe • joeandmotorboat.com Saturday, October 2, 2010
  • 3. • Distributed database built on CouchDB • Real-time Search and Analytics • Sign Up! (Free to 256MB) • cloudant.com • http://github.com/cloudant/bigcouch Saturday, October 2, 2010
  • 4. Bias • Distributed Databases (CouchDB) • Amazon EC2 • Chef • Erlang Saturday, October 2, 2010
  • 6. Availability • What is Availability? Saturday, October 2, 2010
  • 8. Availability “System availability refers to the accessibility of system services to users. A system is available if it is operational for an overwhelming fraction of the time. Unlike reliability, availability is instantaneous.” Saturday, October 2, 2010
  • 9. Availability “System reliability refers to the property of tolerating constituent component failures, for the longest time. A system is perfectly reliable if it never fails.” Saturday, October 2, 2010
  • 10. Availability • Reliability * Availability = Dependability Saturday, October 2, 2010
  • 11. Availability • Availability & Reliability • Mean time to failures • Mean time to repair • Durability • Fault isolation • Fault tolerance Saturday, October 2, 2010
  • 12. Availability • Uptime / Downtime • Perceived • Actual Saturday, October 2, 2010
  • 13. Availability • Probabilistic Risk Assessment • Event Tree Analysis • Fault Tree Analysis Apthorpe (http://www.usenix.org/events/lisa01/tech/apthorpe/apthorpe.ps) Saturday, October 2, 2010
  • 15. The Cloud “It never gets easier, you just go faster.” - Greg Lemond Saturday, October 2, 2010
  • 16. The Cloud • Abstraction • Commoditization • Homogenous • Ephemeral Saturday, October 2, 2010
  • 17. The Cloud • Costs • Loss of Control • Single Points of Failure • Network Partitions / Data Locality • Unreliable • Performance Saturday, October 2, 2010
  • 18. The Cloud • Benefits • API to everything • Fast and Flexible Resource Mgmt • “Unlimited” Resources Saturday, October 2, 2010
  • 19. The Cloud • Bootstrapping • Time and Effort Adam Jacob and Ezra Zygmuntowicz (http://blip.tv/file/2285124/) Saturday, October 2, 2010
  • 20. The Cloud • Nodes are stateless and disposable. Saturday, October 2, 2010
  • 21. The Cloud "Clouds are systems ... and with systems, you have to think hard and know how to deal with issues in that environment. The scale is so much bigger, and you don't have the physical control. But we think people should be optimistic about what we can do here. If we are clever about deploying cloud computing with a clear-eyed notion of what the risk models are, maybe we can actually save the economy through technology." - Security in the Ether By David Talbot - MIT Technology Review Jan/Feb 2010 Saturday, October 2, 2010
  • 22. What’s Next • Distributed Systems • Automation • Data Driven Operations Saturday, October 2, 2010
  • 23. Distributed Systems Baran (http://www.rand.org/pubs/research_memoranda/RM3420/) Saturday, October 2, 2010
  • 24. Distributed Systems • RAID ain’t as redundant as it used to be. Leventhal (http://queue.acm.org/detail.cfm?id=1670144) Saturday, October 2, 2010
  • 25. Distributed Systems • Redundancy • Duplication • Distribution Saturday, October 2, 2010
  • 26. Distributed Systems • Alphabet Soup • ACID, CAP, BASE, 2PC, MVCC • Vector Clocks, Eventual Consistency • Dynamo, Paxos, Chandra, Byzantine Saturday, October 2, 2010
  • 27. Distributed Systems • CAP == Availability Saturday, October 2, 2010
  • 28. Distributed Systems • Erlang • Distributed • Concurrent • Fault Tolerant Saturday, October 2, 2010
  • 29. Distributed Systems • Erlang • Supervision Trees Saturday, October 2, 2010
  • 30. Distributed Systems • Erlang • Hot Code Upgrades • Distributed Upgrades are HARD Saturday, October 2, 2010
  • 31. Distributed Systems • Future Work • Erlang Supervision Trees • PRA / FTA / ETA Apthorpe (http://www.usenix.org/events/lisa01/tech/apthorpe/apthorpe.ps) Saturday, October 2, 2010
  • 33. Automation • Optimal use of the cloud. Saturday, October 2, 2010
  • 34. Automation • Frequent deployment. Saturday, October 2, 2010
  • 35. Automation • Tools • Chef • Puppet • Cfengine • Bcfg2 Saturday, October 2, 2010
  • 36. Automation • Erlang + Chef (as of v0.8) • erl_call Provider Saturday, October 2, 2010
  • 38. Data Driven Operations “What gets measured, gets managed.” -Peter Drucker Saturday, October 2, 2010
  • 39. Data Driven Operations • Instrumentation Saturday, October 2, 2010
  • 40. Data Driven Operations • Logging Saturday, October 2, 2010
  • 41. Data Driven Operations • Visualization Saturday, October 2, 2010
  • 42. Data Driven Operations • Demo! Saturday, October 2, 2010
  • 43. Data Driven Operations • Modeling • Analysis • Universal Law of Computational Scalability • Amdahl’s Law Saturday, October 2, 2010
  • 44. Data Driven Operations • Modeling isn’t just for capacity planning. Montagne (http://queue.acm.org/detail.cfm?id=1862187) Saturday, October 2, 2010
  • 46. Questions? Joe Williams - @williamsjoe Saturday, October 2, 2010