Your SlideShare is downloading. ×
  • Like
  • Save
Deploying distributed software services to the cloud without breaking a sweat
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Deploying distributed software services to the cloud without breaking a sweat

  • 2,161 views
Published

Deploying distributed software services to the cloud without breaking a sweat.

Deploying distributed software services to the cloud without breaking a sweat.

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,161
On SlideShare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
0
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. # finger $(whoami)Login: susan Name: Susan PotterDirectory: /home/susan Shell: /bin/zshOn since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0No mail on me@susanpotter.netPlan: twitter: @SusanPotter github: mbbx6spp
  • 2. # finger $(whoami)Login: susan Name: Susan PotterDirectory: /home/susan Shell: /bin/zshOn since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0No mail on me@susanpotter.netPlan: twitter: @SusanPotter github: mbbx6spp
  • 3. # finger $(whoami)Login: susan Name: Susan PotterDirectory: /home/susan Shell: /bin/zshOn since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0No mail on me@susanpotter.netPlan: twitter: @SusanPotter github: mbbx6spp
  • 4. # finger $(whoami)Login: susan Name: Susan PotterDirectory: /home/susan Shell: /bin/zshOn since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0No mail on me@susanpotter.netPlan: twitter: @SusanPotter github: mbbx6spp
  • 5. # finger $(whoami)Login: susan Name: Susan PotterDirectory: /home/susan Shell: /bin/zshOn since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0No mail on me@susanpotter.netPlan: twitter: @SusanPotter github: mbbx6spp
  • 6. Scope of Talk • Approaches
  • 7. Scope of Talk • Approaches • Best Practices
  • 8. Scope of Talk • Approaches • Best Practices • Pitfalls
  • 9. Scope of Talk • Approaches • Best Practices • Pitfalls • Possibilities
  • 10. Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • Pitfalls • Possibilities
  • 11. Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • (not) Why Cloud? • Pitfalls • Possibilities
  • 12. Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • (not) Why Cloud? • Pitfalls • (not) Why DevOps? • Possibilities
  • 13. Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • (not) Why Cloud? • Pitfalls • (not) Why DevOps? • Possibilities • (not) Which Delivery Model?
  • 14. Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • (not) Why Cloud? • Pitfalls • (not) Why DevOps? • Possibilities • (not) Which Delivery Model? • (not) Release Management
  • 15. Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • (not) Why Cloud? • Pitfalls • (not) Why DevOps? • Possibilities • (not) Which Delivery Model? • (not) Release Management • (not) Monitoring
  • 16. Cloud buzzzzz
  • 17. Cloud: Delivery Models [1/2] Software (as a Service) Platform (as a Service) Infrastructure (as a Service)
  • 18. Cloud: Delivery Models [1/2] Software (as a Service) Platform (as a Service) Infrastructure (as a Service)
  • 19. Cloud: Delivery Models [1/2] Software (as a Service) Platform (as a Service) Infrastructure (as a Service)
  • 20. Cloud: Delivery Models [1/2] Software (as a Service) Platform (as a Service) Infrastructure (as a Service)
  • 21. Cloud: Delivery Models [2/2]
  • 22. Cloud: Characteristics • Instant on-demand
  • 23. Cloud: Characteristics • Instant • Virtualized on-demand performance, reliability
  • 24. Cloud: Characteristics • Instant • Virtualized on-demand performance, reliability • Managed by others
  • 25. Cloud: Characteristics • Instant • Virtualized on-demand performance, reliability • Managed • Lack control by others predictability, reliability, quality
  • 26. Cloud: Characteristics • Instant • Virtualized on-demand performance, reliability • Managed • Lack control by others predictability, reliability, quality • Pay as you go
  • 27. Cloud: Characteristics • Instant • Virtualized on-demand performance, reliability • Managed • Lack control by others predictability, reliability, quality • Pay • Pay as you go as you go!
  • 28. DevOps Is it a command?
  • 29. DevOps: Definition [1/2] • Share responsibility across organizational boundaries
  • 30. DevOps: Definition [1/2] • Share responsibility across organizational boundaries • Invest in people by reducing finger pointing [togetherness] and human error [automation]
  • 31. DevOps: Definition [1/2] • Share responsibility across organizational boundaries • Invest in people by reducing finger pointing [togetherness] and human error [automation] • Manage infrastructure not priority queues of production issues
  • 32. DevOps: Definition [1/2] • Share responsibility across organizational boundaries • Invest in people by reducing finger pointing [togetherness] and human error [automation] • Manage infrastructure not priority queues of production issues • Make infrastructure predictable repeatable, testable, deterministic
  • 33. DevOps: Definition [2/2]
  • 34. Deployment Pipeline commit -> production
  • 35. Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O
  • 36. Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O • Always-ready codebase buildable, testable, deployable
  • 37. Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O • Always-ready codebase buildable, testable, deployable • Managed infrastructure read: SCM and consistent distribution to target nodes
  • 38. Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O • Always-ready codebase buildable, testable, deployable • Managed infrastructure read: SCM and consistent distribution to target nodes • Expect [system] failure handle failures sensibly, policies for timeouts, etc
  • 39. Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O • Always-ready codebase buildable, testable, deployable • Managed infrastructure read: SCM and consistent distribution to target nodes • Expect [system] failure handle failures sensibly, policies for timeouts, etc • Test early and often! outside-in development helps
  • 40. Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O • Always-ready codebase buildable, testable, deployable • Managed infrastructure read: SCM and consistent distribution to target nodes • Expect [system] failure handle failures sensibly, policies for timeouts, etc • Test early and often! outside-in development helps • Build from the ground up layer infrastructure, inject configuration at boot/load time
  • 41. Deployment: Common Bottlenecks • Automation build, provision, configure, integrate Figure: http://www.flickr.com/people/laenulfean/
  • 42. Deployment: Common Bottlenecks • Automation build, provision, configure, integrate • Distribution binaries, assets, configuration Figure: http://www.flickr.com/people/laenulfean/
  • 43. Deployment: Common Bottlenecks • Automation build, provision, configure, integrate • Distribution binaries, assets, configuration • Timeframe restricted window of time Figure: http://www.flickr.com/people/laenulfean/
  • 44. Deployment: Common Bottlenecks • Automation build, provision, configure, integrate • Distribution binaries, assets, configuration • Timeframe restricted window of time • Data schema updates, data migrations Figure: http://www.flickr.com/people/laenulfean/
  • 45. Solution Approaches Solve problems, don’t use tools!
  • 46. Automation Approaches • Full stack server-driven e.g. Chef/Knife, Puppet Master Figure: http://www.flickr.com/people/krazydad/
  • 47. Automation Approaches • Full stack server-driven e.g. Chef/Knife, Puppet Master • Full stack client e.g. Chef Solo Figure: http://www.flickr.com/people/krazydad/
  • 48. Automation Approaches • Full stack server-driven e.g. Chef/Knife, Puppet Master • Full stack client e.g. Chef Solo • Application-tier client e.g. Capistrano, Vlad the Deployer Figure: http://www.flickr.com/people/krazydad/
  • 49. Automation Approaches • Full stack server-driven e.g. Chef/Knife, Puppet Master • Full stack client e.g. Chef Solo • Application-tier client e.g. Capistrano, Vlad the Deployer • Command & control e.g. Vertibrae (inactive), Nanite Figure: http://www.flickr.com/people/krazydad/
  • 50. Distribution Approaches • Shared filesystem less security and reliability in community/public or across zones/regions Figure: http://www.flickr.com/people/nsalt
  • 51. Distribution Approaches • Shared filesystem less security and reliability in community/public or across zones/regions • Pull from source control higher time variance as target nodes increase Figure: http://www.flickr.com/people/nsalt
  • 52. Distribution Approaches • Shared filesystem less security and reliability in community/public or across zones/regions • Pull from source control higher time variance as target nodes increase • Bittorrent or similar e.g. Twitter’s Murder Figure: http://www.flickr.com/people/nsalt
  • 53. Timeframe Approaches • Hot upgrades e.g. Erlang/OTP appup/code_change/3Figure: http://www.flickr.com/people/athenicsword
  • 54. Timeframe Approaches • Hot upgrades e.g. Erlang/OTP appup/code_change/3 • Rolling upgrades Software design considerationsFigure: http://www.flickr.com/people/athenicsword
  • 55. Timeframe Approaches • Hot upgrades e.g. Erlang/OTP appup/code_change/3 • Rolling upgrades Software design considerations • Environment replacement Flip a switch, acceptance <-> productionFigure: http://www.flickr.com/people/athenicsword
  • 56. Data Approaches • Scriptable schema Alternative: NoSQL, but. . .Figure: http://www.flickr.com/people/shanghaidaddy
  • 57. Data Approaches • Scriptable schema Alternative: NoSQL, but. . . • Automated migrations e.g. Rails’ MigrationsFigure: http://www.flickr.com/people/shanghaidaddy
  • 58. Data Approaches • Scriptable schema Alternative: NoSQL, but. . . • Automated migrations e.g. Rails’ Migrations • Sanity checks e.g. cherry picked, consistency checksFigure: http://www.flickr.com/people/shanghaidaddy
  • 59. Common Pitfalls & Workarounds
  • 60. Organizational Culture Figure: http://www.flickr.com/people/lucgaloppin/
  • 61. Organizational Culture [Workaround] Figure: http://www.flickr.com/photos/42682395@N04/
  • 62. Feature Branching Figure: http://www.flickr.com/photos/33939293@N02/
  • 63. Feature Branching [Workaround]
  • 64. Feature Branching [Workaround] DON’T!
  • 65. Live by the meter.Die by the meter. Figure: WARNING
  • 66. Live by the meter.Die by the meter. [Workarounds] Figure: http://www.flickr.com/people/dirkjankraan/
  • 67. Great Expectations Figure: http://www.flickr.com/people/atoxinsocks/
  • 68. Great Expectations [Workaround] Figure: http://www.flickr.com/people/dirkjankraan/
  • 69. Note: Design & Idempotency Figure: A Twitter dialog between me and @jlouis666
  • 70. Possibilities Where.next?
  • 71. Possibilities
  • 72. Possibilities • Dynamic resource allocation allocate based on load, time of day, day of week/month
  • 73. Possibilities • Dynamic resource allocation allocate based on load, time of day, day of week/month • Canary deployments (e.g. A/B testing)
  • 74. Possibilities • Dynamic resource allocation allocate based on load, time of day, day of week/month • Canary deployments (e.g. A/B testing) • Multi-region or multi-provider relocate based on time of day, failover
  • 75. Questions? Figure: http://www.flickr.com/photos/42682395@N04/
  • 76. Questions? Figure: http://www.flickr.com/photos/42682395@N04/ @SusanPotter