Distributed software services to the cloud without breaking a sweat

# finger $(whoami)Login: susan Name: Susan PotterDirectory: /home/susan Shell: /bin/zshOn since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0No mail on me@susanpotter.netPlan: twitter: @SusanPotter github: mbbx6spp

Scope of Talk • Approaches

Scope of Talk • Approaches • Best Practices

Scope of Talk • Approaches • Best Practices • Pitfalls

Scope of Talk • Approaches • Best Practices • Pitfalls • Possibilities

Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • Pitfalls • Possibilities

Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • (not) Why Cloud? • Pitfalls • Possibilities

Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • (not) Why Cloud? • Pitfalls • (not) Why DevOps? • Possibilities

Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • (not) Why Cloud? • Pitfalls • (not) Why DevOps? • Possibilities • (not) Which Delivery Model?

Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • (not) Why Cloud? • Pitfalls • (not) Why DevOps? • Possibilities • (not) Which Delivery Model? • (not) Release Management

Scope of Talk • Approaches • (not) Chef vs Puppet • Best Practices • (not) Why Cloud? • Pitfalls • (not) Why DevOps? • Possibilities • (not) Which Delivery Model? • (not) Release Management • (not) Monitoring

Cloud buzzzzz

Cloud: Delivery Models [1/2] Software (as a Service) Platform (as a Service) Infrastructure (as a Service)

Cloud: Delivery Models [2/2]

Cloud: Characteristics • Instant on-demand

Cloud: Characteristics • Instant • Virtualized on-demand performance, reliability

Cloud: Characteristics • Instant • Virtualized on-demand performance, reliability • Managed by others

Cloud: Characteristics • Instant • Virtualized on-demand performance, reliability • Managed • Lack control by others predictability, reliability, quality

Cloud: Characteristics • Instant • Virtualized on-demand performance, reliability • Managed • Lack control by others predictability, reliability, quality • Pay as you go

Cloud: Characteristics • Instant • Virtualized on-demand performance, reliability • Managed • Lack control by others predictability, reliability, quality • Pay • Pay as you go as you go!

DevOps Is it a command?

DevOps: Deﬁnition [1/2] • Share responsibility across organizational boundaries

DevOps: Deﬁnition [1/2] • Share responsibility across organizational boundaries • Invest in people by reducing ﬁnger pointing [togetherness] and human error [automation]

DevOps: Deﬁnition [1/2] • Share responsibility across organizational boundaries • Invest in people by reducing ﬁnger pointing [togetherness] and human error [automation] • Manage infrastructure not priority queues of production issues

DevOps: Deﬁnition [1/2] • Share responsibility across organizational boundaries • Invest in people by reducing ﬁnger pointing [togetherness] and human error [automation] • Manage infrastructure not priority queues of production issues • Make infrastructure predictable repeatable, testable, deterministic

DevOps: Deﬁnition [2/2]

Deployment Pipeline commit -> production

Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O

Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O • Always-ready codebase buildable, testable, deployable

Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O • Always-ready codebase buildable, testable, deployable • Managed infrastructure read: SCM and consistent distribution to target nodes

Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O • Always-ready codebase buildable, testable, deployable • Managed infrastructure read: SCM and consistent distribution to target nodes • Expect [system] failure handle failures sensibly, policies for timeouts, etc

Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O • Always-ready codebase buildable, testable, deployable • Managed infrastructure read: SCM and consistent distribution to target nodes • Expect [system] failure handle failures sensibly, policies for timeouts, etc • Test early and often! outside-in development helps

Deployment Pipeline: Prerequisites • Design for cloud e.g. decentralized, layered, parallelized, collaborating single purpose services, async I/O • Always-ready codebase buildable, testable, deployable • Managed infrastructure read: SCM and consistent distribution to target nodes • Expect [system] failure handle failures sensibly, policies for timeouts, etc • Test early and often! outside-in development helps • Build from the ground up layer infrastructure, inject conﬁguration at boot/load time

Deployment: Common Bottlenecks • Automation build, provision, conﬁgure, integrate Figure: http://www.ﬂickr.com/people/laenulfean/

Deployment: Common Bottlenecks • Automation build, provision, configure, integrate • Distribution binaries, assets, configuration Figure: http://www.flickr.com/people/laenulfean/

Deployment: Common Bottlenecks • Automation build, provision, configure, integrate • Distribution binaries, assets, configuration • Timeframe restricted window of time Figure: http://www.flickr.com/people/laenulfean/

Deployment: Common Bottlenecks • Automation build, provision, configure, integrate • Distribution binaries, assets, configuration • Timeframe restricted window of time • Data schema updates, data migrations Figure: http://www.flickr.com/people/laenulfean/

Solution Approaches Solve problems, don’t use tools!

Automation Approaches • Full stack server-driven e.g. Chef/Knife, Puppet Master Figure: http://www.ﬂickr.com/people/krazydad/

Automation Approaches • Full stack server-driven e.g. Chef/Knife, Puppet Master • Full stack client e.g. Chef Solo Figure: http://www.ﬂickr.com/people/krazydad/

Automation Approaches • Full stack server-driven e.g. Chef/Knife, Puppet Master • Full stack client e.g. Chef Solo • Application-tier client e.g. Capistrano, Vlad the Deployer Figure: http://www.ﬂickr.com/people/krazydad/

Automation Approaches • Full stack server-driven e.g. Chef/Knife, Puppet Master • Full stack client e.g. Chef Solo • Application-tier client e.g. Capistrano, Vlad the Deployer • Command & control e.g. Vertibrae (inactive), Nanite Figure: http://www.ﬂickr.com/people/krazydad/

Distribution Approaches • Shared ﬁlesystem less security and reliability in community/public or across zones/regions Figure: http://www.ﬂickr.com/people/nsalt

Distribution Approaches • Shared ﬁlesystem less security and reliability in community/public or across zones/regions • Pull from source control higher time variance as target nodes increase Figure: http://www.ﬂickr.com/people/nsalt

Distribution Approaches • Shared ﬁlesystem less security and reliability in community/public or across zones/regions • Pull from source control higher time variance as target nodes increase • Bittorrent or similar e.g. Twitter’s Murder Figure: http://www.ﬂickr.com/people/nsalt

Timeframe Approaches • Hot upgrades e.g. Erlang/OTP appup/code_change/3Figure: http://www.ﬂickr.com/people/athenicsword

Timeframe Approaches • Hot upgrades e.g. Erlang/OTP appup/code_change/3 • Rolling upgrades Software design considerationsFigure: http://www.ﬂickr.com/people/athenicsword

Timeframe Approaches • Hot upgrades e.g. Erlang/OTP appup/code_change/3 • Rolling upgrades Software design considerations • Environment replacement Flip a switch, acceptance <-> productionFigure: http://www.ﬂickr.com/people/athenicsword

Data Approaches • Scriptable schema Alternative: NoSQL, but. . .Figure: http://www.ﬂickr.com/people/shanghaidaddy

Data Approaches • Scriptable schema Alternative: NoSQL, but. . . • Automated migrations e.g. Rails’ MigrationsFigure: http://www.ﬂickr.com/people/shanghaidaddy

Data Approaches • Scriptable schema Alternative: NoSQL, but. . . • Automated migrations e.g. Rails’ Migrations • Sanity checks e.g. cherry picked, consistency checksFigure: http://www.ﬂickr.com/people/shanghaidaddy

Common Pitfalls & Workarounds

Organizational Culture Figure: http://www.ﬂickr.com/people/lucgaloppin/

Organizational Culture [Workaround] Figure: http://www.ﬂickr.com/photos/42682395@N04/

Feature Branching Figure: http://www.ﬂickr.com/photos/33939293@N02/

Feature Branching [Workaround]

Feature Branching [Workaround] DON’T!

Live by the meter.Die by the meter. Figure: WARNING

Live by the meter.Die by the meter. [Workarounds] Figure: http://www.ﬂickr.com/people/dirkjankraan/

Great Expectations Figure: http://www.ﬂickr.com/people/atoxinsocks/

Great Expectations [Workaround] Figure: http://www.ﬂickr.com/people/dirkjankraan/

Note: Design & Idempotency Figure: A Twitter dialog between me and @jlouis666

Possibilities Where.next?

Possibilities

Possibilities • Dynamic resource allocation allocate based on load, time of day, day of week/month

Possibilities • Dynamic resource allocation allocate based on load, time of day, day of week/month • Canary deployments (e.g. A/B testing)

Possibilities • Dynamic resource allocation allocate based on load, time of day, day of week/month • Canary deployments (e.g. A/B testing) • Multi-region or multi-provider relocate based on time of day, failover

Questions? Figure: http://www.ﬂickr.com/photos/42682395@N04/

Questions? Figure: http://www.ﬂickr.com/photos/42682395@N04/ @SusanPotter

Distributed software services to the cloud without breaking a sweat

by José Ferreiro on Jun 23, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 386

Statistics

Distributed software services to the cloud without breaking a sweat — Presentation Transcript

http://magnot.blogspot.com	385
http://magnot.blogspot.in	1