0
Zero to Hero

Pragmatic steps on the
path to continuous delivery

@geoffnettaglich
Continuous Delivery : Theory
ed
at

build / deploy / feedback
10
20
30
40
50
60

a

om
ut

have idea
implement idea
measur...
Continuous Delivery : Reality
● Everyone is doing it (apparently)
○ Amazon, Etsy, Facebook, Netflix
■ N deploys per day / ...
Continuous Delivery : Practice

Suck less tomorrow!
● Focus on the journey
○ it will be ongoing anyway
○ simplify / automa...
Build : Theory
● Dependencies : what we need
lib of Jars / Maven / Ivy

● Artifacts : what we produce
tgz / jar / war

● R...
Build : Reality
● GOOD
○ Using ant + ivy + sh + junit

● NOT SO GOOD
○
○
○
○
○

Issues with build dependencies
No output a...
Build : Practice

checkout / build / run
● Tidy up existing ant build (based on intent)
○ clean / compile / test / package...
Deploy : Theory
● Push button / Zero downtime
○ Not always easy: Java != rails != php/apache

● Deploy from developers mac...
Deploy : Reality
● GOOD
○ Scripted (somewhat)

● NOT SO GOOD
○
○
○
○
○

Only from developers machine [FAIL!]
Ignored error...
Deploy : Practice
● bash script (Capistrano or similar)
○ stop / upload / unpack / restart
○ similar structure to capistra...
Run : Theory
● Elastic scaling
○ Dynamic provisioning
○ Automated service management

● Automated monitoring and alerting
...
Run : Reality
● ssh to servers
○ to start stop services
○ monitor perf via top and ps
○ grep and tail -f of log files
● On...
Run : Practice
● New Relic
○ servers for free
○ app with paid
○ great insight

● Centralized logging
○ rsyslog:unification...
Run : Practice
● Health Metrics (in app)
○ home rolled exposed via web
○ Metrics by CodaHale / Pingdom

● Biz Metrics
○ Du...
Environments : Theory

‘Works on my machine’
● Scale – vertically or horizontally
● automatable, reproducible
○ Green / Bl...
Environments : Reality
You won’t always know what you inherit!
●
●
●
●
●

Private cloud hosting
Limited upgrade ability
Ma...
Environments : Practice
● Picked Chef

(Vi vs Emacs …)

○ Roles for baseline / Web / App / DB nodes
○ Environments for DEV...
Environments : Practice
● DB backup / restore
○ do it and test it regularly
○ scripted
○ hire an expert

Delegate to other...
Environment : Current State
Rackspace Cloud
Hosted chef
Managed by Jenkins
○ Re-Create dev / staging / production
○ deploy...
OH: I deployed to production,
nobody noticed and nothing
went wrong
FTW!
Upcoming SlideShare
Loading in...5
×

Zero to hero - Geoff Webb

444

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
444
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Zero to hero - Geoff Webb"

  1. 1. Zero to Hero Pragmatic steps on the path to continuous delivery @geoffnettaglich
  2. 2. Continuous Delivery : Theory ed at build / deploy / feedback 10 20 30 40 50 60 a om ut have idea implement idea measure effectiveness refine idea GOTO 10 PROFIT
  3. 3. Continuous Delivery : Reality ● Everyone is doing it (apparently) ○ Amazon, Etsy, Facebook, Netflix ■ N deploys per day / min (where N > 0) ○ Great idea (in theory) ● What about me? ○ Existing codebase (smells and all) ○ Existing systems & environments (no access) ○ Existing team and politics (no idea) e in ia rt
  4. 4. Continuous Delivery : Practice Suck less tomorrow! ● Focus on the journey ○ it will be ongoing anyway ○ simplify / automate ● Crawl / Walk / Run ○ Find a place to start ■ most painful ■ simplest entry point
  5. 5. Build : Theory ● Dependencies : what we need lib of Jars / Maven / Ivy ● Artifacts : what we produce tgz / jar / war ● Reproducible : scripted Ant / Maven / Gradle ● Automatable Jenkins
  6. 6. Build : Reality ● GOOD ○ Using ant + ivy + sh + junit ● NOT SO GOOD ○ ○ ○ ○ ○ Issues with build dependencies No output artifact No easy way to create DB Only worked on developers machine Took a day or two to get new team members working
  7. 7. Build : Practice checkout / build / run ● Tidy up existing ant build (based on intent) ○ clean / compile / test / package ○ create / bootstrap DB locally ● Bundle build properties ● Run app locally ● Generate reports ○ Unit tests (JUnit) ○ Static analysis (FindBugs et al)
  8. 8. Deploy : Theory ● Push button / Zero downtime ○ Not always easy: Java != rails != php/apache ● Deploy from developers machine is FAIL ○ Shared env or jump box ○ Then automate (via Jenkins) ● What version is running where ○ Build page ○ All servers up to date with latest version
  9. 9. Deploy : Reality ● GOOD ○ Scripted (somewhat) ● NOT SO GOOD ○ ○ ○ ○ ○ Only from developers machine [FAIL!] Ignored errors (FULL STEAM AHEAD) Rsync of build outcome Manual stop / start Customers may see errors during rollout
  10. 10. Deploy : Practice ● bash script (Capistrano or similar) ○ stop / upload / unpack / restart ○ similar structure to capistrano on filesystem ● Verify ○ Build page with stats (bundled in artifact) ● DB changes ○ Backwards compatible changes (process) ○ Liquibase is great ● Automated (via Jenkins of course)
  11. 11. Run : Theory ● Elastic scaling ○ Dynamic provisioning ○ Automated service management ● Automated monitoring and alerting ○ Sensu / Nagios / ganglia ○ LogStash / GrayLog2 ● Dashboards for EVERYTHING!
  12. 12. Run : Reality ● ssh to servers ○ to start stop services ○ monitor perf via top and ps ○ grep and tail -f of log files ● Only ‘qualified’ admins allowed ● No business metrics (manual reports) ● No visibility ● RESTART ! G N I N R A W Prone to human ERORS
  13. 13. Run : Practice ● New Relic ○ servers for free ○ app with paid ○ great insight ● Centralized logging ○ rsyslog:unification/collection ○ papertrail:aggregation & display
  14. 14. Run : Practice ● Health Metrics (in app) ○ home rolled exposed via web ○ Metrics by CodaHale / Pingdom ● Biz Metrics ○ DucksBoard ■ simple funnel ■ attrition and activity ● Monit ○ alert and restart (if needed)
  15. 15. Environments : Theory ‘Works on my machine’ ● Scale – vertically or horizontally ● automatable, reproducible ○ Green / Blue deploys FTW ● Chef / Puppet / Ansible ... just pick one ● If a server fails, can you rebuild it? ● If an environment fails, can you rebuild it?
  16. 16. Environments : Reality You won’t always know what you inherit! ● ● ● ● ● Private cloud hosting Limited upgrade ability Manual updates Uncertain/misunderstood foundation SSH and rsync to update and deploy ○ overwrote existing codebase
  17. 17. Environments : Practice ● Picked Chef (Vi vs Emacs …) ○ Roles for baseline / Web / App / DB nodes ○ Environments for DEV / STG / PRD ● Vagrant for testing recipes locally ○ config checked into app project ● Automate (via Jenkins of course) ○ validate cookbooks via FoodCritic ○ upload to hosted chef ○ rollout / reprovision via bash / ssh
  18. 18. Environments : Practice ● DB backup / restore ○ do it and test it regularly ○ scripted ○ hire an expert Delegate to others ● Mandrill for outgoing SMTP ● DNSimple for managed DNS ● Pingdom for simple uptime ○ custom health check endpoint
  19. 19. Environment : Current State Rackspace Cloud Hosted chef Managed by Jenkins ○ Re-Create dev / staging / production ○ deploy staging / production Monitored via ○ Pingdom ○ New Relic ○ PaperTrail app Rolled out to iOS app dev TOO! ○ Xcode CLI / OCUnit / Testflight via Jenkins
  20. 20. OH: I deployed to production, nobody noticed and nothing went wrong FTW!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×