Practicing Continuous Deployment
Upcoming SlideShare
Loading in...5
×
 

Practicing Continuous Deployment

on

  • 3,188 views

 

Statistics

Views

Total Views
3,188
Views on SlideShare
3,187
Embed Views
1

Actions

Likes
6
Downloads
30
Comments
0

1 Embed 1

http://74.6.238.254 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Practicing Continuous Deployment Practicing Continuous Deployment Presentation Transcript

  • DISQUS Practicing Continuous Deployment David Cramer @zeegSaturday, March 10, 12
  • Shipping new code as soon as it’s readySaturday, March 10, 12
  • Continuous Deployment # Update the site every 5 minutes */5 * * * * cd /www/example.com && git pull && service apache restartSaturday, March 10, 12
  • Saturday, March 10, 12
  • When it’s readySaturday, March 10, 12
  • When is it ready? - Reviewed by peers - Passes automated tests - Some level of QASaturday, March 10, 12
  • Focus on Stability and IterationSaturday, March 10, 12
  • Workflow Review Commit Integration Failed Build Deploy Reporting RollbackSaturday, March 10, 12
  • The Good - Develop features incrementally - Release frequently - Smaller doses of QA The Bad - Culture Shock - Stability depends on test coverage - Initial time investmentSaturday, March 10, 12
  • Keep Development SimpleSaturday, March 10, 12
  • Development - Automate testing of complicated processes and architecture - Simple can be better than complete - Especially for local development - python setup.py {develop,test} - Puppet, Chef, Buildout, Fabric, etc.Saturday, March 10, 12
  • Production Staging PostgreSQL PostgreSQL Memcache Memcache Redis Redis Solr Solr Apache Apache Nginx Nginx RabbitMQ RabbitMQ (and 100 other painful-to-configure services) CI Server Macbook PostgreSQL PostgreSQL Memcache Apache Redis Memcache Solr Redis Apache Solr Nginx Nginx RabbitMQ RabbitMQSaturday, March 10, 12
  • Bootstrapping Local - Simplify local setup - git clone dcramer@disqus:disqus.git - make - python manage.py runserver - Need to test dependancies? - virtualbox + vagrant upSaturday, March 10, 12
  • Progressive Rollout We actively use early versions of features before public releaseSaturday, March 10, 12
  • Deploy features to portions of a user base at a time to ensure smooth, measurable releases https://github.com/disqus/gargoyleSaturday, March 10, 12
  • • Iterate quickly by hiding features • Early adopters are free QA from gargoyle import gargoyle def my_view(request): if gargoyle.is_active(awesome, request): return new happy version :D else: return old sad version :(Saturday, March 10, 12
  • SWITCHES = { # enable my_feature for 50% my_feature: range(0, 50), } def is_active(switch): try: pct_range = SWITCHES[switch] except KeyError: return False ip_hash = sum([int(x) for x in ip_address.split(.)]) return (ip_hash % 100 in pct_range)Saturday, March 10, 12
  • Review ALL the Commits phabricator.orgSaturday, March 10, 12
  • Saturday, March 10, 12
  • Saturday, March 10, 12
  • Saturday, March 10, 12
  • Integration (or as we like to call it)Saturday, March 10, 12
  • Saturday, March 10, 12
  • Integration Requirements - Developers must know when they’ve broken something - IRC, Email, IM - Support proper reporting - XUnit, Pylint, Coverage.py - Painless setup - apt-get install jenkins * https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins+on+UbuntuSaturday, March 10, 12
  • Shortcomings - False positives - Reporting isnt accurate - Services fail - Bad Tests - Test coverage - Regressions on untested code - Feedback delay - Integration tests vs Unit testsSaturday, March 10, 12
  • Fixing False Positives - Re-run tests several times on a failure - Report continually failing tests - Replace external service tests with a functional test suiteSaturday, March 10, 12
  • Maintaining Coverage - Raise awareness with reporting - Fail/alert when coverage drops on a build - Commit tests with code - Coverage against commit diff for untested regressions - Utilize code reviewSaturday, March 10, 12
  • Speeding Up Tests - Write true unit tests - vs slower integration tests - Mock external services - Distributed and parallel testing - Matrix buildsSaturday, March 10, 12
  • ReportingSaturday, March 10, 12
  • <You> Why is mongodb-1 down? <Ops> It’s down? Must have crashed againSaturday, March 10, 12
  • Meaningful Metrics - Rate of traffic (not just hits!) - Business vs system - Response time (database, web) - Exceptions - Social media - TwitterSaturday, March 10, 12
  • Graphite (Trafficgraphite.wikidot.com across a cluster of servers)Saturday, March 10, 12
  • Sentry sentry.readthedocs.orgSaturday, March 10, 12
  • Wrap UpSaturday, March 10, 12
  • Getting Started - Package your app - Value code review - Ease deployment; fast rollbacks - Setup automated tests - Gather some easy metricsSaturday, March 10, 12
  • Going Further - Build an immune system - Automate deploys, rollbacks (maybe) - Adjust to your culture - There is no “right way” - SOA == great successSaturday, March 10, 12
  • DISQUS Questions? psst, we’re hiring disqus.com/jobsSaturday, March 10, 12
  • References - Gargoyle (feature switches) https://github.com/disqus/gargoyle - Sentry (log aggregation) https://github.com/dcramer/sentry - Jenkins CI (continuous integration) http://jenkins-ci.org/ - Phabricator (code reviews, bug tracking) https://phabricator.org - Graphite (metrics) http://graphite.wikidot.com/ code.disqus.comSaturday, March 10, 12