Your SlideShare is downloading. ×
0
DISQUS                         Continuous Deployment Everything                                    David Cramer           ...
Disclaimer: We don’t use PylonsFriday, April 29, 2011
Continuous Deployment           Shipping new code as soon                  as it’s ready                         (It’s rea...
Workflow                  1. Developer commits code                  2. CI server runs tests automatically                 ...
Pros                         Cons                 •       Develop features         •   Culture Shock                      ...
Painless DevelopmentFriday, April 29, 2011
Development                  •      Production > Staging > CI > Dev                         •   Automate testing of compli...
Production            Staging                         •    PostgreSQL   •   PostgreSQL                         •    Memcac...
Bootstrapping Local                  •      Simplify local setup                         •   git clone dcramer@disqus:disq...
“Under Construction”                  •      Iterate quickly by hiding features                  •      Early adopters are...
Gargoyle                         Deploy features to portions of a user base at a                          time to ensure s...
Integration                         (or as we like to call it)Friday, April 29, 2011
Integration is Required                         Deploy only when things wont breakFriday, April 29, 2011
Reporting is CriticalFriday, April 29, 2011
CI Requirements                  ✓      Developers must know when they’ve                         broken something        ...
Shortcomings                  •      False positives lower awareness                         •   Reporting isnt accurate  ...
Fixing False Positives                  •      Re-run tests several times on a failure                  •      Report cont...
Maintaining Coverage                  •      Raise awareness with reporting                         •   Fail/alert when co...
Speeding Up Tests                  •      Write true unit tests                         •   vs slower integration tests   ...
Mule                  •      Unstable (we’re working on it)                  •      Mostly Django right now               ...
Deploy (finally)Friday, April 29, 2011
How DISQUS Does It                  •      Incremental deploy with Fabric                  •      Drop server from pool   ...
Challenges                  •      PyPi works on server A, but not B                  •      Scale, or lack of            ...
PyPi is Down                  •      http://github.com/disqus/chishopFriday, April 29, 2011
Help, we have 100 servers!                  •      Incremental (ours) vs Fanout                  •      Push vs Pull      ...
SQL Schema Changes                  1. Add column (NULLable)                  2. Add app code to fill column               ...
DISQUS                           Questions?                           psst, we’re hiring                          jobs@dis...
References                  •      Mule (distributed test runner)                         http://github.com/disqus/mule   ...
Upcoming SlideShare
Loading in...5
×

Continuous Deployment at Disqus (Pylons Minicon)

5,126

Published on

Published in: Technology
1 Comment
13 Likes
Statistics
Notes
  • welcome to it !!hi gh quali ty☆ reaso nable pric e☆
    free shi pping accept pay pal,
    if you have interest in it , take action !!!!!
    ***** {{w w w }} {{ happyshopping100 }} {{ com }} ***
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
5,126
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
90
Comments
1
Likes
13
Embeds 0
No embeds

No notes for slide

Transcript of "Continuous Deployment at Disqus (Pylons Minicon)"

  1. 1. DISQUS Continuous Deployment Everything David Cramer @zeegFriday, April 29, 2011
  2. 2. Disclaimer: We don’t use PylonsFriday, April 29, 2011
  3. 3. Continuous Deployment Shipping new code as soon as it’s ready (It’s really just super awesome buildbots)Friday, April 29, 2011
  4. 4. Workflow 1. Developer commits code 2. CI server runs tests automatically 2.1.Build passes, code deploys 2.2.Build fails, block deploy 3. Developer tests feature on productionFriday, April 29, 2011
  5. 5. Pros Cons • Develop features • Culture Shock incrementally • Stability depends on • Release frequently test coverage • Less QA! (maybe) • Initial time investment We mostly just care about iteration and stabilityFriday, April 29, 2011
  6. 6. Painless DevelopmentFriday, April 29, 2011
  7. 7. Development • Production > Staging > CI > Dev • Automate testing of complicated processes and architecture • Simple is arguably better than complete • Especially for local development • python setup.py {develop,test} • Use Puppet/Chef or build a simple bootstrap.{py,sh}Friday, April 29, 2011
  8. 8. Production Staging • PostgreSQL • PostgreSQL • Memcache • Memcache • Redis • Redis • Solr • Solr • Apache • Apache • Nginx • Nginx • RabbitMQ • RabbitMQ CI Server Macbook • Memcache • PostgreSQL • PostgreSQL • Apache • Redis • Memcache • Solr • Redis • Apache • Solr • Nginx • Nginx • RabbitMQ • RabbitMQFriday, April 29, 2011
  9. 9. Bootstrapping Local • Simplify local setup • git clone dcramer@disqus:disqus.git • python setup.py develop • python manage.py runserver • Need to test dependancies? • virtualbox + vagrant upFriday, April 29, 2011
  10. 10. “Under Construction” • Iterate quickly by hiding features • Early adopters are free QA from gargoyle import gargoyle def my_view(request): if gargoyle.is_active(awesome, request): return new happy version :D else: return old sad version :(Friday, April 29, 2011
  11. 11. Gargoyle Deploy features to portions of a user base at a time to ensure smooth, measurable releases Being users of our product, we actively use early versions of features before public releaseFriday, April 29, 2011
  12. 12. Integration (or as we like to call it)Friday, April 29, 2011
  13. 13. Integration is Required Deploy only when things wont breakFriday, April 29, 2011
  14. 14. Reporting is CriticalFriday, April 29, 2011
  15. 15. CI Requirements ✓ Developers must know when they’ve broken something ✓ IRC, Email, IM ✓ Support proper reporting ✓ XUnit, Pylint, Coverage.py ✓ Painless setup ✓ apt-get install jenkinsFriday, April 29, 2011
  16. 16. Shortcomings • False positives lower awareness • Reporting isnt accurate • Services fail • Bad Tests • Not enough code coverage • Regressions on untested code • Test suite takes too long • Integration tests vs Unit testsFriday, April 29, 2011
  17. 17. Fixing False Positives • Re-run tests several times on a failure • Report continually failing tests • Fix continually failing tests • Rely less on 3rd parties • Mock/DingusFriday, April 29, 2011
  18. 18. Maintaining Coverage • Raise awareness with reporting • Fail/alert when coverage drops on a build • Commit tests with code • Drive it into your cultureFriday, April 29, 2011
  19. 19. Speeding Up Tests • Write true unit tests • vs slower integration tests • Mock 3rd party APIs • Distributed and parallel testing • http://github.com/disqus/muleFriday, April 29, 2011
  20. 20. Mule • Unstable (we’re working on it) • Mostly Django right now • Generic interfaces for unittest2 • Works with multi-processing and Celery • More complex than normal Celery usage • Full XUnit integration • Simple workflow • mule test --runner="python manage.py mule --worker $TEST"Friday, April 29, 2011
  21. 21. Deploy (finally)Friday, April 29, 2011
  22. 22. How DISQUS Does It • Incremental deploy with Fabric • Drop server from pool • Pull in requirements on each server • Isolated virtualenv’s built on each server • Push server back onlineFriday, April 29, 2011
  23. 23. Challenges • PyPi works on server A, but not B • Scale, or lack of • CPU cost per server • Schema changes, data model changes • Backwards compatibilityFriday, April 29, 2011
  24. 24. PyPi is Down • http://github.com/disqus/chishopFriday, April 29, 2011
  25. 25. Help, we have 100 servers! • Incremental (ours) vs Fanout • Push vs Pull • Twitter uses BitTorrent • Isolation vs Packaging (Complexity)Friday, April 29, 2011
  26. 26. SQL Schema Changes 1. Add column (NULLable) 2. Add app code to fill column 3. Deploy 4. Backfill column 5. Add app code to read column 6. Deploy Cached Data Changes • Have a global version number • Have a data model cache version • maybe md5(cls.__dict__)?Friday, April 29, 2011
  27. 27. DISQUS Questions? psst, we’re hiring jobs@disqus.comFriday, April 29, 2011
  28. 28. References • Mule (distributed test runner) http://github.com/disqus/mule • Gargoyle (feature switches) https://github.com/disqus/gargoyle • Jenkins CI http://jenkins-ci.org/ code.disqus.comFriday, April 29, 2011
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×