Puppet at Constant Contact


Published on

Talk at PuppetConf 2011
Dave Connors & Hawk Newton

Adopting Puppet over the last couple of years and how we're handing over the keys to the Puppet Kingdom to Development.

And what we are doing so it works for Development.

Published in: Technology, Spiritual
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Quick snapshot of CTCTLittle bit on how we adopted PuppetThen over to the cleanup hitter
  • Fast growingMicro businessesOffices in Loveland, Boca, SF and NYCLots of apps, at decent scale --- Top 20 email senders worldwide
  • Business DriversDiverse: not all uniform web app servers multiple applicationsDistributed: memcached, sso, coherence, cassandraManage it cost-effectively, keep things consistent in order to max uptime and mean time to recovery - don’t slow down the rate of changeAnd oh yeah, improve gross margins
  • We hit a bit of a tipping point on managing servers, couple of hundred or soStarted with a small piece of the puzzleSystem stuff resolve.conf, host…etcWork out the basic process, gain trust
  • Getting adoption by leaving control in hands of adminsThe declarative model was challenging to many at firstPush via func was a good interim step
  • Good news along the way thoughConfidence gained – Automated System Warning!Moved mail subsystem to daemon modeStandard well understood configuration postfix and centosSystems folks were onboard
  • Change in control, ownership, processesRetooling without stopping rate of changeGaining trust while changing folx mindset
  • Manage social data cost-effectively3 months cycle, cheaperBut still had challenges around branching and testingIntegration in pre-productionIndependent upgradesSo we had some work to do, including orchestration
  • Quick Shout Out 
  • We could only go so far pulling from the Systems sideBut it had to work for Developers, so…Here you go Dev, own the config, trust in Hawk
  • The monolith!We had some significant challenges:New appsUgly handoffsEnvironmentsDecreased velocityFUDFrustrated parties on both sides of the house…
  • Puppet at Constant Contact

    1. 1. Puppet at Constant Contact<br />Adoption, the process<br />
    2. 2. Constant Contact<br />Market leader for Small Businesses<br /><ul><li>Email, Event, Survey & Social Campaigns
    3. 3. ~500k paying customers
    4. 4. Many customers pay as little as $15 a month</li></ul>Technology<br /><ul><li>87 applications, mix of old/new codebase
    5. 5. Two datacenters
    6. 6. ~2 million database transactions per minute</li></li></ul><li>Why Puppet?<br /><ul><li>Scaling
    7. 7. Decentralization
    8. 8. Diverse app servers
    9. 9. Distributed systems
    10. 10. Agile Kanbandev teams
    11. 11. Service architecture
    12. 12. Social apps need to iterate faster</li></li></ul><li>How to manage all these servers?<br />
    13. 13. Control of Change<br />
    14. 14. Daemon mode, at last<br />
    15. 15. Cultural Change, part one<br />Ok, <br />let’s start with<br />Incremental Config<br />& <br />Push<br />
    16. 16. Time to market better, but…<br />
    17. 17. Special thanks to:<br />Our DevOps, “we can do this with Puppet” guy, Mark Schena<br />& our puppetmaster, AleksMargan<br />
    18. 18. Cultural Change, part two<br />
    19. 19. It works, right?<br />Yes, errr. Well… kinda.<br />
    20. 20. 12<br />
    21. 21.
    22. 22. The dream of CI<br />
    23. 23. Fix The Flow<br />
    24. 24. I Я COW!<br />
    25. 25. Configuration as code?<br />
    26. 26. Configuration as code!<br />
    27. 27. 19<br />
    28. 28. puppet repository<br />application repository<br /><ul><li>common puppet modules
    29. 29. system puppet modules
    30. 30. code
    31. 31. puppet module
    32. 32. integration tests
    33. 33. database schema</li></ul>Repository layout<br />Source Control<br />
    34. 34. Two types of puppet modules <br />+<br />=<br />common module<br />running application<br />application module<br /><ul><li>Common modules provide non-application-specific classes and resources.
    35. 35. Application modules leverage common modules to configure an environment for a given application.</li></li></ul><li>Okay, I lied, there’s three types of puppet modules<br />
    36. 36. Module development workflow<br />
    37. 37. complete set of puppet modules<br />environment<br />environment<br />environment<br />environment<br />QA-1<br />QA-2<br />PROD<br />Deployment<br />puppetmaster<br />
    38. 38. Thanks!<br />“Dave Connors” <dconnors@constantcontact.com><br />VP, Operations<br />“Hawk Newton” <hnewton@constantcontact.com> <br />Software Architect<br />