Your SlideShare is downloading. ×
0
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

DevOps and Continuous Deployment @ WWPS Government, Education, and Non-profit Symposium 2014

1,089

Published on

Leo Zhadanovsky and I talked to a group of Government, Non-profit, and Education representatives about DevOps and how they could use it to speed up their organization.

Leo Zhadanovsky and I talked to a group of Government, Non-profit, and Education representatives about DevOps and how they could use it to speed up their organization.

Published in: Internet, Travel, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,089
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
3
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • All canonical software life cycles were designed by people who build installed software
  • Not your normal office environment
  • A few friends in high places
  • Problem

    Infrastructure a bottleneck, very long lead time, upfront expense
    Software delivery hindered
    No test environment
    No ability to load test
    Ideas for projects shelved due to infrastructure bottleneck

  • Solution

    Move the web infrastructure to AWS
    Incumbent Ops team unable to support more infrastructure
    Only 3 Software Engineers available
    AWS SDK
    We have to AUTOMATE


  • In VPC, you also have the option of creating an internal load balancer, which does not have public IP addresses.

    In this configuration, only the internet-facing load balancer has public IP addresses; everything else can use private IP addresses.

  • The world’s handmade and vintage marketplace.
  • The world’s handmade and vintage marketplace.
  • Deploying code is the very first thing engineers learn to do at Etsy.
  • No authorization required, Detect problems quickly, gain confidence
  • Que ce soit iteration ou pivot, pas le choix, il faut innover
  • Pour illustrer le launch early
  • Transcript

    • 1. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 DevOps on AWS Continuous Integration and Deployment Best Practices on AWS Leo Zhadanovsky, Senior Solutions Architect, AWS | @leozh JP Schneider, DevOps / Internet Jedi, Mozilla Foundation | @jdotp
    • 2. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Who Am I? I work for AWS I worked for the DNC 2009-2012 I was embedded in the DevOps Team @ OFA AWS does not endorse political candidates
    • 3. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Who Am I? JP, DevOps for Mozilla Foundation Previous gig DevOps at OFA 2012 Before that, Ops at Threadless @jdotp Mozilla Foundation does endorse animated cats
    • 4. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 CONTINUOUS INTEGRATION
    • 5. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 What is Continuous Integration? Changes to code automatically deployed to mainline branch • After passing unit and mock tests Makes changes to code, and deployments iterative, not monolithic Bugs are detected quickly Allows rapid development Helps automate deployments
    • 6. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 DEVELOPER
    • 7. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 SOURCE CODE REPOSITORY
    • 8. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER
    • 9. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER
    • 10. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER PICK TASKS
    • 11. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER SUBMIT CODE
    • 12. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER SCHEDULE BUILD
    • 13. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER RECURRENT BUILDS
    • 14. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER CODE FETCH
    • 15. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER CODE QUALITY TESTS TEST RESULTS
    • 16. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER BUILD OUTPUT
    • 17. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER DOCS BINARIES & PACKAGES DEV FACING NOTIFICATIONS
    • 18. SOURCE CODE REPOSITORY DNS CONTINUOUS INTEGRATION SERVER PROJECT MANAGEMENT SERVER BUILDS
    • 19. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 PAIN POINTS • UNIT TESTS INCOMPLETE • MOCK TESTS MAINTENANCE • EXPENSIVE TEST ENVIRONMENT • TEST ENVIRONMENT ≠ PRODUCTION • DEPLOYMENT CYCLES
    • 20. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 ON-DEMAND PAY AS YOU GO ELASTIC
    • 21. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 = PROGRAMMABLE PLATFORM
    • 22. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014
    • 23. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014
    • 24. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014
    • 25. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 IF YOU CAN PROGRAM IT YOU CAN AUTOMATE IT
    • 26. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 A lot of options… Configuration Management Systems • Puppet • Chef • Saltstack Deployment Frameworks • Elastic Beanstalk • OpsWorks • Ansible • Fabric • Capistrano Infrastructure Management • CloudFormation
    • 27. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Bake an AMI Configure dynamically Time consuming configuration (startup time) Static configurations (less change management) Bootstrapping
    • 28. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Continuous deployment (latest code) Environment specific (dev-test-prod) Bootstrapping Bake an AMI Configure dynamically
    • 29. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Obama for America awsofa.info
    • 30. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 So here’s the Idea ~30th biggest E-commerce operation, globally ~200 distinct new applications, many mobile Hundreds of new, untested analytical approaches Processing hundreds of TB of data on thousands of servers Spikes of hundreds of thousands of concurrent users
    • 31. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 a few constraints… ~30th biggest E-commerce operation, globally ~200 distinct applications, many mobile Hundreds of new, untested analytical approaches Processing hundreds of TB of data on thousands of servers Spikes of hundreds of thousands of concurrent users Critically compressed budget Less than a year to execute Volunteer and near-volunteer development team Core systems will be used for a single critical day Constitutionally-mandated completion date
    • 32. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Web-Scale Applications
    • 33. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 500k+ IOPS DB Systems
    • 34. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Services API
    • 35. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014
    • 36. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Business as usual.. …for a technology startup
    • 37. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Election Day – OFA Headquarters
    • 38. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Typical Charts
    • 39. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014
    • 40. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 How?
    • 41. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 The old approach, even from Amazon 
    • 42. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 The old approach.. Might have some problems..
    • 43. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 OFA’s Infrastructure awsofa.info
    • 44. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Ingredients Ubuntu nginx boundary Unity jQuery SQLServer hbase NewRelic EC2 node.js Cybersource hive ElasticSearch Ruby Twilio EE S3 ELB boto Magento PHP EMR SES Route53 SimpleDB Campfire nagios Paypal CentOS CloudSearch levelDB mongoDB python securitygroups Usahidhi PostgresSQL Github apache bootstrap SNS OpsView Jekyll RoR EBS FPS VPC Mashery Vertica RDS Optimizely MySQL puppet tsunamiUDP R asgard cloudwatch ElastiCache cloudopt SQS cloudinit DirectConnect BSD rsync STS Objective-C DynamoDB
    • 45. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Infrastructure, Configuration Management & Monitoring Ubuntu nginx boundary Unity jQuery SQLServer hbase NewRelic EC2 node.js Cybersource hive ElasticSearch Ruby Twilio EE S3 ELB boto Magento PHP EMR SES Route53 SimpleDB Campfire nagios Paypal CentOS CloudSearch levelDB mongoDB python securitygroups Usahidhi PostgresSQL Github apache bootstrap SNS OpsView Jekyll RoR EBS FPS VPC Mashery Vertica RDS Optimizely MySQL puppet tsunamiUDP R asgard cloudwatch ElastiCache cloudopt SQS cloudinit DirectConnect BSD rsync STS Objective-C DynamoDB
    • 46. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Configuration Management: Puppet In mid-2011, we looked at options for configuration management and chose Puppet We needed to make it scale, and to get it to work with state-less, horizontally scalable infrastructure How did we do this?
    • 47. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Bootstrapping Puppet with CloudInit CloudInit is built into Ubuntu and Amazon Linux • Allows you to pass bootstrap parameters in Amazon EC2 user-data field, in YAML format
    • 48. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Bootstrapping Puppet with CloudInit Don’t store creds in puppet manifests, store them in private Amazon S3 buckets Either pass Amazon S3 creds through CloudInit: Even better – avoid this by using AWS Identity and Access Management (IAM) roles and AWS Unified CLI’s S3 client
    • 49. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Bootstrapping Puppet with CloudInit Built-in puppet support Use certname with %i for instance id to name the node Puppetmaster must have auto sign turned on • Use security groups and/or NACLs for network-level security In nodes.pp, use regex to match node names
    • 50. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Puppet Tips Use a base class to define your standard install
    • 51. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Use runstages Don’t store credentials in puppet, store them in private Amazon S3 buckets • Use AWS IAM to secure the credentials bucket/folders within that bucket Puppet Tips
    • 52. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Puppet Tips Use puppet only for configuration files and what makes your apps unique For undifferentiated parts of apps, use Amazon S3 backed RPM/Debian repositories • Can be either public or private repos, depending on your needs • Amazon S3 Private RPM Repos: http://git.io/YAcsbg • Amazon S3 Private Debian Repos: http://git.io/ecCjWQ
    • 53. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Puppet Tips By using packages for applications deploys, you can set ensure => latest, and just bump the package in the repo to update Log everything with rsyslog/graylog/loggly/NewRelic/splunk
    • 54. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Scaling the Puppet Masters Use an Auto Scaling group for puppet masters • Min size => 2, use multiple Availability Zones Either have them build themselves off of existing puppet masters in the group or off packages stored in Amazon S3 and bootstrapped through user-data Auto-sign must be on
    • 55. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 One thing that is difficult to prepare for…
    • 56. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 They had this built for the previous 3 months, all on the East Coast.
    • 57. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 They had this built for the previous 3 months, all on the East Coast. We built this part in 9 hours to be safe. AWS + Puppet + Netflix Asgard + WAN Optimization Software + DevOps = Cross-Continent Fault- Tolerance On-Demand
    • 58. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 If OFA was run on AWS today, what might that look like? Take advantage of OpsWorks • This would simplify configuration management Use CloudFormation to recreate environments for prod/dev/test quickly Utilize PostgreSQL RDS instead of the many manually installed PostgreSQL instances Analytics data warehouse took months to build, RedShift would take minutes Docker containers could be used to easily allow developers to replicate their dev environments in prod or staging Use of immutable infrastructure More use of CloudFront, resulting in less load on EC2 resources
    • 59. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Mozilla Foundation
    • 60. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Webmaker.org circa 2012 • Included Apps, non-SOA: Thimble, Popcorn, Goggles • ~20 pushes of new software in 2012 • Operations and Development interacted mostly through bugzilla tickets for deploys. • Hosting in physical datacenter at Mozilla Webmaker.org circa early 2013 • Deciding to go 12-factor, SOA in app layer • Weekly pushes of Popcorn on train model • Operations and Development interacted mostly through bugzilla tickets for deploys.
    • 61. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 April 2013 Webmaker begins rebuilding entire platform SOA, 12-factor in node.js exclusively Moving apps into AWS and DevOps / CI
    • 62. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Since April 2013…. Openbadges, Webmaker combine for: 1339 Pushes Pushes Per Day to Staging / Prod Pushes Per Day (Staging and Prod)
    • 63. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Who? • ~30 Paid Developers • Hundreds of Students • Thousands of Contributors • One DevOps / Internet Jedi • Multiple Teams How? • Puppet, Jenkins, Fabric • Tight feedback loops: Newrelic, Opsview • Culture Shift • Staging Envs • Brave devs iterate, keeping work in-context • Visible Ops • Cross-train developers in operations
    • 64. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 What changed?
    • 65. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 1) Know How You Were Doing Before 2) Know What Changed When 3) Know How You Are Now Doing = The confidence to try more things and try them faster, with minimum viable planning.
    • 66. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Deployment Pipeline
    • 67. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 AWS CLOUDFORMATION STACK-BASED DEPLOYMENT SERVICE
    • 68. CLOUDFORMATION TEMPLATE
    • 69. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014
    • 70. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 { "Description" : "Create RDS with username and password", "Resources" : { "MyDB" : { "Type" : "AWS::RDS::DBInstance", "Properties" : { "AllocatedStorage" : "500", "DBInstanceClass" : "db.m1.small", "Engine" : "MySQL", "EngineVersion" : "5.5", "MasterUsername" : "MyName", "MasterUserPassword" : "MyPassword" } } } }
    • 71. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 "AWS::CloudFormation::Init" : { "config" : { "packages" : { "yum" : { "mysql" : [], "mysql-server" : [], "httpd" : [], "php" : [], "php-mysql" : [] } }, "sources" : { "/var/www/html" : "https://s3.amazonaws.com/my-builds/build-v4.zip" } }
    • 72. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 { "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" } }, }
    • 73. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 CLOUDFORMATION TEMPLATE PROCEDURAL DEFINITION Create it programmatically KNOWN CONFIGURATION Store stack configuration in source control PARAMETER DRIVEN Dynamic and user-driven templates COLLABORATION Share templates with ease as just files
    • 74. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014
    • 75. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 APPLICATION VERSIONS + INFRASTRUCTURE VERSIONS
    • 76. CLOUDFORMATION TEMPLATE
    • 77. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 NASA/Jet Propulsion Laboratory California Institute of Technology
    • 78. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Mars Rover Landing by the #’s NASA TV = HD stream, 1080p, ~1 mb/s per viewer Expecting peak of ~1m viewers All playback devices (iOS, Android, Flash, HTML5, blah) Once in a lifetime moment in history (no crashy crashy) FUN FUN FUN
    • 79. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Mars Rover Landing by the #’s NASA TV = HD stream, 1080p, ~1 mb/s per viewer Expecting peak of ~1m viewers All playback devices (iOS, Android, Flash, HTML5, blah) Once in a lifetime moment in history (no crashy crashy) NASA says we can’t use their live stream setup It’s 6 days before the landing It’s the same week as the Olympics Available technical resources from JPL: Brett and Khawaja
    • 80. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 The sticky wicket: HD Video stream size = 4mb never changes easy to cache manifest.f4m size = 4kb New every 4 sec caching difficult
    • 81. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 The plan Design a solution around our limits • Max connections to origin = 6 • Max streams per cache node = 20 • Local Latency = critical • US-WEST-1 capacity reserved for S3 static images Test the snot out of it Hang on!
    • 82. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014
    • 83. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 ok, so one of them is a rocket scientist..
    • 84. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Mars Science Laboratory - Live Video Streaming “The 42 pack”
    • 85. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 LOAD TESTING
    • 86. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Battle Testing our Deployment Benchmarking
    • 87. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Dynamic Traffic Scaling US-East Cache Node Performance 25.3 Gbps
    • 88. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 Only ~42Mbps Dynamic Traffic Scaling Impact on US-East FMS Origin Servers
    • 89. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014
    • 90. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 CONTINUOUS DEPLOYMENT SMALL, FREQUENT CHANGES CONSTANTLY INTEGRATING INTO PRODUCTION.
    • 91. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 KEY = ITERATION
    • 92. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 ITERATION = MODIFY THE SYSTEM TO BETTER MEET THE EXPECTATIONS OF YOUR USERS
    • 93. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 11.6s Mean time between deployments (weekday) 1,079 Max number of deployments in a single hour 10,000 Mean number of hosts simultaneously receiving a deployment 30,000 Max number of hosts simultaneously receiving a deployment DEPLOYMENTS AT AMAZON.COM
    • 94. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 SOFTWARE DEPLOY ≠ PRODUCT LAUNCH
    • 95. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014
    • 96. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 1.5 BILLION PAGE VIEWS OCTOBER 2012 $83 MILLION IN TRANSACTIONS 4.2 MILLION ITEMS SOLD
    • 97. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 30 DEPLOYS PER DAY 1 DEPLOY EVERY 20 MINUTES
    • 98. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 "Production is truly the only place you can validate your code."
    • 99. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014
    • 100. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014
    • 101. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014
    • 102. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014
    • 103. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 AWS OPSWORKS INTEGRATED APPLICATION MANAGEMENT AWS OPSWORKS UNDER THE HOOD
    • 104. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 DATA-DRIVEN ARCHITECTURES
    • 105. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 METRICS @ETSY
    • 106. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 METRICS @OBAMA FOR AMERICA
    • 107. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 COST-ORIENTED ARCHITECTURES
    • 108. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 PHP+APACHE+VARNISH NGINX+NODEJS
    • 109. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 CONTINUOUS INTEGRATION CONTINUOUS DEPLOYMENT
    • 110. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 CONTINUOUS DEPLOYMENT = CONTINUOUS EXPERIMENTATION
    • 111. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 CONTINUOUS DEPLOYMENT = CONTINUOUS IMPROVEMENT
    • 112. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 INNOVATE
    • 113. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 « Want to increase innovation? Lower the cost of failure » Joi Ito
    • 114. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 – June 26, 2014 SPEED AND AGILITY Experiment Often Fail quickly at a low cost More Innovation Experiment Infrequently Failure is expensive Less Innovation “ON-PREMISE”
    • 115. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 Q & A
    • 116. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 aws.amazon.com/training Expand your technical expertise to design, deploy, and operate scalable, efficient applications on AWS Training aws.amazon.com/certification Certification 20% off AWS Instructor-Led Training Class* • Code MK20PSSYM314 • Expires September 30, 2014 Get Savings on AWS Training! * Discount only applies to classes delivered by AWS. Discount does not apply to classes delivered by an APN Training Partner. aws.amazon.com/training/self-paced-labs Self- Paced Labs Get hands-on practice working with AWS technologies in a live environment Validate your proven technical expertise with the AWS platform Expand your skills with AWS
    • 117. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 DevOps on AWS Continuous Integration and Deployment Best Practices on AWS Thank You Leo Zhadanovsky, Senior Solutions Architect, AWS | @leozh JP Schneider, DevOps / Internet Jedi, Mozilla Foundation | @jdotp

    ×