Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DevOps automation for AWS

I had a great audience at the Illinois Java Uses Group at Nielsen in Schaumburg for this presentation. It is about Tvarit, a maven plugin that can fit nicely in a continuous delivery pipeline for the AWS platform. Feedback, comments and questions are all very welcome! Thank you.
Source and instruction are on github: https://github.com/sdole/tvarit-maven

  • Login to see the comments

  • Be the first to like this

DevOps automation for AWS

  1. 1. AWS SDK DevOps Automation Sachin Dole
  2. 2. About me ▪ AWS Certified Solutions Architect ▪ Independent AWS Consultant – Genvega Inc. ▪ Trelair.com – now defunct startup ▪ www.cricclubs.com – ½ million+ page views per month ▪ Backup & Archival on S3 – project for law firm ▪ EC2 automation for image processing ▪ Available for AWS consulting – find me: www.genvega.com
  3. 3. About CricClubs (My customer) 30th Jan 2016 @2015 CRICCLUBS.COM | www.cricclubs,.com | CONFIDENTIAL 3 Started in 2008 Geographic Usage 20+ Countries 600+ Leagues 100,000+ Players Headquarters Milwaukee, WI, USA Features 50+ features on Website and iOS, Android, Windows Mobile Apps Premium Support 24/7 via E-mail, Phone and Ticketing Systems Data Centers High Availability with servers across 12 Data Centers 4 million pageviews/month
  4. 4. About you? ▪ Have day to day AWS experience? ▪ Have personal projects on AWS? ▪ Want a good introduction to DevOps & AWS?
  5. 5. Console access ▪ https://tvarit.signin.aws.amazon.com/console ▪ ijugdemo ▪ devops123 ▪ Read only access to all services
  6. 6. Topics ▪ DevOps for Trelair ▪ Lessons learnt ▪ Tvarit.io & AWS Java SDK ▪ Questions?
  7. 7. Trelair.com
  8. 8. System Architecture Clients ELB Wildfly on EC2 S3 & Postgres External Services
  9. 9. Building Trelair Commit Build FTP Deploy Test Backup & Release 1 to 8 person-hrs DML Disk Compute DB App server Backups
  10. 10. Commit Build FTP Deploy Test Release Enterprises? 1 to 8 person years DML Disk Compute DB App server Backups Dev EA Test Ops EA
  11. 11. Trelair Automation GitCommit • Teamcity • JBoss DMR Auto- Deploy Selenium Test & Release DevOps + 0 person hours Custom Maven Plugin Used AWS Java SDK AMI DML/DDL DNS Snapshots Scaling Auto
  12. 12. Automated Flow A. Commit Code (the only manual step) B. CI build C. DB DML patches are applied to replica D. Deploy code to candidate server E. UI tests run F. DML patches applied to production G. Candidate server is added to cluster H. Old servers are terminated
  13. 13. AWS API us - east AWS Java SDK JBoss DMR Wildfly AS Provision Deploy
  14. 14. Lessons ▪ Automation is easy – mindset is difficult ▪ Better to automate in small increments – big bang is a waste ▪ Automate everything – partial automation is overhead ▪ Release frequently – rely on automation ▪ Use selenium automation (e.g. saucelabs.com) ▪ Rely on AWS products
  15. 15. Tvarit ▪ http://www.tvarit.io, https://github.com/sdole/tvarit-maven ▪ Trelair automation on steroids, opensource ▪ Automated provisioning, deployment and scaling ▪ Alpha release available in Maven Central ▪ Free support through GitHub issues ▪ Working α version & a deep roadmap
  16. 16. Console access ▪ https://tvarit.signin.aws.amazon.com/console ▪ ijugdemo ▪ devops123 ▪ Read only access to all services
  17. 17. Tvarit – demo ▪ CloudFormation templates ▪ Maven goals ▪ VPC, IAM console ▪ EC2: LC, ASG, ELB ▪ S3 bucket ▪ Cloudwatch monitoring, flow logs ▪ Code walkthrough
  18. 18. API structure ▪ Many flavors – Java, C++, JS, Ruby, Python, CLI etc. ▪ APIs are RESTful ▪ Deep documentation
  19. 19. API Structure AmazonEc2Client ec2 = new AmazonEc2Client(new BasicAwsCredentials()); SomeEc2Request req = new SomeEc2Request(); req.withSomething(thing1).withSomething(attr1); SomeEc2Result result = ec2.submitRequest(req); inspectWithYourLogic(result);
  20. 20. Automation Security - Individuals IAM us - east us - west Access Keys groups users roles Authorized
  21. 21. Automation Security - Services IAM Other services Other services ec2 lambda S3 Assumed Role Authorized
  22. 22. AWS Automation Features ▪ Autoscaling ▪ Monitoring ▪ Alarms ▪ Cloudwatch Events ▪ Lambda Functions ▪ S3 Events, SQS and SNS events ▪ SWF ▪ CloudFormation
  23. 23. Tvarit Target Architecture Clients WAF ELB EC2 ELB EC2 Lambda Java SDK Autoscaling Cloudwatch Zookeeper S3, RDS S3, RDS Cloudformation ELB Router App 1 Version 1 App 2 Version 1
  24. 24. Other concepts: Blue-Green Deployment http://martinfowler.com/bliki/BlueGreenDeployment.html
  25. 25. Other concepts: Canary Release http://martinfowler.com/bliki/CanaryRelease.html
  26. 26. Revisit Tvarit Target Architecture Clients WAF ELB EC2 ELB EC2 Lambda Java SDK Autoscaling Cloudwatch Zookeeper S3, RDS S3, RDS Cloudformation ELB Router App 1 Version 1 App 2 Version 1
  27. 27. Tvarit Release concept (future) ELB ELB Router App Versions Router: routes customers based on Http Headers to correct app version
  28. 28. Step Back – what is (not) DevOps? Is… Infrastructure as code Not… Auto-scale, failover, monitoring Continuous delivery Dedicated teams, Dev, Test, Ops Forms, Emails, Release plans Infrastructure Architects/EA
  29. 29. Soft side of DevOps ▪ Cloud-native vs Lift ‘n shift ▪ CALMS ▪ C – Culture ▪ A – Automation ▪ L – Lean ▪ M – Measurement ▪ S – Sharing ▪ Automation as Kaizen – not a single project
  30. 30. Costs of AWS Automation ▪ In most cases – FREE ▪ Priced as usual: Compute, Storage, Monitoring, Network ▪ Monitoring is a must ▪ CPU, Memory, Network, Disk… ▪ Health checks, failed hardware, traffic spikes ▪ Audit trail – API actions, modifications ▪ Configuration changes
  31. 31. How to improve Tvarit ▪ Open an Issue on GitHub: “We use a customized version of tomcat – can you make the app server container configurable so we can use our own?”  Fork it on GitHub and DIY  Email me
  32. 32. Other things ▪ Chef, Puppet, Ansible, SaltStack ▪ AWS OpsWorks, Beanstalk ▪ AWS CodeCommit, CodeDeploy
  33. 33. How to find me @sdoledotcom https://linkedin.com/in/sdole sdole@genvega.com http://genvega.com/blog/ https://github.com/sdole/
  34. 34. Ω Thank you

×