Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Automated DevOps Workflows with Chef on AWS

4,120 views

Published on

Staying competitive in a turbulent market requires more than traditional practices of manual tests and siloed development – it requires maintaining accurate, repeatable builds and predictable deployment times. Chef Automate gives you a single comprehensive workflow across your entire organization, allowing you to treat infrastructure as code and facilitates DevOps automation. In this webinar, we’ll cover some of the latest Chef integrations with AWS. Gannet, a leading media company and publisher of USA Today, will also join us to talk about how they build, test, and deliver over 400 cookbooks on AWS. They'll talk tools and process for building AMI's and managing 1,000 jobs to continuously deliver their Chef environment.

Join us to learn:
• How to develop at high velocity with Chef on AWS
• How to create a culture of treating your AWS infrastructure as code • How Gannet uses Chef "cookbooks" on AWS to manage their USA Today infrastructure

Who should attend:
• CTOs, CIOs, CISOs, Directors and Managers of Security, IT Administrators, IT Architects and IT Security Engineers.

Published in: Technology
  • Be the first to comment

Automated DevOps Workflows with Chef on AWS

  1. 1. DevOps on the AWS Cloud
  2. 2. Introductions George Miranda Product Marketing Manager Chef @gmiranda23 Erik Rogneby Senior DevOps Engineer Gannett Isaiah Weiner Solutions Architect AWS
  3. 3. Traditional development models are obsolete  Business is increasingly software-driven  End-users expect both continuous improvement and stability from applications  IT needs to be able to provision infrastructure as rapidly as developers demand it  An organization’s pace of innovation is largely constrained by their ability to develop applications
  4. 4. Increase  Business agility  Application stability  Ability to meet customer demand  Time spent on innovation  Security Decrease  Length of development cycles  Time to market  Deployment failures and rollbacks  Time to recover upon failure DevOps can help DevOps practices enable companies to innovate at a higher velocity for customers
  5. 5. Infrastructure as Code Microservices Logging and Monitoring Continuous Integration/ Continuous Delivery DevOps on AWS AWS provides on-demand infrastructure resources and tooling built to enable common DevOps practices
  6. 6.  Provision the server, storage, and networking capacity you need on demand  Deploy independently, as a single service, or a group of services  Make configuration changes repeatable and standardized  Build custom templates to provision resources in a controlled and predictable way  Use version control to keep track of all changes made to your infrastructure and application stack Infrastructure as Code Replace traditional infrastructure provisioning and management with code-based techniques
  7. 7.  Build services around the business capabilities you require  Scale up and down as required with virtually no notice  Make configuration code changes repeatable and standardized  API-driven model enables management of infrastructure with language typically used in application code  Free developers from manually configuring operating systems, system applications, and server software Microservices Build applications as a set of small services that communicates with other services through APIs
  8. 8.  Maintain visibility and auditability of activity in your application infrastructure  Assess how application and infrastructure performance impact end-user experience  Gain insight into the root causes of problems or unexpected changes  Support services that must be available 24/7 as a result of continuous integration/ continuous delivery  Create alerts based on thresholds you define Logging and Monitoring Capture, categorize, and analyze data and logs generated by applications and infrastructure
  9. 9.  Model and visualize your own custom release workflow  Automate deployments of new code  Improve developer productivity and deliver updates faster  Find and address bugs quicker with more frequent and comprehensive testing  Store anything from source code to binaries using existing Git tools Continuous Integration and Continuous Delivery Rapidly and reliably build, test, and deploy your applications, while improving quality and reducing time to market.
  10. 10. Get started quickly and pay as you go Automate systems operations Scale without infrastructure constraints Improve visibility and security Leverage fully managed services Benefits of DevOps on AWS
  11. 11. Automated Workflow with Chef & AWS
  12. 12. Agenda  Brief Chef overview  Chef and AWS: your path to DevOps  Gannett’s path to AWS with Chef  Next steps  Q&A
  13. 13. Chef: Leader in the DevOps Market Born with the DevOps movement Provider to web leaders and the enterprise Understands DevOps success patterns Distilled these patterns into the Chef platform
  14. 14. Products by Chef Software Inc. Infrastructure Automation Application Automation Compliance Automation Workflow Visibility Compliance
  15. 15. Chef & AWS Integration  1 hour webinar, May 2016  https://www.chef.io/blog/event/webinar-chef-and-aws-your-path- to-devops/  End-to-end view of test-driven development with Chef & AWS  Showcase of AWS integration points
  16. 16. AWS Marketplace  Fast and convenient way to try Chef on your own w/ Enterprise features  Pay-as-you-go – Per hour billing for annual Chef server licensing – unique to AWS – $0.008 per node, per hour  Flexible consumption pricing – Retired license-pack model – Billed only for the nodes in use
  17. 17. Chef Provisioning for AWS  Provides convergent test & repair resources for managing AWS objects require 'chef/provisioning/aws_driver' with_driver 'aws::eu-west-1' aws_vpc 'test-vpc' do cidr_block '10.0.0.0/24' internet_gateway true end aws_route_table 'ref-public1' do vpc 'test-vpc' routes '0.0.0.0/0' => :internet_gateway end aws_s3_bucket 'name' do enable_website_hosting true options({ :acl => 'private' }) website_options :index_document => { :suffix => 'index.html' }, :error_document => { :key => 'not_found.html' } end
  18. 18. Chef Provisioning for AWS Provides convergent test & repair resources for managing AWS objects  Amazon EC2 instances  Security groups  EBS volumes  Elastic IP addresses  Autoscaling groups  Launch configs  Key pairs  Amazon VPC  VPC options (subnets, peering, routes, acl’s, etc)  Elastic load balancers  IAM roles  IAM instance profiles  Amazon S3 buckets  Amazon RDS instances  Amazon Route53  SNS topics  SQS queues  ElasticSearch domains  Amazon CloudWatch alarms  and more https://docs.chef.io/release/devkit/provisioning_aws.html
  19. 19. Chef and AWS – Provisioning Frameworks  Chef Provisioning  AWS CloudFormation  Terraform  Use your own, but account for bootstrapping necessities – https://docs.chef.io/install_bootstrap.html
  20. 20. Chef manages change across the AWS development pipeline Chef Compliance Available via AWS Marketplace ChefDK (test-kitchen) Open Source & Generally Available Chef Automate Available via Chef Software, Inc. Chef Server Available via AWS Marketplace Chef Compliance Available via AWS Marketplace Scan for Compliance Build & Test Locally Build & Test CI/CD Remediate Verify
  21. 21. Gannett with Chef & AWS
  22. 22.  National and Local Newspaper and Media company  National brand USATODAY  108 media companies in 33 states
  23. 23. Chef Pipeline Tools at Gannett  Enterprise Chef Server – all users share one org  Private Supermarket – CI keeps supermarket in sync with chef-server  Jenkins CI Server – the only way to publish cookbooks at Gannett  Private gems repository on Artifactory  Amazon EC2 AMIs available for CI testing  Vagrant Images available for local testing  Packer – for publishing and storing images  Scalr – Cloud Management provider with governance
  24. 24. What are We Testing?  Foodcritic – Chef linting, we fail on everything except FC005: Avoid repetition of resource declarations  Rubocop – Ruby linting, we exclude our tests and set max line length 160  Chefspec – Unit testing, target 100% coverage with accurate context and platforms  Serverspec – Integration testing, expected end state and audit for best practices
  25. 25. Our Internal Tool Chain  Rake – shared rakefile for common understanding of how to test and parallelize kitchen suites  Kitchen-test-helper – cookbook for storing node attributes and mocking databags from kitchen attributes in serverspec • https://supermarket.chef.io/cookbooks/kitchen-test-helper  Chef-Skeleton – built on the chef generate cookbook command  check_pr_versions – validate metadata version bump, changelog entry and jira tickets in commits  terminate-orphans – lambda script to remove untagged instances and leftovers from failed kitchen runs https://github.com/GannettDigital/chefconf2016
  26. 26. The Gannett Workflow Cookbook Pipeline Application Pipeline Image Pipeline Github repo with packer scripts and config Jenkins kicks off Packer builds from ISO on repo changes Packer runs chef-zero to configure image Packer import image to Amazon EC2 Use the Scalr API to publish images Create instances in Scalr with the new images Validate existing cookbooks can converge on the new image Test with remote serverspec from Jenkins and promote images on success Create feature branch/repo in Github Develop locally using vagrant images Push branch to Github and create pull request to master Jenkins kicks off testing for all PRs Peer review of successful test and merge Jenkins tests changes to master Publish to internal supermarket Publish to chef- server Create feature branch/repo in Github Develop locally Push branch to Github and create pull request to master Jenkins kicks off testing for all PRs Peer review of successful test and merge Jenkins publishes to Artifactory and kicks off development deployment Remove old instances and create new Amazon EC2 instances and deploy with Chef Validate application and move to staging or production environments
  27. 27. A Path to DevOps  Test-Driven Development – Infrastructure is code. Your code should be tested. – Verify your infrastructure works as intended – Accept contributions with confidence – Test-kitchen provides a rapid feedback cycle – Critical component in a continuous delivery pipeline  In-depth coverage – https://www.chef.io/blog/event/webinar-chef-and-aws-your-path-to-devops/  Try a tutorial for yourself at LearnChef  http://learn.chef.io
  28. 28. Q&A

×