This document discusses building a DevOps pipeline on AWS using AWS services like CodeCommit, CodeBuild, CodePipeline, and OpsWorks for Puppet Enterprise. It provides an overview of configuring a release process with continuous integration and delivery. It also demonstrates setting up a sample pipeline to deploy Puppet code to an EC2 node managed by OpsWorks for Puppet Enterprise.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Building a DevOps Pipeline on AWS
D E V 3 2 6
Mark Rambow
SDM
AWS OpsWorks
Jens Bräuer
Sr. Software Engineer
AWS OpsWorks
Darko Meszaros
Solutions Architect
EMEA DevOps

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Management Tools
Integrated & interoperable

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
• DevOps
• Configuration as code
• Puppet
• Auto configuration of nodes
• DevOps pipeline

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Drive securely and fast

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DevOps pillars
• Informational silos
• Accept failures
• Frequent and small changes
• Automation, tooling
• Monitoring

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DevOps pillars
• Informational silos
• Accept failures
• Frequent and small changes
• Automation, tooling
• Monitoring

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CloudFormation
• Infrastructure as code for AWS Services
• Similar to Terraform but AWS optimized
• Templates in YAML or JSON
Amazon
VPC
AWS
Lambda
Amazon
EC2
Amazon
S3
Amazon
DynamoDB
Elastic Load
Balancing
Amazon
CloudFront
Amazon
CloudWatch
Elastic Load
Balancing
AWS
Elastic
Beanstalk
Amazon
ECS
Amazon
RDS
Amazon
Redshift
Amazon
VPC

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Configuration management
• Managing changes on hosts
• Run commands across all hosts
• Monitoring and insights for all running resources
• Enforcing compliance and policies
Files Packages Configuration Keys

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A simple class to configure a web server
### OpsWorks for Puppet Enterprise example for a static website
class role::nginx_webserver {
# All the profiles needed for a webserver
include profile::base
include profile::website
include profile::logrotate
}

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Configure the default website
class profile::website {
include 'nginx’
nginx::resource::server{'localhost':
use_default_location => false,
www_root => '/var/www/demo-website',
}

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
OpsWorks for Puppet Enterprise overview
• Managed experience
• Single server
• All in your account, full root access
• Undifferentiated Puppet experience
• Includes Puppet support through AWS
• Pay as you go, AWS Free Tier of 10 attached nodes

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
OpsWorks for Puppet Enterprise – Management
• Heavy lifting - Setup of Puppet Enterprise
• Maintenance by OpsWorks
• Continuous health checks
• Automated and manual backups
• Restoration and recreation
• Weekly security patches of the server
• Automatic upgrades

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
OpsWorks for Puppet Enterprise – Integration
• Integrated unattended node provisioning
• IAM for permission handling (Authorization)
• User data for
• Amazon EC2 Auto Scaling
• AWS CloudFormation
• Terraform and more
• StarterKit to get started quickly

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Identity and Access Management (IAM): ServiceRole
and InstanceProfile
• ServiceRole
• Assume Role gives permissions to AWS OpsWorks
• AWS OpsWorks can use other AWS Services on your behalf
• CloudFormation, Amazon Elastic Compute Cloud (Amazon EC2), Amazon Simple Storage Service
(Amazon S3), SSM
• InstanceProfile
• Applied to your server
• Lets your server use CloudFormation, Amazon S3, SSM

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Overview
1. Create AWS OpsWorks for Puppet Enterprise using the console wizard
2. !! Use the ControlRepoHTTPSURL as you R10K Remote
Building a DevOps Pipeline on AWS

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The AWS OpsWorks StarterKit
• Example and Getting-Started guide
• Simple NGinX Web-Server with static page
• Ready-to-use user data script
• First provisioned managed node in minutes
• Customizable through tags

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Puppet
Enterprise
Native Puppet node association
NodeDownload
agent install.sh
The operator establishes trust
between node and server manually
1. Modify
/etc/hosts and
/etc/puppet/puppet.conf
to resolve the Puppet Master
2. Sign the puppet agent
SSH to the Puppet Master and run:
puppet cert sign
hostname.example.com
systemctl restart puppet

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Node association of AWS OpsWorks
• API: aws opsworks-cm associate-node
• Security
• Permission granted through nodes IAM InstanceProfile
• Trust is established through OpsWorksCM API
• Approved by AWS security audit
• Benefit
• Can be integrated in a user data script (example is part of StarterKit)
• Auto Scaling Groups
• Amazon EC2 launch configurations
• AWS CloudFormation
• Terraform …

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Puppet
Enterprise
Puppet node association OpsWorksCM API
(1) Associate-node
(3) Run Agent
AWS OpsWorksCM API
AWS IAM
NodeNodeNodeNodeNodeNode
(2) Establish trust
OpsWorks establishes
trust between node and server
Check
permissions

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Associate node with user data
• Nodes are authenticated through AWS
• Unattended bootstrapping
• Only available on AWS OpsWorks
• Works for Amazon EC2 and on-premises

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Have your first Puppet Node
• Push your example Puppet code to your Puppet master
• Create your nodes instance profile
• Create and attach your first Amazon EC2 node to the Puppet master
• See a fully configured NGinX with an example website deployed
Building a DevOps Pipeline on AWS

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Check out your node
• Open the URL from Amazon EC2 console
• Open the Puppet Enterprise UI from AWS OpsWorks console
• View results of Puppet agent run

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Release processes have four major phases
Source Build Test Production
• Check-in source
code such as .java
files.
• Peer review new
code
• Compile code
• Unit tests
• Style checkers
• Code metrics
• Create container
images
• Integration with
other systems
• Load testing
• UI tests
• Penetration testing
• Deployment to
production
environments

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Release processes levels
Continuous integration
Continuous delivery
Continuous deployment
Source Build Test Production

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Code Services
AWS CodePipeline AWS CodeCommit AWS CodeBuildAWS CodeDeployAWS CodeStar

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Code Services
Software Release Steps
Source Build Test Production

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CodeCommit
AWS Code Services
Software Release Steps
Source Build Test Production

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CodeBuild
AWS Code Services
Software Release Steps:
Source Build Test Production

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CodeBuild
puppet parser validate
puppet-lint
rspec-puppet
...
AWS Code Services
Software Release Steps
Source Build Test Production

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Code Services
Software Release Steps
AWS CodeBuild
puppet code deploy
Source Build Test Production

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Source Build Test Production
AWS CodeCommit
AWS CodeBuild
AWS CodePipeline
AWS CodeBuild
AWS Code Services
Software Release Steps

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Component/service overview
Nodes
AWS Cloud

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Sneak peak

47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Steps overview
• Store your PuppetAccessToken in Systems Manager Parameter Store
• Store buildspec files in control-repo-example/aws
• Push your buildspec to your control-repo
• Launch a AWS CloudFormation stack from template provided
• Go to CodePipeline console and view progression
• Approve build
Building a DevOps Pipeline on AWS

49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Enjoy
• Access Puppet console
• Trigger a puppet-run on your node
• Access attached node in the browser
• See your change
Building a DevOps Pipeline on AWS

50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Clean-up
• Delete contents of Amazon S3 bucket
• <stackname>-pipelineartifactstorebucket-<randomhash>
• Delete AWS CloudFormation stack
• Be proud of yourself

51. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nick Alteen
Darko Meszaros
Jens Bräuer
Mark Rambow

52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.