With AWS, organizations now have the ability to develop and run their applications with speed and flexibility like never before. Working with an infrastructure that can be 100% API-driven enables organizations to use lean methodologies and realize these benefits. In this session, we will explore some key concepts and design patterns for continuous deployment and continuous integration, two elements of lean application and infrastructure development. We will look at several use cases where IT organizations leveraged AWS to rapidly develop and iterate on applications for scale, high availability and cost optimization. Speaker: Adrian White, Solutions Architect, Amazon Web Services

1. AWS Government, Education, &
Nonprofits Symposium
Canberra, Australia | May 6, 2015
Continuous Integration and Deployment Best Practices
on AWS
Adrian White
Solutions Architect
Amazon Web Services

2. What are we covering today?
• Consistency through the development, test and
release lifecycle
• Improve quality over time
• Increase velocity of application change
• AWS deployment and management approaches
• What does deployment look like in the future?

3. Task tracker
Continuous integration /
Automated testing /
Release management
Artifact repository
Source code
management
Deployment service
AWS OpsWorks
Elastic Beanstalk
AWS
CloudFormation
AWS
CodeDeploy
Local
development
Application lifecycle management workflow

4. AWS Code and Deployment Tools
MonitorProvisionDeployTestBuildCode
Elastic Beanstalk
OpsWorks
Cloud
Watch
Cloud
Formation
Code
Deploy
Code
Commit
Code
Pipeline

5. Local development challenges
• Source code management design
• “But it works on my machine”
• Portable development environments
• Distributed teams work on tasks in parallel

6. Source code management features
• Fast and easy branching
• Pull requests for distributed development workflows
• Code review
• Audit, logging, security

7. Feature branching, Gitflow and Pull requests
Inspired by https://www.atlassian.com/git/tutorials/comparing-workflows/feature-branch-workflow/
Master Develop Release Feature Hotfix

8. Container image factories
(for consistency)
…

9. CodeCommit
• Private Git on AWS
• Massive scaled version controlled projects
• High service availability and resiliency
• Encrypted at rest
• Pay as you go pricing
• Import from SVN, Git, Microsoft TFS
• Use IAM to control access to repositories

10. CodeCommit workflow
git clone
create repository Create repository
Receive clone request
Sync local / remote repos
modify local files
git add / commit / push
Receive push request
Update remote repo
git pull
Receive push request
Update remote repo
List repos, list branches
Display response
Receive requests
Send responses

11. Why use a release automation service?
Automate
workflow
Release
quickly
Ensure
quality

12. CodePipeline
• Customizable workflow engine
• Integrate with partner and custom systems
• Visual editor and status
Continuous delivery and release automation, just like Amazon
Build
1) Build
2) Unit test
1) Deploy
2) UI test
Source Beta Production
1) Deploy
2) Load test
Gamma
1) Deploy region1
2) Deploy region2
3) Deploy region3

13. How do you ship application changes?
• Deployment approaches
– In place vs discrete stacks
• Where is state in the system?
– Stateless vs stateful application components
• Frequency and speed of change

14. Why use a deployment service?
Automate
deployments
Manage
complexity
Avoid
downtime

15. CodeDeploy
• Scale from 1 instance to thousands
• Deploy without downtime
• Centralize deployment control and monitoring
Staging
CodeDeployv1, v2, v3
Production
Dev
Coordinate automated deployments, just like Amazon
Application
revisions
Deployment groups

16. Step 1: Package your application (with an AppSpec
file)
version: 0.0
os: linux
files:
- source: chef/
destination: /etc/chef/codedeploy
- source: target/hello.war
destination: /var/lib/tomcat6/webapps
hooks:
ApplicationStop:
- location: deploy_hooks/stop-tomcat.sh
BeforeInstall:
- location: deploy_hooks/install-chef.sh
AfterInstall:
- location: deploy_hooks/librarian-install.sh
ApplicationStart:
- location: deploy_hooks/chef-solo.sh
ValidateService:
- location: deploy_hooks/verify_service.sh

17. Step 1: Package your application (with an AppSpec
file)
version: 0.0
os: linux
files:
- source: chef/
destination: /etc/chef/codedeploy
- source: target/hello.war
destination: /var/lib/tomcat6/webapps
hooks:
ApplicationStop:
- location: deploy_hooks/stop-tomcat.sh
BeforeInstall:
- location: deploy_hooks/install-chef.sh
AfterInstall:
- location: deploy_hooks/librarian-install.sh
ApplicationStart:
- location: deploy_hooks/chef-solo.sh
ValidateService:
- location: deploy_hooks/verify_service.sh

18. Step 1: Package your application (with an AppSpec
file)
version: 0.0
os: linux
files:
- source: chef/
destination: /etc/chef/codedeploy
- source: target/hello.war
destination: /var/lib/tomcat6/webapps
hooks:
ApplicationStop:
- location: deploy_hooks/stop-tomcat.sh
BeforeInstall:
- location: deploy_hooks/install-chef.sh
AfterInstall:
- location: deploy_hooks/librarian-install.sh
ApplicationStart:
- location: deploy_hooks/chef-solo.sh
ValidateService:
- location: deploy_hooks/verify_service.sh

19. Step 2: Set up your target environments
Agent Agent Agent
Staging
Agent Agent
Agent Agent
Agent
Agent
Production
Deployment groupDeployment group
Group instances by:
• Auto Scaling group
• Amazon EC2 tag
• On-premises tag

20. Step 3: Deploy!
aws deploy create-deployment
--application-name MyApp
--deployment-group-name TargetGroup
--s3-location bucket=MyBucket,key=MyApp.zip
AWS CLI & SDKs
AWS Console
CI / CD Partners
GitHub

21. Deployment config – Choose speed
v2 v1 v1 v1 v1 v1 v1 v1
v2 v2 v2 v2 v1 v1 v1 v1
v2 v2 v2 v2 v2 v2 v2 v2
One-at-a-time
Half-at-a-time
All-at-once

22. Rolling update – Deploy without downtime
v1v1 v1
Load Balancer

23. Rolling update – Deploy without downtime
v1v2 v1
Load Balancer

24. Rolling update – Deploy without downtime
v2v2 v1
Load Balancer

25. Rolling update – Deploy without downtime
v2v2v2
Load Balancer

26. Rolling update – Deploy without downtime
v2v2 v2
Load Balancer

27. Health Tracking – Catch deployment problems
v2v2 v2
Load Balancer

28. Health tracking – Catch deployment problems
v3 v2 v2Stop
Load Balancer

29. Health tracking – Catch deployment problems
v2v2 v2
Load Balancer
Rollback

30. Product integrations

31. Demo: CodeDeploy & Atlassian Bamboo

32. Shipping artifacts to new environments
• What if we can quickly and easily build new environments
every time?
• CloudFormation
– Deploying AMIs
– Deploying containers
• CodeDeploy to manage discrete application versions

33. Shipping artifacts – Discrete environments
Tasks for AWS
AWS CloudFormation
Discrete stacks

34. Shipping artifacts – Immutability via containers
ECS cluster
Amazon EC2
Container Service
Immutable infrastructure with Docker and
Amazon EC2 Container Service (ECS)
Tasks for AWS

35. Shipping artifacts – Immutability via containers
Tasks for AWS
ECS now supports ELB, health checks,
scale-up and scale-down and update
management
ECS Task
Definition
ECS cluster with tasks

36. ECS Task Definitions
{
"containerDefinitions": [
{
"name": "wordpress",
"links": [
"mysql"
],
"image": "wordpress",
"essential": true,
"portMappings": [
{
"containerPort": 80,
"hostPort": 80
}
],
"memory": 500,
"cpu": 10
},
{
"environment": [
{
"name": "MYSQL_ROOT_PASSWORD",
"value": "password"
}
],
"name": "mysql",
"image": "mysql",
"cpu": 10,
"memory": 500,
"essential": true
}
],
"family": "hello_world"
}

37. CLASSIFICATION: INTERNAL
Cameron Gough, General Manager – Digital Delivery Centre
Andrew Dilenis, Head of Digital Systems
Moving to AWS – our story
Cameron.gough@auspost.com.au, @cam_gough
Andrew.Dilenis@auspost.com.au

38. © 2015 Australia Post
60 Applications
2.4M Registered Users
140M Visits Per year
250 Digital Natives
15 Agile teams
3 Tribes

39. © 2015 Australia Post
Demonstration

40. © 2015 Australia Post
The
Opportunity
Bundesarchiv, B 145 Bild-F038791-0035 / Schaack, Lothar / CC-BY-SA [CC BY-SA 3.0 de (http://creativecommons.org/licenses/by-sa/3.0/de/deed.en)], via Wikimedia Commons

41. © 2015 Australia Post
AWS Pilot
3 tier simple app
Time boxed
3 smart people
Allowed 8 weeks
No plan, just go

42. © 2015 Australia Post
What we found – the good
>95% time and cost saving
9 days to 15 minutes
Automation
New Way of Working
Quality
Quick entry

43. © 2015 Australia Post
What we found – gotchas
Design in cost controls
Security
Licencing
Re-architect
New thinking

44. © 2015 Australia Post
Approach
Start doing, stop planning
Agile
Rework = learning
Think devops
AWS help
Ask “Why?” – often
Be brave!

45. © 2015 Australia Post
From Pets…
…to Cattle

46. © 2015 Australia Post
Two Artefacts
repo automation-code configuration
artefact automation-code-2.3.0.tgz configuration-2.3.0-1.tgz
purpose Automation logic Configuration that describes the
stack and its environment
technologies • AWS CloudFormation for
infrastructure provisioning
• Puppet for Configuration
Management
• Python scripts for deployment
process
• Puppet Hiera
Automation
Config
2.3.0-1
Automation
Code
2.3.0

47. © 2015 Australia Post
Streamlined Processes
I want to deploy my application including
infrastructure changes
2.1 2.1

48. © 2015 Australia Post
Streamlined Processes
I want to deploy my application including
infrastructure changes
1.Code – commit, push and pull request
2.1 2.1

49. © 2015 Australia Post
Streamlined Processes
I want to deploy my application including
infrastructure changes
1.Code – commit, push and pull request
2.Update stack with new versions of artefacts
2.1 2.1

50. © 2015 Australia Post
Streamlined Processes
I want to deploy my application including
infrastructure changes
1.Code – commit, push and pull request
2.Update stack with new versions of artefacts
3.Scale up autoscale group (2 > 4)
2.1 2.1 2.2 2.2

51. © 2015 Australia Post
Streamlined Processes
I want to deploy my application including
infrastructure changes
1.Code – commit, push and pull request
2.Update stack with new versions of artefacts
3.Scale up autoscale group (2 > 4)
4.Scale down autoscale group (4 > 2)2.1 2.1 2.2 2.2

52. © 2015 Australia Post
Streamlined Processes
I want to deploy my application including
infrastructure changes
1.Code – commit, push and pull request
2.Update stack with new versions of artefacts
3.Scale up autoscale group (2 > 4)
4.Scale down autoscale group (4 > 2)
5.Done!
2.2 2.2

53. © 2015 Australia Post
Streamlined Processes
I want to deploy my application including
infrastructure changes
1.Code – commit, push and pull request
2.Update stack with new versions of artefacts
3.Scale up autoscale group (2 > 4)
4.Scale down autoscale group (4 > 2)
5.Done!
2.2 2.2
Exact same process for:

54. © 2015 Australia Post
Streamlined Processes
I want to deploy my application including
infrastructure changes
1.Code – commit, push and pull request
2.Update stack with new versions of artefacts
3.Scale up autoscale group (2 > 4)
4.Scale down autoscale group (4 > 2)
5.Done!
2.2 2.2
Exact same process for:

55. © 2015 Australia Post
Streamlined Processes
I want to deploy my application including
infrastructure changes
1.Code – commit, push and pull request
2.Update stack with new versions of artefacts
3.Scale up autoscale group (2 > 4)
4.Scale down autoscale group (4 > 2)
5.Done!
2.2 2.2
Exact same process for:
• Modifying application configuration

56. © 2015 Australia Post
Streamlined Processes
I want to deploy my application including
infrastructure changes
1.Code – commit, push and pull request
2.Update stack with new versions of artefacts
3.Scale up autoscale group (2 > 4)
4.Scale down autoscale group (4 > 2)
5.Done!
2.2 2.2
Exact same process for:
• Modifying application configuration
• Patching for security vulnerabilities

57. © 2015 Australia Post
Streamlined Processes
I want to deploy my application including
infrastructure changes
1.Code – commit, push and pull request
2.Update stack with new versions of artefacts
3.Scale up autoscale group (2 > 4)
4.Scale down autoscale group (4 > 2)
5.Done!
2.2 2.2
Exact same process for:
• Modifying application configuration
• Patching for security vulnerabilities
• Certificate rotation

58. © 2015 Australia Post
Streamlined Processes
I want to deploy my application including
infrastructure changes
1.Code – commit, push and pull request
2.Update stack with new versions of artefacts
3.Scale up autoscale group (2 > 4)
4.Scale down autoscale group (4 > 2)
5.Done!
2.2 2.2
Exact same process for:
• Modifying application configuration
• Patching for security vulnerabilities
• Certificate rotation
• Stack enhancements

59. © 2015 Australia Post
Provisioning, how it fits…

60. © 2015 Australia Post
Provisioning, how it fits…

61. © 2015 Australia Post
Provisioning, how it fits…

62. © 2015 Australia Post
Provisioning, how it fits…

63. © 2015 Australia Post
Provisioning, how it fits…

64. © 2015 Australia Post
Provisioning, how it fits…

65. © 2015 Australia Post
Provisioning, how it fits…

66. © 2015 Australia Post
Provisions “application stacks”

67. © 2015 Australia Post
Provisions “application stacks”
Web Stack
with Nginx

68. © 2015 Australia Post
Provisions “application stacks”
Web Stack
with Nginx
Web Stack with
Apache

69. © 2015 Australia Post
Provisions “application stacks”
Web Stack
with Nginx
Web Stack with
Apache
App Stack with
Tomcat/JRE
Amazon
S3
Dynamo DB

70. © 2015 Australia Post
Provisions “application stacks”
Web Stack
with Nginx
Web Stack with
Apache
Every Stack…
App Stack with
Tomcat/JRE
Amazon
S3
Dynamo DB

71. © 2015 Australia Post
Provisions “application stacks”
Web Stack
with Nginx
Web Stack with
Apache
Every Stack…
CloudWatch
Monitoring, metrics… Logging…
In-transit
encryption…
App Stack with
Tomcat/JRE
Amazon
S3
Dynamo DB

72. © 2015 Australia Post
Results
All apps in AWS
500+ full stacks per month
Consistent Environments
Patching
Variable spend
Teams self-serve
0
400
800
1200
1600
2013-11 2014-01 2014-03 2014-05 2014-07 2014-09 2014-11 2015-01
Production Stacks
Transient Stacks

73. © 2015 Australia Post
Demonstration

74. © 2015 Australia Post
Working Differently
Safe to experiment
Iterate and release more often
Micro-services
Focus on value creation

75. © 2015 Australia Post
Customer Outcomes
Better experiences,
delivered more often

76. For further information, please contact:
Thank you
© 2015 Australia Post
Cameron Gough, General Manager – Digital Delivery Centre
cameron.gough@auspost.com.au
@cam_gough
Andrew Dilenis, Head of Digital Systems
andrew.dilenis@auspost.com.au

77. The Future – what are you really deploying?
• Application bundles?
• AMIs?
• Docker containers?
• Lambda functions?
• Microservices architectures…

78. • Service-Oriented
Architecture (SOA)
• Everything gets a
service interface
• Primitives
• “Microservices”

80. Thousands of teams +
Microservices architecture +
Multiple environments +
Continuous delivery

81. = 50 million deployments a year
Thousands of teams +
Microservices architecture +
Multiple environments +
Continuous delivery

82. Where to go next…
• AWS Training & Certification
– http://aws.amazon.com/training/
• Deployment and Management at AWS
– http://aws.amazon.com/application-management/
• Code Management and Deployment
– https://aws.amazon.com/blogs/aws/code-management-and-
deployment/
• Amazon EC2 Container Service
– http://aws.amazon.com/ecs/

83. Thank You