1. Continuous Deployment
Pierre Gilot
Solutions Architect, Amazon Web Services

2. CONTINUOUS
INTEGRATION

3. DEVELOPER

4. SOURCE CODE
REPOSITORY

5. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER

6. SOURCE CODE
REPOSITORY
CONTINUOUS
INTEGRATION SERVER
PROJECT MANAGEMENT
SERVER

7. SOURCE CODE
REPOSITORY
CONTINUOUS
INTEGRATION SERVER
PROJECT MANAGEMENT
SERVER
PICK
TASKS

8. SOURCE CODE
REPOSITORY
SUBMIT
CODE
CONTINUOUS
INTEGRATION SERVER
PROJECT MANAGEMENT
SERVER

9. SOURCE CODE
REPOSITORY
CONTINUOUS
INTEGRATION SERVER
SCHEDULE
BUILD
PROJECT MANAGEMENT
SERVER

10. SOURCE CODE
REPOSITORY
CONTINUOUS
INTEGRATION SERVER
RECURRENT
BUILDS
PROJECT MANAGEMENT
SERVER

11. SOURCE CODE
REPOSITORY
CONTINUOUS
INTEGRATION SERVER
CODE
FETCH
PROJECT MANAGEMENT
SERVER

12. SOURCE CODE
REPOSITORY
CONTINUOUS
INTEGRATION SERVER
CODE QUALITY
TESTS
PROJECT MANAGEMENT
SERVER
TEST
RESULTS

13. SOURCE CODE
REPOSITORY
CONTINUOUS
INTEGRATION SERVER
BUILD OUTPUT
PROJECT MANAGEMENT
SERVER

14. SOURCE CODE
REPOSITORY
DOCS
CONTINUOUS
INTEGRATION SERVER
BINARIES
& PACKAGES
PROJECT MANAGEMENT
SERVER

16. SOURCE CODE
REPOSITORY
BUILDS
CONTINUOUS
INTEGRATION SERVER
DNS
PROJECT
MANAGEMENT SERVER

17. PAIN POINTS
•
•
•
•
•
UNIT TESTS INCOMPLETE
MOCK TESTS MAINTENANCE
EXPENSIVE TEST ENVIRONMENT
TEST ENVIRONMENT ≠ PRODUCTION
DEPLOYMENT CYCLES

18. ON-DEMAND
ELASTIC
PAY AS YOU GO

19. =
PROGRAMMABLE
PLATFORM

23. IF YOU CAN PROGRAM IT
YOU CAN AUTOMATE IT

28. HTTP://SORCERY.SMUGMUG.COM/

29. AWS
CLOUDFORMATION
STACK-BASED DEPLOYMENT
SERVICE

30. CLOUDFORMATION
TEMPLATE

32. {
"Description" : "Create RDS with username and password",
"Resources" : {
"MyDB" : {
"Type" : "AWS::RDS::DBInstance",
"Properties" : {
"AllocatedStorage" : "500",
"DBInstanceClass" : "db.m1.small",
"Engine" : "MySQL",
"EngineVersion" : "5.5",
"MasterUsername" : "MyName",
"MasterUserPassword" : "MyPassword"
}
}
}
}

33. "AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"mysql"
: [],
"mysql-server" : [],
"httpd"
: [],
"php"
: [],
"php-mysql"
: []
}
},
"sources" : {
"/var/www/html" :
"https://s3.amazonaws.com/my-builds/build-v4.zip"
}
}

34. {
"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2
KeyPair to enable SSH
access to the instance",
"Type" : "String"
}
},
}

35. PROCEDURAL
DEFINITION
KNOWN
CONFIGURATION
Create it programmatically
Store stack configuration in
source control
CLOUDFORMATION
TEMPLATE
PARAMETER
DRIVEN
Dynamic and user-driven
templates
COLLABORATION
Share templates with ease
as just files

37. APPLICATION
VERSIONS
+
INFRASTRUCTURE
VERSIONS

38. CLOUDFORMATION
TEMPLATE

39. TEST ENVIRONMENTS

40. 30,000 REQUESTS / SECOND
1 TB TRAFFIC / DAY

41. “…AWS seemed to be the best solution available to allow a
small, independent company to rapidly develop and test a
completely new infrastructure, and host it.
We also loved the flexibility that AWS allowed us, when
spinning up smaller test environments, for beta trials, QA,
localization, and during development. The low initial cost
was also crucial.”
Alex Evans, CTO

42. LOAD TESTING

48. USING AMAZON EC2 TO SIMULATE
2.4 MILLION PLAYERS

49. CONTINUOUS
DEPLOYMENT
SMALL, FREQUENT CHANGES
CONSTANTLY INTEGRATING INTO
PRODUCTION

51. KEY = ITERATION

52. ITERATION
=
MODIFY THE SYSTEM TO BETTER
MEET THE EXPECTATIONS OF
YOUR USERS

53. DEPLOYMENTS AT
AMAZON.COM
11.6s
1,079
10,000
30,000
Mean time
between
deployments
(weekday)
Max number of
deployments in a
single hour
Mean number of
hosts
simultaneously
receiving a
deployment
Max number of
hosts
simultaneously
receiving a
deployment

54. SOFTWARE DEPLOY
≠
PRODUCT LAUNCH

56. 1.5 BILLION PAGE VIEWS
$83 MILLION IN TRANSACTIONS
4.2 MILLION ITEMS SOLD
OCTOBER 2012

57. 30 DEPLOYS PER DAY
1 DEPLOY EVERY 20 MINUTES

58. "Production is truly the only place you
can validate your code."

63. A / B TESTING

65. 14 BILLION REQUESTS/MONTH
50 000 DATABASE UPDATES / SEC
NO CACHE

70. DATA-DRIVEN
ARCHITECTURES

71. METRICS @ETSY

72. COST-ORIENTED
ARCHITECTURES

73. PHP+APACHE+VARNISH
NGINX+NODEJS

75. # USERS
APPLICATION
GENERATED
REVENUE
APPLICATION
COST
$ COST

76. CONTINUOUS
DEPLOYMENT
CONTINUOUS
INTEGRATION

77. CONTINUOUS
DEPLOYMENT
=
CONTINUOUS
EXPERIMENTATION

78. CONTINUOUS
DEPLOYMENT
=
CONTINUOUS
IMPROVEMENT

79. INNOVATE

80. « Want to increase innovation?
Lower the cost of failure »
Joi Ito

81. SPEED AND AGILITY
“ON-PREMISE”
Experiment
Infrequently
Experiment
Often
Failure is
expensive
Fail quickly at
a low cost
Less
Innovation
More
Innovation

82. Continuous Deployment
at
Marco Pracucci

83. Platform for Creating
and Sharing Audio
8 people (4 developers)
2M unique listeners / month
3.5M audio tracks created
500 reqs / sec

84. we run 80% of our
traffic on AWS
about 60 EC2 instances and 50 TB on S3

85. we deploy frequently
[even on friday]
51 deploys in the last 30 days

86. how we deploy
the web app

87. Deploy Task

88. code repository
fab www.deploy
1. ssh
EC2 instances

89. code repository
2. checkout
fab www.deploy
EC2 instances

90. Checklist
code repository
- local cache warmup
- db schema migrations
fab www.deploy
- check consistency
3. warmup
EC2 instances

91. code repository
fab www.deploy
4. switch code
everything ok?
EC2 instances

92. Checklist
code repository
- iptables –dport 80 –j DROP
- ln –sfn /var/www/new /var/www/curr
fab www.deploy
- clear bytecode cache
4. switch code
- iptables –dport 80 –j ACCEPT
EC2 instances

93. generate report
fab www.deploy
notify team on IRC

94. which instances to deploy?
we do auto-scaling, they change dinamically

95. we exploited
DNS

96. Split infrastructure into roles
EC2 instances
api1
worker1
www1
api2
www3
role:
api
worker2
{
www2
{
{
api3
role:
web
role:
worker-web

97. Tag each instance with roles
[space separated roles for multi-role instances]

98. Map names and roles to DNS
https://github.com/airbnb/zonify
Creates Route 53 DNS entries
for all instances and roles

99. Address a single instance

100. Address instances by role

101. Easy to integrate with Fabric

102. Do you remember the starting point?
Internally does
fab –R web www.deploy

103. few good reasons
Easy, Fast, Robust

104. few good reasons
Easy, Fast, Robust
Warning: an attacker can easier get your instances IPs
(use an unguessable DNS zone name)

105. Thank you
www.spreaker.com/marco
@pracucci

106. Thank You
Pierre Gilot
Solutions Architect, Amazon Web Services