code.talks 2016 Hamburg - Plesk - AutoScaling WordPress with Docker & AWS - by Jan Löffler

Jan Löffler, CTO Plesk
AutoScaling WordPress with
Docker & AWS
code.talks 2016

Usain Bolt Katy Perry Snoop Dog LL Cool J
Kim Kardashian The Rolling Stones Sylvester Stallone Mötley Crüe
Beyoncé Justin Bieber Justin Timberlake Alanis Morissette

Usain Bolt Katy Perry Snoop Dog LL Cool J
Kim Kardashian Mötley Crüe
Beyoncé Alanis MorissetteJustin Bieber

TechCrunch Metro The New Yorker BBC America
MTV News The Official Star Wars Blog Sony Music Facebook Newsroom
Time.com CNN UPS Mozilla Blog

http://w3techs.com/technologies/history_overview/content_management/all/y

# WEBSITES WORLDWIDE
1.085.358.537
http://www.internetlivestats.com/total-number-of-websites/

# WEBSITES WORLDWIDE
1.085.358.692
http://www.internetlivestats.com/total-number-of-websites/

my-wordpress-site.com
MYSQL ServerLoad-Balancer

MYSQL ServerLoad-Balancer
File Storage

MYSQL Server
Load-Balancer
File Storage
CDN
MYSQL Server

doubled to 30%
http://www.rightscale.com/blog/cloud-industry-insights/cloud-computing-trends-2016-state-cloud-survey

IMMUTABLE STACKS
ELB myapp-v1
jans-wordpress.com
EC2
+ Docker
EC2
+ Docker
EC2
+ Docker
100%

IMMUTABLE STACKS
ELB myapp-v1
EC2
+ Docker
EC2
+ Docker
EC2
+ Docker
ELB myapp-v2
EC2
+ Docker
EC2
+ Docker
jans-wordpress.com
90% 10%

IMMUTABLE STACKS
ELB myapp-v1
EC2
+ Docker
EC2
+ Docker
EC2
+ Docker
ELB myapp-v2
EC2
+ Docker
EC2
+ Docker
jans-wordpress.com
0% 100%

APP INSTANCE
VM (EC2)
Basis Image (AMI)
Docker Container
Application
✓ VPC
✓ Security
Groups
✓ AutoScaling
✓ CloudWatch
✓ CloudFront
✓ Route53
✓ Docker Runtime
✓ Logging
✓ Monitoring
✓ Security config
✓ Debian
✓ Nginx
✓ php7-fpm
✓ NewRelic

AWS
API
Docker Registry
docker pull
docker push
AMI
THE WISH

$ accept-vpc-peering-connection$
$ allocate-address$
$ allocate-hosts$
$ assign-private-ip-addresses$
$ associate-address$
$ associate-dhcp-options$
$ associate-route-table$
$ attach-classic-link-vpc$
$ attach-internet-gateway$
$ attach-network-interface$
$ attach-volume$
$ attach-vpn-gateway$
$ authorize-security-group-egress$
$ authorize-security-group-ingress$
$ bundle-instance$
$ cancel-bundle-task$
$ cancel-conversion-task$
$ cancel-export-task$
$ cancel-import-task$
$ cancel-reserved-instances-listing$
$ cancel-spot-fleet-requests$
$ cancel-spot-instance-requests$
$ confirm-product-instance$
$ copy-image$
$ copy-snapshot$
$ create-customer-gateway$
$ create-dhcp-options$
$ create-flow-logs$
$ create-image$
$ create-instance-export-task$
$ create-internet-gateway$
$ create-key-pair$
$ create-nat-gateway$
$ create-network-acl$
$ create-network-acl-entry$
$ create-network-interface
$ create-reserved-instances-listing$
$ create-route$
$ create-route-table$
$ create-security-group$
$ create-snapshot$
$ create-spot-datafeed-subscription$
$ create-subnet$
$ create-tags$
$ create-volume$
$ create-vpc$
$ create-vpc-endpoint$
$ create-vpc-peering-connection$
$ create-vpn-connection$
$ create-vpn-connection-route$
$ create-vpn-gateway$
$ delete-customer-gateway$
$ delete-dhcp-options$
$ delete-flow-logs$
$ delete-internet-gateway$
$ delete-key-pair$
$ delete-nat-gateway$
$ delete-network-acl$
$ delete-network-acl-entry$
$ delete-network-interface$
$ delete-placement-group$
$ delete-route$
$ delete-route-table$
$ delete-security-group$
$ delete-snapshot$
$ delete-spot-datafeed-subscription$
$ delete-subnet$
$ delete-tags$
$ delete-volume$
$ delete-vpc$
$ delete-vpc-endpoints$
$ delete-vpc-peering-connection$
$ delete-vpn-connection$
$ delete-vpn-connection-route$
$ delete-vpn-gateway$
$ deregister-image$
$ describe-account-attributes$
$ describe-addresses$
$ describe-availability-zones$
$ describe-bundle-tasks$
$ describe-classic-link-instances$
$ describe-conversion-tasks$
$ describe-customer-gateways$
$ describe-dhcp-options$
$ describe-export-tasks$
$ describe-flow-logs$
$ describe-host-reservation-offerings$
$ describe-host-reservations$
$ describe-hosts$
$ describe-id-format$
$ describe-identity-id-format$
$ describe-image-attribute$
$ describe-images$
$ describe-import-image-tasks$
$ describe-import-snapshot-tasks$
$ describe-instance-attribute$
$ describe-instance-status$
$ describe-instances$
$ describe-internet-gateways$
$ describe-key-pairs$
$ describe-moving-addresses$
$ describe-nat-gateways$
$ describe-network-acls$
$ describe-network-interface-attribute$
$ describe-network-interfaces$
$ describe-placement-groups$
$ describe-prefix-lists$
$ describe-regions$
$ describe-reserved-
modifications$
$ describe-route-tab
$ describe-scheduled
$ describe-security-
$ describe-snapshot-
$ describe-snapshots
$ describe-spot-data
$ describe-spot-flee
$ describe-spot-inst
$ describe-spot-pric
$ describe-stale-sec
$ describe-subnets$
$ describe-tags$
$ describe-volume-at
$ describe-volume-st
$ describe-volumes$
$ describe-vpc-attri
$ describe-vpc-class
$ describe-vpc-endpo
$ describe-vpc-peeri
$ describe-vpcs$
$ describe-vpn-conne
$ describe-vpn-gatew
$ detach-classic-lin
$ detach-internet-ga
$ detach-network-int
$ detach-volume$
210 calls for EC2 only

$ accept-vpc-peering-connection$
$ allocate-address$
$ allocate-hosts$
$ assign-private-ip-addresses$
$ associate-address$
$ associate-dhcp-options$
$ associate-route-table$
$ attach-classic-link-vpc$
$ attach-internet-gateway$
$ attach-network-interface$
$ attach-volume$
$ attach-vpn-gateway$
$ authorize-security-group-egress$
$ authorize-security-group-ingress$
$ bundle-instance$
$ cancel-bundle-task$
$ cancel-conversion-task$
$ cancel-export-task$
$ cancel-import-task$
$ cancel-reserved-instances-listing$
$ cancel-spot-fleet-requests$
$ cancel-spot-instance-requests$
$ confirm-product-instance$
$ copy-image$
$ copy-snapshot$
$ create-customer-gateway$
$ create-dhcp-options$
$ create-flow-logs$
$ create-image$
$ create-instance-export-task$
$ create-internet-gateway$
$ create-key-pair$
$ create-nat-gateway$
$ create-network-acl$
$ create-network-acl-entry$
$ create-network-interface
$ create-reserved-instances-listing$
$ create-route$
$ create-route-table$
$ create-security-group$
$ create-snapshot$
$ create-spot-datafeed-subscription$
$ create-subnet$
$ create-tags$
$ create-volume$
$ create-vpc$
$ create-vpc-endpoint$
$ create-vpc-peering-connection$
$ create-vpn-connection$
$ create-vpn-connection-route$
$ create-vpn-gateway$
$ delete-customer-gateway$
$ delete-dhcp-options$
$ delete-flow-logs$
$ delete-internet-gateway$
$ delete-key-pair$
$ delete-nat-gateway$
$ delete-network-acl$
$ delete-network-acl-entry$
$ delete-network-interface$
$ delete-placement-group$
$ delete-route$
$ delete-route-table$
$ delete-security-group$
$ delete-snapshot$
$ delete-spot-datafeed-subscription$
$ delete-subnet$
$ delete-tags$
$ delete-volume$
$ delete-vpc$
$ delete-vpc-endpoints$
$ delete-vpc-peering-connection$
$ delete-vpn-connection$
$ delete-vpn-connection-route$
$ delete-vpn-gateway$
$ deregister-image$
$ describe-account-attributes$
$ describe-addresses$
$ describe-availability-zones$
$ describe-bundle-tasks$
$ describe-classic-link-instances$
$ describe-conversion-tasks$
$ describe-customer-gateways$
$ describe-dhcp-options$
$ describe-export-tasks$
$ describe-flow-logs$
$ describe-host-reservation-offerings$
$ describe-host-reservations$
$ describe-hosts$
$ describe-id-format$
$ describe-identity-id-format$
$ describe-image-attribute$
$ describe-images$
$ describe-import-image-tasks$
$ describe-import-snapshot-tasks$
$ describe-instance-attribute$
$ describe-instance-status$
$ describe-instances$
$ describe-internet-gateways$
$ describe-key-pairs$
$ describe-moving-addresses$
$ describe-nat-gateways$
$ describe-network-acls$
$ describe-network-interface-attribute$
$ describe-network-interfaces$
$ describe-placement-groups$
$ describe-prefix-lists$
$ describe-regions$
modifications$
$ describe-route-tab
$ describe-snapshot-
$ describe-snapshots
$ describe-spot-data
$ describe-spot-inst
$ describe-spot-pric
$ describe-stale-sec
$ describe-subnets$
$ describe-tags$
$ describe-volume-at
$ describe-volume-st
$ describe-volumes$
$ describe-vpc-attri
$ describe-vpc-peeri
$ describe-vpcs$
$ describe-vpn-conne
$ describe-vpn-gatew
$ detach-classic-lin
$ detach-internet-ga
$ detach-network-int
$ detach-volume$
210 calls for EC2 only
S3
ELB
RDS
CloudFrontCloudWatch
AutoScaling
Route53
Events
IAM
S3api
SNS

https://github.com/plesk/wordpress-aws-scaler

AWS
manage-
wordpress.sh
Docker Registry
docker pull
docker push
AMI

Too quick? How does it work internally?
Install AWS CLI (https://docs.aws.amazon.com/cli/latest/userguide/installing.html)
$ sudo pip install awscli
$ aws configure
$ aws --version
Clone WordPress AWS Scaler git repo (https://github.com/plesk/wordpress-aws-scaler)
$ git clone https://github.com/plesk/wordpress-aws-scaler.git
$ cd wordpress-aws-scaler
Build Docker Image and Upload to Docker Hub (https://hub.docker.com/explore/)
$ docker build -t janloeffler/wordpress-aws-scaler:latest .
$ docker push janloeffler/wordpress-aws-scaler:latest

List existing AWS resources
List existing AWS Resources
$ aws ec2 describe-vps
$ aws ec2 describe-security-groups
$ aws ec2 describe-instances
$ aws rds describe-db-instances
[...]
Do this for all AWS components
$ aws autoscaling
$ aws ec2
$ aws cloudfront
$ aws cloudwatch
$ aws elb
$ aws rds
$ aws s3
$ aws route53
$ aws events

Create new resources
Create new AWS Resources
$ aws ec2 create-vpc
$ aws ec2 create-security-group
[...]
Do this for all AWS components
$ aws autoscaling
$ aws ec2
$ aws cloudfront
$ aws cloudwatch
$ aws elb
$ aws rds
$ aws s3
$ aws route53
$ aws events

Create new Launch Configuration
Generate EC2-User-Data Script to run directly after booting EC2 VM
$ cat >ec2-user-data.sh <<EOL
#!/bin/bash
docker pull janloeffler/wordpress-aws-scaler:latest
docker run -d -p 80:80 -p 443:443 -e WORDPRESS_DB_HOST='${DB}' -e
WORDPRESS_DB_USER='${DB_USERNAME}' -e WORDPRESS_DB_PASSWORD='${DB_PASSWORD}' -e
WORDPRESS_DB_NAME='${DB_NAME}' -e WORDPRESS_DB_PREFIX='${WORDPRESS_DB_PREFIX}' -e
WORDPRESS_URL='http://${ELB}' -e WORDPRESS_TITLE='${WORDPRESS_TITLE}' -e
WORDPRESS_USER_EMAIL='${WORDPRESS_USER_EMAIL}' -e NEWRELIC_KEY='${NEWRELIC_KEY}' -e
NEWRELIC_NAME='${NEWRELIC_NAME}' -e S3_KEY='${IAM_USER_KEY}' -e
S3_SECRET='${IAM_USER_SECRET}' -e S3_BUCKET='${S3_BUCKET_NAME}' -e
S3_BUCKET_URL='${S3_URL}' -it janloeffler/wordpress-aws-scaler:latest
EOL
Create new Launch Configuration for AutoScaling
$ aws autoscaling create-launch-configuration --launch-configuration-name $LC_NAME --
image-id $AMI --instance-type $INSTANCE_TYPE --key-name $KEYNAME --security-groups
$SEC_GROUP_ID --user-data file://ec2-user-data.sh

You must love JSON !!! ;-)
{
"Reservations": [
{
"OwnerId": "699328319947",
"ReservationId": "r-041ac13209e0d4eef",
"Groups": [],
"RequesterId": "226008221399",
"Instances": [
{
"Monitoring": {
"State": "enabled"
},
"PublicDnsName": "ec2-52-210-200-157.eu-west-1.compute.amazonaws.com",
"State": {
"Code": 16,
"Name": "running"
},
"EbsOptimized": false,
"LaunchTime": "2016-09-05T12:19:54.000Z",
"PublicIpAddress": "52.210.200.157",
"PrivateIpAddress": "172.31.25.83",
"ProductCodes": [
{
"ProductCodeId": "4jvb72q6a56js2x7jzd24jar5",
"ProductCodeType": "marketplace"
}
],
"VpcId": "vpc-fffbe19a",
"StateTransitionReason": "",
"InstanceId": "i-0ea3384a24a3c6aba",
"ImageId": "ami-64385917",
"PrivateDnsName": "ip-172-31-25-83.eu-west-1.compute.internal",
"KeyName": "Marko Heijnen",
"SecurityGroups": [
{
"GroupName": "WordPress-Docker-Test",
"GroupId": "sg-1b8dde7c"
}
],
"ClientToken": "70aa81e1-10a3-4b83-acc7-62702015d06e_subnet-cff90cb9_1",
"SubnetId": "subnet-cff90cb9",
"InstanceType": "m3.medium",
"NetworkInterfaces": [
{
"Status": "in-use",
"MacAddress": "06:c0:22:65:b4:ed",
"SourceDestCheck": true,
"VpcId": "vpc-fffbe19a",
"Description": "",
"Association": {
"PublicIp": "52.210.200.157",
"IpOwnerId": "amazon"
},
"NetworkInterfaceId": "eni-2f4efa63",
"PrivateIpAddresses": [
{
"Association": {
"PublicIp": "52.210.200.157",
"IpOwnerId": "amazon"
},
"Primary": true,
"PrivateIpAddress": "172.31.25.83"
}
],
"Attachment": {
"Status": "attached",
"DeviceIndex": 0,
"DeleteOnTermination": true,
"AttachmentId": "eni-attach-7f9031a0",
"AttachTime": "2016-09-05T12:19:54.000Z"
},
[…]

WP CUP – RACE DAY 1
single-server
avg 3.86 sec for 947 hits
auto-scaling
avg 1.45 sec for 9.866 hits
Fastest Lap 216 ms
180 RPS < 3sec
Fastest Lap 99 ms
1.000 RPS < sec
No Caching!!!

Plesk Mission
“We simplify the lives of web
professionals.”

Docker
WordPress CDN
ELB &
AutoScaling
DB
Route53

code.talks 2016, Hamburg
“High traffic sites with WordPress,
Docker & AWS”
How to auto-scale WordPress on AWS
and make it fly!
Passion for
Web Development
Jan Löffler
CTO Plesk
● @jlsoft2
● jan@plesk.com
Marko Heijnen
WordPress Contributor
● @markoheijnen
● mheijnen@plesk.com

https://github.com/plesk/wordpress-aws-scaler
https://www.slideshare.net/plesk

“When you walk through your website architecture and you wonder
who could be so stupid to test it with 10.000 parallel requests per
second. But if you want to be successful and play in the first league, it
is important to think about it before your traffic increases”
Jan Löffler @jlsoft2

Web Development Key Tools – what web developers need and use
59
26,6% of all websites
worldwide
2,7% 2,2%
(high traffic sites)
70% of developers use git as primary
source management solution
Increasing usage by web agencies
38% of all online stores worldwide
runs on top of WordPress as plugin
developed by automattic
used by 30% of IT companies
increased from 13% to 30% in 12 months
further 35% plan introduction
Sources: w3techs.com, Rightscale, StackOverflow Survey 2015, 2016, Netcraft
311% growth
17.2% usage
26.8% growth
17.9% usage
14.9% growth
16% of all webservers
22% of all active sites
continuously growing,
while Apache is shrinking

WordPress Management
Free SSL everywhere
http2 50+ extensions
CLI
Automatic Updates
Site Migration
Free support
30+ languages
nginx
Server Security DNS

✔
Plesk Onyx: Continuous Delivery Deployment Pipeline (sample)
61
Jenkins runs as Docker
container managed by
Plesk
Plesk Multi Server
Extension installs
three environments
Apps managed
by Plesk via Docker

code.talks 2016 Hamburg - Plesk - AutoScaling WordPress with Docker & AWS - by Jan Löffler

Recommended

Recommended

More Related Content

What's hot

What's hot (6)

Similar to code.talks 2016 Hamburg - Plesk - AutoScaling WordPress with Docker & AWS - by Jan Löffler

Similar to code.talks 2016 Hamburg - Plesk - AutoScaling WordPress with Docker & AWS - by Jan Löffler (20)

More from Plesk

More from Plesk (16)

Recently uploaded

Recently uploaded (20)

code.talks 2016 Hamburg - Plesk - AutoScaling WordPress with Docker & AWS - by Jan Löffler