Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

code.talks 2016 Hamburg - Plesk - AutoScaling WordPress with Docker & AWS - by Jan Löffler

307 views

Published on

26,6% of all websites worldwide are running on WordPress! Now that’s a lot of sites! For those of you who thought WordPress was still a CMS relegated to bloggers, think again.  38% of all online stores worldwide run wooCommerce (WordPress’ eCommerce plugin) on top of WordPress– yowzers!  But how do you scale WordPress for high traffic peaks?
In this talk, Marko Heijnen (Core Contributor of WordPress) and Jan Löffler (CTO at Plesk) will demonstrate how to scale WordPress on AWS and Docker using AutoScaling to make your apps and websites fly high!

Published in: Internet
  • Be the first to comment

code.talks 2016 Hamburg - Plesk - AutoScaling WordPress with Docker & AWS - by Jan Löffler

  1. 1. Jan Löffler, CTO Plesk AutoScaling WordPress with Docker & AWS code.talks 2016
  2. 2. Sebastian Vettel
  3. 3. Who uses WordPress?
  4. 4. Usain Bolt Katy Perry Snoop Dog LL Cool J Kim Kardashian The Rolling Stones Sylvester Stallone Mötley Crüe Beyoncé Justin Bieber Justin Timberlake Alanis Morissette
  5. 5. Usain Bolt Katy Perry Snoop Dog LL Cool J Kim Kardashian Mötley Crüe Beyoncé Alanis MorissetteJustin Bieber
  6. 6. TechCrunch Metro The New Yorker BBC America MTV News The Official Star Wars Blog Sony Music Facebook Newsroom Time.com CNN UPS Mozilla Blog
  7. 7. http://w3techs.com/technologies/history_overview/content_management/all/y
  8. 8. # WEBSITES WORLDWIDE 1.085.358.537 http://www.internetlivestats.com/total-number-of-websites/
  9. 9. # WEBSITES WORLDWIDE 1.085.358.692 http://www.internetlivestats.com/total-number-of-websites/
  10. 10. 38% of E-Shops
  11. 11. 26,6% 2,7% 2,2%
  12. 12. #SPEED
  13. 13. High Traffic Sites???
  14. 14. NEED FOR SPEED
  15. 15. my-wordpress-site.com
  16. 16. my-wordpress-site.com MYSQL ServerLoad-Balancer
  17. 17. my-wordpress-site.com MYSQL ServerLoad-Balancer File Storage
  18. 18. my-wordpress-site.com MYSQL Server Load-Balancer File Storage CDN MYSQL Server
  19. 19. Easy-peasy, isn’t it?
  20. 20. DEPLOYMENT
  21. 21. doubled to 30% http://www.rightscale.com/blog/cloud-industry-insights/cloud-computing-trends-2016-state-cloud-survey
  22. 22. my-wordpress-site
  23. 23. Modern Web-Development 2016
  24. 24. IMMUTABLE INFRASTRUCTURE
  25. 25. IMMUTABLE STACKS ELB myapp-v1 jans-wordpress.com EC2 + Docker EC2 + Docker EC2 + Docker 100%
  26. 26. IMMUTABLE STACKS ELB myapp-v1 EC2 + Docker EC2 + Docker EC2 + Docker ELB myapp-v2 EC2 + Docker EC2 + Docker jans-wordpress.com 90% 10%
  27. 27. IMMUTABLE STACKS ELB myapp-v1 EC2 + Docker EC2 + Docker EC2 + Docker ELB myapp-v2 EC2 + Docker EC2 + Docker jans-wordpress.com 0% 100%
  28. 28. APP INSTANCE VM (EC2) Basis Image (AMI) Docker Container Application ✓ VPC ✓ Security Groups ✓ AutoScaling ✓ CloudWatch ✓ CloudFront ✓ Route53 ✓ Docker Runtime ✓ Logging ✓ Monitoring ✓ Security config ✓ Debian ✓ Nginx ✓ php7-fpm ✓ NewRelic
  29. 29. AWS API Docker Registry docker pull docker push AMI THE WISH
  30. 30. THE REALITY
  31. 31. $ accept-vpc-peering-connection$ $ allocate-address$ $ allocate-hosts$ $ assign-private-ip-addresses$ $ associate-address$ $ associate-dhcp-options$ $ associate-route-table$ $ attach-classic-link-vpc$ $ attach-internet-gateway$ $ attach-network-interface$ $ attach-volume$ $ attach-vpn-gateway$ $ authorize-security-group-egress$ $ authorize-security-group-ingress$ $ bundle-instance$ $ cancel-bundle-task$ $ cancel-conversion-task$ $ cancel-export-task$ $ cancel-import-task$ $ cancel-reserved-instances-listing$ $ cancel-spot-fleet-requests$ $ cancel-spot-instance-requests$ $ confirm-product-instance$ $ copy-image$ $ copy-snapshot$ $ create-customer-gateway$ $ create-dhcp-options$ $ create-flow-logs$ $ create-image$ $ create-instance-export-task$ $ create-internet-gateway$ $ create-key-pair$ $ create-nat-gateway$ $ create-network-acl$ $ create-network-acl-entry$ $ create-network-interface $ create-reserved-instances-listing$ $ create-route$ $ create-route-table$ $ create-security-group$ $ create-snapshot$ $ create-spot-datafeed-subscription$ $ create-subnet$ $ create-tags$ $ create-volume$ $ create-vpc$ $ create-vpc-endpoint$ $ create-vpc-peering-connection$ $ create-vpn-connection$ $ create-vpn-connection-route$ $ create-vpn-gateway$ $ delete-customer-gateway$ $ delete-dhcp-options$ $ delete-flow-logs$ $ delete-internet-gateway$ $ delete-key-pair$ $ delete-nat-gateway$ $ delete-network-acl$ $ delete-network-acl-entry$ $ delete-network-interface$ $ delete-placement-group$ $ delete-route$ $ delete-route-table$ $ delete-security-group$ $ delete-snapshot$ $ delete-spot-datafeed-subscription$ $ delete-subnet$ $ delete-tags$ $ delete-volume$ $ delete-vpc$ $ delete-vpc-endpoints$ $ delete-vpc-peering-connection$ $ delete-vpn-connection$ $ delete-vpn-connection-route$ $ delete-vpn-gateway$ $ deregister-image$ $ describe-account-attributes$ $ describe-addresses$ $ describe-availability-zones$ $ describe-bundle-tasks$ $ describe-classic-link-instances$ $ describe-conversion-tasks$ $ describe-customer-gateways$ $ describe-dhcp-options$ $ describe-export-tasks$ $ describe-flow-logs$ $ describe-host-reservation-offerings$ $ describe-host-reservations$ $ describe-hosts$ $ describe-id-format$ $ describe-identity-id-format$ $ describe-image-attribute$ $ describe-images$ $ describe-import-image-tasks$ $ describe-import-snapshot-tasks$ $ describe-instance-attribute$ $ describe-instance-status$ $ describe-instances$ $ describe-internet-gateways$ $ describe-key-pairs$ $ describe-moving-addresses$ $ describe-nat-gateways$ $ describe-network-acls$ $ describe-network-interface-attribute$ $ describe-network-interfaces$ $ describe-placement-groups$ $ describe-prefix-lists$ $ describe-regions$ $ describe-reserved- $ describe-reserved- $ describe-reserved- modifications$ $ describe-reserved- $ describe-route-tab $ describe-scheduled $ describe-scheduled $ describe-security- $ describe-security- $ describe-snapshot- $ describe-snapshots $ describe-spot-data $ describe-spot-flee $ describe-spot-flee $ describe-spot-flee $ describe-spot-inst $ describe-spot-pric $ describe-stale-sec $ describe-subnets$ $ describe-tags$ $ describe-volume-at $ describe-volume-st $ describe-volumes$ $ describe-vpc-attri $ describe-vpc-class $ describe-vpc-class $ describe-vpc-endpo $ describe-vpc-endpo $ describe-vpc-peeri $ describe-vpcs$ $ describe-vpn-conne $ describe-vpn-gatew $ detach-classic-lin $ detach-internet-ga $ detach-network-int $ detach-volume$ 210 calls for EC2 only
  32. 32. $ accept-vpc-peering-connection$ $ allocate-address$ $ allocate-hosts$ $ assign-private-ip-addresses$ $ associate-address$ $ associate-dhcp-options$ $ associate-route-table$ $ attach-classic-link-vpc$ $ attach-internet-gateway$ $ attach-network-interface$ $ attach-volume$ $ attach-vpn-gateway$ $ authorize-security-group-egress$ $ authorize-security-group-ingress$ $ bundle-instance$ $ cancel-bundle-task$ $ cancel-conversion-task$ $ cancel-export-task$ $ cancel-import-task$ $ cancel-reserved-instances-listing$ $ cancel-spot-fleet-requests$ $ cancel-spot-instance-requests$ $ confirm-product-instance$ $ copy-image$ $ copy-snapshot$ $ create-customer-gateway$ $ create-dhcp-options$ $ create-flow-logs$ $ create-image$ $ create-instance-export-task$ $ create-internet-gateway$ $ create-key-pair$ $ create-nat-gateway$ $ create-network-acl$ $ create-network-acl-entry$ $ create-network-interface $ create-reserved-instances-listing$ $ create-route$ $ create-route-table$ $ create-security-group$ $ create-snapshot$ $ create-spot-datafeed-subscription$ $ create-subnet$ $ create-tags$ $ create-volume$ $ create-vpc$ $ create-vpc-endpoint$ $ create-vpc-peering-connection$ $ create-vpn-connection$ $ create-vpn-connection-route$ $ create-vpn-gateway$ $ delete-customer-gateway$ $ delete-dhcp-options$ $ delete-flow-logs$ $ delete-internet-gateway$ $ delete-key-pair$ $ delete-nat-gateway$ $ delete-network-acl$ $ delete-network-acl-entry$ $ delete-network-interface$ $ delete-placement-group$ $ delete-route$ $ delete-route-table$ $ delete-security-group$ $ delete-snapshot$ $ delete-spot-datafeed-subscription$ $ delete-subnet$ $ delete-tags$ $ delete-volume$ $ delete-vpc$ $ delete-vpc-endpoints$ $ delete-vpc-peering-connection$ $ delete-vpn-connection$ $ delete-vpn-connection-route$ $ delete-vpn-gateway$ $ deregister-image$ $ describe-account-attributes$ $ describe-addresses$ $ describe-availability-zones$ $ describe-bundle-tasks$ $ describe-classic-link-instances$ $ describe-conversion-tasks$ $ describe-customer-gateways$ $ describe-dhcp-options$ $ describe-export-tasks$ $ describe-flow-logs$ $ describe-host-reservation-offerings$ $ describe-host-reservations$ $ describe-hosts$ $ describe-id-format$ $ describe-identity-id-format$ $ describe-image-attribute$ $ describe-images$ $ describe-import-image-tasks$ $ describe-import-snapshot-tasks$ $ describe-instance-attribute$ $ describe-instance-status$ $ describe-instances$ $ describe-internet-gateways$ $ describe-key-pairs$ $ describe-moving-addresses$ $ describe-nat-gateways$ $ describe-network-acls$ $ describe-network-interface-attribute$ $ describe-network-interfaces$ $ describe-placement-groups$ $ describe-prefix-lists$ $ describe-regions$ $ describe-reserved- $ describe-reserved- $ describe-reserved- modifications$ $ describe-reserved- $ describe-route-tab $ describe-scheduled $ describe-scheduled $ describe-security- $ describe-security- $ describe-snapshot- $ describe-snapshots $ describe-spot-data $ describe-spot-flee $ describe-spot-flee $ describe-spot-flee $ describe-spot-inst $ describe-spot-pric $ describe-stale-sec $ describe-subnets$ $ describe-tags$ $ describe-volume-at $ describe-volume-st $ describe-volumes$ $ describe-vpc-attri $ describe-vpc-class $ describe-vpc-class $ describe-vpc-endpo $ describe-vpc-endpo $ describe-vpc-peeri $ describe-vpcs$ $ describe-vpn-conne $ describe-vpn-gatew $ detach-classic-lin $ detach-internet-ga $ detach-network-int $ detach-volume$ 210 calls for EC2 only S3 ELB RDS CloudFrontCloudWatch AutoScaling Route53 Events IAM S3api SNS
  33. 33. PLESK WORDPRESS AWS SCALER
  34. 34. https://github.com/plesk/wordpress-aws-scaler
  35. 35. AWS manage- wordpress.sh Docker Registry docker pull docker push AMI
  36. 36. PLESK WORDPRESS AWS SCALER
  37. 37. manage-wordpress.sh create
  38. 38. Too quick? How does it work internally? Install AWS CLI (https://docs.aws.amazon.com/cli/latest/userguide/installing.html) $ sudo pip install awscli $ aws configure $ aws --version Clone WordPress AWS Scaler git repo (https://github.com/plesk/wordpress-aws-scaler) $ git clone https://github.com/plesk/wordpress-aws-scaler.git $ cd wordpress-aws-scaler Build Docker Image and Upload to Docker Hub (https://hub.docker.com/explore/) $ docker build -t janloeffler/wordpress-aws-scaler:latest . $ docker push janloeffler/wordpress-aws-scaler:latest
  39. 39. List existing AWS resources List existing AWS Resources $ aws ec2 describe-vps $ aws ec2 describe-security-groups $ aws ec2 describe-instances $ aws rds describe-db-instances [...] Do this for all AWS components $ aws autoscaling $ aws ec2 $ aws cloudfront $ aws cloudwatch $ aws elb $ aws rds $ aws s3 $ aws route53 $ aws events
  40. 40. Create new resources Create new AWS Resources $ aws ec2 create-vpc $ aws ec2 create-security-group [...] Do this for all AWS components $ aws autoscaling $ aws ec2 $ aws cloudfront $ aws cloudwatch $ aws elb $ aws rds $ aws s3 $ aws route53 $ aws events
  41. 41. Create new Launch Configuration Generate EC2-User-Data Script to run directly after booting EC2 VM $ cat >ec2-user-data.sh <<EOL #!/bin/bash docker pull janloeffler/wordpress-aws-scaler:latest docker run -d -p 80:80 -p 443:443 -e WORDPRESS_DB_HOST='${DB}' -e WORDPRESS_DB_USER='${DB_USERNAME}' -e WORDPRESS_DB_PASSWORD='${DB_PASSWORD}' -e WORDPRESS_DB_NAME='${DB_NAME}' -e WORDPRESS_DB_PREFIX='${WORDPRESS_DB_PREFIX}' -e WORDPRESS_URL='http://${ELB}' -e WORDPRESS_TITLE='${WORDPRESS_TITLE}' -e WORDPRESS_USER_EMAIL='${WORDPRESS_USER_EMAIL}' -e NEWRELIC_KEY='${NEWRELIC_KEY}' -e NEWRELIC_NAME='${NEWRELIC_NAME}' -e S3_KEY='${IAM_USER_KEY}' -e S3_SECRET='${IAM_USER_SECRET}' -e S3_BUCKET='${S3_BUCKET_NAME}' -e S3_BUCKET_URL='${S3_URL}' -it janloeffler/wordpress-aws-scaler:latest EOL Create new Launch Configuration for AutoScaling $ aws autoscaling create-launch-configuration --launch-configuration-name $LC_NAME -- image-id $AMI --instance-type $INSTANCE_TYPE --key-name $KEYNAME --security-groups $SEC_GROUP_ID --user-data file://ec2-user-data.sh
  42. 42. You must love JSON !!! ;-) { "Reservations": [ { "OwnerId": "699328319947", "ReservationId": "r-041ac13209e0d4eef", "Groups": [], "RequesterId": "226008221399", "Instances": [ { "Monitoring": { "State": "enabled" }, "PublicDnsName": "ec2-52-210-200-157.eu-west-1.compute.amazonaws.com", "State": { "Code": 16, "Name": "running" }, "EbsOptimized": false, "LaunchTime": "2016-09-05T12:19:54.000Z", "PublicIpAddress": "52.210.200.157", "PrivateIpAddress": "172.31.25.83", "ProductCodes": [ { "ProductCodeId": "4jvb72q6a56js2x7jzd24jar5", "ProductCodeType": "marketplace" } ], "VpcId": "vpc-fffbe19a", "StateTransitionReason": "", "InstanceId": "i-0ea3384a24a3c6aba", "ImageId": "ami-64385917", "PrivateDnsName": "ip-172-31-25-83.eu-west-1.compute.internal", "KeyName": "Marko Heijnen", "SecurityGroups": [ { "GroupName": "WordPress-Docker-Test", "GroupId": "sg-1b8dde7c" } ], "ClientToken": "70aa81e1-10a3-4b83-acc7-62702015d06e_subnet-cff90cb9_1", "SubnetId": "subnet-cff90cb9", "InstanceType": "m3.medium", "NetworkInterfaces": [ { "Status": "in-use", "MacAddress": "06:c0:22:65:b4:ed", "SourceDestCheck": true, "VpcId": "vpc-fffbe19a", "Description": "", "Association": { "PublicIp": "52.210.200.157", "PublicDnsName": "ec2-52-210-200-157.eu-west-1.compute.amazonaws.com", "IpOwnerId": "amazon" }, "NetworkInterfaceId": "eni-2f4efa63", "PrivateIpAddresses": [ { "PrivateDnsName": "ip-172-31-25-83.eu-west-1.compute.internal", "Association": { "PublicIp": "52.210.200.157", "PublicDnsName": "ec2-52-210-200-157.eu-west-1.compute.amazonaws.com", "IpOwnerId": "amazon" }, "Primary": true, "PrivateIpAddress": "172.31.25.83" } ], "PrivateDnsName": "ip-172-31-25-83.eu-west-1.compute.internal", "Attachment": { "Status": "attached", "DeviceIndex": 0, "DeleteOnTermination": true, "AttachmentId": "eni-attach-7f9031a0", "AttachTime": "2016-09-05T12:19:54.000Z" }, […]
  43. 43. WP CUP – RACE DAY 1 single-server avg 3.86 sec for 947 hits auto-scaling avg 1.45 sec for 9.866 hits Fastest Lap 216 ms 180 RPS < 3sec Fastest Lap 99 ms 1.000 RPS < sec No Caching!!!
  44. 44. Plesk Mission “We simplify the lives of web professionals.”
  45. 45. Docker WordPress CDN ELB & AutoScaling DB Route53
  46. 46. code.talks 2016, Hamburg “High traffic sites with WordPress, Docker & AWS” How to auto-scale WordPress on AWS and make it fly! Passion for Web Development Jan Löffler CTO Plesk ● @jlsoft2 ● jan@plesk.com Marko Heijnen WordPress Contributor ● @markoheijnen ● mheijnen@plesk.com
  47. 47. https://github.com/plesk/wordpress-aws-scaler https://www.slideshare.net/plesk
  48. 48. “When you walk through your website architecture and you wonder who could be so stupid to test it with 10.000 parallel requests per second. But if you want to be successful and play in the first league, it is important to think about it before your traffic increases” Jan Löffler @jlsoft2
  49. 49. https://github.com/plesk/wordpress-aws-scaler
  50. 50. BACKUP
  51. 51. Web Development Key Tools – what web developers need and use 59 26,6% of all websites worldwide 2,7% 2,2% (high traffic sites) 70% of developers use git as primary source management solution Increasing usage by web agencies 38% of all online stores worldwide runs on top of WordPress as plugin developed by automattic used by 30% of IT companies increased from 13% to 30% in 12 months further 35% plan introduction Sources: w3techs.com, Rightscale, StackOverflow Survey 2015, 2016, Netcraft 311% growth 17.2% usage 26.8% growth 17.9% usage 14.9% growth 16% of all webservers 22% of all active sites continuously growing, while Apache is shrinking
  52. 52. WordPress Management Free SSL everywhere http2 50+ extensions CLI Automatic Updates Site Migration Free support 30+ languages nginx Server Security DNS
  53. 53. ✔ Plesk Onyx: Continuous Delivery Deployment Pipeline (sample) 61 Jenkins runs as Docker container managed by Plesk Plesk Multi Server Extension installs three environments Apps managed by Plesk via Docker

×