Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Hopping in clouds - phpuk 17

282 views

Published on

Today there are a lot of cloud providers, with a wide range of offers. Web projects usually have continuously changing needs: what worked well yesterday may not be enough today. These two facts became quite obvious for us while migrating a large PHP application from Rackspace to Amazon. In this session I’d like to share this experience highlighting infrastructure and code evolution, migration steps, cost analisys, issues.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Hopping in clouds - phpuk 17

  1. 1. Hopping a tale of migration from one cloud provider to another in Clouds
  2. 2. Michele Orselli CTO@Ideato _orso_ micheleorselli / ideatosrl mo@ideato.it
  3. 3. Let’s start from the beginning…
  4. 4. What do we Italians like?
  5. 5. “Italians lose wars as if they were football matches, and football matches as if they were wars” Winston Churchill
  6. 6. Peaks on gen - jun - aug up to 70 M pg/mth Peaks during big matches
  7. 7. PaaS IaaS SaaS
  8. 8. PaaS IaaS SaaS
  9. 9. PaaS Platform as a Service Zero configuration (almost) Push the code “on the cloud” and you’re done
  10. 10. Hard limits on resource (e.g 50 db con) Deploy via ftp on shared nfs dir (sf cache mess) Blackbox: No realtime log, no access PHP 5.3
  11. 11. Web: all articles categories, team details, ... symfony1
  12. 12. Mobile: all articles categories, team details, ... Sf2 components
  13. 13. Community Site with user generated content Symfony2
  14. 14. μServices
  15. 15. Macro services
  16. 16. Talk: api for comments, votes, ratings Symfony2
  17. 17. Adv: api for serving ads Symfony2
  18. 18. Media: api for asset mgmt Symfony2
  19. 19. The problems started with talk…
  20. 20. Quick wins Tuning HTTP response headers Caching more endpoints Optimize queries
  21. 21. Tuning HTTP Headers
  22. 22. 1 $date = new DateTime(); 2 $date->modify("+$lifetime seconds"); 3 4 $response->setExpires($date); 5 $response->setMaxAge($lifetime); 6 $response->setSharedMaxAge($lifetime);
  23. 23. PaaS IaaS SaaS
  24. 24. PaaS IaaS SaaS
  25. 25. First candidate for migration was… the talk app
  26. 26. PHP from 5.3 to 5.6 Mysql from 5.0 to 5.6 Apache to nginx (+ php fpm)
  27. 27. Web servers ip are dynamic Can connect only through bastion Share user sessions between servers
  28. 28. Web servers ip are dynamic Can connect only through bastion Share user sessions between servers
  29. 29. 1 'hosts' => function () { 2 $c = Ec2Client::factory([ 3 'profile' => 'calciomercato', 4 'region' => 'eu-central-1', 5 ]); 6 7 $ips = new GetInstancesIps($c); 8 9 return $ips->execute(); 10 }
  30. 30. 1 public function execute() 2 { 3 $instances = $this->ec2Client 4 ->describeInstances( 5 [ 6 'DryRun' => false, 7 'Filters' => [ 8 [ 9 'Name' => 'instance.group-name', 10 'Values' => ['Web Public Auto-assign SG'], 11 ], 12 ], 13 ]); 14 15 return $instancesDescription->getPath( 16 'Reservations/*/Instances/*/NetworkInterfaces/*/ PrivateIpAddresses/*/PrivateIpAddress' 18 ); 19 }
  31. 31. Web servers ip are dynamic Can connect only through bastion Share user sessions between servers
  32. 32. 1 Host cmbastion 2 HostName xx.xx.xx.xx 3 User ec2-user 4 Port 9760 5 StrictHostKeyChecking no 6 UserKnownHostsFile /dev/null 7 IdentityFile ~/.ssh/cm_bastion.pem 8 LogLevel quiet
  33. 33. 10 Host 10.0.14.* 11 User centos 12 StrictHostKeyChecking no 13 UserKnownHostsFile /dev/null 14 IdentityFile ~/.ssh/cm_production.pem 15 ProxyCommand ssh -W %h:%p cmbastion 16 LogLevel quiet 17 18 Host 10.0.24.* 19 User centos 20 StrictHostKeyChecking no 21 UserKnownHostsFile /dev/null 22 IdentityFile ~/.ssh/cm_production.pem 23 ProxyCommand ssh -W %h:%p cmbastion 24 LogLevel quiet
  34. 34. Web servers ip are dynamic Can connect only through bastion Share user sessions between servers
  35. 35. Nginx static cache
  36. 36. 1 fastcgi_cache_key “$scheme$request_method$host$request_uri"; 2 fastcgi_cache_lock on; 3 fastcgi_cache_revalidate on; 4 fastcgi_cache_valid 3m;
  37. 37. 1 if ($request_method ~ ^(POST|PUT|DELETE)$ ) { 2 set $no_cache 1; 3 } 4 5 if ($request_uri ~* "/api/queue") { 6 set $no_cache 1; 7 } 8 9 location ~ ^/(app|dev).php(/|$) { [..] 17 18 # Enable fastcgi_cache 19 add_header X-Cache $upstream_cache_status; 20 fastcgi_cache CALCIOMERCATO_TALK; 21 fastcgi_cache_bypass $no_cache; 22 fastcgi_no_cache $no_cache; 23 }
  38. 38. 8 9 location ~ ^/(app|dev).php(/|$) { 10 fastcgi_split_path_info ^(.+.php)(/.*)$; 11 include fastcgi_params; 12 fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name; 13 fastcgi_param DOCUMENT_ROOT $realpath_root; 14 fastcgi_param HTTPS off; 15 fastcgi_index app.php; 16 fastcgi_intercept_errors on; 17 18 # Enable fastcgi_cache [..] 23 }
  39. 39. Load test using old logs Create AMI Images Deploy latest version of the code Switch DNS
  40. 40. The platform now runs on two clouds: RCS and AWS
  41. 41. Backup
  42. 42. DB and Machine snapshotted every night Copied to another region
  43. 43. snapshot_id=`
 aws --profile snapshotdr rds 
 describe-db-snapshots
 --db-instance-identifier $instance_identifier 
 --region eu-central-1
 | tail -n 1
 | awk -F '{print $5}’
 `
  44. 44. aws rds copy-db-snapshot
 --source-db-snapshot-identifier arn:aws:rds:eu- central-1:$id:snapshot:$snapshot_id
 --region eu-west-1
 --target-db-snapshot-identifier mysqlrds-snap-copy-$NOW
 --copy-tags
  45. 45. snapshot_id=`aws --profile snapshotdr rds 
 describe-db-snapshots
 --region eu-west-1
 --db-instance-identifier $instance_identifier 
 | head -n 1
 | awk -F '{print $4}’` aws rds delete-db-snapshot 
 --region eu-west-1 
 --db-snapshot-identifier $snapshot_id
  46. 46. Adv
  47. 47. Only stateless apis Small database small traffic Infrastructure was already set Easy!
  48. 48. Created a Cloudfront distribution for dynamic content (adv.calciomercato.com)
  49. 49. Mobile
  50. 50. No database, it consumes data from other services High impact, 40% of the total traffic
  51. 51. How to deal with static assets?
  52. 52. s3://com-calciomercato-cdn-mobile/
  53. 53. Created 2 Cloudfront distribution dynamic content (m.calciomercato.com) static content (cdnmobile.calciomercato.com)
  54. 54. Sync asset to s3 via s3cmd s3cmd 
 -m text/javascript 
 --no-preserve sync 
 /var/www/mobile/content/js 
 s3://com-calciomercato-cdn-mobile/
  55. 55. Deploy on a sample machine Performance test based on log Deploy Rebuilding AMI Switch DNS
  56. 56. Tweaking assets caching s3cmd 
 --recursive modify 
 --add-header='Cache-Control:max-age=3600' 
 s3://com-calciomercato-cdn-mobile/assets/
  57. 57. Tweaking assets caching s3cmd 
 --recursive modify 
 --add-header='Cache-Control:max-age=3600' 
 s3://com-calciomercato-cdn-mobile/assets/ Less CF -> S3 requests Less $$$
  58. 58. Community
  59. 59. Allows uses to create their own personal blog First app that can be considered “complete” Exposes api for user related stuff Works as SSO
  60. 60. Web servers ip are dynamic Can connect only through bastion Share user sessions between servers
  61. 61. 1 services: 2 memcache: 3 class: Memcache 4 calls: 5 - [ addServer, [%memc_host%, %memc_port% ]] 6 session.handler.memcache: 7 class: 8 SymfonyComponentHttpFoundationSessionStorageHa ndlerMemcacheSessionHandler 10 arguments: [ 11 @memcache, 12 { prefix: %session_memcache_prefix%, 13 expiretime: %session_memcache_expire% } 14 ]
  62. 62. 1 framework: 2 session: 3 handler_id: %session_handler_id%
  63. 63. Deal with User Generated Content
  64. 64. Created 2 Cloudfront distributions dynamic content (vxl.calciomercato.com) static content (cdnvxl.calciomercato.com)
  65. 65. Gaufrette Filesystem abstraction layer http://knplabs.github.io/Gaufrette/
  66. 66. 1 knp_gaufrette: 2 adapters: 3 photo_storage: 4 aws_s3: 5 service_id: cdn.amazon_s3 6 bucket_name: %amazon_s3_bucket_name% 7 options: 8 directory: data 9 filesystems: 10 photo_storage: 11 adapter: photo_storage 12 alias: photo_storage_filesystem 13
  67. 67. 8 cdn.amazon_s3: 9 class: AwsS3S3Client 10 factory_class: AwsS3S3Client 11 factory_method: 'factory' 12 arguments: 13 - 14 key: %cdn.amazon_s3.aws_key% 15 secret: %cdn.amazon_s3.aws_secret_key% 16 region: eu-central-1
  68. 68. 7 class PhotoUploader 8 { [..] 27 public function upload(File $file, $dir) 28 { 29 $fullPath = $dir.'/'.$file->getFilename(); 30 31 if (!in_array($file->getMimeType(), self::$allowedTypes)) { 32 throw new InvalidArgumentException($file->getMimeType()); 35 } 36 40 return $this->filesystem->write( 41 $fullPath, 42 file_get_contents($file->getPathname()) 43 ); 44 }
  69. 69. 7 class PhotoUploader 8 { [..] 27 public function upload(File $file, $dir) 28 { 29 $fullPath = $dir.'/'.$file->getFilename(); 30 31 if (!in_array($file->getMimeType(), self::$allowedTypes)) { 32 throw new InvalidArgumentException($file->getMimeType()); 35 } 36 40 return $this->filesystem->write( 41 $fullPath, 42 file_get_contents($file->getPathname()) 43 ); 44 } return $this->filesystem->write( 41 $fullPath, 42 file_get_contents($file->getPathname()) 43 );
  70. 70. Deploy on a sample machine Performance test based on log Deploy Rebuilding AMI Copy User Assets on S3 Switch DNS
  71. 71. Web
  72. 72. Oldest and biggest codebase Proxy for mobile calls High traffic 60%
  73. 73. PHP 5.6 not supported by symfony 1
  74. 74. Plan A: try to upgrade sf1 to support php 5.6 Plan B: deploy web on different machines
  75. 75. https://github.com/LExpress/symfony1
  76. 76. 1 protected function camelize($text) 2 { 3 return preg_replace(array('#/(.?)#e', '/(^|_|-)+(.)/e'), array("'::'. 4 strtoupper('1')", "strtoupper('2')"), $text); 5 } 6 7 public static function camelize($text) 8 { 9 return strtr(ucwords(strtr($text, array('/' => 10 ':: ', '_' => ' ', '-' => ' '))), array(' ' => '')); 11 }
  77. 77. Created 1 Cloudfront distribution for static content (cdnweb.calciomercato.com)
  78. 78. https://blog.cloudflare.com/zone-apex-naked-domain-root-domain-cname-supp/ calciomercato.com => cmelb-463612445.eu-central-1.elb.amazonaws.com 
  79. 79. Deploy on a sample machine Performance test based on log Deploy Rebuilding AMI Switch DNS
  80. 80. Media
  81. 81. Only stateful api Handles image thumbailing Pretty big archive (70GB)
  82. 82. 1 public function generateThumbAndUploadToCdn(File $file, $width, $height) 2 { 3 $downloadedFile = $this->downloadFromFileManager($file); 4 5 $cdnKey = $this->generateThumbCdnKey($file, $width, $height); 6 $resizedFile = $this->resizeFilesystemImage($downloadedFile, $width, $height); 8 9 $optimizedFile = $this->optimizeImage($resizedFile); 10 11 $this->uploadFileToCdn($optimizedFile, $cdnKey) 12 13 $this->updateFileInfoTumbs($file, $width, $height, $cdnKey); 14 15 $this->deleteTemporaryFile($downloadedFile); 16 $this->deleteTemporaryFile($optimizedFile); 17 18 return true; 19 }
  83. 83. Transfer from Rackspace CDN to S3
  84. 84. #!/bin/bash
 login="USERNAME_FTP"
 pass="FTP_PASSWORD"
 host="HOST_FTP_RACKSPACE"
 
 remote_dir='/web/content/data'
 local_dir=“/var/www/vhosts/media.calciomercato.pro/data"
 base_name="$(basename "$0")" lftp -u $login,$pass $host << EOF
 set ftp:ssl-allow no
 set mirror:use-pget-n 5 mirror -c -P5 --log="/var/log/$base_name.log" "$remote_dir" "$local_dir" quit
  85. 85. 1 public function slugifyFilename($text) 2 { 3 $text = preg_replace('~[^pLd]+~u', '.', $text); 4 $text = trim($text, '-'); 5 6 if (function_exists('iconv')) { 7 $text = iconv('utf-8', 'us - ascii//TRANSLIT', $text); 8 } 9 10 $text = preg_replace('~[^-w.]+~', '', $text); 11 12 return $text; 13 }
  86. 86. Monitoring
  87. 87. cloudwatch -> nagios -> catci cw retention max 2 sett
  88. 88. 2 weeks retention: integration with cacti and nagios
  89. 89. ...and Slack
  90. 90. Autoscaling
  91. 91. 2 autoscaling groups 3 metrics - CPU % (> 70%) - Response Time (> 100 ms) - # of req/sec (> 10000)
  92. 92. Full migration took 1 year
  93. 93. 50% cost reduction
  94. 94. This talk is not about blaming RCS…
  95. 95. …simply it wasn’t suitable anymore for our needs :-)
  96. 96. Macro services FTW! HTTP cache helped us a lot! Measure measure measure
  97. 97. Rationalising api: less call, less $$$ Reorganise frontend stuff Get rid of sf1
  98. 98. Upgrading to php7 Upgrading to ALB (HTTP/2)
  99. 99. Verona
  100. 100. JsDay: May 10th-11th PHPDay: May 12th-13th Verona
  101. 101. Thank you! https://joind.in/talk/cd6af Michele Orselli CTO@Ideato _orso_ micheleorselli / ideatosrl mo@ideato.it

×