Downloaded 12 times
![1 'hosts' => function () {
2 $c = Ec2Client::factory([
3 'profile' => 'calciomercato',
4 'region' => 'eu-central-1',
5 ]);
6
7 $ips = new GetInstancesIps($c);
8
9 return $ips->execute();
10 }](https://image.slidesharecdn.com/hoppinginclouds-phpuk-170217065955/85/Hopping-in-clouds-phpuk-17-50-320.jpg)
![1 public function execute()
2 {
3 $instances = $this->ec2Client
4 ->describeInstances(
5 [
6 'DryRun' => false,
7 'Filters' => [
8 [
9 'Name' => 'instance.group-name',
10 'Values' => ['Web Public Auto-assign SG'],
11 ],
12 ],
13 ]);
14
15 return $instancesDescription->getPath(
16 'Reservations/*/Instances/*/NetworkInterfaces/*/
PrivateIpAddresses/*/PrivateIpAddress'
18 );
19 }](https://image.slidesharecdn.com/hoppinginclouds-phpuk-170217065955/85/Hopping-in-clouds-phpuk-17-51-320.jpg)
![1 if ($request_method ~ ^(POST|PUT|DELETE)$ ) {
2 set $no_cache 1;
3 }
4
5 if ($request_uri ~* "/api/queue") {
6 set $no_cache 1;
7 }
8
9 location ~ ^/(app|dev).php(/|$) {
[..]
17
18 # Enable fastcgi_cache
19 add_header X-Cache $upstream_cache_status;
20 fastcgi_cache CALCIOMERCATO_TALK;
21 fastcgi_cache_bypass $no_cache;
22 fastcgi_no_cache $no_cache;
23 }](https://image.slidesharecdn.com/hoppinginclouds-phpuk-170217065955/85/Hopping-in-clouds-phpuk-17-58-320.jpg)
![8
9 location ~ ^/(app|dev).php(/|$) {
10 fastcgi_split_path_info ^(.+.php)(/.*)$;
11 include fastcgi_params;
12 fastcgi_param SCRIPT_FILENAME
$realpath_root$fastcgi_script_name;
13 fastcgi_param DOCUMENT_ROOT $realpath_root;
14 fastcgi_param HTTPS off;
15 fastcgi_index app.php;
16 fastcgi_intercept_errors on;
17
18 # Enable fastcgi_cache
[..]
23 }](https://image.slidesharecdn.com/hoppinginclouds-phpuk-170217065955/85/Hopping-in-clouds-phpuk-17-59-320.jpg)
![1 services:
2 memcache:
3 class: Memcache
4 calls:
5 - [ addServer, [%memc_host%, %memc_port% ]]
6 session.handler.memcache:
7 class:
8
SymfonyComponentHttpFoundationSessionStorageHa
ndlerMemcacheSessionHandler
10 arguments: [
11 @memcache,
12 { prefix: %session_memcache_prefix%,
13 expiretime: %session_memcache_expire% }
14 ]](https://image.slidesharecdn.com/hoppinginclouds-phpuk-170217065955/85/Hopping-in-clouds-phpuk-17-88-320.jpg)
![7 class PhotoUploader
8 {
[..]
27 public function upload(File $file, $dir)
28 {
29 $fullPath = $dir.'/'.$file->getFilename();
30
31 if (!in_array($file->getMimeType(), self::$allowedTypes)) {
32 throw new InvalidArgumentException($file->getMimeType());
35 }
36
40 return $this->filesystem->write(
41 $fullPath,
42 file_get_contents($file->getPathname())
43 );
44 }](https://image.slidesharecdn.com/hoppinginclouds-phpuk-170217065955/85/Hopping-in-clouds-phpuk-17-95-320.jpg)
![7 class PhotoUploader
8 {
[..]
27 public function upload(File $file, $dir)
28 {
29 $fullPath = $dir.'/'.$file->getFilename();
30
31 if (!in_array($file->getMimeType(), self::$allowedTypes)) {
32 throw new InvalidArgumentException($file->getMimeType());
35 }
36
40 return $this->filesystem->write(
41 $fullPath,
42 file_get_contents($file->getPathname())
43 );
44 }
return $this->filesystem->write(
41 $fullPath,
42 file_get_contents($file->getPathname())
43 );](https://image.slidesharecdn.com/hoppinginclouds-phpuk-170217065955/85/Hopping-in-clouds-phpuk-17-96-320.jpg)
![1 public function slugifyFilename($text)
2 {
3 $text = preg_replace('~[^pLd]+~u', '.', $text);
4 $text = trim($text, '-');
5
6 if (function_exists('iconv')) {
7 $text = iconv('utf-8', 'us - ascii//TRANSLIT', $text);
8 }
9
10 $text = preg_replace('~[^-w.]+~', '', $text);
11
12 return $text;
13 }](https://image.slidesharecdn.com/hoppinginclouds-phpuk-170217065955/85/Hopping-in-clouds-phpuk-17-115-320.jpg)
Uploaded byMichele Orselli
Hopping in clouds - phpuk 17
The document outlines the migration process of a cloud platform from one provider to another, detailing challenges faced while upgrading PHP and database versions, and the transition from Rackspace to AWS. It includes technical strategies for deploying applications, optimizing performance, and managing user-generated content across different services. The migration resulted in a successful implementation lasting a year with a significant cost reduction of 50%.
More Related Content
Similar to Hopping in clouds - phpuk 17
![20260201 [FOSDEM] gomodjail - library sandboxing for Go modules.pdf](https://cdn.slidesharecdn.com/ss_thumbnails/20260201fosdemgomodjail-librarysandboxingforgomodules-260201225659-76609ec4-thumbnail.jpg?width=640&height=640&fit=bounds)
Hopping in clouds - phpuk 17
- 1. Hopping a tale ofmigration from one cloud provider to another in Clouds
- 2. Michele Orselli CTO@Ideato _orso_ micheleorselli /ideatosrl mo@ideato.it
- 9. Let’s start fromthe beginning…
- 11. What do weItalians like?
- 16. “Italians lose warsas if they were football matches, and football matches as if they were wars” Winston Churchill
- 18. Peaks on gen- jun - aug up to 70 M pg/mth Peaks during big matches
- 20.
- 21.
- 22. PaaS Platform asa Service Zero configuration (almost) Push the code “on the cloud” and you’re done
- 24. Hard limits onresource (e.g 50 db con) Deploy via ftp on shared nfs dir (sf cache mess) Blackbox: No realtime log, no access PHP 5.3
- 25. Web: all articles categories,team details, ... symfony1
- 26. Mobile: all articles categories,team details, ... Sf2 components
- 27. Community Site with user generatedcontent Symfony2
- 28.
- 30.
- 31. Talk: api for comments,votes, ratings Symfony2
- 32. Adv: api for servingads Symfony2
- 33. Media: api for assetmgmt Symfony2
- 35. The problems startedwith talk…
- 37. Quick wins Tuning HTTPresponse headers Caching more endpoints Optimize queries
- 39.
- 40. 1 $date =new DateTime(); 2 $date->modify("+$lifetime seconds"); 3 4 $response->setExpires($date); 5 $response->setMaxAge($lifetime); 6 $response->setSharedMaxAge($lifetime);
- 43.
- 44.
- 45. First candidate formigration was… the talk app
- 47. PHP from 5.3to 5.6 Mysql from 5.0 to 5.6 Apache to nginx (+ php fpm)
- 48. Web servers ipare dynamic Can connect only through bastion Share user sessions between servers
- 49. Web servers ipare dynamic Can connect only through bastion Share user sessions between servers
- 50. 1 'hosts' =>function () { 2 $c = Ec2Client::factory([ 3 'profile' => 'calciomercato', 4 'region' => 'eu-central-1', 5 ]); 6 7 $ips = new GetInstancesIps($c); 8 9 return $ips->execute(); 10 }
- 51. 1 public functionexecute() 2 { 3 $instances = $this->ec2Client 4 ->describeInstances( 5 [ 6 'DryRun' => false, 7 'Filters' => [ 8 [ 9 'Name' => 'instance.group-name', 10 'Values' => ['Web Public Auto-assign SG'], 11 ], 12 ], 13 ]); 14 15 return $instancesDescription->getPath( 16 'Reservations/*/Instances/*/NetworkInterfaces/*/ PrivateIpAddresses/*/PrivateIpAddress' 18 ); 19 }
- 52. Web servers ipare dynamic Can connect only through bastion Share user sessions between servers
- 53. 1 Host cmbastion 2HostName xx.xx.xx.xx 3 User ec2-user 4 Port 9760 5 StrictHostKeyChecking no 6 UserKnownHostsFile /dev/null 7 IdentityFile ~/.ssh/cm_bastion.pem 8 LogLevel quiet
- 54. 10 Host 10.0.14.* 11User centos 12 StrictHostKeyChecking no 13 UserKnownHostsFile /dev/null 14 IdentityFile ~/.ssh/cm_production.pem 15 ProxyCommand ssh -W %h:%p cmbastion 16 LogLevel quiet 17 18 Host 10.0.24.* 19 User centos 20 StrictHostKeyChecking no 21 UserKnownHostsFile /dev/null 22 IdentityFile ~/.ssh/cm_production.pem 23 ProxyCommand ssh -W %h:%p cmbastion 24 LogLevel quiet
- 55. Web servers ipare dynamic Can connect only through bastion Share user sessions between servers
- 56.
- 57. 1 fastcgi_cache_key “$scheme$request_method$host$request_uri"; 2 fastcgi_cache_lockon; 3 fastcgi_cache_revalidate on; 4 fastcgi_cache_valid 3m;
- 58. 1 if ($request_method~ ^(POST|PUT|DELETE)$ ) { 2 set $no_cache 1; 3 } 4 5 if ($request_uri ~* "/api/queue") { 6 set $no_cache 1; 7 } 8 9 location ~ ^/(app|dev).php(/|$) { [..] 17 18 # Enable fastcgi_cache 19 add_header X-Cache $upstream_cache_status; 20 fastcgi_cache CALCIOMERCATO_TALK; 21 fastcgi_cache_bypass $no_cache; 22 fastcgi_no_cache $no_cache; 23 }
- 59. 8 9 location ~^/(app|dev).php(/|$) { 10 fastcgi_split_path_info ^(.+.php)(/.*)$; 11 include fastcgi_params; 12 fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name; 13 fastcgi_param DOCUMENT_ROOT $realpath_root; 14 fastcgi_param HTTPS off; 15 fastcgi_index app.php; 16 fastcgi_intercept_errors on; 17 18 # Enable fastcgi_cache [..] 23 }
- 60. Load test usingold logs Create AMI Images Deploy latest version of the code Switch DNS
- 64. The platform nowruns on two clouds: RCS and AWS
- 65.
- 67. DB and Machinesnapshotted every night Copied to another region
- 68. snapshot_id=` aws --profile snapshotdrrds describe-db-snapshots --db-instance-identifier $instance_identifier --region eu-central-1 | tail -n 1 | awk -F '{print $5}’ `
- 69. aws rds copy-db-snapshot --source-db-snapshot-identifierarn:aws:rds:eu- central-1:$id:snapshot:$snapshot_id --region eu-west-1 --target-db-snapshot-identifier mysqlrds-snap-copy-$NOW --copy-tags
- 70. snapshot_id=`aws --profile snapshotdrrds describe-db-snapshots --region eu-west-1 --db-instance-identifier $instance_identifier | head -n 1 | awk -F '{print $4}’` aws rds delete-db-snapshot --region eu-west-1 --db-snapshot-identifier $snapshot_id
- 71.
- 72. Only stateless apis Smalldatabase small traffic Infrastructure was already set Easy!
- 73. Created a Cloudfrontdistribution for dynamic content (adv.calciomercato.com)
- 75.
- 76. No database, itconsumes data from other services High impact, 40% of the total traffic
- 77. How to dealwith static assets?
- 78.
- 79. Created 2 Cloudfrontdistribution dynamic content (m.calciomercato.com) static content (cdnmobile.calciomercato.com)
- 80. Sync asset tos3 via s3cmd s3cmd -m text/javascript --no-preserve sync /var/www/mobile/content/js s3://com-calciomercato-cdn-mobile/
- 81. Deploy on asample machine Performance test based on log Deploy Rebuilding AMI Switch DNS
- 82. Tweaking assets caching s3cmd --recursive modify --add-header='Cache-Control:max-age=3600' s3://com-calciomercato-cdn-mobile/assets/
- 83. Tweaking assets caching s3cmd --recursive modify --add-header='Cache-Control:max-age=3600' s3://com-calciomercato-cdn-mobile/assets/ Less CF -> S3 requests Less $$$
- 84.
- 85. Allows uses tocreate their own personal blog First app that can be considered “complete” Exposes api for user related stuff Works as SSO
- 86. Web servers ipare dynamic Can connect only through bastion Share user sessions between servers
- 88. 1 services: 2 memcache: 3class: Memcache 4 calls: 5 - [ addServer, [%memc_host%, %memc_port% ]] 6 session.handler.memcache: 7 class: 8 SymfonyComponentHttpFoundationSessionStorageHa ndlerMemcacheSessionHandler 10 arguments: [ 11 @memcache, 12 { prefix: %session_memcache_prefix%, 13 expiretime: %session_memcache_expire% } 14 ]
- 89. 1 framework: 2 session: 3handler_id: %session_handler_id%
- 90. Deal with UserGenerated Content
- 91. Created 2 Cloudfrontdistributions dynamic content (vxl.calciomercato.com) static content (cdnvxl.calciomercato.com)
- 92.
- 93. 1 knp_gaufrette: 2 adapters: 3photo_storage: 4 aws_s3: 5 service_id: cdn.amazon_s3 6 bucket_name: %amazon_s3_bucket_name% 7 options: 8 directory: data 9 filesystems: 10 photo_storage: 11 adapter: photo_storage 12 alias: photo_storage_filesystem 13
- 94. 8 cdn.amazon_s3: 9 class:AwsS3S3Client 10 factory_class: AwsS3S3Client 11 factory_method: 'factory' 12 arguments: 13 - 14 key: %cdn.amazon_s3.aws_key% 15 secret: %cdn.amazon_s3.aws_secret_key% 16 region: eu-central-1
- 95. 7 class PhotoUploader 8{ [..] 27 public function upload(File $file, $dir) 28 { 29 $fullPath = $dir.'/'.$file->getFilename(); 30 31 if (!in_array($file->getMimeType(), self::$allowedTypes)) { 32 throw new InvalidArgumentException($file->getMimeType()); 35 } 36 40 return $this->filesystem->write( 41 $fullPath, 42 file_get_contents($file->getPathname()) 43 ); 44 }
- 96. 7 class PhotoUploader 8{ [..] 27 public function upload(File $file, $dir) 28 { 29 $fullPath = $dir.'/'.$file->getFilename(); 30 31 if (!in_array($file->getMimeType(), self::$allowedTypes)) { 32 throw new InvalidArgumentException($file->getMimeType()); 35 } 36 40 return $this->filesystem->write( 41 $fullPath, 42 file_get_contents($file->getPathname()) 43 ); 44 } return $this->filesystem->write( 41 $fullPath, 42 file_get_contents($file->getPathname()) 43 );
- 97. Deploy on asample machine Performance test based on log Deploy Rebuilding AMI Copy User Assets on S3 Switch DNS
- 99.
- 100. Oldest and biggestcodebase Proxy for mobile calls High traffic 60%
- 101. PHP 5.6 notsupported by symfony 1
- 102. Plan A: tryto upgrade sf1 to support php 5.6 Plan B: deploy web on different machines
- 103.
- 104. 1 protected functioncamelize($text) 2 { 3 return preg_replace(array('#/(.?)#e', '/(^|_|-)+(.)/e'), array("'::'. 4 strtoupper('1')", "strtoupper('2')"), $text); 5 } 6 7 public static function camelize($text) 8 { 9 return strtr(ucwords(strtr($text, array('/' => 10 ':: ', '_' => ' ', '-' => ' '))), array(' ' => '')); 11 }
- 105. Created 1 Cloudfrontdistribution for static content (cdnweb.calciomercato.com)
- 106.
- 107. Deploy on asample machine Performance test based on log Deploy Rebuilding AMI Switch DNS
- 109.
- 110. Only stateful api Handlesimage thumbailing Pretty big archive (70GB)
- 111. 1 public functiongenerateThumbAndUploadToCdn(File $file, $width, $height) 2 { 3 $downloadedFile = $this->downloadFromFileManager($file); 4 5 $cdnKey = $this->generateThumbCdnKey($file, $width, $height); 6 $resizedFile = $this->resizeFilesystemImage($downloadedFile, $width, $height); 8 9 $optimizedFile = $this->optimizeImage($resizedFile); 10 11 $this->uploadFileToCdn($optimizedFile, $cdnKey) 12 13 $this->updateFileInfoTumbs($file, $width, $height, $cdnKey); 14 15 $this->deleteTemporaryFile($downloadedFile); 16 $this->deleteTemporaryFile($optimizedFile); 17 18 return true; 19 }
- 112. Transfer from RackspaceCDN to S3
- 113. #!/bin/bash login="USERNAME_FTP" pass="FTP_PASSWORD" host="HOST_FTP_RACKSPACE" remote_dir='/web/content/data' local_dir=“/var/www/vhosts/media.calciomercato.pro/data" base_name="$(basename "$0")" lftp -u$login,$pass $host << EOF set ftp:ssl-allow no set mirror:use-pget-n 5 mirror -c -P5 --log="/var/log/$base_name.log" "$remote_dir" "$local_dir" quit
- 115. 1 public functionslugifyFilename($text) 2 { 3 $text = preg_replace('~[^pLd]+~u', '.', $text); 4 $text = trim($text, '-'); 5 6 if (function_exists('iconv')) { 7 $text = iconv('utf-8', 'us - ascii//TRANSLIT', $text); 8 } 9 10 $text = preg_replace('~[^-w.]+~', '', $text); 11 12 return $text; 13 }
- 117.
- 118. cloudwatch -> nagios-> catci cw retention max 2 sett
- 120. 2 weeks retention: integrationwith cacti and nagios
- 121.
- 122.
- 123. 2 autoscaling groups 3metrics - CPU % (> 70%) - Response Time (> 100 ms) - # of req/sec (> 10000)
- 125. Full migration took1 year
- 126.
- 127. This talk isnot about blaming RCS…
- 128. …simply it wasn’tsuitable anymore for our needs :-)
- 130. Macro services FTW! HTTPcache helped us a lot! Measure measure measure
- 131. Rationalising api: lesscall, less $$$ Reorganise frontend stuff Get rid of sf1
- 132. Upgrading to php7 Upgradingto ALB (HTTP/2)
- 134.
- 135. JsDay: May 10th-11th PHPDay:May 12th-13th Verona
- 136. Thank you! https://joind.in/talk/cd6af Michele Orselli CTO@Ideato _orso_ micheleorselli/ ideatosrl mo@ideato.it