Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Leveraging a distributed architecture to your advantage

797 views

Published on

The days of a "simple" LAMP stack are behind us. We now rely on different types of technologies, applications and services to run our web based applications. With "the cloud" we have learned how to distribute our operations, but are we resilient when these cloud services are not available?

We have all heard about the major outages of Amazon and Azure in the past and many online services were impacted by those outages. So how can you protect yourself against being "offline" for hours or days and what are the tools you can use to protect yourself against it?

Learn how we protect our customers with distributed systems (cloud and on-prem) to mitigate outages and stay online even when the lights go out.

Published in: Engineering
  • Be the first to comment

Leveraging a distributed architecture to your advantage

  1. 1. TO YOUR ADVANTAGE LEVERAGING A DISTRIBUTED ARCHITECTURE
  2. 2. WHO AM I? MICHELANGELO VAN DAM ▸ Professional PHP architect ▸ President PHPBenelux ▸ Coach at CoderDojo ▸ Contributor to open source projects ▸ Zend Certified Engineer
  3. 3. WHO AM I? MICHELANGELO VAN DAM ▸ Professional PHP architect ▸ President PHPBenelux ▸ Coach at CoderDojo ▸ Contributor to open source projects ▸ Zend Certified Engineer Rafael Dohms @rdohms
  4. 4. WHO AM I? MICHELANGELO VAN DAM ▸ Professional PHP architect ▸ President PHPBenelux ▸ Coach at CoderDojo ▸ Contributor to open source projects ▸ Zend Certified Engineer Rafael Dohms @rdohms Me @DragonBe
  5. 5. SINGLE LAMP STACK
  6. 6. WHAT IS A LAMP STACK? A COMMON ACRONYM ▸ Linux ▸ Apache ▸ MySQL ▸ PHP
  7. 7. PHP MySQL Apache Linux P M A L
  8. 8. WHY USED? SMALL WEB APPLICATIONS ▸ WordPress blogs ▸ Joomla marketing sites ▸ Drupal business sites ▸ TYPO3 publications ▸ A lot more …
  9. 9. SIMPLICITY IS ULTIMATELY A MATTER OF FOCUS. Ann Voskamp TWEETABLE QUOTE
  10. 10. BENEFITS GALORE WE LOVE EASY ▸ Single point of concern ▸ Easy to maintain ▸ Easy to distribute ▸ Easy to protect
  11. 11. BENEFITS GALORE WE LOVE EASY ▸ Single point of concern ▸ Easy to maintain ▸ Easy to distribute ▸ Easy to protect
  12. 12. NOT ALL SUNSHINE DOESN'T SCALE WELL ▸ Increase of users ▸ Huge amount of data ▸ Additional logic or features ▸ Compliance requirements
  13. 13. MORE WE LIKE
  14. 14. Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 77 bytes) in /path/to/MassiveClass.php on line 666
  15. 15. WE NEED TO SCALE!
  16. 16. PHP MySQL Apache Linux P M A L
  17. 17. PHP MySQL Apache Linux P M A L PHP MySQL Apache Linux P M A L PHP MySQL Apache Linux P M A L PHP MySQL Apache Linux P M A L PHP MySQL Apache Linux P M A L PHP MySQL Apache Linux P M A L
  18. 18. PHP MySQL Apache Linux P M A L PHP MySQL Apache Linux P M A L PHP MySQL Apache Linux P M A L PHP MySQL Apache Linux P M A L PHP MySQL Apache Linux P M A L PHP MySQL Apache Linux P M A L
  19. 19. SCALING MISTAKE #1: DON’T ADD MORE MACHINES AND THINK YOU’RE SCALED. Remember this!!! TWEETABLE QUOTE
  20. 20. SOME WISDOM SCALE WITH PURPOSE ▸ Separate based on responsibility ▸ Pinpoint your weakest link ▸ You need more of them ▸ Look for “alternate” (better) solutions ▸ e.g. Nginx vs. Apache ▸ Don’t over-scale
  21. 21. PHP-FPM MySQL LinuxLinux NGINX PHP-FPM Linux NGINX
  22. 22. IS YOUR APP READY TO BE SCALED? Find the “hard” requirements!
  23. 23. ANALYSING YOUR CODE SOMETHING TO LOOK OUT FOR resources.db.params.host = "database.server.tld" resources.db.params.port = 3306 resources.db.params.username = "fda23a84" resources.db.params.password = "b10a8db164e0754105b7a99be72e3fe5" resources.db.params.dbname = "webapp" resources.db.isDefaultTableAdapter = true
  24. 24. REPLACE WITH SOMETHING THAT CHANGES OUTSIDE YOUR APP ENVIRONMENT VARIABLES CAN HELP resources.db.params.host = WEBAPP_DB_HOST resources.db.params.port = WEBAPP_DB_PORT resources.db.params.username = WEBAPP_DB_USERNAME resources.db.params.password = WEBAPP_DB_PASSWORD resources.db.params.dbname = WEBAPP_DB_DBNAME resources.db.isDefaultTableAdapter = WEBAPP_DB_DEFAULT
  25. 25. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED
  26. 26. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED ▸ Caching
  27. 27. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED ▸ Caching ▸ Search engine
  28. 28. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED ▸ Caching ▸ Search engine ▸ Load balancer
  29. 29. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED ▸ Caching ▸ Search engine ▸ Load balancer ▸ Replication
  30. 30. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED ▸ Caching ▸ Search engine ▸ Load balancer ▸ Replication ▸ Queues
  31. 31. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED ▸ Caching ▸ Search engine ▸ Load balancer ▸ Replication ▸ Queues ▸ Workers
  32. 32. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED ▸ Caching ▸ Search engine ▸ Load balancer ▸ Replication ▸ Queues ▸ Workers ▸ API’s
  33. 33. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED ▸ Caching ▸ Search engine ▸ Load balancer ▸ Replication ▸ Queues ▸ Workers ▸ API’s ▸ and even more things …
  34. 34. PHP-FPM NGINX Shared FS MariaDB MariaDB MariaDB MongoDB MongoDB MongoDB Redis Cache Redis Cache Redis Cache ElasticSearch ElasticSearch Varnish RabbitMQ PHP (workers) PHP (workers) PHP-FPM NGINX
  35. 35. EVENT SOURCING: GBC (MARCO PIVETTA) MUST-SEE PRESENTATION
  36. 36. MAKING CHOICES
  37. 37. ON PREM
  38. 38. ON PREM CLOUD
  39. 39. ON PREM HOSTED CLOUD
  40. 40. MOVE TO THE “CLOUD”, TRUST ME I’M A PROFESSIONAL!!!
  41. 41. Source: The Register 2015-09-20 Source: CBS News 2017-02-28 Source: Forbes 2014-11-18 Source: ZDNet 2016-09-15 Source: C|Net 2009-06-29 Source: The Register 2016-04-20 Source: InfoWorld 2013-03-22
  42. 42. THIS IS BAD!!! NO NEED TO SAY…
  43. 43. SCALING MISTAKE #2: PUT ALL YOUR EGGS IN ONE BASKET. Remember this!!! TWEETABLE QUOTE
  44. 44. STRATEGY IS KEY!
  45. 45. ONLINE? WHAT DID WE DO TO STAY
  46. 46. CAN YOU LIVE WITH REDUCED CAPACITY BUT STAY ONLINE? First question to clients
  47. 47. Hosted / On Premise
  48. 48. DynamoDB EC2 EC2 Elas-csearchRDS - MySQL Instance SQS Queue S3 Elas-c Load Balancing CloudFront CloudSearch West Europe DynamoDB EC2 EC2 Elas-csearchRDS - MySQL Instance SQS Queue S3 CloudFront CloudSearch North Europe DynamoDB EC2 EC2 Elas-csearchRDS - MySQL Instance SQS Queue S3 CloudFront CloudSearch South America
  49. 49. West Europe North Europe South America
  50. 50. DynamoDB EC2 EC2 Elas-csearchRDS - MySQL Instance SQS Queue S3 Elas-c Load Balancing CloudFront CloudSearch West Europe DynamoDB EC2 EC2 Elas-csearchRDS - MySQL Instance SQS Queue S3 CloudFront CloudSearch North Europe South America
  51. 51. DynamoDB EC2 EC2 Elas-csearchRDS - MySQL Instance SQS Queue S3 Elas-c Load Balancing CloudFront CloudSearch West Europe DynamoDB EC2 EC2 Elas-csearchRDS - MySQL Instance SQS Queue S3 CloudFront CloudSearch North Europe South America WARNING: THIS MEANS YOU NEED TO REPLICATE ALL TRANSACTIONS OVER BOTH CLOUD PROVIDERS!!! (COST X 2)
  52. 52. SCALING MISTAKE #3: NOT READY FOR DISASTER Remember this!!! TWEETABLE QUOTE
  53. 53. RESILIENCE TESTING NETFLIX SIMIAN ARMY ▸ Chaos Monkey ▸ Chaos Gorilla ▸ Chaos Kong ▸ Janitor Monkey ▸ Doctor Monkey ▸ Compliance Monkey ▸ Latency Monkey ▸ Security Monkey
  54. 54. ALWAYS PREPARE FOR THE WORST… …so you’re ready when it happens.
  55. 55. SCALING MISTAKE #4: DON’T AUTOMATE YOUR DEPLOYMENT PROCESSES Remember this!!! TWEETABLE QUOTE
  56. 56. Build Feedback Execute Delivery SCM Commits SCM Change Polling test staging production
  57. 57. Unit tests Metrics Provisioning new target Provisioning databases Provisioning workers Integration tests Promote branch CI Unit tests Metrics Provisioning new target Provisioning databases Provisioning workers Integration tests Promote branch CI
  58. 58. AUTOMATE YOUR DEPLOYMENT PROCESS So you can deploy multiple times a day.
  59. 59. WHAT TO TAKE HOME FROM THIS TALK RECAP ▸ Prepare you code to scale and distribute ▸ When online is important, scale over multiple hosting options ▸ Test your resilience for failure ▸ Automate your processes ▸ Build pipelines for all tasks ▸ Release with confidence
  60. 60. in it2PROFESSIONAL PHP SERVICES Michelangelo van Dam Zend Certified Engineer contact@in2it.be - www.in2it.be - T in2itvof - F in2itvof Microsoft Azure Zend Framework Consulting Quality Assurance & Disaster Recovery

×