Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Microservices Architecture for Content Management Systems using AWS Lambda and more

6,880 views

Published on

Content Management Systems are by nature resource intensive, expensive to customize, and difficult to manage at scale. What if we can change this perception and help PHP / Drupal developers architect a content platform that is high performance and low cost, high security and low maintenance? This talk will focus on 3 key topics: 1) serverless environment, 2) microservices architecture and 3) hands-on demos. We will describe a serverless solution and propose a scalable architecture that will help Drupal community to adopt cloud-native approach without huge efforts or expensive resources allocation.

Published in: Software
  • Be the first to comment

Microservices Architecture for Content Management Systems using AWS Lambda and more

  1. 1. Microservices Architecture for Content Management Systems using AWS Lambda and more http://www.meetup.com/drupalnyc/events/227188360 Eugene Istrati, Technology Partner eugene@mitocgroup.com www.mitocgroup.com
  2. 2. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Eugene Istrati, Partner @ Mitoc Group Microservices Architecture for Digital Platforms with AWS Lambda, Amazon CloudFront, and Amazon DynamoDB eugene@mitocgroup.com October 2015 ARC201
  3. 3. Web Applications Challenges Note: Credits and thanks are listed at the end of the presentation … … …
  4. 4. Average cost of downtime • $500K - $1M / hour (IDC, Dec 2014) • $140K - $540K / hour (Garner, July 2014) • $474K / hour (Ponemon Inst., Dec 2013) Most commonly reported consequences • Damage to reputation (38%) • Increase in customer churn (37%) • Damage to credit rating (28%) • Increase to insurance premiums (26%) Web Applications Challenges 27% 60% 13% Outage Degradation No impact 0% 20% 40% 60% 80% Impact of DoS/DDoS Attack Note: Credits and thanks are listed at the end of the presentation
  5. 5. Digital Enterprise End-to-end Platform
  6. 6. About Eugene Istrati • eugene@mitocgroup.com • Partner @ Mitoc Group Inc • 15+ years in IT; 7+ years on AWS • AWS Certified Solutions Architect (re-certified at re:Invent 2015) • Companies: Hearst, Amazon, GrubHub, Tenaris (Europe) Mitoc Group Inc • www.mitocgroup.com • Web Development Studio • AWS Technology Partner • Focusing on enterprise applications and platforms • Working with customers from media and entertainment industry
  7. 7. Agenda • Web Apps Hosting on AWS • Reference Architecture • Serverless Architecture • Demo: Setup Serverless AWS • Microservices Architecture • Powered by AWS Lambda • Tips and Tricks • Drupal on Microservices • Demo: todo.deep.mg • Q&A + Next Steps
  8. 8. Demo: todo.deep.mg • Inspired from open source • www.todomvc.com • Go to the GitHub repository • github.com/MitocGroup/deep -microservices-todo-app • Follow the steps from Getting Started to build and deploy • todo.deep.com
  9. 9. Agenda • Web Apps Hosting on AWS • Reference Architecture • Serverless Architecture • Demo: Setup Serverless AWS • Microservices Architecture • Powered by AWS Lambda • Tips and Tricks • Drupal on Microservices • Demo: todo.deep.mg • Q&A + Next Steps
  10. 10. Web Apps Hosting / Reference Architecture Availability Zone A Availability Zone B Auto Scaling Group Auto Scaling Group www.example.com static.example.com web servers web servers app servers app servers • Scales in minutes
  11. 11. Web Apps Hosting / Reference Architecture Availability Zone A Availability Zone B Auto Scaling Group Auto Scaling Group www.example.com static.example.com web servers web servers app servers app servers • Scales in minutes • Huge challenge for breaking news, viral content, or attacks
  12. 12. Web Apps Hosting / Reference Architecture Availability Zone A Availability Zone B Auto Scaling Group Auto Scaling Group www.example.com static.example.com web servers web servers app servers app servers • Scales in minutes • Huge challenge for breaking news, viral content, or attacks • Reduced operational complexity
  13. 13. Web Apps Hosting / Reference Architecture Availability Zone A Availability Zone B Auto Scaling Group Auto Scaling Group www.example.com static.example.com web servers web servers app servers app servers • Scales in minutes • Huge challenge for breaking news, viral content, or attacks • Reduced operational complexity • Requires DevOps with experience
  14. 14. Web Apps Hosting / Reference Architecture Availability Zone A Availability Zone B Auto Scaling Group Auto Scaling Group www.example.com static.example.com web servers web servers app servers app servers • Scales in minutes • Huge challenge for breaking news, viral content, or attacks • Reduced operational complexity • Requires DevOps with experience • Flexible choice of technology
  15. 15. Web Apps Hosting / Reference Architecture Availability Zone A Availability Zone B Auto Scaling Group Auto Scaling Group www.example.com static.example.com web servers web servers app servers app servers • Scales in minutes • Huge challenge for breaking news, viral content, or attacks • Reduced operational complexity • Requires DevOps with experience • Flexible choice of technology • Requires devs with rich skill set
  16. 16. Web Apps Hosting / Reference Architecture Availability Zone A Availability Zone B Auto Scaling Group Auto Scaling Group www.example.com static.example.com web servers web servers app servers app servers • Scales in minutes • Huge challenge for breaking news, viral content, or attacks • Reduced operational complexity • Requires DevOps with experience • Flexible choice of technology • Requires devs with rich skill set • Cost-effective
  17. 17. Web Apps Hosting / Reference Architecture Availability Zone A Availability Zone B Auto Scaling Group Auto Scaling Group www.example.com static.example.com web servers web servers app servers app servers • Scales in minutes • Huge challenge for breaking news, viral content, or attacks • Reduced operational complexity • Requires DevOps with experience • Flexible choice of technology • Requires devs with rich skill set • Cost-effective • Over-provisioning and over-paying
  18. 18. Web Apps Hosting / Reference Architecture Availability Zone A Availability Zone B Auto Scaling Group Auto Scaling Group www.example.com static.example.com web servers web servers app servers app servers • Scales in minutes • Huge challenge for breaking news, viral content, or attacks • Reduced operational complexity • Requires DevOps with experience • Flexible choice of technology • Requires devs with rich skill set • Cost-effective • Over-provisioning and over-paying
  19. 19. AWS re:Invent 2014 Note: Credits and thanks are listed at the end of the presentation
  20. 20. AWS Summit NY 2015 Note: Credits and thanks are listed at the end of the presentation
  21. 21. Web Apps Hosting … Reinvented Availability Zone A Availability Zone B Auto Scaling Group Auto Scaling Group www.example.com static.example.com web servers web servers app servers app servers S3 bucket CloudFront distribution Web Tier Cognito Identity DB Tier SQS DynamoDB LambdaCloudFront logs API Gateway www.example.com static.example.com App Tier AWS Region RDS Aurora
  22. 22. Agenda • Web Apps Hosting on AWS • Reference Architecture • Serverless Architecture • Demo: Setup Serverless AWS • Microservices Architecture • Powered by AWS Lambda • Tips and Tricks • Drupal on Microservices • Demo: todo.deep.mg • Q&A + Next Steps
  23. 23. What does “serverless” mean? Not involving a server; composed only of clients. http://www.wordsense.eu/serverless Serverless doesn’t mean servers are no longer involved. It simply means that developers no longer have to think "that much" about them. Computing resources get used as services without having to manage around physical capacities or limits. https://www.quora.com/What-is-Serverless-Computing
  24. 24. Serverless vs. Reference Availability Zone A Availability Zone B Auto Scaling Group Auto Scaling Group www.example.com static.example.com web servers web servers app servers app servers S3 bucket CloudFront distribution Web Tier Cognito Identity DB Tier SQS DynamoDB LambdaCloudFront logs API Gateway www.example.com static.example.com App Tier AWS Region RDS Aurora vs
  25. 25. Serverless Architecture – Web Tier S3 bucket CloudFront distribution Web Tier Cognito Identity CloudFront logs www.example.com static.example.com Availability Zone A Availability Zone B Auto Scaling Group www.example.com static.example.com web servers web servers
  26. 26. Serverless Architecture – Web Tier S3 bucket CloudFront distribution Web Tier Cognito Identity CloudFront logs www.example.com static.example.com • Static Assets • Same as in reference architecture • css, js, docs, images, videos + html • Dynamic Functionality • Use JS framework (e.g. Angular) • SEO-friendly (Custom Error Response + HTML5 History API) • Completely Serverless • Pre-scaled • Low-cost • Low-maintenance
  27. 27. Serverless Architecture – Web Tier S3 bucket CloudFront distribution Web Tier Cognito Identity CloudFront logs www.example.com static.example.com • Static Assets • Same as in reference architecture • css, js, docs, images, videos + html • Dynamic Functionality • Use JS framework (e.g. Angular) • SEO-friendly (Custom Error Response + HTML5 History API) • Completely Serverless • Pre-scaled • Low-cost • Low-maintenance
  28. 28. Serverless Architecture – Web Tier S3 bucket CloudFront distribution Web Tier Cognito Identity CloudFront logs www.example.com static.example.com • Static Assets • Same as in reference architecture • css, js, docs, images, videos + html • Dynamic Functionality • Use JS framework (e.g. Angular) • SEO-friendly (Custom Error Response + HTML5 History API) • Completely Serverless • Pre-scaled • Low-cost • Low-maintenance
  29. 29. Serverless Architecture – App Tier Cognito Identity SQS Lambda API Gateway App Tier Availability Zone A Availability Zone B Auto Scaling Group app servers app servers
  30. 30. Cognito Identity SQS Lambda API Gateway App Tier • Accelerated Backend • Write node.js functions and load into Lambda • Power up Lambda with RESTful endpoints on API Gateway • Cache, throttle, meter, version, etc. • Completely Serverless • Pre-scaled • Low-cost • Low-maintenance Serverless Architecture – App Tier
  31. 31. • Accelerated Backend • Write node.js functions and load into Lambda • Power up Lambda with RESTful endpoints on API Gateway • Cache, throttle, meter, version, etc. • Completely Serverless • Pre-scaled • Low-cost • Low-maintenance Serverless Architecture – App Tier Cognito Identity SQS Lambda API Gateway App Tier
  32. 32. Availability Zone A Availability Zone B Serverless Architecture – DB Tier DB Tier SQS DynamoDB RDS Aurora
  33. 33. DB Tier SQS DynamoDB RDS Aurora Serverless Architecture – DB Tier • First choice – DynamoDB + SQS • Schema-free • Scale only reads and writes • Completely Serverless • Pre-scaled • Low-cost • Low-maintenance • Next choice – RDS Aurora • Relational • MySQL-like approach, but 5x better
  34. 34. Serverless Architecture – DB Tier • First choice – DynamoDB + SQS • Schema-free • Scale only reads and writes • Completely Serverless • Pre-scaled • Low-cost • Low-maintenance • Next choice – RDS Aurora • Relational • MySQL-like approach, but 5x better DB Tier SQS DynamoDB RDS Aurora
  35. 35. Serverless Architecture – DB Tier • First choice – DynamoDB + SQS • Schema-free • Scale only reads and writes • Completely Serverless • Pre-scaled • Low-cost • Low-maintenance • Next choice – RDS Aurora • Relational • MySQL-like approach, but 5x better DB Tier SQS DynamoDB RDS Aurora
  36. 36. Agenda • Web Apps Hosting on AWS • Reference Architecture • Serverless Architecture • Demo: Setup Serverless AWS • Microservices Architecture • Powered by AWS Lambda • Tips and Tricks • Drupal on Microservices • Demo: todo.deep.mg • Q&A + Next Steps
  37. 37. Demo: Setup Serverless AWS 1. Security - Create IAM roles 2. Front-end - Create S3 bucket - Enable static website hosting - Add bucket policy - Create CloudFront distribution 3. Back-end - Create Lambda function - Upload code into Lambda - Create API Gateway endpoint 4. Database - Create DynamoDB table 5. Code - Load code into S3 bucket - View via CloudFront (S3 as backup) S3 bucket CloudFront distribution Web Tier Cognito Identity DB Tier SQS DynamoDB LambdaCloudFront logs API Gateway www.example.com static.example.com App Tier AWS Region RDS Aurora
  38. 38. Lessons Learned • Serverless approach is challengingly awesome • Frontend is restricted to JS (and JS Frameworks) • Backend is restricted to Python, Java or JS (for now) • SOA and APIs are required by design
  39. 39. Lessons Learned • Serverless approach is challengingly awesome • Frontend is restricted to JS (and JS Frameworks) • Backend is restricted to Python, Java or JS (for now) • SOA and APIs are required by design • Services must be as small as possible • AWS Lambda constrains • Browser limitations (on mobile devices)
  40. 40. Lessons Learned • Serverless approach is challengingly awesome • Frontend is restricted to JS (and JS Frameworks) • Backend is restricted to Python, Java or JS (for now) • SOA and APIs are required by design • Services must be as small as possible => microservices • AWS Lambda constrains • Browser limitations (on mobile devices)
  41. 41. Agenda • Web Apps Hosting on AWS • Reference Architecture • Serverless Architecture • Demo: Setup Serverless AWS • Microservices Architecture • Powered by AWS Lambda • Tips and Tricks • Drupal on Microservices • Demo: todo.deep.mg • Q&A + Next Steps
  42. 42. Google Trends: Microservices
  43. 43. What does “microservices” mean? In computing, microservices is a software architecture style in which complex applications are composed of small, independent processes communicating with each other using language- agnostic APIs. These services are small, highly decoupled and focus on doing a small task, facilitating a modular approach to system- building. https://en.wikipedia.org/wiki/Microservices
  44. 44. Why use “microservices” in Drupal? Drupal is content management software […] Drupal has great standard features, like easy content authoring, reliable performance, and excellent security. But what sets it apart is its flexibility; modularity is one of its core principles. Its tools help you build the versatile, structured content that dynamic web experiences need. https://www.drupal.org/about In computing, microservices is a software architecture style in which complex applications are composed of small, independent processes communicating with each other using language- agnostic APIs. These services are small, highly decoupled and focus on doing a small task, facilitating a modular approach to system- building. https://en.wikipedia.org/wiki/Microservices
  45. 45. Microservices Architecture Keynote GOTO Conference: Microservices by Martin Fowler - https://www.youtube.com/watch?v=wgdBVIX9ifA State of the Art in Microservices by Adrian Cockcroft - https://www.youtube.com/watch?v=nMTaS07i3jk Sam Newman at ThoughtWorks London 2015: Deploying and Operating Microservices - https://www.youtube.com/watch?v=OTSlg7_y3bA
  46. 46. Speeding Up Digital Platforms on AWS Deploy in weeks Live for years Deploy in minutes Live for weeks Deploy in seconds Live for minutes/hours Deploy in milliseconds Live for seconds On-Premises Amazon EC2 Amazon ECS AWS Lambda
  47. 47. Agenda • Web Apps Hosting on AWS • Reference Architecture • Serverless Architecture • Demo: Setup Serverless AWS • Microservices Architecture • Powered by AWS Lambda • Tips and Tricks • Drupal on Microservices • Demo: todo.deep.mg • Q&A + Next Steps
  48. 48. Powered by AWS Lambda
  49. 49. Agenda • Web Apps Hosting on AWS • Reference Architecture • Serverless Architecture • Demo: Setup Serverless AWS • Microservices Architecture • Powered by AWS Lambda • Tips and Tricks • Drupal on Microservices • Demo: todo.deep.mg • Q&A + Next Steps
  50. 50. Tips and Tricks • AWS Lambda is continuously evolving • Set up alarms for all 4 Lambda metrics in Amazon CloudWatch • Avoid S3 throttling by integrating S3 => SNS => Lambda • Beware of potential infinite loops
  51. 51. Tips and Tricks • AWS Lambda is continuously evolving • Set up alarms for all 4 Lambda metrics in Amazon CloudWatch • Avoid S3 throttling by integrating S3 => SNS => Lambda • Beware of potential infinite loops • Microservices are game changers • The shorter TTL, the more secure it becomes • First, build a service or a feature • Next, break it down into microservices
  52. 52. Agenda • Web Apps Hosting on AWS • Reference Architecture • Serverless Architecture • Demo: Setup Serverless AWS • Microservices Architecture • Powered by AWS Lambda • Tips and Tricks • Drupal on Microservices • Demo: todo.deep.mg • Q&A + Next Steps
  53. 53. Drupal on Microservices ?
  54. 54. Evolution from Monolithic Approach … applicationsdevelopers Build Test Release development cycle
  55. 55. … to Microservices Architecture applicationsdevelopers Build Test Release development cycle Build Test Release Build Test Release Build Test Release Build Test Release Build Test Release Build Test Release
  56. 56. Agenda • Web Apps Hosting on AWS • Reference Architecture • Serverless Architecture • Demo: Setup Serverless AWS • Microservices Architecture • Powered by AWS Lambda • Tips and Tricks • Drupal on Microservices • Demo: todo.deep.mg • Q&A + Next Steps
  57. 57. Demo: todo.deep.mg • Inspired from open source • www.todomvc.com • Go to the GitHub repository • github.com/MitocGroup/deep -microservices-todo-app • Follow the steps from Getting Started to build and deploy • todo.deep.mg
  58. 58. DEEP Framework https://github.com/MitocGroup/deep-framework “DEEP Framework is a serverless web framework, core component of the Platform-as-a-Service that abstracts web apps and web services from specific cloud providers. This framework enables developers build cloud-native applications or platforms using microservices architecture in a completely serverless approach”
  59. 59. Agenda • Web Apps Hosting on AWS • Reference Architecture • Serverless Architecture • Demo: Setup Serverless AWS • Microservices Architecture • Powered by AWS Lambda • Tips and Tricks • Drupal on Microservices • Demo: todo.deep.mg • Q&A + Next Steps
  60. 60. Q&A + Next Steps github.com/MitocGroup medium.com/@MitocGroup beta@deep.mg www.deep.mg Thanks: http://www.meetup.com/drupalnyc/events/227188360 Ben & Elijah from Drupal NYC Meetup Hosting Team from NBCUniversal
  61. 61. Credits and Thanks • Slide 3: Digital Platforms Challenges • http://www.buzzfeed.com/daozers/what-its-like-to-work-on-buzzfeeds-tech-team-during-record-t#.axR6WG9Yr • http://www.dailydot.com/crime/new-york-magazine-ddos-bill-cosby-cover/ • http://www.cio.in/topstory/flipkart%E2%80%99s-cto-explains-the-xiaome-launch-outage • Slide 4: Digital Platforms Challenges • http://www.slideshare.net/Radware/radware-cmg2014-tammyevertsslowtimevsdowntime • http://www.statuscast.com/application-downtime-according-to-idc-gartner-and-others • https://press.kaspersky.com/files/2014/11/B2B-International-2014-Survey-DDoS-Summary-Report.pdf • Slide 19: AWS re:Invent 2014 • https://venturebeat.com/wp-content/uploads/2014/11/aws-reinvent-lambda.png • Slide 20: AWS Summit NY 2015 • https://d0.awsstatic.com/events/aws-hosted-events/2015/AWS-Global-Summit-Series/new-york/press-room/introducing-amazon-api- gateway.jpg • Slide 46: Microservices Architecture • https://www.youtube.com/watch?v=nMTaS07i3jk - State of the Art in Microservices by Adrian Cockcroft • https://www.youtube.com/watch?v=wgdBVIX9ifA - Microservices by Martin Fowler • https://www.youtube.com/watch?v=OTSlg7_y3bA - Deploying and Operating Microservices by Sam Newman

×