Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to To Serverless And Beyond!
Similar to To Serverless And Beyond! (20)
More from SheenBrisals
More from SheenBrisals (8)
Recently uploaded
Recently uploaded (20)
To Serverless And Beyond!
- 1. To Serverless and Beyond! Successful eCommerce Migration of LEGO.com Sheen Brisals The LEGO Group @sheenbrisals
- 2. This site is poweredby Serverless
- 3. Is there a life after?
- 4. Is there a life after Serverless? No
- 5. There is life with Serverless
- 6. Agenda ▪ Starting with Serverless ▪ Travelling with Serverless ▪ Accelerating with Serverless
- 7. Starting with Serverless ▪ Motivation ▪ Preparation ▪ Quick win
- 8. Starting with Serverless ▪ Motivation ▪ Preparation ▪ Quick win
- 9. November 2017 – Migration thoughts VPC Node.js + React Elastic Beanstalk container Server contents Server contents Tax Products CRM Fulfillment Rewards VPC Gateway 503 Service Unavailable Corporate data center Server contents
- 10. Starting with Serverless ▪ Motivation ▪ Preparation ▪ Quick win
- 11. Summer 2018 – Serverless thoughts
- 12. No one starts perfect with Serverless. That’s fine, but strive to be better at every iteration, and that is important - Sheen Brisals
- 13. Starting with Serverless ▪ Motivation ▪ Preparation ▪ Quick win
- 14. A journey of a thousand miles begins with a single step - Confucius
- 15. September 2018 – First Lambda function VPC Node.js + React Elastic Beanstalk container Server contents Server contents Tax Products CRM Fulfillment Rewards AWS Cloud SaaS VPC Gateway Server contents Corporate data center Amazon API Gateway AWS Lambda function
- 16. Travelling with Serverless ▪ Principles ▪ Processes ▪ Triumphs
- 17. Serverless requires a "mind-shift" to see the products as event-driven orchestration of managed services, knitted with infrastructure code, to bring the best value for the customer - Sheen Brisals
- 18. Travelling with Serverless ▪ Principles ▪ Processes ▪ Triumphs
- 19. AWS Cloud … cos everything is AWSome • Number of services • Support and active community • Service updates
- 20. Serverless … the why matters more than the what Trigger Push Trigger Push Trigger Push Trigger Product Pricing SKU an ecosystem of managed services granular level optimisation ideal for iterative development brings engineering diversity
- 21. … coz it does what it says on the tin Managed Services • Fastrack development • Event driven & decoupling • Resilient, secure & scalable • Native integrations • Functionless (codeless)
- 22. … coz bitten badly by the monolith Microservices • Start simple (small) and evolve • Independent deploy pipeline • API & event driven communication • Refactor to grow with the technology • Service ownership
- 23. Events PAB Site map Pricing Products Stores Promotion Customer Wish list Payments Prices Rewards Discounts Orders Address BAP Cart Track & Trace Inventory Check Product Feeds
- 24. Travelling with Serverless ▪ Principles ▪ Processes ▪ Triumphs
- 25. … start small. scale fast Agile and Iterative daily standup retro planning grooming review • Practical agile • Speed matters in development • Focus squads • Solution first, perfection next • 2 week sprints
- 26. … agree fast. act fast design review agree develop Design, Review, Decide • Open mindset for ideas • Bold approach & quick decisions • Iterate and improve • Solution detailing, where needed • Make engineers architects
- 27. Architect in set-pieces … accomplish peace by piece • Visualise big but focus small • Event-driven interaction • Rehearse the design • PoC but don’t throw away • Embrace serverless patterns
- 28. A set-piece is a scene or sequence of scenes whose execution requires logistical planning with LEGO
- 29. A set-piece is a scene or sequence of scenes whose execution requires logistical planning in Architecture
- 30. Set-piece architecture requires planning and execution, but can be rehearsed in advance
- 31. Product catalog import and update Commerce platform Feeds transform Products feeds
- 32. Event-driven data pipeline Transform feed Transformed feeds queue Feeds store Commerce platform API Update data Data feeds Trigger Push Trigger Update • Feeds in JSON format • Independent feed flow • Immediate handling of feeds • Decoupled processing • Failure retry process • Failure notification DLQ Thousands per day processed as soon as possible Fine-grained control on throughput and errors Observability
- 33. Event-driven data pipeline Commerce platform API Data feeds Trigger Push Trigger Update Push Trigger Push Trigger Product Pricing SKU Observability DLQs
- 34. Order placement “What’s the status of my order?”
- 35. Order placement Shop POST GET Submit order API Submit order function Submit order queue Process order function Update status Order status API Status provider function Order cache (TTL) Order number Query status Payment providers New commerce platform Frontend visibility Processing Failed Complete Backend visibility Validating Validation failed Payment authorized Payment processing Card declined Sent to fulfillment
- 36. … cos manual is painful Automation • Account separation – Dev, Test, QA, Prod • Integration tests • Test in real environment • Grow the CI/CD pipeline • Developers towards DevOps
- 37. Travelling with Serverless ▪ Principles ▪ Processes ▪ Triumphs
- 38. July 10, 2019 shop.LEGO.com was switched to serverless on AWS
- 39. August 15, 2019 – Unified LEGO.com
- 40. AWS services in use Amazon S3 Amazon SNS AWS Fargate AWS Lambda Amazon Kinesis Data Firehose Amazon DynamoDB Amazon API Gateway AWS Step Functions Amazon SQS Amazon EC2 Amazon ECS Amazon ECR Amazon VPC AWS WAF AWS KMS AWS Identity and Access Management(IAM) AWS X-Ray Amazon Route 53 AWS CloudTrail Amazon ElastiCache Amazon CloudWatch Amazon EventBridge AWS CloudFormation Parameter Store
- 41. Growing usage stats from production 170+ Lambda functions 35+ microservices 40+ API Gateway endpoints 25+ DynamoDB tables 30+ S3 buckets 20+ SNS topics 60+ SQS queues + DLQs 200+ Systems Manager params
- 42. 1. Serverless microservices – 80% logic 2. Frontend app – React, Node.js - Fargate 3. GraphQL query layer – Fargate 4. Headless commerce platform – SaaS 5. Content management system – SaaS 6. Rewards platform - SaaS
- 43. So, we migrated. We are serverless! Hooray! Are we done then?
- 44. Accelerating with Serverless ▪ Team ▪ Value ▪ Technology
- 45. Agile, iterative and automated serverless development on AWS is like a gravity defying take-off. Once launched, no stopping! - Sheen Brisals
- 46. Accelerating with Serverless ▪ Team ▪ Value ▪ Technology
- 47. … grow serverless engineers Accelerating with Serverless ▪ Team ▪ Growth & Diversity ▪ Value ▪ Technology
- 48. https://bit.ly/31zrh1N Serverless Engineer is a software engineer who is innovative, efficient, capable of building secure, cloud first and cost effective functional solutions, iteratively using managed services
- 50. services checkout content infra management Before – Focus Squads
- 51. management Now – Full-stack Squads checkout infra apps payments future content rewards new
- 52. Accelerating with Serverless ▪ Team ▪ Growth & Diversity ▪ DevOps Culture ▪ Value ▪ Technology … you build it. You run it
- 53. • Engineers perform end to end service delivery • Become part of the on-call team • Studying logs and monitoring applications • Work with infra, security and SR specialists • Mission Control dashboard for deployment visibility
- 57. Accelerating with Serverless ▪ Team ▪ Growth & Diversity ▪ DevOps Culture ▪ Sharing ▪ Value … sharing is caring
- 60. Accelerating with Serverless ▪ Team ▪ Value ▪ Technology
- 61. Accelerating with Serverless ▪ Team ▪ Value ▪ Business value ▪ Technology
- 62. You can’t do today’s job with yesterday’s methods and be in business tomorrow - George W Bush
- 63. Black Friday 2018 – one of the best Part Serverless Part On-prem
- 64. Black Friday 2019 Best Ever! Fully Serverless
- 66. Feature – Upload product images securely to CMS Authorised employee Images, videos Assets store Process assets CMS Commerce platform Store Index Delete asset • Restricted AWS access to assets uploader • Processed assets must be deleted immediately Old CMS Migrate assets
- 67. Feature – Upload product images securely to CMS Original assets Assets CMS Commerce platform DLQ S3 events 1. Fetch asset 2. Delete asset Transporter Workflow trigger Delete asset Uploader Processor Account Uploader Account Workflow
- 68. Accelerating with Serverless ▪ Team ▪ Value ▪ Business value ▪ Customer value ▪ Technology
- 69. Customers don’t care how much you know until they know how much you care - Damon Richards
- 70. Feature – Store inventory check
- 71. Store inventory check Inventory feeds Feeds Feed handler Feeds ingest API Store inventory Shop Stock check API Stock check handler Geo coordinates Store locations 1 2 3 SKU Postcode SKU Postcode
- 72. Store inventory check Inventory feeds Feeds Feed handler Feeds ingest API Store inventory Shop Stock check API Stock check handler Geo coordinates Store locations 1 2 3 SKU Postcode SKU Postcode Dashboard Event bus
- 73. Accelerating with Serverless ▪ Team ▪ Value ▪ Technology
- 74. Accelerating with Serverless ▪ Team ▪ Value ▪ Technology ▪ Amazon EventBridge
- 75. Amazon EventBridge is a serverless event bus service that makes it easy to connect your applications with data from a variety of sources
- 79. SNS Limitations Event filtering nightmare Filtering requires custom attributes Can’t control event visibility Leads to creating topic per event type
- 81. Amazon EventBridge Custom event bus Default event bus Partner event bus Event producers (sources) Event consumers (targets) Partner apps Custom apps AWS services Event bus Filtering & routing rules
- 82. Use case: Submit order notification
- 83. Rewards platform Customer Shipment Order All orders Only VIP customer orders
- 84. Shipping Send order to SAP Checkout Submit order Rewards Process rewards New order All orders VIP customer orders
- 85. Shipping Send order to SAP Checkout Submit order Rewards Process rewards { "detail": { "metadata": { “domain”: “LEGO-SHOP”, “service": “service_checkout”, "type": “ORDER”, "status": “SUBMITTED” }, "data": { “orderNumber": “T12346789”, “customerId": “bf3703467718”, “locale”: “en-GB”, “value”: 59.99, “isVIPOrder": “true” } } } Submitted order event
- 86. Shipping Send order to SAP Checkout Submit order Rewards Process rewards { "detail": { "metadata": { “service": [“service_checkout“], "type": [“ORDER“], "status": [“SUBMITTED“] } } } { "detail": { "metadata": { “domain”: “LEGO-SHOP”, “service": “service_checkout”, "type": “ORDER”, "status": “SUBMITTED” }, "data": { “orderNumber": “T12346789”, “customerId": “bf3703467718”, “locale”: “en-GB”, “value”: 59.99, “isVIPOrder": “true” } } } All submitted orders Submitted order event
- 87. Shipping Send order to SAP Checkout Submit order Rewards Process rewards { "detail": { "metadata": { “service": [“service_checkout“], "type": [“ORDER“], "status": [“SUBMITTED“] }, "data": { “isVIPOrder": [“true“], } } } { "detail": { "metadata": { “service": [“service_checkout“], "type": [“ORDER“], "status": [“SUBMITTED“] } } } { "detail": { "metadata": { “domain”: “LEGO-SHOP”, “service": “service_checkout”, "type": “ORDER”, "status": “SUBMITTED” }, "data": { “orderNumber": “T12346789”, “customerId": “bf3703467718”, “locale”: “en-GB”, “value”: 59.99, “isVIPOrder": “true” } } } Only VIP customer orders All submitted orders Submitted order event
- 88. Use case: Payment device status monitor
- 89. LEGO store employee Order Get available devices Instore order App Payment devices
- 90. Payment provider Notifications receiver (web hook) Payment devices Device status notifications Stores the status Updates monitoring Status API Status store Status dashboard
- 91. { "SaleToPOIRequest": { "EventNotification": { "EventDetails": "newstate", "EventToNotify": "Shutdown", "TimeStamp": "2019-08-07…" }, "MessageHeader": { "SaleID": "POSSystemID12345", "ProtocolVersion": "3.0", "MessageType": "Notification", "POIID": "V400m-324688179", "MessageClass": "Event", "MessageCategory": "Event", "DeviceID": "1517998561" } } } { “Entries": [ { “DetailType": “device-status", “Source": “Adyen", “EventBusName": “payments-bus", “Detail": “$util………." } ] } { "version": "0", "id": "73bb6b40", "detail-type": "device-status", "source": "Adyen", "account": "12345", "time": "2020-01-19T14:44:24Z", "region": "eu-central-1", "resources": [], "detail": { "SaleToPOIRequest": { "EventNotification": { "EventDetails": "newstate", "EventToNotify": "Shutdown", "TimeStamp": "2019-08-07..." }, "MessageHeader": { "SaleID": "POSSystemID12345", "ProtocolVersion": "3.0", "MessageType": "Notification", "POIID": "V400m-324688179", "MessageClass": "Event", "MessageCategory": "Event", "DeviceID": "1517998561" } } } } Raw event from the payment provider API Gateway template EventBridge event
- 92. Payment provider Notifications receiver (web hook) Payment devices Device status notifications Stores the status Updates monitoring Status API Status store Status dashboard
- 93. { "version": "0", "id": "73bb6b40", "detail-type": "device-status", "source": "Adyen", "account": "12345", "time": "2020-01-19T14:44:24Z", "region": "eu-central-1", "resources": [], "detail": { "SaleToPOIRequest": { "EventNotification": { "EventDetails": "newstate", "EventToNotify": "Shutdown", "TimeStamp": "2019-08-07..." }, "MessageHeader": { "SaleID": "POSSystemID12345", "ProtocolVersion": "3.0", "MessageType": "Notification", "POIID": "V400m-324688179", "MessageClass": "Event", "MessageCategory": "Event", "DeviceID": "1517998561" } } } } EventBridge event { "detail-type": ["device-status“], "source": ["Adyen“], } Event filter pattern For monitoringtarget – Transformedevent flow { “event": “$.detail.SaleToPOIRequest.EventToNotify“, “device": “$.detail.MessageHeader.DeviceID“ } { “eventName": <event>, “deviceId": <device> } Monitoringevent
- 96. Love is in the air Feb 14 Valentines Day
- 97. Serverless is in the air Lets breath it Lets grow with it #ServerlessForEveryone #ServerlessEverywhere Feb 13 Serverless Day