Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Serverless @ hbc digital real world explorations of lambda - aws meetup - april 4th 2017

309 views

Published on

Learn how our AWS architecture at Gilt / HBC Digital is evolving from micro-services to lambda-based processing. In this talk we discuss some of our early experiences with server-less architecture. Presented at the Dublin AWS User Group meet-up on April 4th, 2017.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Serverless @ hbc digital real world explorations of lambda - aws meetup - april 4th 2017

  1. 1. Serverless@HBC Real-world Explorations of λ AWS Meetup Dublin, April 4th. Ade Trenaman, SVP Engineering, HBC Digital t: @adrian_trenaman Ryan Martin, Snr. Director Engineering g: https://github.com/rlmartin t: @hbcdigital fa: @hbcdigital in: hbc_digital
  2. 2. A minimalist abstraction of our architectural evolution 2007 Monolith 2010 Service Oriented 2012 µ-Services λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ 2016 Rise of λ
  3. 3. I hear ya.
  4. 4. adopt hold trial assess std. https://github.com/gilt/standards CodeDeploy Docker CodePipeline ION Roller ECS sbt-code-deploy AWS Lamda Docker Hub (Open Source) CloudFormation NOVA ECR
  5. 5. We have marketing URLs like: https://gilt.com/loveaws We need to look up the slug ‘loveaws’ and change that to a ‘pkey’ for our login, so we can redirect with 302 to: https://gilt.com/login?pkey=loveaws&... Existing solution routed to legacy Ruby on Rails app: not scalable. not ‘symmetric’ A small-but-important problem: marketing redirects Rails Postgres Zeus :( customer-facing traffic on Rails :(
  6. 6. λ-based solution Replace with solution using API-Gateway + Lambda + KMS 3/80 LOC (cruft/code) .js KMS used for encrypted DB credentials Response cached No longer hits rails! λ Postgres Zeus Tiny λ API Gateway
  7. 7. It’s just code.
  8. 8. Cool: encrypted environment variables (encrypted values grayed out, I’m paranoid)
  9. 9. Integration Pattern: batch to realtime Problem: we’re integrating with a batch upload (CSV/SFTP), but want to treat the payload as real-time updates. Can we future proof our solution to that we ‘contain’ the non real-time element? Can we contain the logic for connecting with SFTP into one job (not N) Can we separate the implementation (bus. logic) from the protocol (batch CSV over SFTP) Can we solve this problem fast? SFTP .csv Organisational Boundary ??? Postgres
  10. 10. Solution Solution: a pipeline of λ, connected by S3 and SNS. LOC 8/335 + 8/109 + 3/66 (cruft/code) .js KMS used to store credentials. Partly open-sourced: https://github.com/gilt/s3-sftp-bridge, https://github.com/gilt/csv-receiver SFTP .csv Organisational Boundary λ Postgres S3 https://github.com/gilt/s3-sftp-bridge λ csv-receiver SNS λ employee_cost_members-createFromSns
  11. 11. github.com/gilt/aws-lambda-config - easy config for lambda
  12. 12. github.com/gilt/node-s3-encryption-client - easy encryption for lambda
  13. 13. Notify all the things: CloudWatch + λ http://tech.gilt.com
  14. 14. Snippet: lambda_handler (python)
  15. 15. svc-herald: event-based auto-scaling Auto-scaling ‘in time’ for large scale sales and marketing events is hard. #srsly We built a service ‘svc-herald’ that leverages λ to auto-scale critical services up and down a-priori. Svc-herald soon to be open-sourced on http://github.com/gilt
  16. 16. Futures: APIDOC (http://apidoc.me) api.json model skeleton service model skeleton λ API Gateway Swagger as intermediate API format?
  17. 17. 5 Things we learn about λ
  18. 18. λ.1: Better github integration would be nice.
  19. 19. λ.2: Love the integration with KMS
  20. 20. λ.3: Code-to-cruft ratio is awesome
  21. 21. λ.4 prediction: complexity will move from ‘the boxes’ to ‘the white-space in between’ the λ’s
  22. 22. λ.5: fast, low-friction time-to-market.
  23. 23. Serverless@HBC Real-world Explorations of λ AWS Meetup Dublin, April 4th. Ade Trenaman, SVP Engineering, HBC Digital t: @adrian_trenaman Ryan Martin, Snr. Director Engineering g: https://github.com/rlmartin t: @hbcdigital fa: @hbcdigital in: hbc_digital

×