Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Go Global with Cloud-Native Architecture: Deploy AdTech Services Across Four Continents (ADT205) - AWS re:Invent 2018

495 views

Published on

Plista, a Germany-based advertising solution provider, discusses how they use a cloud-native architecture, container-first approach to speed up development, increase agility, reduce latency, localize storage, and foster innovation and ownership in their organization. They demonstrate how a cloud blueprint is used to easily roll out their services to new global markets. With this architecture, Plista processes 1.7 billion requests per day, across four continents. They also discuss how they're adapting for GDPR compliance and redesigning parts of their platform to leverage new AWS services.

  • Be the first to comment

Go Global with Cloud-Native Architecture: Deploy AdTech Services Across Four Continents (ADT205) - AWS re:Invent 2018

  1. 1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Using Cloud Native for AdTech Services on AWS to Grow Globally Torben Brodt Global Director Data Engineering plista GmbH A D T 2 0 5 Philipp Hoffmann Chapter Lead Data Core plista GmbH Karl Bunch Worldwide Technical Leader Digital Advertising Amazon Web Services
  2. 2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda AdTech plista’s 3 attempts to grow Deep dives on Agility, Big Data and Scale
  3. 3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  4. 4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AdTech two tiers Backend smart decisions Serving (<=100ms)
  5. 5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AdTech Australia 2014 China 2016 APAC 2016 Germany 2008 North America 2018
  6. 6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AdTech 28 Markets 16 GB/hour 20,000 Recommendations/ sec
  7. 7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  8. 8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. 1. Growing Australia OPTIONS a) Build as autonomous region b) Only do serving from AU DECISION DRIVERS - double infrastructure - double costs - double manpower - unclear growth Germany 2008 Australia 2014
  9. 9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. 1. Growing Australia Serving from Australia + Data processing from Germany* *In DE we run on-prem
  10. 10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Australia 1. Growing Australia Guess what we learned! Germany
  11. 11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Australia 1. Growing Australia CONCLUSION Business was growing rapidly. Tech fears did not come true. We are GOOD! WHAT WE LEARNED - it’s extra operational complexity - a lot of data transfer across regions (costs + delays + problems) - at this stage, still quite expensive Germany
  12. 12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. We had similar uncertainties For less data transfer problems, we split Master Data into regions For reasons of costs and complexity we still wanted to re-use the Big Data cluster in Germany. China 2016 APAC 2016 2. Growing APAC & China
  13. 13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. 2. Growing APAC & China Germany: Big Data Singapore: Master Data + limited backend processing CN: Serving
  14. 14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. China APAC 2. Growing APAC & China Guess what we learned! Germany
  15. 15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. China APAC 2. Growing APAC & China CONCLUSION - we added even more complexity - good cost WHAT WE LEARNED - underestimated Big China firewall - still many problems with data transfer across regions Germany
  16. 16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. 3. Growing America No more fears, age of containers This time we wanted to make it right! - Cloud Native + Autonomous - Terraform Blueprint - Aurora - ElasticCache - Datalake - etc North America 2018
  17. 17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. 3. Growing America Ohio: Serving + Master Data + Data Lake + Backend Processing
  18. 18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. 3. Growing America Guess what we learned! North America
  19. 19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. 3. Growing America CONCLUSION - we are confident in cloud - this now serves as blueprint WHAT WE LEARNED - faster in development / rapid prototyping - focus on business + innovation North America
  20. 20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  21. 21. Thank you! © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Karl Bunch Worldwide Technical Leader Digital Advertising Amazon Web Services Torben Brodt Global Director Data Engineering plista GmbH Philipp Hoffmann Chapter Lead Data Core plista GmbH
  22. 22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  23. 23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  24. 24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. PoC ● Learn AWS ● Build a pilot project ● Build support within organization Data Migration ● Leverage different storage options ● Create a rollback plan ● Migrate Application Migration ● Forklift migration ● Hybrid migration strategy Optimization ● Utilization ● Monitoring ● Efficiency ● Performance ● Re- engineering Assessment ● Assess architecture ● Assess costs ● Assess Security Leverage the Cloud ● Autoscaling ● Automation ● Elasticity ● High-availability Cloud Migration Phases
  25. 25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Terraform Prerequisites - Unified, cloud-agnostic configuration - Avoid a vendor lock-in - Open-source Keys - Single abstraction layer (no more provider’s IDs in our configuration) - Declarative configuration language (don’t need to care about API calls order) - Configuration could be used as inventory source - Open-source! :)
  26. 26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  27. 27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Serverless Data Lake
  28. 28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. GDPR
  29. 29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  30. 30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Spot Instances / Auto-Scaling
  31. 31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Spot Instances / Auto-Scaling ~50% cost saving
  32. 32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  33. 33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Going Cloud Native? On-Premise AWS Comments Caching Redis Redis/ElastiCache No cross-region replication for EC Messaging Kafka Kafka Integration with existing stack Resource Manager Mesos EMR Spark works well on both Container Orch. Mesos ECS/(EKS)
  34. 34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Going Cloud Native? On-Premise AWS Comments Offline Storage HDFS S3 Spark integrates with both SQL MySQL Aurora Compatible interface Data Processing Spark Spark/EMR Load Balancing DNS/haproxy DNS/ELB
  35. 35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Data Processing (Single-Region)
  36. 36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Data Processing (Multi-Region)
  37. 37. Thank you! © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Karl Bunch Worldwide Technical Leader Digital Advertising Amazon Web Services Torben Brodt Global Director Data Engineering plista GmbH Philipp Hoffmann Chapter Lead Data Core plista GmbH

×