Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DAT322_The Nanoservices Architecture That Powers BBC Online


Published on

The BBC’s website and apps are used around the world by an audience of millions who read, watch, and interact with a range of content. The BBC handles this scale with an innovative website platform, built on Amazon ElastiCache and Amazon EC2 and based on nanoservices. The BBC has over a thousand nanoservices, powering many of its biggest webpages. Explore its nanoservices platform and use of ElastiCache. Learn how Redis’s ultra-fast queues and pub/sub allow thousands of nanoservices to interact efficiently with low latency. Discover intelligent caching strategies to optimize rendering costs and ensure lightning fast performance. Together, ElastiCache and nanoservices can make real-time systems that can handle thousands of requests per second.

  • Be the first to comment

DAT322_The Nanoservices Architecture That Powers BBC Online

  1. 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. The Nanoservices Architecture That Powers BBC Online M a t t h e w C l a r k , B B C N o v e m b e r 2 8 , 2 0 1 7 AWS re:INVENT D A T 3 2 2
  2. 2. Background Pattern Our implementation Pros & cons This is the story of the nanoservices pattern. Small units of logic that are easily understood, deployed, operated, and shared. Built on an AWS-hosted platform that makes extensive use of Amazon ElastiCache. 4
  3. 3. AWS’s hosted service for Redis (and Memcached) Automates common administrative tasks (e.g., failed nodes, server optimisation) Multiple options available (different memory and network sizes) Redis is much more than a key-value store: multiple data types, transactions, Lua scripting, pub/sub, … Can grow with sharding and/or replication Amazon ElastiCache 5
  4. 4. Make the common things easy and the special things possible
  5. 5. 8
  6. 6. Multiple teams Multiple technologies Smaller and safer deploys Shareable and flexible Easily scalable Pros 9 Microservices Cons Maintenance overhead Comms overhead Multi-version challenges Harder to refactor Cost
  7. 7. [Microservices] bring an enormous amount of operational complexity to the table. Suddenly, you need to continuously deploy many different (possibly containerized) services. New concerns arise: service discovery, distributed logging, tracing and so on. You are now even more prone to the fallacies of distributed computing. Versioning of interfaces and configuration management become a major concern. The list goes on and on. –Sander Mak 10
  8. 8.
  9. 9. Responsible for a single, non-trivial, business-understandable task Smaller than a microservice, larger than a function Likely to call other nanoservices, to offer rich functionality The ideal sized thing to release (or MVT) in a CD environment Example: – Get customer order history – Place order Nanoservice 12
  10. 10.
  11. 11. Event page Top Left Centre Right Medals module Medals page Medals data Mobile alert DB Multi- lingual medals Internet connected TV
  12. 12. Architecture 18 Developer Data (APIs, etc.) User Nanoservice store Nanoservice platform Metadata store Content cache Queue
  13. 13. Every output from every run of every nanoservice is stored Cheap, simple, ephemeral Maximises reuse, speed, reliability, and scalability Redis for content 19
  14. 14. 4x m4.xlarge (sharded) Redis: We store 6–8 million items 20
  15. 15. Metadata about every request, and its ‘tree’, is stored in Redis If the data changes, the metadata informs us what nanoservices to rerun The ‘relationship’ is stored both ways:  ‘A uses B’ and ‘B is used by A’  Not ‘normal form’, but fast  Lua can help make atomic  ‘Hexastore’ approach can do full graphs Redis for metadata 21 Medals page Medals module Medals Data Mobile alert DB
  16. 16. Redis, as a queue, is extraordinarily fast (<5 ms end-to-end journey) It’s the only solution fast enough for this architecture We comfortably do 10k messages per second (r3.4xlarge) Redis for queues 22 We use Lua to also write to a set, to then catch de-duplicates producer LPUSH queue msg BRPOP queue consumer
  17. 17. Doesn’t really work with replication  Can be a SPOF Lossy (due to consumer failure), though can be offset with:  BRPOPLPUSH <queue> <in-progress-queue> <timeout>  And something to analyse the in-progress-queue Metrics not so obvious Harder to scale No advanced features (dead-letter, prioritization, data pushes) Redis for queues: Trade-offs 23
  18. 18. Cross-AZ Redis calls can be 4x slower* (though still very quick) Replication helps (but you become eventually consistent) To minimise latency, consider multiple single-AZ deployments: An alternative cross-AZ structure 24 * redis-benchmark -c 1 -n 10000 -t GET on m4.large against m4.large ElastiCache, does 7,200/sec same AZ, 1,800/sec different AZ Availability Zone #1 Availability Zone #2
  19. 19. Each nanoservice is developed & managed independently Nanoservice Owner Tests Connected nanoservices Single-click deploy Analytics UsersStatic files CodeVersions
  20. 20. Demo 26
  21. 21. Great ‘Lego bricks’ for sharing Easy to understand concepts, reduces barrier-to-entry Should be good value (sharing infrastructure, highly cacheable) Looks after much of the operational side (scaling, resilience, monitoring, etc.) Fast deployment  Greater development speed Nanoservices: Advantages 27
  22. 22. Single point of failure Must be conservative, isolate and limit Can be restrictive Not very fast Needs great tooling to manage and understand usage Challenges 28
  23. 23. Multiple teams Multiple technologies Smaller and safer deploys Shareable and flexible More scalable Pros 29 Microservices Cons Maintenance overhead Comms overhead Multi-version challenges Harder to refactor Cost ✓ ✓ ✓ ✓ ?
  24. 24. Serverless AWS Lambda functions Azure functions Google Cloud functions 30 Why should they have all the fun? The world needs powerful serverless platforms.
  25. 25. Is this serverless? 31 We didn’t use Lambda because:  The need for consistent, <10 ms response  Instantiation from a Redis queue  Twiddling thumbs (waiting for dependencies) We’re considering ‘burst to Lambda’ possibilities To the developers, this is serverless
  26. 26. Serverless will fundamentally change how we build business around technology and how you code. … You thought devops was big but it’s chicken feed compared to this… ...expect to see the whole world being overtaken by serverless by 2025 –Simon Wardley, in Why the fuss about serverless? 32
  27. 27. 33 Is nanoservices a paradigm for future serverless architecture?
  28. 28. a8bbd5b899a0 It’s not just us 34
  29. 29. But this is (if you want to read more) 35
  30. 30. Nanoservices have brought the BBC: Sharing Flexibility Efficiency Velocity
  31. 31. Summary Using Redis, you can achieve phenomenal performance, not just in caching content, but as a control and communication layer. Which can let you build fast and flexible serverless-like platforms that allow your organisation to move faster. The nanoservice pattern is an interesting way to design for serverless, and can maximise the potential of continuous delivery, reusability, and flexibility. 37
  32. 32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank you! M a t t h e w C l a r k m a t t h e w . c l a r k @ b b c . c o . u k @ m a t t h e w 1 0 0 0