Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

From Winning the Microservice War to Keeping the Peace

134 views

Published on

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2RLYvFy.

Andrew McVeigh explains how transitioning to a microservice architecture went for a top-tier video service & also a major games studio. Operating and evolving such significant systems reveals a number of challenges: monitoring and root cause analysis, cascading failures and scaling for major events are just some. McVeigh discusses how to avoid common pitfalls when working with microservices. Filmed at qconsf.com.

Andrew McVeigh is VP Architecture at Hulu. He is an experienced technical leader & architect, with a background in many different languages, platforms, and technologies. He's worked successfully at the highest level across a number of domains including gaming, investment banking, digital broadcasting, speech research, and telecoms.

Published in: Technology
  • Be the first to comment

From Winning the Microservice War to Keeping the Peace

  1. 1. Microservices From Winning the War …To Keeping the Peace Andrew McVeigh QConSF October 2018
  2. 2. InfoQ.com: News & Community Site • Over 1,000,000 software developers, architects and CTOs read the site world- wide every month • 250,000 senior developers subscribe to our weekly newsletter • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • 2 dedicated podcast channels: The InfoQ Podcast, with a focus on Architecture and The Engineering Culture Podcast, with a focus on building • 96 deep dives on innovative topics packed as downloadable emags and minibooks • Over 40 new content items per week Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ microservices-pitfalls-lessons
  3. 3. Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon San Francisco www.qconsf.com
  4. 4. Background A bit about Andrew’s work... ◎ Many different domains ◎ Trading & risk systems ◎ PhD on software components ◎ Riot Games ◎ Hulu 2
  5. 5. Microservices Work! But make sure you prepare for the challenges... 3
  6. 6. 1. Microservices in Gaming 2. Microservices for Video 3. Believe the Hype 4. Believe the Challenges 4
  7. 7. 1. Microservices in Gaming A microservice architecture @ scale 5
  8. 8. League of Legends ◎ Here you have a list of items ◎ And some text ◎ But remember not to overload your slides with content Your audience will listen to you or read the content, but won’t do both. 6
  9. 9. @ Scale 7
  10. 10. Gaming Particulars ◎ Low latency ◎ Need to match up players = shared state ◎ Rapid development cycles ◎ Lots of engineers working on 1 game 8
  11. 11. 1A. Winning the War Evolving from a monolith 9
  12. 12. Evolving the Architecture ◎ 2009: Large service monolith ○ Matchmaking, game selection, inventory etc ◎ 2012: Started evolving to microservices ○ http://bit.ly/evolving-league 10
  13. 13. Front End Box League of Legends 11 platform.war Distributed in-memory cache MySQL Matchmaking Game Server Mgr platform.warplatform.warplatform.war platform.warplatform.war MySQL
  14. 14. Microservices for New Features 12 Platform platform.war Distributed in-memory cache MySQL Service Proxy / API MySQL Match Making tb.jar Cache
  15. 15. REST SWAGGER Dashboards Service Discovery Config Service Standard Infrastructure 13 Client of Service Metrics Client REST library S/W Load Balancer Service Metrics Server REST library HTTP Server
  16. 16. 1B. Keeping the Peace Held back by remains of the monolith 14
  17. 17. Not Quite Free of the Monolith... 15 Platform ● Inventory ● Stats ● Summoner details Distributed in-memory cache MySQL MySQL Microservice Cache
  18. 18. Ouch - Hitting the Wall 16
  19. 19. If We Could Redo? ◎ Decouple state completely ○ Inventory service ○ Catalog service ○ Runes service ◎ Socialize to get prioritization ◎ Simplify infrastructure ○ Config System too “clever” ○ Too much “smarts” in fat libraries 17
  20. 20. 2. Microservices for Internet Video Hundreds of tiny pieces... 18
  21. 21. Hulu ◎ 2016: Full microservices architecture ○ Evolved / replaced existing VOD architecture ○ Live TV ○ 20m+ total subs ○ 1m+ live subs ◎ 15 month development! ○ 800+ microservices http://bit.ly/hulu-landscape 19
  22. 22. Video System Particulars ◎ Lots of caching to support browsing ◎ TV show metadata needed everywhere ◎ Real-time playback to support live TV ◎ Lots of integration (billing, ads etc) 20
  23. 23. Infrastructure for Microservices 21 Donki PaaS AppsGithub Jenkins Apps Apps Redis Cluster MySQL Cluster Provisioning System Apps Apps VMs Load Balancers DNS Statsd Graphite
  24. 24. Microservice Ownership 22 Playback team Browse team Recording team Etc...
  25. 25. Data Distribution Pattern 23 Master Cache CacheCacheCache Ingestion + Admin Reading Pub-sub / read-through
  26. 26. Scaling for Launch Day 24 Browse Queries Per Second College kids falling asleep with XBOX on!
  27. 27. 2B. Keeping the Peace Every issue gets magnified... 25
  28. 28. Arch / Ops Dashboard 26 Login Browse & Search Playback Ads DVR … etcPlayback
  29. 29. http://bit.ly/hulu-scaling Scaling For Growth 27
  30. 30. Circuit Breakers avoid Firestorms 28 A B C X YY X A B C Z
  31. 31. Cross-Cutting Requirements 29
  32. 32. Cloud versus DC ◎ Could target cloud or DC ○ But no elasticity, must overprovision ◎ Abstracting cloud & DC == Lowest common denominator ◎ Hard to do proper blue-green in DC 30
  33. 33. 3. Believe the Hype The many benefits of microservices 31
  34. 34. They Actually Work! ◎ Ownership & Independence ◎ Development velocity ◎ Operational & Development scaling 32
  35. 35. They Actually Work! ◎ Granular deployment ◎ Evolution ◎ Organizational alignment 33
  36. 36. 4. Believe the Challenges Standing on the shoulders of constantly improving infrastructure... 34
  37. 37. Common CICD + Operating Env Runtime Infra 35 Build + CICD Operational Infra Deploy & Rollback Jenkins Spinnaker / Harness Terraform K8s, istio Stackdriver Cloud!
  38. 38. Cloud Approach Preferred approach ◎ Pick one cloud provider (per workflow?) ◎ Consider costs early ◎ Multi-region, multi-account on day 1 Cloud has so many advantages over DC ◎ Elasticity ◎ Easy environments (blue-green) ◎ Better shared services (db, queues etc) 36
  39. 39. Circuit Breakers by Default! 37
  40. 40. API Gateways 38 Gateway ServiceService Envoy API Docs
  41. 41. Platforms For things that need holistic treatment … but still allow self-service ◎ Load testing ◎ Billing ◎ Browse caching ◎ A / B testing ◎ UI layout 39
  42. 42. Takeaways Microservices offer many benefits ◎ Isolation & independence ◎ Granular deployment, scaling & evolution Use infra to protect against common issues ◎ Full CICD ◎ Infrastructure as code ◎ Circuit breakers to prevent firestorms ◎ Istio to help with monitoring + more ◎ Cloud elasticity FTW 40
  43. 43. Thanks for Listening! Any questions? andrew.mcveigh@gmail.com 41 AMA Decomposing the Monolith Tues 2:55pm Boardroom C
  44. 44. 42
  45. 45. Scaling For Growth 43But look out!!!
  46. 46. Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ microservices-pitfalls-lessons

×