Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

34

Share

Download to read offline

Concurrency at Scale: Evolution to Micro-Services

Download to read offline

Most large-scale web companies have evolved their system architecture from a monolithic application and monolithic database to a set of loosely coupled micro-services. Using examples from Google, eBay, and KIXEYE, this talk outlines the pros and cons of these different stages of evolution, and makes practical suggestions about when and how other organizations should consider migrating to micro-services. It concludes with some more advanced implications of a micro-services architecture, including SLAs, cost-allocation, and vendor-customer relationships within the organization.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Concurrency at Scale: Evolution to Micro-Services

  1. 1. Concurrency at Scale: The Evolution to Micro-Services Randy Shoup @randyshoup linkedin.com/in/randyshoup
  2. 2. Background • CTO at KIXEYE o Real-time strategy games for web and mobile • Director of Engineering for Google App Engine o World’s largest Platform-as-a-Service o Part of Google Cloud Platform • Chief Engineer at eBay o Multiple generations of eBay’s real-time search infrastructure
  3. 3. Evolution in Action • eBay • 5th generation today • Monolithic Perl  Monolithic C++  Java  microservices • Twitter • 3rd generation today • Monolithic Rails  JS / Rails / Scala  microservices • Amazon • Nth generation today • Monolithic C++  Perl / C++  Java / Scala  microservices
  4. 4. Evolution to Micro-Services • The Monolith • Micro-Services • Reactive Systems • Migrating to Micro-Services
  5. 5. Evolution to Micro-Services • The Monolith • Micro-Services • Reactive Systems • Migrating to Micro-Services
  6. 6. The Monolithic Architecture 2-3 monolithic tiers • {JS, iOS, Android} • {PHP, Ruby, Python} • {MySQL, Mongo} Presentation Application Database
  7. 7. The Monolithic Application Pros Simple at first In-process latencies Single codebase, deploy unit Resource-efficient at small scale Cons Coordination overhead as team grows Poor enforcement of modularity Poor scaling (vertical only) All-or-nothing deploy (downtime, failures) Long build times
  8. 8. The Monolithic Database Pros Simple at first Join queries are easy Single schema, deployment Resource-efficient at small scale Cons Coupling over time Poor scaling and redundancy (all-or-nothing, vertical only) Difficult to tune properly All-or-nothing schema management
  9. 9. “If you don’t end up regretting your early technology decisions, you probably over-engineered” -- me
  10. 10. Evolution to Micro-Services • The Monolith • Micro-Services • Reactive Systems • Migrating to Micro-Services
  11. 11. Micro-Services “Loosely-coupled service oriented architecture with bounded contexts” -- Adrian Cockcroft
  12. 12. Micro-Services “Loosely-coupled service oriented architecture with bounded contexts” -- Adrian Cockcroft
  13. 13. Micro-Services “Loosely-coupled service oriented architecture with bounded contexts” -- Adrian Cockcroft
  14. 14. Micro-Services “Loosely-coupled service oriented architecture with bounded contexts” -- Adrian Cockcroft
  15. 15. Micro-Services • Single-purpose • Simple, well-defined interface • Modular and independent • More graph of relationships than tiers • Fullest expression of encapsulation and modularity • Isolated persistence (!) A B C D E
  16. 16. Micro-Services Pros Each unit is simple Independent scaling and performance Independent testing and deployment Can optimally tune performance (caching, replication, etc.) Cons Many cooperating units Many small repos Requires more sophisticated tooling and dependency management Network latencies
  17. 17. Google Services • All engineering groups organized into “services” • Gmail, App Engine, Bigtable, etc. • Self-sufficient and autonomous • Layered on one another  Very small teams achieve great things
  18. 18. Google Cloud Datastore • Cloud Datastore: NoSQL service o Highly scalable and resilient o Strong transactional consistency o SQL-like rich query capabilities • Megastore: geo-scale structured database o Multi-row transactions o Synchronous cross-datacenter replication • Bigtable: cluster-level structured storage o (row, column, timestamp) -> cell contents • Colossus: next-generation clustered file system o Block distribution and replication • Cluster management infrastructure o Task scheduling, machine assignment Cloud Datastore Megastore Bigtable Colossus Cluster manager
  19. 19. Evolution to Micro-Services • The Monolith • Micro-Services • Reactive Systems • Migrating to Micro-Services
  20. 20. Reactive Micro-Services • Responsive o Predictable performance at 99%ile trumps low mean latency (!) o Tail latencies far more important than mean or median o Client protects itself with asynchronous, non-blocking calls • Resilient o Redundancy for machine / cluster / data center failures o Load-balancing and flow control for service invocations o Client protects itself with standard failure management patterns: timeouts, retries, circuit breakers
  21. 21. Reactive Micro-Services • Elastic o Scale up and down service instances according to load o Gracefully handle spiky workloads o Predictive and reactive scaling • Message-Driven o Asynchronous message-passing over synchronous request-response o Often custom protocol over TCP / UDP or WebSockets over HTTP
  22. 22. KIXEYE Game Services • Minimize request latency o Respond as rapidly as possible to client • Functional Reactive + Actor model o Highly asynchronous, never block (!) o Queue events / messages for complex work o Heavy use of Scala / Akka and RxJava at KIXEYE • Highly Scalable and Productive o (-) eBay uses threaded synchronous model o (-) Google uses complicated callback-based asynchronous model
  23. 23. KIXEYE Service Chassis • Goal: Make it easy to build and deploy micro-services • Chassis core • Configuration integration • Registration and Discovery • Monitoring and Metrics • Load-balancing for downstream services • Failure management for downstream services • Development / Deployment Pipeline • Transport layer over REST / JSON or WebSockets • Service template in Maven • Build pipeline through Puppet -> Packer -> AMI • Red-black deployment via Asgard
  24. 24. KIXEYE Service Chassis • Heavy use of NetflixOSS • Asgard • Hystrix • Ribbon + WebSockets  Janus • Eureka  Results • 15 minutes from no code to running service in AWS (!) • Open-sourced at https://github.com/Kixeye
  25. 25. Evolution to Micro-Services • The Monolith • Micro-Services • Reactive Systems • Migrating to Micro-Services
  26. 26. Migrating Incrementally • Find your worst scaling bottleneck • Wall it off behind an interface • Replace it •  Rinse and Repeat
  27. 27. Building Micro-Services • Common Chassis / Framework o Make it trivially easy to build and maintain a service • Define Service Interface (Formally!) o Propose o Discuss with client(s) o Agree • Prototype Implementation o Simplest thing that could possibly work o Client can integrate with prototype o Implementor can learn what works and what does not
  28. 28. Building Micro-Services • Real Implementation o Throw away the prototype (!) •  Rinse and Repeat
  29. 29. “So you are really serious about this …” • Distributed tracing • Trace a request chain through multiple service invocations • Network visualization • “Weighted” communication paths between microservices / instances • Latency, error rates, connection failures • Dashboard metrics • Quickly scan operational health of many services • Median, 99%ile, 99.9%ile, etc. • Netflix Hystrix / Turbine
  30. 30. Micro-Service Organization • Small, focused teams • Single service or set of related services • Minimal, well-defined “interface” • Autonomy • Freedom to choose technology, methodology, working environment • Responsibility for the results of those choices • Accountability • Give a team a goal, not a solution • Let team own the best way to achieve the goal
  31. 31. Micro-Service Relationships • Vendor – Customer Relationship o Friendly and cooperative, but structured o Clear ownership and division of responsibility o Customer can choose to use service or not (!) • Service-Level Agreement (SLA) o Promise of service levels by the provider o Customer needs to be able to rely on the service, like a utility • Charging and Cost Allocation o Charge customers for *usage* of the service o Aligns economic incentives of customer and provider o Motivates both sides to optimize
  32. 32. Recap: Evolution to Micro-Services • The Monolith • Micro-Services • Reactive Systems • Migrating to Micro-Services
  33. 33. Thank You! • @randyshoup • linkedin.com/in/randyshoup • Slides will be at slideshare.net/randyshoup
  • RajuGottupalli1

    Jul. 9, 2019
  • RajendraAlluri

    Sep. 5, 2018
  • ReddyGurram

    May. 11, 2016
  • szlamb

    Apr. 25, 2016
  • DcCdd

    Nov. 23, 2015
  • passionlim

    Aug. 16, 2015
  • MikeGarrett2

    Jun. 1, 2015
  • unclenate

    Apr. 24, 2015
  • caseorganic

    Apr. 24, 2015
  • alexlarikov

    Mar. 31, 2015
  • janyeh

    Mar. 31, 2015
  • TrestYeung

    Mar. 30, 2015
  • eleriane

    Mar. 25, 2015
  • ssuser8db482

    Mar. 24, 2015
  • acoronadoiruegas

    Feb. 19, 2015
  • hrxone

    Feb. 16, 2015
  • MichaelMcClure

    Feb. 16, 2015
  • zouyanjian

    Feb. 11, 2015
  • bunkertor

    Feb. 2, 2015
  • tagyangyang

    Feb. 1, 2015

Most large-scale web companies have evolved their system architecture from a monolithic application and monolithic database to a set of loosely coupled micro-services. Using examples from Google, eBay, and KIXEYE, this talk outlines the pros and cons of these different stages of evolution, and makes practical suggestions about when and how other organizations should consider migrating to micro-services. It concludes with some more advanced implications of a micro-services architecture, including SLAs, cost-allocation, and vendor-customer relationships within the organization.

Views

Total views

9,260

On Slideshare

0

From embeds

0

Number of embeds

756

Actions

Downloads

174

Shares

0

Comments

0

Likes

34

×