Decompose That WAR! Architecting for Adaptability, Scalability, and Deployability (jmaghreb, jmaghreb2013)


Published on

It’s no longer acceptable to develop large, monolithic, single-language, single-framework Web applications. In this session, you will learn how to use the scale cube to decompose your monolithic Web application into a set of narrowly focused, independently deployable services. The presentation discusses how a modular architecture makes it easy to adopt newer and better languages and technologies. You will learn about the various communication mechanisms—synchronous and asynchronous—that these services can use.

Published in: Technology, Education

Decompose That WAR! Architecting for Adaptability, Scalability, and Deployability (jmaghreb, jmaghreb2013)

  1. 1. Decompose That WAR! Architecting for Adaptability, Scalability, and Deployability Chris Richardson Author of POJOs in Action Founder of the original @crichardson @crichardson
  2. 2. Presentation goal How decomposing applications improves deployability and scalability and simplifies the adoption of new technologies @crichardson
  3. 3. (About Chris) @crichardson
  4. 4. About Chris() @crichardson
  5. 5. About Chris @crichardson
  6. 6. About Chris @crichardson
  7. 7. Agenda The (sometimes evil) monolith Decomposing applications into services API gateway design Refactoring the monolith @crichardson
  8. 8. Let’s imagine you are building an online store @crichardson
  9. 9. Traditional application architecture WAR/EAR StoreFrontUI Browser Apache/ Load balancer Spring MVC Product Info Service Recommendation Service Spring Hibernate MySQL Database Review Service Simple to develop test deploy scale Order Service Tomcat @crichardson
  10. 10. But there are problems with a monolithic architecture @crichardson
  11. 11. Users expect a rich, dynamic and interactive experience @crichardson
  12. 12. Presentation layer evolution.... WAR Browser Old HTML / HTTPre tu e tyl s Ia U h ug o StoreFrontUI en d oo ’t g Controller isn View c ite rch Model @crichardson
  13. 13. ...Presentation layer evolution Browser - desktop and mobile View Web application Controller Model Static content JSON-REST HTML 5 - JavaScript Native mobile client IoS or Android Events RESTful Endpoints Event publisher No elaborate, server-side web framework required @crichardson
  14. 14. Intimidates developers @crichardson
  15. 15. Obstacle to frequent deployments Need to redeploy everything to change one component Interrupts long running background (e.g. Quartz) jobs Increases risk of failure Eggs in one basket Fear of change Updates will happen less often - really long QA cycles e.g. Makes A/B testing UI really difficult @crichardson
  16. 16. Overloads your IDE and container Slows down development @crichardson
  17. 17. Obstacle to scaling development Accounting Engineering E-commerce application Shipping team @crichardson
  18. 18. Obstacle to scaling development WAR UI team StoreFrontUI Product Info team Product Info service Recommendations team Recommendation service Reviews team Review service Orders team Order service @crichardson
  19. 19. Obstacle to scaling development I want to update the UI But the backend is not working yet! Lots of coordination and communication required @crichardson
  20. 20. Requires long-term commitment to a technology stack @crichardson
  21. 21. Agenda The (sometimes evil) monolith Decomposing applications into services API gateway design Refactoring the monolith @crichardson
  22. 22. @crichardson
  23. 23. The scale cube X axis - horizontal duplication Sc Z ax is Scale by splitting different things ale - d by ata sp pa th litt rtit in in ion gs g sim ing ila r Y axis functional decomposition @crichardson
  24. 24. Y-axis scaling - application level WAR StoreFrontUI Product Info Service Recommendation Service Review Service Order Service @crichardson
  25. 25. Y-axis scaling - application level product info application Product Info Service recommendations application Store front application StoreFrontUI Recommendation Service reviews application Review Service orders application Order Service Apply X axis cloning and/or Z axis partitioning to each service @crichardson
  26. 26. Partitioning strategies... Partition by noun, e.g. product info service Partition by verb, e.g. shipping service Single Responsibility Principle Unix utilities - do one focussed thing well @crichardson
  27. 27. Partitioning strategies Too few Drawbacks of the monolithic architecture Too many - a.k.a. Nano-service anti-pattern Runtime overhead Potential risk of excessive network hops Potentially difficult to understand system Something of an art @crichardson
  28. 28. Inter-service communication options Synchronous HTTP asynchronous AMQP Formats: JSON, XML, Protocol Buffers, Thrift, ... Asynchronous is preferred but REST is popular too JSON is fashionable but binary format is more efficient @crichardson
  29. 29. Example micro-service class RegistrationSmsServlet extends RegistrationSmsScalatraStack { } post("/") { val phoneNumber = request.getParameter("From") val registrationUrl = System.getenv("REGISTRATION_URL") + "?phoneNumber=" + encodeForUrl(phoneNumber) <Response> <Sms>To complete registration please go to {registrationUrl} </Sms> </Response> } For more on micro-services see http:/ / @crichardson
  30. 30. Real world examples eBaySDForum2006-11-29.pdf @crichardson
  31. 31. There are drawbacks @crichardson
  32. 32. Complexity See Steve Yegge’s Google Platforms Rant re @crichardson
  33. 33. Multiple databases & Transaction management @crichardson
  34. 34. Implementing features that span multiple services @crichardson
  35. 35. When to use it? In the beginning: •You don’t need it •It will slow you down Later on: •You need it •Refactoring is painful @crichardson
  36. 36. But there are many benefits @crichardson
  37. 37. Smaller, simpler apps Easier to understand and develop Reduced startup time - important for GAE Less jar/classpath hell Who needs OSGI? @crichardson
  38. 38. Scales development: develop, deploy and scale each service independently @crichardson
  39. 39. Improves fault isolation @crichardson
  40. 40. Eliminates long-term commitment to a single technology stack Modular, polyglot, multiframework applications @crichardson
  41. 41. Two levels of architecture System-level Services Inter-service glue: interfaces and communication mechanisms Slow changing Service-level Internal architecture of each service Each service could use a different technology stack Pick the best tool for the job Rapidly evolving @crichardson
  42. 42. Easily try other technologies ... and fail safely @crichardson
  43. 43. The human body as a system @crichardson
  44. 44. 50 to 70 billion of your cells die each day @crichardson
  45. 45. Yet you (the system) remain you @crichardson
  46. 46. Can we build software systems with these characteristics? DesignBeyondHumanAbilitiesSimp.pdf @crichardson
  47. 47. Agenda The (sometimes evil) monolith Decomposing applications into services API gateway design Refactoring the monolith @crichardson
  48. 48. Directly connecting the front-end to the backend Chatty API View REST Product Info service REST Recommendation Service AMQP Controller Review service Model Traditional web application View Controller Model Browser/Native App Web unfriendly protocols @crichardson
  49. 49. Use an API gateway Single entry point View REST Product Info service REST Recommendation Service AMQP Controller Review service Model Traditional web application View API Gateway Controller Model Browser/Native App Client specific APIs Protocol translation @crichardson
  50. 50. Optimized client-specific APIs Desktop Browser getProductInfo() getRecomm...() getReviews() NodeJS API Gateway REST proxy Mobile App getProductDetails() Event publishing REST Product Info service REST Recommendation Service AMQP Review service @crichardson
  51. 51. Netflix API Gateway Device specific end points @crichardson
  52. 52. Developing an API gateway Java EE web technologies Netty-based technology stack Non-Java options: e.g. Node.JS @crichardson
  53. 53. API gateway design issues @crichardson
  54. 54. The need for parallelism Product Info getProductDetails() get Product Details Product Details API getRecomendations() Recommendations Call in parallel getReviews() Reviews @crichardson
  55. 55. Futures are a great concurrency abstraction An object that will contain the result of a concurrent computation Future<Integer> result = executorService.submit(new Callable<Integer>() {... }); Java has basic futures. We can do much better.... @crichardson
  56. 56. Better: Futures with callbacks val f : Future[Int] = Future { ... } f onSuccess { case x : Int => println(x) } f onFailure { case e : Exception => println("exception thrown") } Guava ListenableFutures, Java 8 CompletableFuture, Scala Futures @crichardson
  57. 57. Even better: Composable Futures val f1 = Future { val f2 = Future { ... ; 1 } ... ; 2 } Transforms Future val f4 = * 2) assertEquals(4, Await.result(f4, 1 second)) Combines two futures val fzip = f1 zip f2 assertEquals((1, 2), Await.result(fzip, 1 second)) def asyncOp(x : Int) = Future { x * x} val f = Future.sequence((1 to 5).map { x => asyncOp(x) }) assertEquals(List(1, 4, 9, 16, 25), Await.result(f, 1 second)) Transforms list of futures to a future containing a list Scala Futures @crichardson
  58. 58. Composing concurrent requests using Scala Futures class ProductDetailsService @Autowired()(....) { def getProductDetails(productId: Long) = { val productInfoFuture = productInfoService.getProductInfo(productId) val recommendationsFuture = recommendationService.getRecommendations(productId) val reviewsFuture = reviewService.getReviews(productId) for (((productInfo, recommendations), reviews) <productInfoFuture zip recommendationsFuture zip reviewsFuture) yield ProductDetails(productInfo, recommendations, reviews) } } Asynchronously creates a Future containing result @crichardson
  59. 59. Handling partial failures Product Info getProductDetails() get Product Details Product Details Controller getRecomendations() getReviews() X Recommendations Reviews @crichardson
  60. 60. About Netflix > 1B API calls/day 1 API call average 6 service calls Fault tolerance is essential @crichardson
  61. 61. How to run out of threads Thread 1 X HTTP Request Thread 2 X X Thread 3 API Gateway Code X Recommendation Service Thread n X Execute thread pool Tomcat Eventually all threads will be blocked If service is down then thread will be blocked @crichardson
  62. 62. Their approach Network timeouts Invoke remote services via a bounded thread pool Use the Circuit Breaker pattern On failure: return default/cached data return error to caller @crichardson
  63. 63. Agenda The (sometimes evil) monolith Decomposing applications into services API gateway design Refactoring the monolith @crichardson
  64. 64. How do you decompose your big, scary monolithic application? @crichardson
  65. 65. Strategy #1: Stop digging @crichardson
  66. 66. New functionality = service Pristine Monolith Anti-corruption layer Service Glue code @crichardson
  67. 67. Sounds simple but... Dependencies between monolith and service e.g. Common entities Building the anti-corruption layer can be challenging Must replicate data between systems ... @crichardson
  68. 68. Strategy #2: Split front-end & backend @crichardson
  69. 69. Split front-end & backend Independently deployable Monolith Presentation layer Front-end Backend Backend @crichardson
  70. 70. Strategy #3: Decompose front-end @crichardson
  71. 71. Decompose front-end Front-end Catalog /catalog Catalog Checkout /checkout Checkout Account /account Account Orders /orders Orders @crichardson
  72. 72. Issues and challenges Server-side Session state: Need to use a separate session state server ... @crichardson
  73. 73. Strategy #4: extract services @crichardson
  74. 74. Module service ... WAR Module @crichardson
  75. 75. ... Module WAR service Anti-corruption layer Service @crichardson
  76. 76. What to extract? Have the ideal partitioned architecture in mind: Partitioned by verb or noun Start with troublesome modules: Frequently updated Stateful components that prevent clustering Conflicting resource requirements @crichardson
  77. 77. Untangling dependencies Monolith Service API A API C API B Trouble! Domain model 1 X Domain model 2 C A Y B Z @crichardson
  78. 78. Useful idea: Bounded context Different services have a different view of a domain object, e.g. User Management = complex view of user Rest of application: User = PK + ACL + Name Different services can have a different domain model Services exchange messages to synchronize data @crichardson
  79. 79. Untangling orders and customers Customer Service Order Service availableCredit() updateCustomer() placeOrder() Order management Order Customer management belongs to has orders creditLimit ... total Invariant: sum( <= creditLimit Customer Trouble! available credit= creditLimit sum( @crichardson
  80. 80. Replicating the credit limit Order Service sum( <= creditLimit placeOrder() Order management updateCustomer() Customer management Customer Order Simplified Customer Service creditLimit ... total CreditLimitChangedEvent Customer’ creditLimit @crichardson
  81. 81. Maintaining the openOrderTotal Customer Service Order Service placeOrder() = creditLimit - openOrderTotal Order management availableCredit() Customer management Customer Order creditLimit openOrderTotal ... customerId total OpenOrderTotalUpdated @crichardson
  82. 82. Refactoring a monolith is not easy BUT the alternative is far worse @crichardson
  83. 83. Summary @crichardson
  84. 84. Monolithic applications are simple to develop and deploy BUT have significant drawbacks @crichardson
  85. 85. Apply the scale cube Modular, polyglot, and scalable applications Services developed, deployed and scaled independently @crichardson
  86. 86. Use a modular, polyglot architecture View REST Controller Product Info service REST Recommendation Service AMQP Review service Model Traditional web application View Controller API Gateway Model @crichardson
  87. 87. @crichardson Questions? @crichardson