Decomposing Applications for Scalability and Deployability (April 2012)


Published on

Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers. In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as NodeJS and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application.

Published in: Technology, Education
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Decomposing Applications for Scalability and Deployability (April 2012)

  1. 1. Decomposing applications fordeployability and scalability Chris Richardson Author of POJOs in Action Founder of the original @crichardson 1
  2. 2. Presentation goal Decomposing applications to improve deployability and scalability and How Cloud Foundry helps 2
  3. 3. About Chris 3
  4. 4. (About Chris) 4
  5. 5. About Chris() 5
  6. 6. About Chris 6
  7. 7. About Chris 7
  8. 8. vmc push About-Chris Developer Advocate for CloudFoundry.comSignup at http://cloudfoundry.comPromo code: CFOpenTour2012 8
  9. 9. Agenda§ The (sometimes evil) monolith§ Decomposing applications into services§ How do services communicate?§ Presentation layer design§ Decomposing a monolithic application§ How Cloud Foundry helps 9
  10. 10. Let’s imagine you are building an e- commerce application 10
  11. 11. Traditional web application architecture WAR StoreFrontUI Accounting Service MySQL Browser Apache Database InventoryService Shipping ServiceSimple to Tomcat develop test deploy scale 11
  12. 12. But there are problems 12
  13. 13. Users expect a rich, dynamic andinteractive experience on mobiledevices and desktop h oug d en oo ’t g HTTP Request e isn r ctu Java Web Browser e hit HTML/Javascript Application c I ar ty le U s Old Real-time web ≅ NodeJS 13
  14. 14. One dimensional scalability§ Scales to handle transaction volumes§ Higher transaction rate required run more instances§ Some clouds will do this automatically, others easily§ No code changes required BUT§ Does not scale with increasing data volumes§ Caching is less effective§ Requires more memory§ Requires more I/O 14
  15. 15. Components that don’t cluster§ Most application components are easily clustered: • Most services are stateless • Server-side session state: sticky session routing/state server BUT there are exceptions§ Singletons that maintain global application state: • e.g. CEP engine analyzing event stream over HTTP§ Commercial software with a per-server license fee§ If one application component can’t be clustered you can’t cluster the application! 15
  16. 16. Obstacle to frequent deployments§ Need to redeploy everything to change one component§ Interrupts long running background (e.g. Quartz) jobs§ Increases risk of failure Fear of change§ Updates will happen less often§ e.g. Makes A/B testing UI really difficult 16
  17. 17. Large application Slow IDESlow start times for web container 17
  18. 18. Scaling development WAR StoreFrontUI Accounting Scalable != InventoryService development Shipping§ Forces teams to synchronize development efforts§ Teams need to coordinate updates 18
  19. 19. Long-term commitment to a single technology stack§ Let’s suppose you go with the JVM • Some polyglot support • Many (but not all) JVM languages interoperate easily • e.g. Scala, Groovy modules within a Java application • But you can’t use non-JVM languages§ Application depends on a particular: • Framework - e.g. Spring or Java EE • Container - e.g. Tomcat§ Switching technology stack touches entire application • Painful • Rarely done 19
  20. 20. Agenda§ The (sometimes evil) monolith§ Decomposing applications into services§ How do services communicate?§ Presentation layer design§ Decomposing a monolithic application§ How Cloud Foundry helps 20
  21. 21. 3 dimensions to scalingY axis -functionaldecompositionScale by s ngsplitting ila g n hi g s oni rtdifferent things iti rt im pa ta in da ittl s- sp i ax by Z ale X axis - horizontal duplication Sc Scale by cloning 21
  22. 22. X axis scaling - application level Load WAR Balancer Tomcat 22
  23. 23. X axis scaling - database level Writes Consistent reads MySQL Master Inconsistent reads MySQL MySQL MySQL Slave 1 Slave 2 Slave N Slaves are clones 23
  24. 24. Benefits and drawbacks of X axis splits§ Benefits • Easy and fast to implement • Handles increasing transaction volumes • Simple to manage§ Drawbacks • Doesn’t scale with increasing data volumes • Doesn’t address application/development complexity • Couples entire application to a particular technology stack 24
  25. 25. Z axis scaling - data partitioning§ Each server runs the identical code base§ Different servers responsible for different data§ Route request based on • Attribute of the request • Identity/attribute of the requestor/customer§ For example: • Sharding - partition data by primary key • Free customer vs. paying customer Server 1 Request: ... Some attribute ... “Router” Server 2 ... 25
  26. 26. Z axis scaling - database sharding Partition column RESTAURANT table ID Name … 1 Ajanta 2 Eggshop … X axis scaling too! Database server 1 Database server 2 ID Name … ID Name … 1 Ajanta 2 Eggshop … … 26
  27. 27. Z axis scaling - application example Content Query Update Router Query Aggregator Search Service Search Service Search Service Search Service Cache Cache Database server 1 Database server 2 Partition 1 Partition 2 27
  28. 28. Benefits and drawbacks of Z axis splits§ Benefits • Each server only deals with a subset of the data • Improved caching • Reduced memory usage and reduced I/O • Improved transaction scalability • Improved fault isolation - failure only makes part of the data inaccessible§ Drawbacks • Doesn’t scale with increasing development and application complexity • Increased management/deployment complexity 28
  29. 29. Y axis scaling - functional partitioning§ Splits monolithic application into a set of services§ Each service implements related set of functionality§ Partitioning schemes: • Partition functionality by noun or by verb • Single Responsibility Principle • e.g. Unix utilities - do one focussed thing well 29
  30. 30. Y axis scaling - application level billing web application Accounting Service Store front web application inventory web application Store Front InventoryService shipping web application ShippingServiceApply X axis cloning and/or Z axis partitioning to each service 30
  31. 31. Y axis scaling - databases Application MySQL MySQL MySQL MySQL Order Customer table table .... .... Apply X axis cloning and/or Z axis partitioning to each database 31
  32. 32. Real world examples Between 100-150 services are accessed to build a page. eBaySDForum2006-11-29.pdf 32
  33. 33. Benefits and drawbacks of Y axis splits§ Benefits • Scales development: focussed two pizza devops teams • Deploy services independently • Scale services independently • Improves fault isolation • Eliminates long-term commitment to a single technology stack • Enforces well defined interfaces between components§ Drawbacks • Implementation complexity • Deployment complexity 33
  34. 34. Two levels of architecture§ System level • Defines the inter-service glue: interfaces and communication mechanisms • Slow changing Versus§ Service level • Defines the internal architecture of each service • Far fewer constraints on technology • Each service could use a different technology stack • Rapid evolving 34
  35. 35. If services are small...§ Regularly rewrite using a better technology stack§ Pick the best developers rather than best <pick a language> developers§ Adapt system to changing requirements and better technology without a total rewrite Fred George “Developer Anarchy” 35
  36. 36. Moreover: you are not the same you ...§ Cell lifetimes: Can we build software • hours - some white blood cells systems with these characteristics? • days - stomach lining cells • years - bone cells Too much technical debt • lifetime - brain cells component death?§ 50 to 70 billion of your cells die each day§ Yet you (the system) remains intact DesignBeyondHumanAbilitiesSimp.pdf 36
  37. 37. Agenda§ The (sometimes evil) monolith§ Decomposing applications into services§ How do services communicate?§ Presentation layer design§ Decomposing a monolithic application§ How Cloud Foundry helps 37
  38. 38. Inter-service communication options§ Multiple collaborating services need a communication mechanism§ Many choices: • Synchronous asynchronous • Transports: HTTP, AMQP, ... • Formats: JSON, XML, Protocol Buffers, Thrift, ... • Even via the database§ Distributed application error handling strategies 38
  39. 39. Synchronous communication wgrus-billing.war Accounting Service RESTwgrus-store.war wgrus-shipping.war SOAP StoreFrontUI Shipping MySQL Service Thrift ... wgrus-inventory.war InventoryService Lots of choices 39
  40. 40. Service discovery options§ Clients need to know the coordinates of their dependencies§ Option #1: Caller is configured with dependencies’s URL • e.g. Environment variables, system properties§ Option #2: Services announce their location • Relies on ‘broadcast’ protocol or Gossip protocol to announce location • e.g. RabbitMQ message broker§ Option #3: configuration server • Maintains all the configuration information • e.g. Zookeeper 40
  41. 41. Benefits and drawbacks§ Benefits • Simple to implement and use§ Drawbacks • Poor fault isolation: caller is blocked if server is down/slow • Caller needs to know server’s coordinates (URL, ...) • Less scalable than asynchronous protocols 41
  42. 42. Asynchronous message-based communication wgrus-billing.war Accounting Servicewgrus-store.war RabbitMQ wgrus-inventory.war StoreFrontUI (Message MySQL Broker) Widget InventoryService wgrus-inventory.war InventoryService 42
  43. 43. Benefits and drawbacks§ Benefits • Decouples caller from server • Caller unaware of server’s coordinates (URL) • Message broker buffers message when server is down/slow§ Drawbacks • Additional complexity of message broker • RPC using messaging is more complex 43
  44. 44. Shared databases? Order Management Customer Management Database Orders Customers 44
  45. 45. Benefits and drawbacks§ Benefits • Simple • Single (consistent) source of data§ Drawbacks • Defeats the purpose of having multiple services • Less scalable • More coupling 45
  46. 46. Better: separate databases Customer Update Event Order Management Customer Management Database Database Orders Customers Customers (copy) 46
  47. 47. Maintaining consistency without 2PCbegin transaction update order tables save intent to update customer tablescommit transactionfor each saved intent begin transaction delete intent queue message to update customer Customer management commit transaction dequeue messageOrder management begin transaction if (update has not been applied) { record update as applied update customer tablesBASE: An Acid Alternative by Dan Pritchett } commit transaction acknowledge message 47
  48. 48. Writing code that calls services 48
  49. 49. Composable futures§ Problem: • Service A needs to call services B and C and then D • Makes sense to call B and C parallel • Yet most concurrency APIs are low-level, error-prone etc§ Solution: • Use Akka composable futures = really nice abstraction val futureB = callB() Two calls execute in parallel val futureC = callC() val futureD = for { b <- futureB.mapTo[SomeType] c <- futureC.mapTo[SomeType] d <- callD(b, c) And then invokes D } yield d val result = Await.result(futureD, 1 second). asInstanceOf[SomeOtherType] Get the result of D 49
  50. 50. Spring Integration§ Builds on Spring framework§ High-level of abstraction for building message based applications§ Implements EAI patterns§ Provides plumbing for exchanging messages between application components§ Promotes loosely coupled components§ Integrates with external messaging infrastructure: JMS, AMQP, HTTP, Email, File transfer 50
  51. 51. Spring Integration concepts§ Message channel • Virtual pipe connecting producer and consumer§ Message endpoints • The filter of a pipes-and-filter architecture • Read from and/or write to channel§ Endpoint types: • Transformer • Filter • Router • Splitter • Aggregator • ServiceActivator • Inbound channel adapter - read from external source, writes to channel • Outbound channel adapter - read from channel write to external destination 51
  52. 52. Spring Integration insulatescomponents from the underlying communication mechanism 52
  53. 53. Example of reconfigurability - local@Servicepublic class OrderServiceImpl { @Service public class ShippingServiceImpl {@Autowiredprivate ShippingService shippingService; public void shipOrder(String orderId) { System.out.println("shipped order: " + orderId);public void placeOrder() { } String orderId = generateOrderId(); … } shippingService.shipOrder(orderId);}} Order Shipping Service service Messaging Gateway Channel Service Activator 53
  54. 54. Example of reconfigurability - distributed Code unchanged in new deployment Order Shipping Service serviceMessaging Gateway RabbitMQ Channel AMQP AMQP Channel Service Activator 54
  55. 55. Handling failure§ Errors happen (especially in distributed systems)§ Use timeouts and retries • Never wait forever • Some errors are transient so retry§ Use per-dependency bounded thread pool with bounded queue • Limits number of outstanding requests • Fails fast if service is slow or down§ Use circuit breaker • High rate of errors stop calling temporarily • Avoids calling service that has issues§ On failure • Returned cached or default data • Invoke custom error handler 55
  56. 56. Agenda§ The (sometimes evil) monolith§ Decomposing applications into services§ How do services communicate?§ Presentation layer design§ Decomposing a monolithic application§ How Cloud Foundry helps 56
  57. 57. Modular applicationChoice of presentation layer technology + Redeploy UI frequently/independently 57
  58. 58. NodeJS is the fashionable technologyMany JavaScript client frameworks have a NodeJS counterparte.g. 58
  59. 59. NodeJS isn’t the only game in town JVM-based 59
  60. 60. A modern web application Service 1 RESTful WS Node JSBrowser HTML 5 Server Application Application Service 2 Events client server ... 60
  61. 61. Alternatively: multiple front-end servers Node JS Service 1Browser HTML 5 Node JS Service 2 Application Node JS ... Needs single-sign-on 61
  62. 62. NodeJS - using RESTful WS and AMQP REST ServiceRequests Node JS Events AMQP AMQP RabbitMQ Service 62
  63. 63. Updating the UI is easy§ Update the UI independently of rest of system§ Easily run A/B tests§ Enables fast iteration of the UI 63
  64. 64. But coordination with backend changes required§ Let’s imagine that you are deploying an advanced search feature: • Enhancements to search service • Enhancements to UI§ Before • Deploy new war§ Now: • Some coordination required • Deploy updated backend service • Deploy updated NodeJS and browser code • Enable feature using feature switch 64
  65. 65. Agenda§ The (sometimes evil) monolith§ Decomposing applications into services§ How do services communicate?§ Presentation layer design§ Decomposing a monolithic application§ How Cloud Foundry helps 65
  66. 66. The challenge§ In the beginning: • It would be easy to use Y axis/functional decomposition • But you don’t need it and it would slow you down§ Later on: • You do need it • But your application logic is tangled and decomposition is difficult 66
  67. 67. Lower cost way to start with Y axis decomposition§ Enforce boundaries between services, e.g. DTOs§ But run in a single JVM with a single database§ Use Spring Integration • Course-grained message-based communication • Via in-memory channels§ Downside: • Implementing the glue layers are extra work 67
  68. 68. Special case: Components that don’t cluster Application Monitoring Rest of domain classes A C B D Esper Tangled dependencies prevent Y axis scaling Doesn’t cluster No X axis scaling/cloning 68
  69. 69. Don’t let this happen!Use well-defined interfacesEnsure loose coupling 69
  70. 70. Traditional application architectureMonolithic application Service A Service B Service C Shared domain model A C Difficult to untangle B X Z Y 70
  71. 71. Extracting a service = untangling dependenciesApplication Service Service C Service A Service B Trouble! Domain model 1 Domain model 2 X A C Y B Z 71
  72. 72. Untangling dependencies§ Domain object is part of an association class A { • Read only - replicate private B other; • Update only - notification interface } • Update then read - replicate§ Domain object is a request parameter class A { • Read-only - replace with DTO void m(B other) { ... • Update only - notification interface } • Update then read - tricky }§ Note - Need acyclic dependencies • If used object isn’t part of a service = assumption is that it belongs in calling service 72
  73. 73. Bounded context is a useful idea§ Different parts of a system different domain models§ Different services have a different view of some domain objects, e.g. • User Management = complex view of user • Rest of application = PK + ACL + Name§ Different services can have a different domain model§ Services exchange messages to synchronize data 73
  74. 74. Untangling dependencies 1 Service A Service B Service CDomain model 1 Domain model 2 X A C doSomething(Y) doSomethingElse() Y B getSomething() Z 74
  75. 75. Untangling dependencies 2 Service A Service B Service CDomain model 1 Domain model 2 doSomething(YDTO) C X A Facade A doSomethingElse() Y B Facade B Z 75
  76. 76. Untangling dependencies 3 Service A Service B Service CDomain model 1 Domain model 2 C doSomething(YDTO) X Facade A A Message Broker Listener Y Listener B doSomethingElse() Z Facade B 76
  77. 77. Agenda§ The (sometimes evil) monolith§ Decomposing applications into services§ How do services communicate?§ Presentation layer design§ Decomposing a monolithic application§ How Cloud Foundry helps 77
  78. 78. Traditional tools: monolithic applications 78
  79. 79. Developing service-oriented apps is harder§ Many more moving parts to manage • Infrastructure services: SQL, NoSQL, RabbitMQ • Application services§ Who is going to setup the environments: • the developer sandbox? • ... • QA environments? 79
  80. 80. Cloud Foundry makes it easier§ Easy to deploy applications§ Easy to provision services§ Manifest describes the structure of a set of services§ vmc env-add for binding app services together§ Micro Cloud Foundry is your personal cloud/sandbox§ Caldecott exposes services for use by integration tests 80
  81. 81. Services available on Cloud Foundry 81
  82. 82. Creating a service instance$ vmc create-service mysql --name mysql1Creating Service: OK$ vmc services......=========== Provisioned Services ============+-------------+---------+| Name | Service |+-------------+---------+| mysql1 | mysql |+-------------+---------+
  83. 83. Multi-application manifest - part 1 --- applications: inventory/target: Path to application name: inventory url: framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: si-rabbit: Required services type: :rabbitmq si-mongo: type: :mongodb si-redis: type: :redis 83
  84. 84. Multi-application manifest - part 2 store/target: Path to application name: store url: framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: Required services si-mongo: type: :mongodb si-rabbit: type: :rabbitmq 84
  85. 85. One command to create services and deploy application $ vmc push Would you like to deploy from the current directory? [Yn]: Pushing application inventory... Creating Application: OK Creating Service [si-rabbit]: OK Binding Service [si-rabbit]: OK Creating Service [si-mongo]: OK Binding Service [si-mongo]: OK Creating Service [si-redis]: OK Binding Service [si-redis]: OK Uploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (12K): OK vmc push: Push Status: OK •Reads the manifest file •Creates the required services Staging Application inventory: OK Starting Application inventory: OK Pushing application store... Creating Application: OK •Deploys all the applications Binding Service [si-mongo]: OK Binding Service [si-rabbit]: OK Uploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (5K): OK Push Status: OK Staging Application store: OK 85 Starting Application store: ...
  86. 86. Application configuration via environment variables $ vmc env-add cf1 PAYMENT_SVC=http://... Adding Environment Variable [PAYMENT_SVC=http://...]: OK Stopping Application: OK Staging Application: OK String value = System.getenv("PAYMENT_SVC") @Value("#{systemEnvironment[PAYMENT_SVC]}") private String envVariable;
  87. 87. Micro Cloud Foundry: new developer sandbox App Instances Services Open source Platform as a Service project 10.04 A PaaS packaged as a VMware Virtual Machine Use as a developer sandbox • Use the services from Junit integration tests • Deploy your application for functional testing • Remote debugging from STS 87
  88. 88. Using Caldecott…$ vmc tunnel1: mysql-135e02: mysql1Which service to tunnel to?: 2Password: ********Stopping Application: OKRedeploying tunnel application caldecott.Uploading Application: Checking for available resources: OK Packing application: OK Uploading (1K): OKPush Status: OKBinding Service [mysql1]: OKStaging Application: OKStarting Application: OKGetting tunnel connection info: OKService connection info: username : uMe6Apgw00AhS password : pKcD76PcZR7GZ name : d7cb8afb52f084f3d9bdc269e7d99ab50Starting tunnel to mysql1 on port 10000.1: none2: mysqlWhich client would you like to start?: 2
  89. 89. …Using CaldecottLaunching mysql --protocol=TCP --host=localhost --port=10000 -- user=uMe6Apgw00AhS --password=pKcD76PcZR7GZ d7cb8afb52f084f3d9bdc269e7d99ab50Welcome to the MySQL monitor. Commands end with ; or g.Your MySQL connection id is 10944342Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL), Release 12.5, Revision 188Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type help; or h for help. Type c to clear the current input statement.mysql>
  90. 90. Running JUnit test with Caldecott Configure your test code to use port + connection info 90
  91. 91. Summary§ Monolithic applications are simple to develop and deploy BUT applying the scale cube§ Decomposes your application into services§ Enables scaling for transactions and data volumes§ Tackles application complexity§ Enables scaling for development§ Enables frequent, independent deployments§ Make it easy to leverage other technologies AND§ Cloud Foundry simplifies the development and deployment of “service-oriented” applications 91
  92. 92. @crichardson Questions? promo code CFOpenTour2012