Decomposing applications for scalability and deployability (devnexus 2013)


Published on

Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers.

In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as NodeJS and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Decomposing applications for scalability and deployability (devnexus 2013)

  1. 1. Decomposing applications fordeployability and scalability Chris Richardson Author of POJOs in Action Founder of the original @crichardson @crichardson
  2. 2. Presentation goalHow decomposing applications improves deployability and scalability and How Cloud Foundry helps @crichardson
  3. 3. About Chris @crichardson
  4. 4. (About Chris) @crichardson
  5. 5. About Chris() @crichardson
  6. 6. About Chris @crichardson
  7. 7. About Chris @crichardson
  8. 8. vmc push About-Chris Developer Advocate Signup at @crichardson
  9. 9. AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps @crichardson
  10. 10. Let’s imagine you are buildingan e-commerce application @crichardson
  11. 11. Traditional web applicationarchitecture WAR StoreFrontUI Accounting Service MySQLBrowser Apache Database Inventory Service Shipping ServiceSimple to develop Tomcat test deploy scale @crichardson
  12. 12. But there are problems with a monolithic architecture @crichardson
  13. 13. Users expect a rich, dynamicand interactive experience h oug d en goo HTTP Request is n’t e ec tur Java Web hit Browser HTML/Javascript Application I arc ty le U s Old Real-time web ≅ NodeJS @crichardson
  14. 14. Intimidates developers @crichardson
  15. 15. Obstacle to frequentdeployments Need to redeploy everything to change one component Interrupts long running background (e.g. Quartz) jobs Increases risk of failure Fear of change Updates will happen less often e.g. Makes A/B testing UI really difficult @crichardson
  16. 16. Overloads your IDE andcontainer Slows down development @crichardson
  17. 17. Obstacle to scalingdevelopment Accounting E-commerce Engineering application Shipping team @crichardson
  18. 18. Obstacle to scalingdevelopment WAR UI team StoreFrontUI Accounting Accounting Inventory team InventoryService Shipping team Shipping @crichardson
  19. 19. Obstacle to scalingdevelopment But I want the backend is not working to update the UI yet! Lots of coordination and communication required @crichardson
  20. 20. Requires long-term commitmentto a technology stack @crichardson
  21. 21. AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps @crichardson
  22. 22. @crichardson
  23. 23. The scale cube Y axis - functionaldecomposition Scale by sim ing splitting r n ila litt rtitiodifferent things a ng ing by ta p s da sp Sc is - thi ax ale Z X axis - horizontal duplication @crichardson
  24. 24. Y-axis scaling - application level WAR StoreFrontUI Accounting Service Inventory Service Shipping Service @crichardson
  25. 25. Y-axis scaling - application level accounting web application Accounting Service Store front web application inventory web application StoreFrontUI Inventory Service shipping web application Shipping Service Apply X axis cloning and/or Z axis partitioning to each service @crichardson
  26. 26. Partitioning strategies... Partition by verb, e.g. shipping service Partition by noun, e.g. inventory service Single Responsibility Principle Unix utilities - do one focussed thing well @crichardson
  27. 27. ...Partitioning strategies Too few Drawbacks of the monolithic architecture Too many - a.k.a. Nano-service anti-pattern Runtime overhead Potential risk of excessive network hops Potentially difficult to understand system Something of an art @crichardson
  28. 28. Example micro-servicerequire sinatrapost / do phone_number = params[:From] registration_url = "#{ENV[REGISTRATION_URL]}?phoneNumber=#{URI.encode(phone_number, "+")}" <<-eof <Response> <Sms>To complete registration please go to #{registration_url}</Sms> </Response>eofend For more on micro-services see @fgeorge52 @crichardson
  29. 29. Service deployment options Isolation, manageabilityVMLinux Container/LXCJVMJAR/WAR/OSGI bundle/... Density/efficiency @crichardson
  30. 30. Real world examples eBaySDForum2006-11-29.pdf @crichardson
  31. 31. There are drawbacks @crichardson
  32. 32. ComplexitySee Steve Yegge’s Google Platforms Rant re @crichardson
  33. 33. Multiple databases &Transaction management @crichardson
  34. 34. Implementing features that span multiple services @crichardson
  35. 35. When to use it? In the beginning: •You don’t need it •It will slow you down Later on: •You need it •Refactoring is painful @crichardson
  36. 36. But there are many benefitsScales development: develop, deploy and scale each serviceindependentlyUpdate UI independentlyImproves fault isolationEliminates long-term commitment to a single technology stack Modular, polyglot, multi- framework applications @crichardson
  37. 37. Two levels of architecture System-level ServicesInter-service glue: interfaces and communication mechanisms Slow changing Service-level Internal architecture of each service Each service could use a different technology stack Pick the best tool for the job Rapidly evolving @crichardson
  38. 38. If services are small... Regularly rewrite using a better technology stack Adapt system to changing requirements and better technology without a total rewrite Pick the best developers rather than best <pick a language> developers polyglot culture @crichardson
  39. 39. The human body as a system @crichardson
  40. 40. 50 to 70 billion of your cells die each day @crichardson
  41. 41. Yet you (the system) remain you @crichardson
  42. 42. Can we build software systems with these characteristics? DesignBeyondHumanAbilitiesSimp.pdf @crichardson
  43. 43. AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps @crichardson
  44. 44. Inter-service communicationoptions Synchronous HTTP asynchronous AMQP Formats: JSON, XML, Protocol Buffers, Thrift, ... Even via the database Asynchronous is preferredJSON is fashionable but binary format is more efficient @crichardson
  45. 45. Asynchronous message-based communication wgrus-billing.war Accounting Service wgrus-store.war wgrus-inventory.war RabbitMQ StoreFrontUI (Message InventoryService MySQL Broker) wgrus-shipping.war ShippingService @crichardson
  46. 46. BenefitsDecouples caller from serverCaller unaware of server’s coordinates (URL)Message broker buffers message when server is down/slowSupports a variety of communication patterns, e.g. point-to-point, pub-sub, ... @crichardson
  47. 47. DrawbacksAdditional complexity of message brokerRequest/reply-style communication is more complex @crichardson
  48. 48. Writing code that calls services @crichardson
  49. 49. The need for parallelism Service B b = serviceB() Call in parallel c = serviceC()Service A Service C d = serviceD(b, c) Service D @crichardson
  50. 50. Java Futures are a greatconcurrency abstraction Futures_and_promises @crichardson
  51. 51. Using Java Futurespublic class Client { private ExecutorService executorService; private RemoteServiceProxy remoteServiceProxy; Eventually contains result public void doSomething() throws ... { Future<Integer> result = executorService.submit(new Callable<Integer>() { @Override public Integer call() throws Exception { return remoteServiceProxy.invokeRemoteService(); } }); /// Do other things When needed wait for result int r = result.get(500, TimeUnit.MILLISECONDS); System.out.println(r); }} @crichardson
  52. 52. Better future implementations Guava ListenableFutures = better Scala’s composable Futures = really good Java 8 CompletableFuture = great @crichardson
  53. 53. Composable Futuresval f1 = Future { ... ; 1 }val f2 = Future { ... ; 2 } Transforms Futureval f4 = * 2)assertEquals(4, Await.result(f4, 1 second)) Combines two futuresval fzip = f1 zip f2assertEquals((1, 2), Await.result(fzip, 1 second)) @crichardson
  54. 54. Using Scala futuresdef callB() : Future[...] = ...def callC() : Future[...] = ...def callD() : Future[...] = ... Two calls execute in parallelval future = for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d And then invokes Dval result = Await.result(future, 1 second) Get the result of D @crichardson
  55. 55. Spring IntegrationProvides the building blocks for a pipesand filters architectureEnables development of applicationcomponents that are loosely coupled insulated from messaging infrastructureMessaging defined declaratively @crichardson
  56. 56. Handling partial failures Down? Slow? Service A Service B Down? Slow? @crichardson
  57. 57. About Netflix > 1B API calls/day 1 API call average 6 service calls Fault tolerance is essential @crichardson
  58. 58. How to run out of threadsHTTP Request X X X Thread 1 Thread 2 Thread 3 X Thread n Execute thread Service A X Service B pool Eventually all threads If service B is down then thread Tomcat will be blocked will be blocked @crichardson
  59. 59. Their approach Network timeouts and retries Invoke remote services via a bounded thread pool Use the Circuit Breaker pattern On failure: return default/cached data return error to caller my talk tomorrow for more details @crichardson
  60. 60. AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps @crichardson
  61. 61. Presentation layer evolution.... WAR StoreFrontUI HTML / HTTP View Controller Browser Model @crichardson
  62. 62. ...Presentation layer evolutionBrowser Web application Static View Controller content JSON-REST RESTful Model Endpoints HTML 5 - Events Event JavaScript publisherNo elaborate, server-side web framework required @crichardson
  63. 63. How to publish events to browser? @crichardson
  64. 64. NodeJS is the fashionabletechnology @crichardson
  65. 65. Why NodeJS?Familiar JavascriptHigh-performance, scalable event-driven, non-blocking I/OmodelCompact runtimeOver 17,000 modules developed by the communitySimple event publishing using or SockJS @crichardson
  66. 66. Why not NodeJS? a.k.a. callback hell @crichardson
  67. 67. A modern web application Browser Node JS RESTful WS Service 1HTML 5/ Server Java application Service 2 Script Events client server Service 3 @crichardson
  68. 68. NodeJS - using RESTful WS and AMQP REST Service RESTRequests Node JS Events AMQP AMQP RabbitMQ Service @crichardson
  69. 69. server-side var express = require(express) , http = require(http) , amqp = require(‘amqp’) ....; Handle server.listen(8081); ... var amqpCon = amqp.createConnection(...); connection io.sockets.on(connection, function (socket) { function amqpMessageHandler(message, headers, deliveryInfo) { var m = JSON.parse(; Republish socket.emit(‘tick’, m); as }; amqpCon.queue(“”, {}, event function(queue) { queue.bind(“myExchange”, “”); queue.subscribe(amqpMessageHandler); }); Subscribe to }); AMQP queue @crichardson
  70. 70. - client side<html><body>The event is <span data-bind="text: ticker"></span><script src="/"></script><script src="/knockout-2.0.0.js"></script><script src="/clock.js"></script> Bind to model Connect to</body></html> clock.jsvar socket = io.connect(location.hostname);function ClockModel() { self.ticker = ko.observable(1); Subscribe socket.on(tick, function (data) { self.ticker(data); to tick event });}; Updateko.applyBindings(new ClockModel()); model @crichardson
  71. 71. AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps @crichardson
  72. 72. Original architecture WAR StoreFrontUI Accounting Service MySQLBrowser Apache Database InventoryService Shipping Tomcat @crichardson
  73. 73. Modern architecture p, lo Native Mobile HTML5 mobile Desktop Browser application application ve StoreUI StoreUI StoreUI st de NodeJS ?Asynchronous, Javascript is scalablecommunication StoreUI te e th w RabbitMQ oy do Spring/Scala pl Standaloneweb application Inventory Shipping “headless” ow Service Service de Spring/Java applications H d MySQL an Billing Service Customer Redis Inventory Mongo Order Database Database Database @crichardson
  74. 74. Traditional tools: monolithicapplications @crichardson
  75. 75. Developing modular apps ismore difficultMany more moving parts to manage Platform services: SQL, NoSQL, RabbitMQ Application services: your codeWho is going to setup the environments: the developer sandbox? ...But Cloud Foundry helps... @crichardson
  76. 76. Easy polyglot application deployment andservice provisioning OSS community Ap p lica o vFabric Postgres Private   n  S Clouds   Data Services erv vFabric Public i ce Msg RabbitMQTM Clouds Services  In Micro ter Other Services Clouds fac Additional partners e services … @crichardson
  77. 77. Creating a platform service instance$ vmc create-service mysql mysql1Creating Service: OK$ vmc services......=========== Provisioned Services ============+-------------+---------+| Name | Service |+-------------+---------+| mysql1 | mysql |+-------------+---------+ @crichardson
  78. 78. Multi-application manifest -part 1 --- applications: inventory/target: Path to application name: inventory url: framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: Required platform services si-rabbit: type: :rabbitmq si-mongo: type: :mongodb si-redis: type: :redis @crichardson
  79. 79. Multi-application manifest -part 2store/target: Path to application name: store url: framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 Required platform services services: si-mongo: type: :mongodb si-rabbit: type: :rabbitmq @crichardson
  80. 80. One command to create platformservices and deploy application $ vmc push Would you like to deploy from the current directory? [Yn]: Pushing application inventory... Creating Application: OK Creating Service [si-rabbit]: OK Binding Service [si-rabbit]: OK Creating Service [si-mongo]: OK vmc push: Binding Service [si-mongo]: OK •Reads the manifest file •Creates the required platform services Creating Service [si-redis]: OK Binding Service [si-redis]: OK Uploading Application: Checking for available resources: OK •Deploys all the applications Processing resources: OK Packing application: OK Uploading (12K): OK Push Status: OK Staging Application inventory: OK Starting Application inventory: OK Pushing application store... @crichardson
  81. 81. Micro Cloud Foundry: new developer sandbox App Instances Services Open source Platform as a Service project 10.04 A PaaS packaged as a VMware Virtual Machine Use as a developer sandbox • Use the services from Junit integration tests • Deploy your application for functional testing • Remote debugging from STS @crichardson
  82. 82. Using Caldecott to tunnelinto your services @crichardson
  83. 83. Caldecott = TCP over HTTP native protocol native HTTPService Caldecott Caldecott protocol Service client Port gem application NNN Your computer Cloud Foundry @crichardson
  84. 84. Using Caldecott…$ vmc tunnel1: mysql-135e02: mysql1Which service to tunnel to?: 2Password: ********Stopping Application: OKRedeploying tunnel application caldecott.Uploading Application: Checking for available resources: OK Packing application: OK Uploading (1K): OKPush Status: OKBinding Service [mysql1]: OKStaging Application: OKStarting Application: OKGetting tunnel connection info: OKService connection info: username : uMe6Apgw00AhS password : pKcD76PcZR7GZ name : d7cb8afb52f084f3d9bdc269e7d99ab50Starting tunnel to mysql1 on port 10000.1: none2: mysqlWhich client would you like to start?: 2 @crichardson
  85. 85. …Using CaldecottLaunching mysql --protocol=TCP --host=localhost --port=10000 -- user=uMe6Apgw00AhS --password=pKcD76PcZR7GZ d7cb8afb52f084f3d9bdc269e7d99ab50Welcome to the MySQL monitor. Commands end with ; or g.Your MySQL connection id is 10944342Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL), Release 12.5, Revision 188Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type help; or h for help. Type c to clear the current input statement.mysql> @crichardson
  86. 86. Running JUnit test withCaldecott Configure your test code to use port + connection info @crichardson
  87. 87. Summary @crichardson
  88. 88. Monolithic applications aresimple to develop and deploy BUT have significant drawbacks @crichardson
  89. 89. Apply the scale cube Modular, polyglot, and scalable applications Services developed, deployed and scaled independently @crichardson
  90. 90. Cloud Foundry helps .js Ap p e lica Private   fac Data Services o ter Clouds   n  S r  In erv ide Public i ov ce Msg Services  In  Pr Clouds ter ud Cl o fac Micro e Other Services Clouds @crichardson
  91. 91. @crichardson Questions? @crichardson