Your SlideShare is downloading. ×

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Decomposing applications for scalability and deployability (devnexus 2013)


Published on

Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile …

Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers.

In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as NodeJS and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application.

Published in: Technology

  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Decomposing applications fordeployability and scalability Chris Richardson Author of POJOs in Action Founder of the original @crichardson @crichardson
  • 2. Presentation goalHow decomposing applications improves deployability and scalability and How Cloud Foundry helps @crichardson
  • 3. About Chris @crichardson
  • 4. (About Chris) @crichardson
  • 5. About Chris() @crichardson
  • 6. About Chris @crichardson
  • 7. About Chris @crichardson
  • 8. vmc push About-Chris Developer Advocate Signup at @crichardson
  • 9. AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps @crichardson
  • 10. Let’s imagine you are buildingan e-commerce application @crichardson
  • 11. Traditional web applicationarchitecture WAR StoreFrontUI Accounting Service MySQLBrowser Apache Database Inventory Service Shipping ServiceSimple to develop Tomcat test deploy scale @crichardson
  • 12. But there are problems with a monolithic architecture @crichardson
  • 13. Users expect a rich, dynamicand interactive experience h oug d en goo HTTP Request is n’t e ec tur Java Web hit Browser HTML/Javascript Application I arc ty le U s Old Real-time web ≅ NodeJS @crichardson
  • 14. Intimidates developers @crichardson
  • 15. Obstacle to frequentdeployments Need to redeploy everything to change one component Interrupts long running background (e.g. Quartz) jobs Increases risk of failure Fear of change Updates will happen less often e.g. Makes A/B testing UI really difficult @crichardson
  • 16. Overloads your IDE andcontainer Slows down development @crichardson
  • 17. Obstacle to scalingdevelopment Accounting E-commerce Engineering application Shipping team @crichardson
  • 18. Obstacle to scalingdevelopment WAR UI team StoreFrontUI Accounting Accounting Inventory team InventoryService Shipping team Shipping @crichardson
  • 19. Obstacle to scalingdevelopment But I want the backend is not working to update the UI yet! Lots of coordination and communication required @crichardson
  • 20. Requires long-term commitmentto a technology stack @crichardson
  • 21. AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps @crichardson
  • 22. @crichardson
  • 23. The scale cube Y axis - functionaldecomposition Scale by sim ing splitting r n ila litt rtitiodifferent things a ng ing by ta p s da sp Sc is - thi ax ale Z X axis - horizontal duplication @crichardson
  • 24. Y-axis scaling - application level WAR StoreFrontUI Accounting Service Inventory Service Shipping Service @crichardson
  • 25. Y-axis scaling - application level accounting web application Accounting Service Store front web application inventory web application StoreFrontUI Inventory Service shipping web application Shipping Service Apply X axis cloning and/or Z axis partitioning to each service @crichardson
  • 26. Partitioning strategies... Partition by verb, e.g. shipping service Partition by noun, e.g. inventory service Single Responsibility Principle Unix utilities - do one focussed thing well @crichardson
  • 27. ...Partitioning strategies Too few Drawbacks of the monolithic architecture Too many - a.k.a. Nano-service anti-pattern Runtime overhead Potential risk of excessive network hops Potentially difficult to understand system Something of an art @crichardson
  • 28. Example micro-servicerequire sinatrapost / do phone_number = params[:From] registration_url = "#{ENV[REGISTRATION_URL]}?phoneNumber=#{URI.encode(phone_number, "+")}" <<-eof <Response> <Sms>To complete registration please go to #{registration_url}</Sms> </Response>eofend For more on micro-services see @fgeorge52 @crichardson
  • 29. Service deployment options Isolation, manageabilityVMLinux Container/LXCJVMJAR/WAR/OSGI bundle/... Density/efficiency @crichardson
  • 30. Real world examples eBaySDForum2006-11-29.pdf @crichardson
  • 31. There are drawbacks @crichardson
  • 32. ComplexitySee Steve Yegge’s Google Platforms Rant re @crichardson
  • 33. Multiple databases &Transaction management @crichardson
  • 34. Implementing features that span multiple services @crichardson
  • 35. When to use it? In the beginning: •You don’t need it •It will slow you down Later on: •You need it •Refactoring is painful @crichardson
  • 36. But there are many benefitsScales development: develop, deploy and scale each serviceindependentlyUpdate UI independentlyImproves fault isolationEliminates long-term commitment to a single technology stack Modular, polyglot, multi- framework applications @crichardson
  • 37. Two levels of architecture System-level ServicesInter-service glue: interfaces and communication mechanisms Slow changing Service-level Internal architecture of each service Each service could use a different technology stack Pick the best tool for the job Rapidly evolving @crichardson
  • 38. If services are small... Regularly rewrite using a better technology stack Adapt system to changing requirements and better technology without a total rewrite Pick the best developers rather than best <pick a language> developers polyglot culture @crichardson
  • 39. The human body as a system @crichardson
  • 40. 50 to 70 billion of your cells die each day @crichardson
  • 41. Yet you (the system) remain you @crichardson
  • 42. Can we build software systems with these characteristics? DesignBeyondHumanAbilitiesSimp.pdf @crichardson
  • 43. AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps @crichardson
  • 44. Inter-service communicationoptions Synchronous HTTP asynchronous AMQP Formats: JSON, XML, Protocol Buffers, Thrift, ... Even via the database Asynchronous is preferredJSON is fashionable but binary format is more efficient @crichardson
  • 45. Asynchronous message-based communication wgrus-billing.war Accounting Service wgrus-store.war wgrus-inventory.war RabbitMQ StoreFrontUI (Message InventoryService MySQL Broker) wgrus-shipping.war ShippingService @crichardson
  • 46. BenefitsDecouples caller from serverCaller unaware of server’s coordinates (URL)Message broker buffers message when server is down/slowSupports a variety of communication patterns, e.g. point-to-point, pub-sub, ... @crichardson
  • 47. DrawbacksAdditional complexity of message brokerRequest/reply-style communication is more complex @crichardson
  • 48. Writing code that calls services @crichardson
  • 49. The need for parallelism Service B b = serviceB() Call in parallel c = serviceC()Service A Service C d = serviceD(b, c) Service D @crichardson
  • 50. Java Futures are a greatconcurrency abstraction Futures_and_promises @crichardson
  • 51. Using Java Futurespublic class Client { private ExecutorService executorService; private RemoteServiceProxy remoteServiceProxy; Eventually contains result public void doSomething() throws ... { Future<Integer> result = executorService.submit(new Callable<Integer>() { @Override public Integer call() throws Exception { return remoteServiceProxy.invokeRemoteService(); } }); /// Do other things When needed wait for result int r = result.get(500, TimeUnit.MILLISECONDS); System.out.println(r); }} @crichardson
  • 52. Better future implementations Guava ListenableFutures = better Scala’s composable Futures = really good Java 8 CompletableFuture = great @crichardson
  • 53. Composable Futuresval f1 = Future { ... ; 1 }val f2 = Future { ... ; 2 } Transforms Futureval f4 = * 2)assertEquals(4, Await.result(f4, 1 second)) Combines two futuresval fzip = f1 zip f2assertEquals((1, 2), Await.result(fzip, 1 second)) @crichardson
  • 54. Using Scala futuresdef callB() : Future[...] = ...def callC() : Future[...] = ...def callD() : Future[...] = ... Two calls execute in parallelval future = for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d And then invokes Dval result = Await.result(future, 1 second) Get the result of D @crichardson
  • 55. Spring IntegrationProvides the building blocks for a pipesand filters architectureEnables development of applicationcomponents that are loosely coupled insulated from messaging infrastructureMessaging defined declaratively @crichardson
  • 56. Handling partial failures Down? Slow? Service A Service B Down? Slow? @crichardson
  • 57. About Netflix > 1B API calls/day 1 API call average 6 service calls Fault tolerance is essential @crichardson
  • 58. How to run out of threadsHTTP Request X X X Thread 1 Thread 2 Thread 3 X Thread n Execute thread Service A X Service B pool Eventually all threads If service B is down then thread Tomcat will be blocked will be blocked @crichardson
  • 59. Their approach Network timeouts and retries Invoke remote services via a bounded thread pool Use the Circuit Breaker pattern On failure: return default/cached data return error to caller my talk tomorrow for more details @crichardson
  • 60. AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps @crichardson
  • 61. Presentation layer evolution.... WAR StoreFrontUI HTML / HTTP View Controller Browser Model @crichardson
  • 62. ...Presentation layer evolutionBrowser Web application Static View Controller content JSON-REST RESTful Model Endpoints HTML 5 - Events Event JavaScript publisherNo elaborate, server-side web framework required @crichardson
  • 63. How to publish events to browser? @crichardson
  • 64. NodeJS is the fashionabletechnology @crichardson
  • 65. Why NodeJS?Familiar JavascriptHigh-performance, scalable event-driven, non-blocking I/OmodelCompact runtimeOver 17,000 modules developed by the communitySimple event publishing using or SockJS @crichardson
  • 66. Why not NodeJS? a.k.a. callback hell @crichardson
  • 67. A modern web application Browser Node JS RESTful WS Service 1HTML 5/ Server Java application Service 2 Script Events client server Service 3 @crichardson
  • 68. NodeJS - using RESTful WS and AMQP REST Service RESTRequests Node JS Events AMQP AMQP RabbitMQ Service @crichardson
  • 69. server-side var express = require(express) , http = require(http) , amqp = require(‘amqp’) ....; Handle server.listen(8081); ... var amqpCon = amqp.createConnection(...); connection io.sockets.on(connection, function (socket) { function amqpMessageHandler(message, headers, deliveryInfo) { var m = JSON.parse(; Republish socket.emit(‘tick’, m); as }; amqpCon.queue(“”, {}, event function(queue) { queue.bind(“myExchange”, “”); queue.subscribe(amqpMessageHandler); }); Subscribe to }); AMQP queue @crichardson
  • 70. - client side<html><body>The event is <span data-bind="text: ticker"></span><script src="/"></script><script src="/knockout-2.0.0.js"></script><script src="/clock.js"></script> Bind to model Connect to</body></html> clock.jsvar socket = io.connect(location.hostname);function ClockModel() { self.ticker = ko.observable(1); Subscribe socket.on(tick, function (data) { self.ticker(data); to tick event });}; Updateko.applyBindings(new ClockModel()); model @crichardson
  • 71. AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps @crichardson
  • 72. Original architecture WAR StoreFrontUI Accounting Service MySQLBrowser Apache Database InventoryService Shipping Tomcat @crichardson
  • 73. Modern architecture p, lo Native Mobile HTML5 mobile Desktop Browser application application ve StoreUI StoreUI StoreUI st de NodeJS ?Asynchronous, Javascript is scalablecommunication StoreUI te e th w RabbitMQ oy do Spring/Scala pl Standaloneweb application Inventory Shipping “headless” ow Service Service de Spring/Java applications H d MySQL an Billing Service Customer Redis Inventory Mongo Order Database Database Database @crichardson
  • 74. Traditional tools: monolithicapplications @crichardson
  • 75. Developing modular apps ismore difficultMany more moving parts to manage Platform services: SQL, NoSQL, RabbitMQ Application services: your codeWho is going to setup the environments: the developer sandbox? ...But Cloud Foundry helps... @crichardson
  • 76. Easy polyglot application deployment andservice provisioning OSS community Ap p lica o vFabric Postgres Private   n  S Clouds   Data Services erv vFabric Public i ce Msg RabbitMQTM Clouds Services  In Micro ter Other Services Clouds fac Additional partners e services … @crichardson
  • 77. Creating a platform service instance$ vmc create-service mysql mysql1Creating Service: OK$ vmc services......=========== Provisioned Services ============+-------------+---------+| Name | Service |+-------------+---------+| mysql1 | mysql |+-------------+---------+ @crichardson
  • 78. Multi-application manifest -part 1 --- applications: inventory/target: Path to application name: inventory url: framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: Required platform services si-rabbit: type: :rabbitmq si-mongo: type: :mongodb si-redis: type: :redis @crichardson
  • 79. Multi-application manifest -part 2store/target: Path to application name: store url: framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 Required platform services services: si-mongo: type: :mongodb si-rabbit: type: :rabbitmq @crichardson
  • 80. One command to create platformservices and deploy application $ vmc push Would you like to deploy from the current directory? [Yn]: Pushing application inventory... Creating Application: OK Creating Service [si-rabbit]: OK Binding Service [si-rabbit]: OK Creating Service [si-mongo]: OK vmc push: Binding Service [si-mongo]: OK •Reads the manifest file •Creates the required platform services Creating Service [si-redis]: OK Binding Service [si-redis]: OK Uploading Application: Checking for available resources: OK •Deploys all the applications Processing resources: OK Packing application: OK Uploading (12K): OK Push Status: OK Staging Application inventory: OK Starting Application inventory: OK Pushing application store... @crichardson
  • 81. Micro Cloud Foundry: new developer sandbox App Instances Services Open source Platform as a Service project 10.04 A PaaS packaged as a VMware Virtual Machine Use as a developer sandbox • Use the services from Junit integration tests • Deploy your application for functional testing • Remote debugging from STS @crichardson
  • 82. Using Caldecott to tunnelinto your services @crichardson
  • 83. Caldecott = TCP over HTTP native protocol native HTTPService Caldecott Caldecott protocol Service client Port gem application NNN Your computer Cloud Foundry @crichardson
  • 84. Using Caldecott…$ vmc tunnel1: mysql-135e02: mysql1Which service to tunnel to?: 2Password: ********Stopping Application: OKRedeploying tunnel application caldecott.Uploading Application: Checking for available resources: OK Packing application: OK Uploading (1K): OKPush Status: OKBinding Service [mysql1]: OKStaging Application: OKStarting Application: OKGetting tunnel connection info: OKService connection info: username : uMe6Apgw00AhS password : pKcD76PcZR7GZ name : d7cb8afb52f084f3d9bdc269e7d99ab50Starting tunnel to mysql1 on port 10000.1: none2: mysqlWhich client would you like to start?: 2 @crichardson
  • 85. …Using CaldecottLaunching mysql --protocol=TCP --host=localhost --port=10000 -- user=uMe6Apgw00AhS --password=pKcD76PcZR7GZ d7cb8afb52f084f3d9bdc269e7d99ab50Welcome to the MySQL monitor. Commands end with ; or g.Your MySQL connection id is 10944342Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL), Release 12.5, Revision 188Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type help; or h for help. Type c to clear the current input statement.mysql> @crichardson
  • 86. Running JUnit test withCaldecott Configure your test code to use port + connection info @crichardson
  • 87. Summary @crichardson
  • 88. Monolithic applications aresimple to develop and deploy BUT have significant drawbacks @crichardson
  • 89. Apply the scale cube Modular, polyglot, and scalable applications Services developed, deployed and scaled independently @crichardson
  • 90. Cloud Foundry helps .js Ap p e lica Private   fac Data Services o ter Clouds   n  S r  In erv ide Public i ov ce Msg Services  In  Pr Clouds ter ud Cl o fac Micro e Other Services Clouds @crichardson
  • 91. @crichardson Questions? @crichardson