Decomposing applications for deployability and scalability #springone2gx #s12gx
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


Decomposing applications for deployability and scalability #springone2gx #s12gx



Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile ...

Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers.

In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as NodeJS and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application.



Total Views
Views on SlideShare
Embed Views



4 Embeds 1,517 1371 143 2 1



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Decomposing applications for deployability and scalability #springone2gx #s12gx Presentation Transcript

  • 1. Decomposing applications for deployability and scalabilityChris Richardson,Author of POJOs in Action, Founder of the original @crichardsoncrichardson@vmware.com
  • 2. Presentation goal How decomposing applications improves deployability and scalability and How Cloud Foundry helps 2
  • 3. About Chris 3
  • 4. (About Chris) 4
  • 5. About Chris() 5
  • 6. About Chris 6
  • 7. About Chris 7
  • 8. vmc push About-Chris Developer Advocate Signup at 8
  • 9. AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps 9
  • 10. Let’s imagine you are building an e-commerce application 10
  • 11. Traditional web application architecture WAR StoreFrontUI Accounting Service MySQL Browser Apache InventoryService Database Shipping Service Simple to Tomcat develop test deploy scale 11
  • 12. But there are problems with a monolithic architecture 12
  • 13. Users expect a rich, dynamicand interactive experience h oug en HTTP Request od ’t go Browser isn Java Web Application re ctu HTML/Javascript e rc hit I a ty le U s Old Real-time web ≅ NodeJS 13
  • 14. Intimidates developers 14
  • 15. Obstacle to frequent deployments Need to redeploy everything to change one component Interrupts long running background (e.g. Quartz) jobs Increases risk of failure Fear of change Updates will happen less often e.g. Makes A/B testing UI really difficult 15
  • 16. Overloads your IDE and container Slows down development 16
  • 17. Obstacle to scaling development Accounting team E-commerce Engineering application Shipping team 17
  • 18. Obstacle to scaling development WAR UI team StoreFrontUI Accounting team Accounting Inventory team InventoryService Shipping team Shipping 18
  • 19. Obstacle to scaling development UI team Accounting team Lots of coordination and communication required 19
  • 20. Requires long-term commitment to a technology stack 20
  • 21. AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps 21
  • 22. 22
  • 23. The scale cube Y axis - functional decomposition ila ng Scale by i m ni g s itio r splitting tin rt lit pa different things sp a gs y dat th ale is - Sc ax in b X axis Z - horizontal duplication 23
  • 24. Y-axis scaling - application level WAR StoreFrontUI Accounting Service InventoryService Shipping Service 24
  • 25. Y-axis scaling - application level accounting web application Accounting Service Store front web application inventory web application StoreFrontUI InventoryService shipping web application Shipping Service Apply X axis cloning and/or Z axis partitioning to each service 25
  • 26. Partitioning strategiesPartition by verb, e.g. shipping servicePartition by noun, e.g. inventory serviceSingle Responsibility PrincipleUnix utilities - do one focussed thing well Something of an art 26
  • 27. Real world examples Between 100-150 services are accessed to build a page. eBaySDForum2006-11-29.pdf 27
  • 28. There are drawbacks 28
  • 29. ComplexitySee Steve Yegge’s Google Platforms Rant re 29
  • 30. Multiple databases =Transaction management challenges 30
  • 31. When to use it? In the beginning: •You don’t need it •It will slow you down Later on: •You need it •Refactoring is painful 31
  • 32. But there are many benefitsScales development: develop, deploy and scale each service independentlyUpdate UI independentlyImproves fault isolationEliminates long-term commitment to a single technology stack Modular, polyglot, multi-framework applications 32
  • 33. Two levels of architecture System-levelServicesInter-service glue: interfaces and communication mechanismsSlow changing Service-levelInternal architecture of each serviceEach service could use a different technology stackPick the best tool for the jobRapidly evolving 33
  • 34. If services are small...Regularly rewrite using a better technology stackAdapt system to changing requirements and better technology without a total rewritePick the best developers rather than best <pick a language> developers polyglot culture Fred George “Developer Anarchy” 34
  • 35. The human body as a system 35
  • 36. 50 to 70 billion of your cells die each day 36
  • 37. Yet you (the system) remain you 37
  • 38. Can we build software systems with these characteristics? DesignBeyondHumanAbilitiesSimp.pdf 38
  • 39. AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps 39
  • 40. Inter-service communication optionsSynchronous HTTP asynchronous AMQPFormats: JSON, XML, Protocol Buffers, Thrift, ...Even via the database Asynchronous is preferred JSON is fashionable but binary format is more efficient 40
  • 41. Asynchronous message-based communication wgrus-billing.war Accounting Servicewgrus-store.war wgrus-inventory.war RabbitMQ StoreFrontUI (Message InventoryService MySQL Broker) wgrus-shipping.war ShippingService 41
  • 42. BenefitsDecouples caller from serverCaller unaware of server’s coordinates (URL)Message broker buffers message when server is down/slow 42
  • 43. DrawbacksAdditional complexity of message brokerRPC using messaging is more complex 43
  • 44. Writing code that calls services 44
  • 45. The need for parallelism Service B b = serviceB() Call in parallel c = serviceC() Service A Service C d = serviceD(b, c) Service D 45
  • 46. Java Futures are a great concurrency abstraction 46
  • 47. Using Java Futurespublic class Client { private ExecutorService executorService; private RemoteServiceProxy remoteServiceProxy; public void doSomething() throws ... { Eventually contains result Future<Integer> result = executorService.submit(new Callable<Integer>() { @Override public Integer call() throws Exception { return remoteServiceProxy.invokeRemoteService(); } }); /// Do other things When needed wait for result int r = result.get(500, TimeUnit.MILLISECONDS); System.out.println(r); 47 }}
  • 48. Akka’s composable futures are even better 48
  • 49. Composable Futuresval f1 = Future { ... ; 1 }val f2 = Future { ... ; 2 } Transforms Futureval f4 = * 2)assertEquals(4, Await.result(f4, 1 second)) Combines two futuresval fzip = f1 zip f2assertEquals((1, 2), Await.result(fzip, 1 second)) 49
  • 50. Using Akka futures def callB() : Future[...] = ... def callC() : Future[...] = ... def callD() : Future[...] = ... Two calls execute in parallel val future = for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d And then invokes D val result = Await.result(future, 1 second) Get the result of D 50
  • 51. Spring IntegrationProvides the building blocks for a pipes and filters architectureEnables development of application components that are •loosely coupled •insulated from messaging infrastructureMessaging defined declaratively 51
  • 52. Handling failure Service A Service B Errors happen in distributed systems 52
  • 53. About Netflix > 1B API calls/day 1 API call average 6 service calls Fault tolerance is essential 53
  • 54. How to run out of threadsHTTP Request X Thread 1 X Thread 2 ... X X Thread N Execute thread Service A XService B pool Eventually all threads If service B is down then thread will be Tomcat will be blocked blocked 54
  • 55. Use timeouts and retries Never wait forever Errors can be transient retry 55
  • 56. Use per-dependency bounded thread poolService A Runnable 1 Task 1 Runnable 2 Task 2 Service B Runnable ... Task ... bounded queue bounded thread pool Fails fast if Limits number of service is slow or down outstanding requests 56
  • 57. Use a circuit breaker High error rate stop calling temporarily Down wait for it to come back up Slow gives it a chance to recover 57
  • 58. On failure Return cached data Avoid Failing Return default data Fail fast 58
  • 59. Aspects + Actors Dependency Caller Invoker Aspect Implements circuit breaker state machine CircuitBreakerEquivalent of thread Actor pool Worker Dependency Worker Actor Stub Actor 59
  • 60. @DependencyProxy annotationtrait RestaurantService { def findNearbyRestaurants(location: Location) : Future[FindNearbyRestaurantResponse]} @Service @DependencyProxy(circuitBreaker = "factualCircuitBreaker", timeoutInMilliseconds=750) class FactualRestaurantService extends RestaurantService { ... } 60
  • 61. Aspect-based Async Execution@Aspectclass DependencyInvokingAspect { @Pointcut("execution(* (@DependencyProxy *).*(..))") def dependencyProxyMethods {} @Around("dependencyProxyMethods()") def invoke(jp: ProceedingJoinPoint) = { val a = AnnotationUtils.findAnnotation(jp.getTarget.getClass, classOf[DependencyProxy]) val actor = findActor(a) val timeout = Timeout(a.timeoutInMilliseconds milliseconds) actor.ask(InvokeDependency( () => jp.proceed())) (timeout) }} Ask actor to invoke jp.proceed() and return Future 61
  • 62. See for the Actor code 62
  • 63. AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps 63
  • 64. Modular applicationChoice of presentation layer technology 64
  • 65. NodeJS is the fashionable technology 65
  • 66. Why NodeJS?Familiar JavascriptHigh-performance, scalable event-driven, non-blocking I/O modelOver 13,000 modules developed by the communityMany JavaScript client frameworks have a NodeJS counterpart, e.g. 66
  • 67. NodeJS example Handle HTTP Handle file readvar http = require(http); requestvar fs = require("fs");http.createServer(function (req, res) { fs.readFile(somefile.txt, function (err, data) { if (err) throw err; res.writeHead(200, {Content-Type: text/plain}); res.end(data); });}).listen(1337,;console.log(Server running at; 67
  • 68. - server-sidevar express = require(express) , http = require(http) , app = express() Serve static content from public dir , server = http.createServer(app) , io = require( ;app.configure(function(){ Listen on port 8081 app.use(express.static(__dirname + /public));});server.listen(8081); Create a timer for clientio.sockets.on(connection, function (socket) { var counter = 0; function tick() { counter = counter + 1; Publish event socket.emit(tick, counter); }; setInterval(tick, 1000);}); 68
  • 69. - client-side<html><body>The time is <span data-bind="text: ticker"></span><script src="/"></script><script src="/knockout-2.0.0.js"></script><script src="/clock.js"></script></body> Bind to model Connect to</html> server clock.jsvar socket = io.connect(location.hostname);function ClockModel() { self.ticker = ko.observable(1); Subscribe to tick socket.on(tick, function (data) { event self.ticker(data); });}; Update modelko.applyBindings(new ClockModel()); 69
  • 70. NodeJS isn’t the only game in town JVM-based 70
  • 71. A modern web application Service 1 RESTful WS Node JSBrowser HTML 5 Server Application Application Service 2 Events client server ... 71
  • 72. NodeJS - using RESTful WS and AMQP REST ServiceRESTRequests Node JS Events AMQP AMQP RabbitMQ Service 72
  • 73. AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps 73
  • 74. Original architecture WAR StoreFrontUI Accounting Service MySQLBrowser Apache InventoryService Database Shipping Service Tomcat 74
  • 75. Modern architecture Desktop Browser Native Mobile application HTML5 mobile application StoreUI StoreUI StoreUI NodeJSAsynchronous, scalable NodeJS Javascript communication StoreUI RabbitMQ Spring/Scala web application Inventory Service Shipping Service Standalone Inventory Service Shipping Service “headless” Spring/ Java applications MySQL Customer Redis Inventory Mongo Order Billing Service Database Database Database 75
  • 76. Traditional tools: monolithic applications 76
  • 77. Developing modular apps is more difficultMany more moving parts to manage •Platform services: SQL, NoSQL, RabbitMQ •Application services: your codeWho is going to setup the environments: •the developer sandbox? •... •QA environments? But Cloud Foundry helps... 77
  • 78. Easy polyglot application deployment and serviceprovisioning OSS community vFabric Postgres Ap p lica Private   o Data Services Clouds   n  S erv i ce  In ter fac vFabric e RabbitMQTM Public Clouds Msg Services Micro Clouds Other Services Additional partners services …
  • 79. Creating a platform service instance$ vmc create-service mysql --name mysql1Creating Service: OK$ vmc services......=========== Provisioned Services ============+-------------+---------+| Name | Service |+-------------+---------+| mysql1 | mysql |+-------------+---------+
  • 80. Multi-application manifest - part 1---applications: inventory/target: Path to application name: inventory url: framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: Required platform services si-rabbit: type: :rabbitmq si-mongo: type: :mongodb si-redis: type: :redis 80
  • 81. Multi-application manifest - part 2store/target: Path to application name: store url: framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: Required platform services si-mongo: type: :mongodb si-rabbit: type: :rabbitmq 81
  • 82. One command to create platform services and deployapplication$ vmc pushWould you like to deploy from the current directory? [Yn]:Pushing application inventory...Creating Application: OKCreating Service [si-rabbit]: OKBinding Service [si-rabbit]: OKCreating Service [si-mongo]: OKBinding Service [si-mongo]: OKCreating Service [si-redis]: OKBinding Service [si-redis]: OKUploading Application: Checking for available resources: OK vmc push: Processing resources: OK •Reads the manifest file Packing application: OK Uploading (12K): OK •Creates the required platform servicesPush Status: OK •Deploys all the applicationsStaging Application inventory: OKStarting Application inventory: OKPushing application store...Creating Application: OKBinding Service [si-mongo]: OKBinding Service [si-rabbit]: OKUploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK 82
  • 83. Micro Cloud Foundry: new developer sandbox App Instances Services Open source Platform as a Service project 10.04 A PaaS packaged as a VMware Virtual Machine Use as a developer sandbox • Use the services from Junit integration tests • Deploy your application for functional testing • Remote debugging from STS 83
  • 84. Using Caldecott to tunnel into your services 84
  • 85. Caldecott = TCP over HTTP native protocol native HTTP Service Caldecott protocol Caldecott gem Service client Port application NNNYour computer Cloud Foundry 85
  • 86. Using Caldecott…$ vmc tunnel1: mysql-135e02: mysql1Which service to tunnel to?: 2Password: ********Stopping Application: OKRedeploying tunnel application caldecott.Uploading Application: Checking for available resources: OK Packing application: OK Uploading (1K): OKPush Status: OKBinding Service [mysql1]: OKStaging Application: OKStarting Application: OKGetting tunnel connection info: OKService connection info: username : uMe6Apgw00AhS password : pKcD76PcZR7GZ name : d7cb8afb52f084f3d9bdc269e7d99ab50Starting tunnel to mysql1 on port 10000.1: none2: mysqlWhich client would you like to start?: 2
  • 87. …Using CaldecottLaunching mysql --protocol=TCP --host=localhost --port=10000 --user=uMe6Apgw00AhS -- password=pKcD76PcZR7GZ d7cb8afb52f084f3d9bdc269e7d99ab50Welcome to the MySQL monitor. Commands end with ; or g.Your MySQL connection id is 10944342Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL), Release 12.5, Revision 188Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type help; or h for help. Type c to clear the current input statement.mysql>
  • 88. Running JUnit test with Caldecott Configure your test code to use port + connection info 88
  • 89. Summary 89
  • 90. Monolithic applications are simple to develop and deployBUT have significant drawbacks 90
  • 91. Apply the scale cube Modular, polyglot, and scalable applications Y axis - functional decomposition Services developed, deployed and scaled independently ngi on iti rt pa a at -d is ax Z X axis - horizontal duplication 91
  • 92. Cloud Foundry helps Ap .js p lic a on Private    Se Data Services Clouds   rv ice  In te rfa ce Public Clouds ce rfa Msg Services te  In er vid ro d  P ou Micro Cl Clouds Other Services
  • 93. @crichardson Questions? @cloudfoundry