Your SlideShare is downloading. ×
Decomposing applications for deployability and scalability (SpringOne China 2012)
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Decomposing applications for deployability and scalability (SpringOne China 2012)


Published on

A look at how decomposing applications improves deployability and scalability …

A look at how decomposing applications improves deployability and scalability

Published in: Technology

1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Decomposing applications for deployability and scalabilityChris Richardson,Author of POJOs in Action, Founder of the original @crichardson
  • 2. Presentation goal How decomposing applications improves deployability and scalability and How Cloud Foundry helps 2
  • 3. About Chris 3
  • 4. (About Chris) 4
  • 5. About Chris() 5
  • 6. About Chris 6
  • 7. About Chris 7
  • 8. vmc push About-Chris Developer Advocate Signup at 8
  • 9. Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?§Presentation layer design§How Cloud Foundry helps 9
  • 10. Let’s imagine you are building an e-commerce application 10
  • 11. Traditional web application architecture WAR StoreFrontUI Accounting Service MySQL Browser Apache InventoryService Database Shipping Service Simple to Tomcat develop test deploy scale 11
  • 12. But there are problems with a monolithic architecture 12
  • 13. Users expect a rich, dynamicand interactive experience h oug en HTTP Request od ’t go Browser isn Java Web Application re ctu HTML/Javascript e rc hit I a ty le U s Old Real-time web ≅ NodeJS 13
  • 14. Intimidates developers 14
  • 15. Obstacle to frequent deployments§ Need to redeploy everything to change one component§ Interrupts long running background (e.g. Quartz) jobs§ Increases risk of failure Fear of change§ Updates will happen less often§ e.g. Makes A/B testing UI really difficult 15
  • 16. Overloads your IDE and container Slows down development 16
  • 17. Obstacle to scaling development Accounting team E-commerce Engineering application Shipping team 17
  • 18. Obstacle to scaling development WAR UI team StoreFrontUI Accounting team Accounting Inventory team InventoryService Shipping team Shipping 18
  • 19. Obstacle to scaling development But the I want to update backend is not the UI working yet! Lots of coordination and communication required 19
  • 20. Requires long-term commitment to a technology stack 20
  • 21. Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?§Presentation layer design§How Cloud Foundry helps 21
  • 22. 22
  • 23. The scale cube Y axis - functional decomposition ila ng Scale by i m ni g s itio r splitting tin rt lit pa different things sp a gs y dat th ale is - Sc ax in b X axis Z - horizontal duplication 23
  • 24. Y-axis scaling - application level WAR StoreFrontUI Accounting Service InventoryService Shipping Service 24
  • 25. Y-axis scaling - application level accounting web application Accounting Service Store front web application inventory web application StoreFrontUI InventoryService shipping web application Shipping Service Apply X axis cloning and/or Z axis partitioning to each service 25
  • 26. Partitioning strategies§Partition by verb, e.g. shipping service§Partition by noun, e.g. inventory service§Single Responsibility Principle§Unix utilities - do one focussed thing well Something of an art 26
  • 27. Real world examples Between 100-150 services are accessed to build a page. eBaySDForum2006-11-29.pdf 27
  • 28. There are drawbacks 28
  • 29. ComplexitySee Steve Yegge’s Google Platforms Rant re 29
  • 30. Multiple databases =Transaction management challenges 30
  • 31. When to use it? In the beginning: •You don’t need it •It will slow you down Later on: •You need it •Refactoring is painful 31
  • 32. But there are many benefits§Scales development: develop, deploy and scale each service independently§Update UI independently§Improves fault isolation§Eliminates long-term commitment to a single technology stack Modular, polyglot, multi-framework applications 32
  • 33. Two levels of architecture System-levelServicesInter-service glue: interfaces and communication mechanismsSlow changing Service-levelInternal architecture of each serviceEach service could use a different technology stackPick the best tool for the jobRapidly evolving 33
  • 34. If services are small...§Regularly rewrite using a better technology stack§Adapt system to changing requirements and better technology without a total rewrite§Pick the best developers rather than best <pick a language> developers polyglot culture 34
  • 35. The human body as a system 35
  • 36. 50 to 70 billion of your cells die each day 36
  • 37. Yet you (the system) remain you 37
  • 38. Can we build software systems with these characteristics? DesignBeyondHumanAbilitiesSimp.pdf 38
  • 39. Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?§Presentation layer design§How Cloud Foundry helps 39
  • 40. Inter-service communication options§Synchronous HTTP asynchronous AMQP§Formats: JSON, XML, Protocol Buffers, Thrift, ...§Even via the database Asynchronous is preferred JSON is fashionable but binary format is more efficient 40
  • 41. Asynchronous message-based communication wgrus-billing.war Accounting Servicewgrus-store.war wgrus-inventory.war RabbitMQ StoreFrontUI (Message InventoryService MySQL Broker) wgrus-shipping.war ShippingService 41
  • 42. Benefits§Decouples caller from server§Caller unaware of server’s coordinates (URL)§Message broker buffers message when server is down/slow§Supports a variety of communication patterns, e.g. point-to-point, pub-sub, ... 42
  • 43. Drawbacks§Additional complexity of message broker§Request/reply-style communication is more complex 43
  • 44. Writing code that calls services 44
  • 45. The need for parallelism Service B b = serviceB() Call in parallel c = serviceC() Service A Service C d = serviceD(b, c) Service D 45
  • 46. Java Futures are a great concurrency abstraction 46
  • 47. Using Java Futurespublic class Client { private ExecutorService executorService; private RemoteServiceProxy remoteServiceProxy; public void doSomething() throws ... { Eventually contains result Future<Integer> result = executorService.submit(new Callable<Integer>() { @Override public Integer call() throws Exception { return remoteServiceProxy.invokeRemoteService(); } }); /// Do other things When needed wait for result int r = result.get(500, TimeUnit.MILLISECONDS); System.out.println(r); 47 }}
  • 48. Akka’s composable futures are even better 48
  • 49. Composable Futuresval f1 = Future { ... ; 1 }val f2 = Future { ... ; 2 } Transforms Futureval f4 = * 2)assertEquals(4, Await.result(f4, 1 second)) Combines two futuresval fzip = f1 zip f2assertEquals((1, 2), Await.result(fzip, 1 second)) 49
  • 50. Using Akka futures def callB() : Future[...] = ... def callC() : Future[...] = ... def callD() : Future[...] = ... Two calls execute in parallel val future = for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d And then invokes D val result = Await.result(future, 1 second) Get the result of D 50
  • 51. Spring Integration§Provides the building blocks for a pipes and filters architecture§Enables development of application components that are •loosely coupled •insulated from messaging infrastructure§Messaging defined declaratively 51
  • 52. Handling failure Service A Service B Errors happen in distributed systems 52
  • 53. About Netflix > 1B API calls/day 1 API call average 6 service calls Fault tolerance is essential 53
  • 54. How to run out of threadsHTTP Request X Thread 1 X Thread 2 ... X X Thread N Execute thread Service A XService B pool Eventually all threads If service B is down then thread will be Tomcat will be blocked blocked 54
  • 55. Use timeouts and retries Never wait forever Errors can be transient retry 55
  • 56. Use per-dependency bounded thread poolService A Runnable 1 Task 1 Runnable 2 Task 2 Service B Runnable ... Task ... bounded queue bounded thread pool Fails fast if Limits number of service is slow or down outstanding requests 56
  • 57. Use a circuit breaker High error rate stop calling temporarily Down wait for it to come back up Slow gives it a chance to recover 57
  • 58. On failure Return cached data Avoid Failing Return default data Fail fast 58
  • 59. Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?§Presentation layer design§How Cloud Foundry helps 59
  • 60. Modular applicationChoice of presentation layer technology 60
  • 61. NodeJS is the fashionable technology 61
  • 62. Why NodeJS?§Familiar Javascript§High-performance, scalable event-driven, non-blocking I/O model§Compact runtime§Over 17,000 modules developed by the community§Many JavaScript client frameworks have a NodeJS counterpart, e.g. and SockJS 62
  • 63. Why not NodeJS? a.k.a. callback hell 63
  • 64. A modern web application Service 1 RESTful WS Node JSBrowser HTML 5 Server Application Application Service 2 Events client server ... 64
  • 65. NodeJS - using RESTful WS and AMQP REST ServiceRESTRequests Node JS Events AMQP AMQP RabbitMQ Service 65
  • 66. server-side var express = require(express) , http = require(http) , amqp = require(‘amqp’) ....; server.listen(8081); Handle ... connection var amqpCon = amqp.createConnection(...); io.sockets.on(connection, function (socket) { Republish as function amqpMessageHandler(message, headers, deliveryInfo) { event var m = JSON.parse(; socket.emit(‘tick’, m); }; amqpCon.queue(“”, {}, function(queue) { Subscribe to AMQP queue.bind(“myExchange”, “”); queue.subscribe(amqpMessageHandler); queue }); }); 66
  • 67. - client side<html><body>The event is <span data-bind="text: ticker"></span><script src="/"></script><script src="/knockout-2.0.0.js"></script><script src="/clock.js"></script></body> Bind to model Connect to</html> server clock.jsvar socket = io.connect(location.hostname);function ClockModel() { self.ticker = ko.observable(1); Subscribe to tick socket.on(tick, function (data) { event self.ticker(data); });}; Update modelko.applyBindings(new ClockModel()); 67
  • 68. Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?§Presentation layer design§How Cloud Foundry helps 68
  • 69. Original architecture WAR StoreFrontUI Accounting Service MySQLBrowser Apache InventoryService Database Shipping Service Tomcat 69
  • 70. Modern architecture p, lo Desktop Browser Native Mobile application HTML5 mobile application ve StoreUI StoreUI StoreUI st de ? is NodeJSAsynchronous, scalable NodeJS Javascript communication StoreUI th te e w RabbitMQ oy do Spring/Scala web pl application Inventory Service Shipping Service Standalone Inventory Service Shipping Service “headless” Spring/ ow Java applications de H MySQL Customer Redis Inventory Mongo Order d Billing Service Database Database Database an 70
  • 71. Traditional tools: monolithic applications 71
  • 72. Developing modular apps is more difficult§Many more moving parts to manage •Platform services: SQL, NoSQL, RabbitMQ •Application services: your code§Who is going to setup the environments: •the developer sandbox? •... •QA environments? But Cloud Foundry helps... 72
  • 73. Easy polyglot application deployment and serviceprovisioning OSS community vFabric Postgres Ap p lica Private   o Data Services Clouds   n  S erv ice  In ter fac vFabric e RabbitMQTM Public Clouds Msg Services Micro Clouds Other Services Additional partners services …
  • 74. Creating a platform service instance$ vmc create-service mysql --name mysql1Creating Service: OK$ vmc services......=========== Provisioned Services ============+-------------+---------+| Name | Service |+-------------+---------+| mysql1 | mysql |+-------------+---------+
  • 75. Multi-application manifest - part 1---applications: inventory/target: Path to application name: inventory url: framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: Required platform services si-rabbit: type: :rabbitmq si-mongo: type: :mongodb si-redis: type: :redis 75
  • 76. Multi-application manifest - part 2store/target: Path to application name: store url: framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: Required platform services si-mongo: type: :mongodb si-rabbit: type: :rabbitmq 76
  • 77. One command to create platform services and deployapplication$ vmc pushWould you like to deploy from the current directory? [Yn]:Pushing application inventory...Creating Application: OKCreating Service [si-rabbit]: OKBinding Service [si-rabbit]: OKCreating Service [si-mongo]: OKBinding Service [si-mongo]: OKCreating Service [si-redis]: OKBinding Service [si-redis]: OKUploading Application: Checking for available resources: OK vmc push: Processing resources: OK •Reads the manifest file Packing application: OK Uploading (12K): OK •Creates the required platform servicesPush Status: OK •Deploys all the applicationsStaging Application inventory: OKStarting Application inventory: OKPushing application store...Creating Application: OKBinding Service [si-mongo]: OKBinding Service [si-rabbit]: OKUploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK 77
  • 78. Micro Cloud Foundry: new developer sandbox App Instances Services Open source Platform as a Service project 10.04 A PaaS packaged as a VMware Virtual Machine Use as a developer sandbox • Use the services from Junit integration tests • Deploy your application for functional testing • Remote debugging from STS 78
  • 79. Using Caldecott to tunnel into your services 79
  • 80. Caldecott = TCP over HTTP native protocol native HTTP Service Caldecott protocol Caldecott gem Service client Port application NNNYour computer Cloud Foundry 80
  • 81. Using Caldecott…$ vmc tunnel1: mysql-135e02: mysql1Which service to tunnel to?: 2Password: ********Stopping Application: OKRedeploying tunnel application caldecott.Uploading Application: Checking for available resources: OK Packing application: OK Uploading (1K): OKPush Status: OKBinding Service [mysql1]: OKStaging Application: OKStarting Application: OKGetting tunnel connection info: OKService connection info: username : uMe6Apgw00AhS password : pKcD76PcZR7GZ name : d7cb8afb52f084f3d9bdc269e7d99ab50Starting tunnel to mysql1 on port 10000.1: none2: mysqlWhich client would you like to start?: 2
  • 82. …Using CaldecottLaunching mysql --protocol=TCP --host=localhost --port=10000 --user=uMe6Apgw00AhS -- password=pKcD76PcZR7GZ d7cb8afb52f084f3d9bdc269e7d99ab50Welcome to the MySQL monitor. Commands end with ; or g.Your MySQL connection id is 10944342Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL), Release 12.5, Revision 188Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type help; or h for help. Type c to clear the current input statement.mysql>
  • 83. Running JUnit test with Caldecott Configure your test code to use port + connection info 83
  • 84. Summary 84
  • 85. Monolithic applications are simple to develop and deployBUT have significant drawbacks 85
  • 86. Apply the scale cube §Modular, polyglot, and scalable applications Y axis - functional decomposition §Services developed, deployed and scaled independently ngi on iti rt pa a at -d is ax Z X axis - horizontal duplication 86
  • 87. Cloud Foundry helps Ap .js p lic a on Private    Se Data Services Clouds   rv ice  In te rfa ce Public Clouds ce rfa Msg Services te  In er vid ro d  P ou Micro Cl Clouds Other Services
  • 88. @crichardson Questions? @cloudfoundry
  • 89. Cloud Foundry 启动营在www.cloudfoundry.com注册账号并成功上传应用程序,即可于12月8日中午后凭账号ID和应用URL到签到处换取Cloud Foundry主题卫衣一件。89
  • 90. iPhone5 等你拿第二天大会结束前,请不要提前离 ,将填写完整的意见反馈表投到签到处的抽奖箱内,即可参与“iPhone5”抽奖活动。90
  • 91. Birds of a Feather 专家面对面所有讲师都会在课程结束后,到紫兰厅与来宾讨论课程上的问题91