Decomposing applications for deployability and scalability (SpringOne China 2012)

  • 1,670 views
Uploaded on

A look at how decomposing applications improves deployability and scalability …

A look at how decomposing applications improves deployability and scalability

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,670
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
9
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Decomposing applications for deployability and scalabilityChris Richardson,Author of POJOs in Action, Founder of the original CloudFoundry.com @crichardson chris.richardson@springsource.com http://plainoldobjects.com/
  • 2. Presentation goal How decomposing applications improves deployability and scalability and How Cloud Foundry helps 2
  • 3. About Chris 3
  • 4. (About Chris) 4
  • 5. About Chris() 5
  • 6. About Chris 6
  • 7. About Chris http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/ 7
  • 8. vmc push About-Chris Developer Advocate Signup at http://cloudfoundry.com 8
  • 9. Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?§Presentation layer design§How Cloud Foundry helps 9
  • 10. Let’s imagine you are building an e-commerce application 10
  • 11. Traditional web application architecture WAR StoreFrontUI Accounting Service MySQL Browser Apache InventoryService Database Shipping Service Simple to Tomcat develop test deploy scale 11
  • 12. But there are problems with a monolithic architecture 12
  • 13. Users expect a rich, dynamicand interactive experience h oug en HTTP Request od ’t go Browser isn Java Web Application re ctu HTML/Javascript e rc hit I a ty le U s Old Real-time web ≅ NodeJS 13
  • 14. Intimidates developers 14
  • 15. Obstacle to frequent deployments§ Need to redeploy everything to change one component§ Interrupts long running background (e.g. Quartz) jobs§ Increases risk of failure Fear of change§ Updates will happen less often§ e.g. Makes A/B testing UI really difficult 15
  • 16. Overloads your IDE and container Slows down development 16
  • 17. Obstacle to scaling development Accounting team E-commerce Engineering application Shipping team 17
  • 18. Obstacle to scaling development WAR UI team StoreFrontUI Accounting team Accounting Inventory team InventoryService Shipping team Shipping 18
  • 19. Obstacle to scaling development But the I want to update backend is not the UI working yet! Lots of coordination and communication required 19
  • 20. Requires long-term commitment to a technology stack 20
  • 21. Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?§Presentation layer design§How Cloud Foundry helps 21
  • 22. 22
  • 23. The scale cube Y axis - functional decomposition ila ng Scale by i m ni g s itio r splitting tin rt lit pa different things sp a gs y dat th ale is - Sc ax in b X axis Z - horizontal duplication 23
  • 24. Y-axis scaling - application level WAR StoreFrontUI Accounting Service InventoryService Shipping Service 24
  • 25. Y-axis scaling - application level accounting web application Accounting Service Store front web application inventory web application StoreFrontUI InventoryService shipping web application Shipping Service Apply X axis cloning and/or Z axis partitioning to each service 25
  • 26. Partitioning strategies§Partition by verb, e.g. shipping service§Partition by noun, e.g. inventory service§Single Responsibility Principle§Unix utilities - do one focussed thing well Something of an art 26
  • 27. Real world examples http://techblog.netflix.com/ Between 100-150 services are accessed to build a page. http://highscalability.com/amazon-architecture http://www.addsimplicity.com/downloads/ eBaySDForum2006-11-29.pdf http://queue.acm.org/detail.cfm?id=1394128 27
  • 28. There are drawbacks 28
  • 29. ComplexitySee Steve Yegge’s Google Platforms Rant re Amazon.com 29
  • 30. Multiple databases =Transaction management challenges 30
  • 31. When to use it? In the beginning: •You don’t need it •It will slow you down Later on: •You need it •Refactoring is painful 31
  • 32. But there are many benefits§Scales development: develop, deploy and scale each service independently§Update UI independently§Improves fault isolation§Eliminates long-term commitment to a single technology stack Modular, polyglot, multi-framework applications 32
  • 33. Two levels of architecture System-levelServicesInter-service glue: interfaces and communication mechanismsSlow changing Service-levelInternal architecture of each serviceEach service could use a different technology stackPick the best tool for the jobRapidly evolving 33
  • 34. If services are small...§Regularly rewrite using a better technology stack§Adapt system to changing requirements and better technology without a total rewrite§Pick the best developers rather than best <pick a language> developers polyglot culture 34
  • 35. The human body as a system 35
  • 36. 50 to 70 billion of your cells die each day 36
  • 37. Yet you (the system) remain you 37
  • 38. Can we build software systems with these characteristics? http://dreamsongs.com/Files/ DesignBeyondHumanAbilitiesSimp.pdf http://dreamsongs.com/Files/WhitherSoftware.pdf 38
  • 39. Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?§Presentation layer design§How Cloud Foundry helps 39
  • 40. Inter-service communication options§Synchronous HTTP asynchronous AMQP§Formats: JSON, XML, Protocol Buffers, Thrift, ...§Even via the database Asynchronous is preferred JSON is fashionable but binary format is more efficient 40
  • 41. Asynchronous message-based communication wgrus-billing.war Accounting Servicewgrus-store.war wgrus-inventory.war RabbitMQ StoreFrontUI (Message InventoryService MySQL Broker) wgrus-shipping.war ShippingService 41
  • 42. Benefits§Decouples caller from server§Caller unaware of server’s coordinates (URL)§Message broker buffers message when server is down/slow§Supports a variety of communication patterns, e.g. point-to-point, pub-sub, ... 42
  • 43. Drawbacks§Additional complexity of message broker§Request/reply-style communication is more complex 43
  • 44. Writing code that calls services 44
  • 45. The need for parallelism Service B b = serviceB() Call in parallel c = serviceC() Service A Service C d = serviceD(b, c) Service D 45
  • 46. Java Futures are a great concurrency abstractionhttp://en.wikipedia.org/wiki/Futures_and_promises 46
  • 47. Using Java Futurespublic class Client { private ExecutorService executorService; private RemoteServiceProxy remoteServiceProxy; public void doSomething() throws ... { Eventually contains result Future<Integer> result = executorService.submit(new Callable<Integer>() { @Override public Integer call() throws Exception { return remoteServiceProxy.invokeRemoteService(); } }); /// Do other things When needed wait for result int r = result.get(500, TimeUnit.MILLISECONDS); System.out.println(r); 47 }}
  • 48. Akka’s composable futures are even better 48
  • 49. Composable Futuresval f1 = Future { ... ; 1 }val f2 = Future { ... ; 2 } Transforms Futureval f4 = f2.map(_ * 2)assertEquals(4, Await.result(f4, 1 second)) Combines two futuresval fzip = f1 zip f2assertEquals((1, 2), Await.result(fzip, 1 second)) http://doc.akka.io/docs/akka/2.0.1/scala/futures.html 49
  • 50. Using Akka futures def callB() : Future[...] = ... def callC() : Future[...] = ... def callD() : Future[...] = ... Two calls execute in parallel val future = for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d And then invokes D val result = Await.result(future, 1 second) http://doc.akka.io/docs/akka/2.0.1/scala/futures.html Get the result of D 50
  • 51. Spring Integration§Provides the building blocks for a pipes and filters architecture§Enables development of application components that are •loosely coupled •insulated from messaging infrastructure§Messaging defined declaratively 51
  • 52. Handling failure Service A Service B Errors happen in distributed systems 52
  • 53. About Netflix > 1B API calls/day 1 API call average 6 service calls Fault tolerance is essential http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 53
  • 54. How to run out of threadsHTTP Request X Thread 1 X Thread 2 ... X X Thread N Execute thread Service A XService B pool Eventually all threads If service B is down then thread will be Tomcat will be blocked blocked 54
  • 55. Use timeouts and retries Never wait forever Errors can be transient retry http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 55
  • 56. Use per-dependency bounded thread poolService A Runnable 1 Task 1 Runnable 2 Task 2 Service B Runnable ... Task ... bounded queue bounded thread pool Fails fast if Limits number of service is slow or down outstanding requests http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 56
  • 57. Use a circuit breaker High error rate stop calling temporarily Down wait for it to come back up Slow gives it a chance to recover http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 57
  • 58. On failure Return cached data Avoid Failing Return default data Fail fast http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 58
  • 59. Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?§Presentation layer design§How Cloud Foundry helps 59
  • 60. Modular applicationChoice of presentation layer technology 60
  • 61. NodeJS is the fashionable technology 61
  • 62. Why NodeJS?§Familiar Javascript§High-performance, scalable event-driven, non-blocking I/O model§Compact runtime§Over 17,000 modules developed by the community§Many JavaScript client frameworks have a NodeJS counterpart, e.g. socket.io and SockJS 62
  • 63. Why not NodeJS? a.k.a. callback hell 63
  • 64. A modern web application Service 1 RESTful WS Node JSBrowser HTML 5 Server Application Application Service 2 Events Socket.io Socket.io client server ... 64
  • 65. NodeJS - using RESTful WS and AMQP REST ServiceRESTRequests Node JS Events socket.io AMQP AMQP RabbitMQ Service 65
  • 66. Socket.io server-side var express = require(express) , http = require(http) , amqp = require(‘amqp’) ....; server.listen(8081); Handle socket.io ... connection var amqpCon = amqp.createConnection(...); io.sockets.on(connection, function (socket) { Republish as function amqpMessageHandler(message, headers, deliveryInfo) { socket.io event var m = JSON.parse(message.data.toString()); socket.emit(‘tick’, m); }; amqpCon.queue(“”, {}, function(queue) { Subscribe to AMQP queue.bind(“myExchange”, “”); queue.subscribe(amqpMessageHandler); queue }); }); 66
  • 67. Socket.io - client side<html><body>The event is <span data-bind="text: ticker"></span><script src="/socket.io/socket.io.js"></script><script src="/knockout-2.0.0.js"></script><script src="/clock.js"></script></body> Bind to model Connect to</html> socket.io server clock.jsvar socket = io.connect(location.hostname);function ClockModel() { self.ticker = ko.observable(1); Subscribe to tick socket.on(tick, function (data) { event self.ticker(data); });}; Update modelko.applyBindings(new ClockModel()); 67
  • 68. Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?§Presentation layer design§How Cloud Foundry helps 68
  • 69. Original architecture WAR StoreFrontUI Accounting Service MySQLBrowser Apache InventoryService Database Shipping Service Tomcat 69
  • 70. Modern architecture p, lo Desktop Browser Native Mobile application HTML5 mobile application ve StoreUI StoreUI StoreUI st de ? is NodeJSAsynchronous, scalable NodeJS Javascript communication StoreUI th te e w RabbitMQ oy do Spring/Scala web pl application Inventory Service Shipping Service Standalone Inventory Service Shipping Service “headless” Spring/ ow Java applications de H MySQL Customer Redis Inventory Mongo Order d Billing Service Database Database Database an 70
  • 71. Traditional tools: monolithic applications 71
  • 72. Developing modular apps is more difficult§Many more moving parts to manage •Platform services: SQL, NoSQL, RabbitMQ •Application services: your code§Who is going to setup the environments: •the developer sandbox? •... •QA environments? But Cloud Foundry helps... 72
  • 73. Easy polyglot application deployment and serviceprovisioning OSS community vFabric Postgres Ap p lica Private   o Data Services Clouds   n  S erv ice  In ter fac vFabric e RabbitMQTM Public Clouds Msg Services Micro Clouds Other Services Additional partners services …
  • 74. Creating a platform service instance$ vmc create-service mysql --name mysql1Creating Service: OK$ vmc services......=========== Provisioned Services ============+-------------+---------+| Name | Service |+-------------+---------+| mysql1 | mysql |+-------------+---------+
  • 75. Multi-application manifest - part 1---applications: inventory/target: Path to application name: inventory url: cer-inventory.chrisr.cloudfoundry.me framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: Required platform services si-rabbit: type: :rabbitmq si-mongo: type: :mongodb si-redis: type: :redis 75
  • 76. Multi-application manifest - part 2store/target: Path to application name: store url: cer-store.chrisr.cloudfoundry.me framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: Required platform services si-mongo: type: :mongodb si-rabbit: type: :rabbitmq 76
  • 77. One command to create platform services and deployapplication$ vmc pushWould you like to deploy from the current directory? [Yn]:Pushing application inventory...Creating Application: OKCreating Service [si-rabbit]: OKBinding Service [si-rabbit]: OKCreating Service [si-mongo]: OKBinding Service [si-mongo]: OKCreating Service [si-redis]: OKBinding Service [si-redis]: OKUploading Application: Checking for available resources: OK vmc push: Processing resources: OK •Reads the manifest file Packing application: OK Uploading (12K): OK •Creates the required platform servicesPush Status: OK •Deploys all the applicationsStaging Application inventory: OKStarting Application inventory: OKPushing application store...Creating Application: OKBinding Service [si-mongo]: OKBinding Service [si-rabbit]: OKUploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK 77
  • 78. Micro Cloud Foundry: new developer sandbox App Instances Services Open source Platform as a Service project 10.04 A PaaS packaged as a VMware Virtual Machine Use as a developer sandbox • Use the services from Junit integration tests • Deploy your application for functional testing • Remote debugging from STS 78
  • 79. Using Caldecott to tunnel into your services 79
  • 80. Caldecott = TCP over HTTP native protocol native HTTP Service Caldecott protocol Caldecott gem Service client Port application NNNYour computer Cloud Foundry 80
  • 81. Using Caldecott…$ vmc tunnel1: mysql-135e02: mysql1Which service to tunnel to?: 2Password: ********Stopping Application: OKRedeploying tunnel application caldecott.Uploading Application: Checking for available resources: OK Packing application: OK Uploading (1K): OKPush Status: OKBinding Service [mysql1]: OKStaging Application: OKStarting Application: OKGetting tunnel connection info: OKService connection info: username : uMe6Apgw00AhS password : pKcD76PcZR7GZ name : d7cb8afb52f084f3d9bdc269e7d99ab50Starting tunnel to mysql1 on port 10000.1: none2: mysqlWhich client would you like to start?: 2
  • 82. …Using CaldecottLaunching mysql --protocol=TCP --host=localhost --port=10000 --user=uMe6Apgw00AhS -- password=pKcD76PcZR7GZ d7cb8afb52f084f3d9bdc269e7d99ab50Welcome to the MySQL monitor. Commands end with ; or g.Your MySQL connection id is 10944342Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL), Release 12.5, Revision 188Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type help; or h for help. Type c to clear the current input statement.mysql>
  • 83. Running JUnit test with Caldecott Configure your test code to use port + connection info 83
  • 84. Summary 84
  • 85. Monolithic applications are simple to develop and deployBUT have significant drawbacks 85
  • 86. Apply the scale cube §Modular, polyglot, and scalable applications Y axis - functional decomposition §Services developed, deployed and scaled independently ngi on iti rt pa a at -d is ax Z X axis - horizontal duplication 86
  • 87. Cloud Foundry helps Ap .js p lic a on Private    Se Data Services Clouds   rv ice  In te rfa ce Public Clouds ce rfa Msg Services te  In er vid ro d  P ou Micro Cl Clouds Other Services
  • 88. @crichardson chris.richardson@springsource.com http://plainoldobjects.com/presentations/ Questions?www.cloudfoundry.com @cloudfoundry
  • 89. Cloud Foundry 启动营在www.cloudfoundry.com注册账号并成功上传应用程序,即可于12月8日中午后凭账号ID和应用URL到签到处换取Cloud Foundry主题卫衣一件。89
  • 90. iPhone5 等你拿第二天大会结束前,请不要提前离 ,将填写完整的意见反馈表投到签到处的抽奖箱内,即可参与“iPhone5”抽奖活动。90
  • 91. Birds of a Feather 专家面对面所有讲师都会在课程结束后,到紫兰厅与来宾讨论课程上的问题91