Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Multi cloud appcustomscale-appgroups-slideshare

329 views

Published on

Multi-Cloud Management of Micro-Services using CloudFoundry APIs

Published in: Internet
  • Blog posting can be found here: https://cloudfoundryideas.wordpress.com/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Multi cloud appcustomscale-appgroups-slideshare

  1. 1. Multi-Cloud Management & Application Customized Scaling with CloudFoundry Edison Ting Solutions Architect / Field Engineering October 23, 2014 eting@pivotal.io
  2. 2. Motivation: Change Capture: • On Average = 1,500 – 5,500 ops/sec (inserts/updates/deletes) captured • Highly variable and bursty • Real time data flow problem A few months ago, we built a distributed replication solution for a customer, using distributed processes that read from queues populated with change capture data Each Replicator would read from a set of queues, then write data into a in-memory data grid, while periodically retiring batches of changes into a distributed database Replicator: On Average can handle 1,000+ ops/sec
  3. 3. On average each Replicator can process 1,000+ operations per second and so to keep up with the change capture process, we would add as many Replicator processes as needed to support the highest rate observed Adding Replicators allows the overall task to scale linearly Distributed Replication
  4. 4. We created a performance dashboard for monitoring of the overall throughput of the system, as replicators are manually added/removed to process the changing demands of the system. Distributed Replication Monitoring
  5. 5. We could not dynamically scale our replication processes to react fast enough to the demands of the incoming transactions We couldn’t take advantage of resources that might be available from other clusters to scale out the replication processes We needed an automated way to monitor whether the replication processes had reach peak loads so that we can begin to decide how to add or remove replication processes that are not needed We needed a way to allocate resources properly to processes so as to not to take away resources that can be used by other processes replicating other parts of the system Challenges:
  6. 6. Goals  Define multi-cloud management and application customized scaling and how it begins to solve some of the challenges  What is CloudF0undry: a PaaS that provides a polyglot and cloud agnostic environment with application lifecycle recovery and resource management capabilities  Demo of Multi-Cloud Management and Custom App Scaling Prototype  Discuss the architecture of the MCC and Custom App Scaling solution using CloudFoundry capabilities and APIs.
  7. 7. Pivotal - Our Mission
  8. 8. mccController can analyze resources globally, across multiple clusters managed by a PaaS, to determine where applications can run or where resource can be adjusted for provide optimal use of the system Application #1 PaaS (CloudFoundry) Multi-Cloud Controller Architecture mccMonitor mccController Application #2 Application #N Application #1 PaaS (CloudFoundry) mccMonitor Application #2 Application #N
  9. 9. mccController tracks the resource utilization of each PaaS that it manages, and based on the data can provide recommendations on where to deploy new applications and scale resources Application #1 PaaS (CloudFoundry) Multi-Cloud Controller Architecture mccMonitor mccController Application #2 Application #N Application #1 PaaS (CloudFoundry) mccMonitor Application #2 Application #N
  10. 10. Applications periodically reports resource utilization to mccMonitor as well as changing workload requirements. mccMonitor will manage the information and report to mccController for workload analysis Application #1 PaaS (CloudFoundry) Application #2 Application #N mccMonitor mccController Multi-Cloud Controller Architecture
  11. 11. mccController communicates with mccMonitors periodically while mccMonitors runs locally on each PaaS to allows application to communicate workload status as well as scaling requirements. mccController then communicates with the PaaS (CloudFoundry) to manage resources of applications and scale the number of application instances. Application #1 PaaS (CloudFoundry) Application #2 Application #N mccMonitor mccController Multi-Cloud Controller Architecture
  12. 12. mccController will communicate with multiple PaaS instances with different load characteristics, so that it can scale applications and add or drop app instances as necessary to achieve desired throughput and balance resource utilization between different PaaS instances. Application #1 PaaS (CloudFoundry) Multi-Cloud Controller Architecture mccMonitor mccController Application #2 Application #N Application #1 PaaS (CloudFoundry) mccMonitor Application #2 Application #N Application #1 Application #1 Application #2
  13. 13. CloudFoundry: An Operating System for the Cloud
  14. 14. CloudFoundry and Bosh provides an IaaS Agnostic Infrastructure +
  15. 15. Bosh manages other distributed processes besides CloudFoundry, such as Hadoop
  16. 16. CloudFoundry (PaaS) Architecture Containers Containers Containers Containers
  17. 17. What does CloudFoundry do with an Application ? Containers Containers Containers Containers Containers + + Kernel Containers
  18. 18. CloudFoundry manages the applications placement for effective scaling , high availability and recovery Containers
  19. 19. CloudFoundry promotes loose coupling of applications and services Containers Containers Environment variables Provision Services CloudFoundry provision services and make them available for use to applications. Applications access services using environment variables configured thru CloudFoundry
  20. 20. How CloudFoundry enables the Multi-Cloud Controller Architecture mccMonitor mccController Environment variables mccController can (1) deploy mccMonitor’s to each CloudFoundry instance, and have applications bind to the mccMonitor, enabling the application to communicate performance metrics, then (2) enable mccController to dynamically scale their resources mccMonitor Environment variables
  21. 21. Our Tools: CloudFoundry Client APIs How can we use these to accomplish what we need ?  Package: org.cloudfoundry.client.lib  CloudFoundryClient  CloudCredentials  Package: org.cloudfoundry.client.lib.domain  CloudInfo  CloudApplication  CloudService  ApplicationStats  InstanceStats  InstanceInfo  InstanceState
  22. 22. Demo
  23. 23. Pushing (deploying) Application(s) Containers Containers + + CloudFoundryClient client = new CloudFoundryClient(new CloudCredentials(user, pw), cfURL, org, space,..); client.login(); List<CloudDomain> domains = client.getDomains(); String appURL = “myApp” + domains(0).getName(); // myApp.cfapps.io client.createApplication( “myApp”, new Staging(), memSize, appURLs, services ); client.uploadApplication( “myApp”, file.getCanonicalPath() );
  24. 24. Pushing (deploying) Application(s) Containers Containers + + Create CloudFoundry client object Assign application name and URL Upload (push) and start application
  25. 25. Creating Service(s) Containers CloudFoundryClient client = new CloudFoundryClient( … ); String appURL = “myApp” + client.getDomains(0).getName(); // myApp.cfapps.io CloudService service = new CloudService(metaData, “myService”); Map<String, Object> credentials = new HashMap<String, Object>(); credentials.put( “myServiceAPI”, appURL ); client.createUserProvidedService( service, credentials ); myApp.cfapps.io
  26. 26. Creating Service(s) Containers myApp.cfapps.io Assign application name and URL Create Service object Provide credentials information to allow applications to use the service
  27. 27. Binding Service(s) to Application(s) Containers CloudApplication yourApp = client.getApplication( “yourApp” ); client.bindService( yourApp, “myService” ); client.stopApplication( “yourApp” ); client.startApplication( “yourApp” ); Environment variables Containers
  28. 28. Binding Service(s) to Application(s) Containers Environment variables Containers Get application object related of the app to be bound Bind application to service Stop and Start application
  29. 29. How can Applications find ‘bound’ Services ? Containers String myEnv = System.getenv( “VCAP_SERVICES” ); JSONObject obj = JSONValue.parse( myEnv ); JSONArray arrySrv = obj.get( “yourService” ); for( JSONObject srvObj: arrySrv ) { JSONObject credentials = srvObj.get( “credentials” ); String serviceUrl = credentials.get( “yourServiceAPI” ); Environment variables Containers
  30. 30. How can Applications find ‘bound’ Services ? Containers Environment variables Containers Get environment variable VCAP_SERVICES Find the service’s entry and get credentials information to be used to communicate with the service
  31. 31. How do we gather application statistics ? CloudFoundryClient client = new CloudFoundryClient(..); Map<> env = client.getApplication().getEnvAsMap(); int instCnt = client.getApplication().getInstances(); ApplicationStats stats = client.getApplicationStats(“yourApp”); for( InstanceStats is : stats.getRecords()) { InstanceStats.Usage usage = is.getUsage(); long memuse = usage.getMem(); long cpuuse = usage.getCpu(); long diskuse = usage.getDisk(); long memQuota = is.getMemQuota(); long diskQuota = is.getDiskQuota(); … } mccController
  32. 32. How do we gather application statistics ? mccController Create CloudFoundry Client Object Get Application Stats Object Iterate Applications Instances Stats object to get memory, disk, and instance count information
  33. 33. How do we scale application resources ? CloudFoundryClient clnt = new CloudFoundryClient(); clnt.stopApplication(“yourApp”); clnt.updateApplicationEnv(“yourApp”, Map<> env); clnt.updateApplicationMemory(“yourApp”, newVal); clnt.updateApplicationDiskQuota(“yourApp” , nDisk); clnt.updateApplicationInstances(“yourApp”, nInstns); clnt.startApplication(“yourApp”); mccController
  34. 34. How do we scale application resources ? mccController Create CloudFoundry Client Object Update application memory, disks, and instances Stop and Start application
  35. 35. How can we apply these APIs to the Multi-Cloud Controller Architecture ? mccMonitor mccController Environment variables mccMonitor Environment variables
  36. 36. Let us deploy the monitor and bind it to applications running on each CloudFoundry instance mccMonitor mccController Environment variables /* Push mccMonitor to CloudFoundry Instances */ pushMccMonitor( cloudFoundryURL ); /* Bind CloudFoundry Applications to MccMonitor */ bindApplicationsToMccMonitor( cloudFoundryURL, MccMonitorUrl ); /* Next Applications will provide workload specfiic statistics to MccMonitor… */
  37. 37. How do we enable the application to customize its scaling characteristics ? mccMonitor Environment variables /* Applications get mccMonitor URL via VCAP environment variable */ mccURL = getMccURL( System.getenv( “VCAP_SERVICES” ) ); /* Application provides known workload thresholds for each instance to mccURL */ URL( mccURL + “&invokeThreshold=1000&updateThreshold=500” ); /* Application periodically reports accumulated statistics about specific workloads */ URL( mccURL + “&binc=1&invokeCount=27&updateCount=4” ); App1 :{ invokeCount: 34 invokeThreshold: 1000 updateThreshold: 500 }
  38. 38. Let us follow the protocol between the applications and mccMonitor mccMonitor Environment variables /* Applications get mccMonitor URL via VCAP environment variable */ mccURL = getMccURL( System.getenv( “VCAP_SERVICES” ) ); /* Application provides known workload thresholds for each instance to mccURL */ URL( mccURL + “&invokeThreshold=1000&updateThreshold=500” ); /* Application periodically reports accumulated statistics about specific workloads */ URL( mccURL + “&binc=1&invokeCount=27&updateCount=4” ); App1 :{ invokeCount: 34 invokeThreshold: 1000 updateThreshold: 500 }
  39. 39. mccMonitor mccController Environment variables Let us follow the protocol between mccMonitor and mccController /* mccController loops thru all applications and checks last pollDateTime environment variable that it sets on every application to compute elapsed time since last poll */ CloudApplication appl = client.getApplication(..); Map<String,String> env = appl.getEnvAsMap(); prevPoll = env.get(“lastpoll”); prevPollDateTime = new Date(prevPoll); elapsedTime = currDate - prevPollDateTime /* mccController get applStats from monitor */ String mccMonitorUrl = App1 :{ invokeCount: 34 invokeThreshold: 1000 updateThreshold: 500 }
  40. 40. mccMonitor mccController Environment variables Protocol between mccMonitor and mccController (contd…) /* mccController takes variables ending in Count and Threshold and computes overall rate */ String jsonStruct = post2Monitor(mccMonitorUrl); JSONObject obj = JSONValue.parse(jsonStruct); Iterator it = obj.entrySet().iterator(); while( it.hasNext() ) if( it.next().getKey().endsWith(“Count”)) { thres = thresKey(key); … // compute rate achieved per instance ratePerInst = getRate(elapsed,numIns); // Add instance if threshold exceeded App1 :{ invokeCount: 34 invokeThreshold: 1000 updateThreshold: 500 }
  41. 41. mccMonitor mccController Environment variables Protocol between mccMonitor and mccController (contd…) /* mccController also reads in pctGrowTrigger and pctShrinkTrigger variables to scale memory */ String jsonStruct = post2Monitor(mccMonitorUrl); JSONObject obj = JSONValue.parse(jsonStruct); pctGrowTrigger = obj.get(“pctGrowTrigger”) pctGrowAmount = obj.get(“pctGrowAmount”); pctShrinkTrigger = obj.get(“pctShrinkTrigger”); pctShrinkAmount = obj.get(“pctShrinkAmount”); App1 :{ invokeCount: 34 invokeThreshold: 1000 updateThreshold: 500 }
  42. 42. Multi-Cloud Controller gathers resource utilization data directly from multiple CloudFoundry Instances for( cloudFoundryUrl : cfUrls ) { CloudFoundryClient client = new CloudFoundryClient(cloudFoundryUrL); List<CloudApplication> apps = client.getApplications(); for( CloudApplication app: apps ) { ApplicationStats stats = client.getApplicationStats( app.getName() ); for( InstanceStats is : stats.getRecords()) { InstanceStats.Usage usage = is.getUsage(); long memuse = usage.getMem(); long memQuota = mccController
  43. 43. Multi-Cloud Controller provides global view of how each CloudFoundry Instance utilize their resources mccController
  44. 44. mccController This CloudFoundry Instance shows a large portion of overall memory not being used, possible candidate for scaling down some instances ?
  45. 45. mccController This CloudFoundry Instance shows hosts only a few low footprint applications, should these be moved to the other instance that can host more instances ?
  46. 46. How do we adapt these tools to the distributed replication solution ? mccMonitor mccControllermccMonitor
  47. 47. How do we adapt these tools to the distributed replication solution ? mccMonitor mccController mccMonitor
  48. 48. How do we deploy the distributed apps shown below ?
  49. 49. Summary  Multi-cloud management and App customized scaling architecture makes applications active participants in the way they are scaled and how they consume resources within a collection of PaaS instances  Applications automatically scale up and down based on specific workloads they need to process  Distributed processes and Micro-services can self deploy and self bind to one another achieving loose coupling and independent scaling  Applications have full control of their destiny
  50. 50. Where can you find more information ?  CloudFoundry Client Java APIs  https://github.com/cloudfoundry/cf-java-client  CloudFoundry Documentation  http://docs.cloudfoundry.org/  https://github.com/cloudfoundry  Bosh Documentation  http://docs.cloudfoundry.org/bosh/  http://www.think-foundry.com/cloud-foundry-bosh-introduction/  https://github.com/cloudfoundry/bosh-lite  MicroServices Architectures  http://www.activestate.com/blog/2014/09/microservices-resources
  51. 51. Thank you!

×