JVM Support for Multitenant Applications - Steve Poole (IBM)

1,508 views

Published on

Presented at JAX London 2013

Per-tenant resource management can help ensure that collocated tenants peacefully share computational resources based on individual quotas. This session begins with a comparison of deployment models (shared: hardware, OS, middleware, everything) to motivate the multitenant approach. The main topic is an exploration of experimental data isolation and resource management primitives in IBM’s JDK that combine to help make multitenant applications smaller and more predictable.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,508
On SlideShare
0
From Embeds
0
Number of Embeds
33
Actions
Shares
0
Downloads
41
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

JVM Support for Multitenant Applications - Steve Poole (IBM)

  1. 1. Steve Poole IBM JVM Support for Multitenant Applications
  2. 2. Important Disclaimers THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES. ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE. IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE. IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF: - CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS
  3. 3. Steve Poole ! Works at IBM’s Hursley Laboratory in the UK Involved in IBM Java VM development since before Java was 1 Currently leading IBM’s OpenJDK technical engagement
  4. 4. What this talk is about ! ! ! ! JVM Support for Multi-Tenant Applications This experimental technology is being developed to help address important pressures on Java ! Google ‘IBM Java 8 beta’ for more information ! ■ By the end of this session, you should be able to: ! – Understand what multitenancy is and what it’s good for ! – Describe the challenges of multitenant Java deployments ! – Understand ideas for new JDK features to convert existing applications into multitenant deployments !
  5. 5. Agenda 1. What Multi-tenancy is all about aka how to simplify to save time and money 2. The challenges of building a multi-tenant application in Java 3. How to keep your application running in a multi-tenant world dealing with bad behaviour 4. Risk vs. Reward: How dense can we go? 5. Wrap-up: Summary, and next steps
  6. 6. Part 1 What is Multitenancy? ! (simplify to save time and money)
  7. 7. Who owns one of these?
  8. 8. Who owns one of these?
  9. 9. Who owns one of these?
  10. 10. Who owns one of these? At a basic level Multitenancy is a drive to reduce unnecessary complexity and duplication this is sophisticated but expensive this is simpler and cheaper
  11. 11. Who owns one of these? At a basic level Multitenancy is a drive to reduce unnecessary complexity and duplication this is sophisticated but expensive Multitenancy helps with finding the right balance this is simpler and cheaper
  12. 12. Its not just a hardware story Simplifying the software stack by removing all extraneous pieces makes better use of hardware and the people who run it. ! Simple == Cheaper == Predictable == Robust
  13. 13. Simple Dont Repeat Yourself “Every piece of knowledge must have a single, unambiguous, authoritative representation within a system” 
 
 Pragmatic Programmer (Hunt & Thomas) (or: copy-and-paste encourages problems) We want to avoid the clone army
  14. 14. Are you already using Multitenancy •Are your choices only •fully dedicated machine or shared hardware? •Multitenancy isn’t just a checkbox. •There are various levels of ‘tenancy’ defined today.
  15. 15. SaaS Tenancy Spectrum
  16. 16. SaaS Tenancy Spectrum Your basic laptop, desktop or server machine. yours, all yours..
  17. 17. SaaS Tenancy Spectrum A simple shared machine LPAR’d or VM’d into multiple complete stacks.
  18. 18. SaaS Tenancy Spectrum Reduced overhead by running multiple app servers on one O/S Keep everything else separate though. “just in case”
  19. 19. SaaS Tenancy Spectrum Multiple applications running on one app server sharing db server and middleware!
  20. 20. SaaS Tenancy Spectrum sharing one app!
  21. 21. SaaS Tenancy Spectrum Sharing everything including database tables.
  22. 22. SaaS Tenancy Spectrum A Multitenant JVM covers theses levels of sharing
  23. 23. SaaS Tenancy Spectrum Merging these bubbles saves money, reduces complexity and can be a real business differentiator. But it requires engineering effort
  24. 24. SaaS Tenancy Spectrum Remember, from the end user point of view, they see it like this
  25. 25. Efficencies ■ Customer viewpoint ! – Cost: provider offers the service more cheaply – Time to Value: up and running fast, typically upgraded often & quickly – Quality of Service: focus on SLA needed not your ability to run infrastructure – Bypass IT Backlog: streamlined deployment (handled by provider) ! ■ Provider viewpoint – Cost: Minimal moving parts / duplication – Agility: Upgrades, backups, on-boarding Fewer Parts == Better Density == £££ for the service provider
  26. 26. Part 2 Getting there.. !
  27. 27. Climbing MT tenant Increasing density of the software stack is impacting us all. ! Cost of hosting is a business differentiator already ! Its only going to increase in importance ! If you’re in the cloud you already know ! ! Moving from ‘free’ to ‘not free’ hosting is an intensive exercise. Your software choices are starting to be decided by CPU and memory usage. With your own hardware you could let it run slow. Can’t do that in the cloud! ! So how do move forward with Java and Multitenancy?
  28. 28. Challenge #1 Isolation The fear factor is loosing isolation ■ unexpected side effects from other parts ■ This is a reasonable concern ■ Same number of eggs (apps), fewer baskets ■ You want really good baskets arranged carefully http://circa71.wordpress.com/2010/07/ ■ But it’s not a new problem ■ We can reduce the change of failure (more later) ■ We can reduce the impact with good choreography ■ You need to think about choreography today ■ Today we use clusters to provide fail-over and arrange clusters to avoid same machine/rack/site failures http://bit.ly/e7G1jb ■ Apply the same ideas to Multitenancy ■
  29. 29. Challenge #2 Cost of Entry merge merge J Hypervisor sharing only simplest (hence most popular) Still have neighbours but they are far away J Port Collisions J Data Isolation between apps J File System Collisions J Control over resource hogs J Security Challenges J Easy == No app changes J JVM can help!! Saves footprint (GB) But now we have collisions Ops guys can help with O/S JVMs can use -Xshareclasses More footprint savings But now we have real isolation concerns (closer neighbours) We need control over resource hogs
  30. 30. Challenge #2.5 Building isolation Java Heap consumes 100’s of MB of memory –Heap objects cannot be shared between JVMs –GC has a helper thread-per-core by default
 Just-in-Time Compiler consumes 10’s of MB of memory –Generated code is private and big –Generated code is expensive to produce • Steals time from application • Multiple compilation threads by default
 No choreography between JVM instances –Compilation or GC activity can happen at identical (and bad) times
  31. 31. Challenge #2.5 Building isolation We need to fix the following ! L Data Isolation between applications ! L Control over resource hogs Without forcing people to change their applications!
  32. 32. Data Isolation Challenges ■ ■ Applications embed deployment information like url patterns in code Wait! What happens if we try to deploy two copies of this servlet to a single server?
  33. 33. ■ Static variables are bad (for sharing) ■ Most libraries are full of static variables Wait! What happens if
 each tenant needs a 
 different default locale?
  34. 34. Wait! What happens if
 each tenant needs a 
 different default locale? •Usual solutions: – Wrap the whole thing in a ClassLoader (i.e. rewrite your code) – Get ride of the static variable (i.e. rewrite your code) – Use BCI to rewrite the code automatically (erodes robustness) ! ! We need a silver bullet!
  35. 35. Multitenant JDK ■ Concept: Add a single argument (–Xmt for multi-tenant) to your Java command-line to opt into sharing a runtime with others. 
 ■ Result: Your application behaves exactly as it if had a dedicated JVM, but in reality it runs side-by-side with other applications.
 ■ Benefits: Smaller, faster, and eventually smarter –Less duplication: (1 GC, 1 JIT), Heap object sharing –Fast Startup: JVM is already running and warm when starting apps ■ Required: No code Changes!
  36. 36. Part 3 Demo (of a demo)
  37. 37. Launch your application ■ Opt-in to multitenancy by adding –Xmt
  38. 38. Register with javad daemon ■ JVM will locate/start daemon automatically locate
  39. 39. Create a new tenant ■ New tenant created inside the javad daemon locate Tenant1
  40. 40. Create a 2nd tenant ■ New tenant created inside the javad daemon locate Tenant1
  41. 41. Create a 2nd tenant ■ New tenant created inside the javad daemon locate Tenant1 Tenant2 One copy of common code
 lives in the javad process.
 
 Most runtime structures
 are shared.
  42. 42. Providing data isolation ■ ■ What if … the JVM knew about tenants and provided each one with a different view of static variables? Meet the @TenantScope annotation. Tenant1 …
 LocaleSettings.setDefaultLocale(
 LocaleSettings.UK ); … Tenant2 …
 LocaleSettings.setDefaultLocale(
 LocaleSettings.US ); … ■ @TenantScope Semantics: Static variable values are stored per-tenant ■ Each tenant has their own LocaleSettings.defaultLocale ■ Now many tenants can share a single LocaleSettings class
  43. 43. Did I say ‘no code changes?’ ■ ■ @TenantScope markup gets added automatically as classes are loaded Tenants see dedicated middleware – but behind the curtains classes (and JIT’ed code) are actually shared Application
 Changes merge
  44. 44. If it’s invisible - why have @TenantScope? Allows middleware to opt out ■ Opportunities for even more density. ■ ! ■ Basic operations on Tenants available to the middleware –Data Isolation –Resource Management (more in this in a minute) ! ■ Ability for the middleware to differentiate between Tenants –Which one is causing the problem? ! ■ Querying the state of Tenants –How much free memory do you have?
  45. 45. Part 4 Hey, aren’t we done yet?
  46. 46. Dealing with bad behaviour http://bit.ly/ficwkl images from http://www.rra.memberlodge.org/Neighbourhood-Watch-Reporting
  47. 47. Shared environments need resource ctl The closer your neighbours the better your controls must be ■ Multitenant JDK provides controls on ■ –CPU time –Heap size –Thread count –File IO: read b/w, write b/w –Socket IO: read b/w, write b/w !
  48. 48. Resource Control Ergonomics ■ Simple command-line switches for new resources – -Xlimit:cpu=10-30 // 10% minimum CPU, 30% max // 30% max CPU – -Xlimit:cpu=30 –-Xlimit:netIO=20M // Max bandwidth of 20 Mbps ■ Existing options get mapped for free – -Xms8m –Xmx64m ■ // Initial 8M heap, 64M max Plus some JMX beans to see how much of each resource you are using – i.e. understand how your code uses resources by wrapping in a tenant
  49. 49. Building on JSR 284 JSR-284 Resource Consumption Mgmt API ■ Throttling at Java layer for portability ■ Or, leveraging OS WLM directly for efficiency (Linux & AIX) – Note: many WLMs tend to like processes, not groups of threads Tenant Tenant Tenant Tenant Tenant JVM Resource Management JSR 284 API JVM Memory CPU GC 
 (Heap Mgmt) Thread File I/O Socket I/O CPU File I/O Socket I/O Resource Throttle Layer OS Level Resources Management OS resources Resource native API OS Thread Handler OS Workload Manager (WLM) Socket Hardware resources CPU: XXX GHZ Memory: XXX GB DISK: XXXKB/S Network: XXXKB/S
  50. 50. CPU Throttling Round 1239s 1452s 1390s 1094s 1122s 1139s 1123s 6 1244s 1134s Average 1243s 1212s cpu throttling in jvm controller 90.00 60.00 67.50 cpu% 80.00 cpu% 1167s 5 cpu throttling in os controller 1267s 4 Accuracy 1362s 3 • Duration comparison: Linux AMD64, run a CPU-intensive app with 10 threads with 100% CPU quota, each thread doing the same Fibonacci calculation, benchmark the duration • Accuracy comparison: Linux AMD64, run two CPU-intensive apps each doing the same Fibonacci calculation, but with different CPU quota: 60% vs 30%, benchmark the accuracy JVM as controller 2 Benchmark setting OS as controller 1 Duration 40.00 20.00 45.00 60% throttling 30% throttling 22.50 0.00 0.00 00:0000:3001:0001:3002:0002:3003:0003:3004:0004:3005:0005:3006:0006:3007:0007:3008:0008:3009:0009:30 00:0100:3101:0101:3102:0102:3103:0103:3104:0104:3105:0105:3106:0106:3107:0107:3108:0108:3109:0109:31 00:0200:3201:0201:3202:0202:3203:0203:3204:0204:3205:0205:3206:0206:3207:0207:3208:0208:3209:0209:32 00:0300:3301:0301:3302:0302:3303:0303:3304:0304:3305:0305:3306:0306:3307:0307:3308:0308:3309:0309:33 00:0400:3401:0401:3402:0402:3403:0403:3404:0404:3405:0405:3406:0406:3407:0407:3408:0408:3409:0409:34 00:0500:3501:0501:3502:0502:3503:0503:3504:0504:3505:0505:3506:0506:3507:0507:3508:0508:3509:0509:35 00:0600:3601:0601:3602:0602:3603:0603:3604:0604:3605:0605:3606:0606:3607:0607:3608:0608:3609:0609:36 00:0700:3701:0701:3702:0702:3703:0703:3704:0704:3705:0705:3706:0706:3707:0707:3708:0708:3709:0709:37 00:0800:3801:0801:3802:0802:3803:0803:3804:0804:3805:0805:3806:0806:3807:0807:3808:0808:3809:0809:38 00:0900:3901:0901:3902:0902:3903:0903:3904:0904:3905:0905:3906:0906:3907:0907:3908:0908:3909:0909:39 00:1000:4001:1001:4002:1002:4003:1003:4004:1004:4005:1005:4006:1006:4007:1007:4008:1008:4009:1009:40 00:1100:4101:1101:4102:1102:4103:1103:4104:1104:4105:1105:4106:1106:4107:1107:4108:1108:4109:1109:41 00:1200:4201:1201:4202:1202:4203:1203:4204:1204:4205:1205:4206:1206:4207:1207:4208:1208:4209:1209:42 00:1300:4301:1301:4302:1302:4303:1303:4304:1304:4305:1305:4306:1306:4307:1307:4308:1308:4309:1309:43 00:1400:4401:1401:4402:1402:4403:1403:4404:1404:4405:1405:4406:1406:4407:1407:4408:1408:4409:1409:44 00:1500:4501:1501:4502:1502:4503:1503:4504:1504:4505:1505:4506:1506:4507:1507:4508:1508:4509:1509:45 00:1600:4601:1601:4602:1602:4603:1603:4604:1604:4605:1605:4606:1606:4607:1607:4608:1608:4609:1609:46 00:1700:4701:1701:4702:1702:4703:1703:4704:1704:4705:1705:4706:1706:4707:1707:4708:1708:4709:1709:47 00:1800:4801:1801:4802:1802:4803:1803:4804:1804:4805:1805:4806:1806:4807:1807:4808:1808:4809:1809:48 00:1900:4901:1901:4902:1902:4903:1903:4904:1904:4905:1905:4906:1906:4907:1907:4908:1908:4909:1909:49 00:2000:5001:2001:5002:2002:5003:2003:5004:2004:5005:2005:5006:2006:5007:2007:5008:2008:5009:2009:50 00:2100:5101:2101:5102:2102:5103:2103:5104:2104:5105:2105:5106:2106:5107:2107:5108:2108:5109:2109:51 00:2200:5201:2201:5202:2202:5203:2203:5204:2204:5205:2205:5206:2206:5207:2207:5208:2208:5209:2209:52 00:2300:5301:2301:5302:2302:5303:2303:5304:2304:5305:2305:5306:2306:5307:2307:5308:2308:5309:2309:53 00:2400:5401:2401:5402:2402:5403:2403:5404:2404:5405:2405:5406:2406:5407:2407:5408:2408:5409:2409:54 00:2500:5501:2501:5502:2502:5503:2503:5504:2504:5505:2505:5506:2506:5507:2507:5508:2508:5509:2509:55 00:2600:5601:2601:5602:2602:5603:2603:5604:2604:5605:2605:5606:2606:5607:2607:5608:2608:5609:2609:56 00:2700:5701:2701:5702:2702:5703:2703:5704:2704:5705:2705:5706:2706:5707:2707:5708:2708:5709:2709:57 00:2800:5801:2801:5802:2802:5803:2803:5804:2804:5805:2805:5806:2806:5807:2807:5808:2808:5809:2809:58 00:2900:5901:2901:5902:2902:5903:2903:5904:2904:5905:2905:5906:2906:5907:2907:5908:2908:5909:29 00:0000:4201:2402:0602:4803:3004:1204:5405:3606:1807:0007:4208:2409:0609:48 00:0100:4301:2502:0702:4903:3104:1304:5505:3706:1907:0107:4308:2509:0709:49 00:0200:4401:2602:0802:5003:3204:1404:5605:3806:2007:0207:4408:2609:0809:50 00:0300:4501:2702:0902:5103:3304:1504:5705:3906:2107:0307:4508:2709:0909:51 00:0400:4601:2802:1002:5203:3404:1604:5805:4006:2207:0407:4608:2809:1009:52 00:0500:4701:2902:1102:5303:3504:1704:5905:4106:2307:0507:4708:2909:1109:53 00:0600:4801:3002:1202:5403:3604:1805:0005:4206:2407:0607:4808:3009:1209:54 00:0700:4901:3102:1302:5503:3704:1905:0105:4306:2507:0707:4908:3109:1309:55 00:0800:5001:3202:1402:5603:3804:2005:0205:4406:2607:0807:5008:3209:1409:56 00:0900:5101:3302:1502:5703:3904:2105:0305:4506:2707:0907:5108:3309:1509:57 00:1000:5201:3402:1602:5803:4004:2205:0405:4606:2807:1007:5208:3409:1609:58 00:1100:5301:3502:1702:5903:4104:2305:0505:4706:2907:1107:5308:3509:17 00:1200:5401:3602:1803:0003:4204:2405:0605:4806:3007:1207:5408:3609:18 00:1300:5501:3702:1903:0103:4304:2505:0705:4906:3107:1307:5508:3709:19 00:1400:5601:3802:2003:0203:4404:2605:0805:5006:3207:1407:5608:3809:20 00:1500:5701:3902:2103:0303:4504:2705:0905:5106:3307:1507:5708:3909:21 00:1600:5801:4002:2203:0403:4604:2805:1005:5206:3407:1607:5808:4009:22 00:1700:5901:4102:2303:0503:4704:2905:1105:5306:3507:1707:5908:4109:23 00:1801:0001:4202:2403:0603:4804:3005:1205:5406:3607:1808:0008:4209:24 00:1901:0101:4302:2503:0703:4904:3105:1305:5506:3707:1908:0108:4309:25 00:2001:0201:4402:2603:0803:5004:3205:1405:5606:3807:2008:0208:4409:26 00:2101:0301:4502:2703:0903:5104:3305:1505:5706:3907:2108:0308:4509:27 00:2201:0401:4602:2803:1003:5204:3405:1605:5806:4007:2208:0408:4609:28 00:2301:0501:4702:2903:1103:5304:3505:1705:5906:4107:2308:0508:4709:29 00:2401:0601:4802:3003:1203:5404:3605:1806:0006:4207:2408:0608:4809:30 00:2501:0701:4902:3103:1303:5504:3705:1906:0106:4307:2508:0708:4909:31 00:2601:0801:5002:3203:1403:5604:3805:2006:0206:4407:2608:0808:5009:32 00:2701:0901:5102:3303:1503:5704:3905:2106:0306:4507:2708:0908:5109:33 00:2801:1001:5202:3403:1603:5804:4005:2206:0406:4607:2808:1008:5209:34 00:2901:1101:5302:3503:1703:5904:4105:2306:0506:4707:2908:1108:5309:35 00:3001:1201:5402:3603:1804:0004:4205:2406:0606:4807:3008:1208:5409:36 00:3101:1301:5502:3703:1904:0104:4305:2506:0706:4907:3108:1308:5509:37 00:3201:1401:5602:3803:2004:0204:4405:2606:0806:5007:3208:1408:5609:38 00:3301:1501:5702:3903:2104:0304:4505:2706:0906:5107:3308:1508:5709:39 00:3401:1601:5802:4003:2204:0404:4605:2806:1006:5207:3408:1608:5809:40 00:3501:1701:5902:4103:2304:0504:4705:2906:1106:5307:3508:1708:5909:41 00:3601:1802:0002:4203:2404:0604:4805:3006:1206:5407:3608:1809:0009:42 00:3701:1902:0102:4303:2504:0704:4905:3106:1306:5507:3708:1909:0109:43 00:3801:2002:0202:4403:2604:0804:5005:3206:1406:5607:3808:2009:0209:44 00:3901:2102:0302:4503:2704:0904:5105:3306:1506:5707:3908:2109:0309:45 00:4001:2202:0402:4603:2804:1004:5205:3406:1606:5807:4008:2209:0409:46 00:4101:2302:0502:4703:2904:1104:5305:3506:1706:5907:4108:2309:0509:47 time time os throttling jvm throttling Result: JVM control achieves comparable performance, but less accuracy.
  51. 51. Part 5 Performance so far
  52. 52. Current Performance Data ■ Environment: Measure standard benchmarks in a 1 GB + 1 core VirtualBox guest – Advantage: Easy to control, highly reproducible ■ Methodology: Add applications until the system swaps, then it’s ‘full’ – More applications is better – Per tenant cost is amount of RAM / # tenants
  53. 53. Standard JVM 20 15 10 5 0 instances Standard JVM Hand tuned Standard JVM MT support
  54. 54. Best hand-tuned JVM config 60 45 30 15 0 instances Standard JVM Hand tuned Standard JVM MT support
  55. 55. Simple out of the box -Xmt 300 4MB
 tenant 225 150 75 0 instances Standard JVM Hand tuned Standard JVM MT support
  56. 56. Part 6 Wrap up
  57. 57. This is still ‘experimental’ ■ Focus to date has been ‘zero application changes’ – We can do even better with tenant-aware middleware ■ API’s used to provide isolation & throttling are available to stack products – JSR-284 (Resource Management) – JSR-121 (Isolates) – @TenantScope fields ■ Java language and frameworks (EclipseLink) are evolving to have first-class multitenant support ■ Stay tuned for progress: watch the IBM Java 8 beta program
  58. 58. This is still ‘experimental’ Simplifying the software stack by removing all extraneous pieces makes better use of hardware (and people who run it). ! Multitenancy can make us more efficient: –Trades isolation for footprint and agility –JVM support makes multitenancy safer and easier –Measuring resource usage and load patterns is critical –Multitenant JDK primitives give us room for future growth
  59. 59. Conclusion Now that you’ve completed this session, you are able to:
 – Understand what multitenancy is and what it’s good for • Per-tenant costs measured in single-digit MB are possible
 – Describe challenges of multitenant Java deployments • Hard for VM guys, should be easy for you • Choreography of load / deployment is up to you
 – Understand new JDK features to convert existing applications into multitenant deployments • Are we on the right track? Could you use this in your business? Thank you - any questions?

×