JVM Support for Multitenant Applications - Steve Poole (IBM)
Upcoming SlideShare
Loading in...5
×
 

JVM Support for Multitenant Applications - Steve Poole (IBM)

on

  • 806 views

Presented at JAX London 2013 ...

Presented at JAX London 2013

Per-tenant resource management can help ensure that collocated tenants peacefully share computational resources based on individual quotas. This session begins with a comparison of deployment models (shared: hardware, OS, middleware, everything) to motivate the multitenant approach. The main topic is an exploration of experimental data isolation and resource management primitives in IBM’s JDK that combine to help make multitenant applications smaller and more predictable.

Statistics

Views

Total Views
806
Views on SlideShare
777
Embed Views
29

Actions

Likes
1
Downloads
20
Comments
0

2 Embeds 29

http://jmmiddleware.wordpress.com 25
https://twitter.com 4

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

JVM Support for Multitenant Applications - Steve Poole (IBM) JVM Support for Multitenant Applications - Steve Poole (IBM) Presentation Transcript

  • Steve Poole IBM JVM Support for Multitenant Applications
  • Important Disclaimers THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES. ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE. IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE. IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF: - CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS
  • Steve Poole ! Works at IBM’s Hursley Laboratory in the UK Involved in IBM Java VM development since before Java was 1 Currently leading IBM’s OpenJDK technical engagement
  • What this talk is about ! ! ! ! JVM Support for Multi-Tenant Applications This experimental technology is being developed to help address important pressures on Java ! Google ‘IBM Java 8 beta’ for more information ! ■ By the end of this session, you should be able to: ! – Understand what multitenancy is and what it’s good for ! – Describe the challenges of multitenant Java deployments ! – Understand ideas for new JDK features to convert existing applications into multitenant deployments !
  • Agenda 1. What Multi-tenancy is all about aka how to simplify to save time and money 2. The challenges of building a multi-tenant application in Java 3. How to keep your application running in a multi-tenant world dealing with bad behaviour 4. Risk vs. Reward: How dense can we go? 5. Wrap-up: Summary, and next steps
  • Part 1 What is Multitenancy? ! (simplify to save time and money)
  • Who owns one of these?
  • Who owns one of these?
  • Who owns one of these?
  • Who owns one of these? At a basic level Multitenancy is a drive to reduce unnecessary complexity and duplication this is sophisticated but expensive this is simpler and cheaper
  • Who owns one of these? At a basic level Multitenancy is a drive to reduce unnecessary complexity and duplication this is sophisticated but expensive Multitenancy helps with finding the right balance this is simpler and cheaper
  • Its not just a hardware story Simplifying the software stack by removing all extraneous pieces makes better use of hardware and the people who run it. ! Simple == Cheaper == Predictable == Robust
  • Simple Dont Repeat Yourself “Every piece of knowledge must have a single, unambiguous, authoritative representation within a system” 
 
 Pragmatic Programmer (Hunt & Thomas) (or: copy-and-paste encourages problems) We want to avoid the clone army
  • Are you already using Multitenancy •Are your choices only •fully dedicated machine or shared hardware? •Multitenancy isn’t just a checkbox. •There are various levels of ‘tenancy’ defined today.
  • SaaS Tenancy Spectrum
  • SaaS Tenancy Spectrum Your basic laptop, desktop or server machine. yours, all yours..
  • SaaS Tenancy Spectrum A simple shared machine LPAR’d or VM’d into multiple complete stacks.
  • SaaS Tenancy Spectrum Reduced overhead by running multiple app servers on one O/S Keep everything else separate though. “just in case”
  • SaaS Tenancy Spectrum Multiple applications running on one app server sharing db server and middleware!
  • SaaS Tenancy Spectrum sharing one app!
  • SaaS Tenancy Spectrum Sharing everything including database tables.
  • SaaS Tenancy Spectrum A Multitenant JVM covers theses levels of sharing
  • SaaS Tenancy Spectrum Merging these bubbles saves money, reduces complexity and can be a real business differentiator. But it requires engineering effort
  • SaaS Tenancy Spectrum Remember, from the end user point of view, they see it like this
  • Efficencies ■ Customer viewpoint ! – Cost: provider offers the service more cheaply – Time to Value: up and running fast, typically upgraded often & quickly – Quality of Service: focus on SLA needed not your ability to run infrastructure – Bypass IT Backlog: streamlined deployment (handled by provider) ! ■ Provider viewpoint – Cost: Minimal moving parts / duplication – Agility: Upgrades, backups, on-boarding Fewer Parts == Better Density == £££ for the service provider
  • Part 2 Getting there.. !
  • Climbing MT tenant Increasing density of the software stack is impacting us all. ! Cost of hosting is a business differentiator already ! Its only going to increase in importance ! If you’re in the cloud you already know ! ! Moving from ‘free’ to ‘not free’ hosting is an intensive exercise. Your software choices are starting to be decided by CPU and memory usage. With your own hardware you could let it run slow. Can’t do that in the cloud! ! So how do move forward with Java and Multitenancy?
  • Challenge #1 Isolation The fear factor is loosing isolation ■ unexpected side effects from other parts ■ This is a reasonable concern ■ Same number of eggs (apps), fewer baskets ■ You want really good baskets arranged carefully http://circa71.wordpress.com/2010/07/ ■ But it’s not a new problem ■ We can reduce the change of failure (more later) ■ We can reduce the impact with good choreography ■ You need to think about choreography today ■ Today we use clusters to provide fail-over and arrange clusters to avoid same machine/rack/site failures http://bit.ly/e7G1jb ■ Apply the same ideas to Multitenancy ■
  • Challenge #2 Cost of Entry merge merge J Hypervisor sharing only simplest (hence most popular) Still have neighbours but they are far away J Port Collisions J Data Isolation between apps J File System Collisions J Control over resource hogs J Security Challenges J Easy == No app changes J JVM can help!! Saves footprint (GB) But now we have collisions Ops guys can help with O/S JVMs can use -Xshareclasses More footprint savings But now we have real isolation concerns (closer neighbours) We need control over resource hogs
  • Challenge #2.5 Building isolation Java Heap consumes 100’s of MB of memory –Heap objects cannot be shared between JVMs –GC has a helper thread-per-core by default
 Just-in-Time Compiler consumes 10’s of MB of memory –Generated code is private and big –Generated code is expensive to produce • Steals time from application • Multiple compilation threads by default
 No choreography between JVM instances –Compilation or GC activity can happen at identical (and bad) times
  • Challenge #2.5 Building isolation We need to fix the following ! L Data Isolation between applications ! L Control over resource hogs Without forcing people to change their applications!
  • Data Isolation Challenges ■ ■ Applications embed deployment information like url patterns in code Wait! What happens if we try to deploy two copies of this servlet to a single server?
  • ■ Static variables are bad (for sharing) ■ Most libraries are full of static variables Wait! What happens if
 each tenant needs a 
 different default locale?
  • Wait! What happens if
 each tenant needs a 
 different default locale? •Usual solutions: – Wrap the whole thing in a ClassLoader (i.e. rewrite your code) – Get ride of the static variable (i.e. rewrite your code) – Use BCI to rewrite the code automatically (erodes robustness) ! ! We need a silver bullet!
  • Multitenant JDK ■ Concept: Add a single argument (–Xmt for multi-tenant) to your Java command-line to opt into sharing a runtime with others. 
 ■ Result: Your application behaves exactly as it if had a dedicated JVM, but in reality it runs side-by-side with other applications.
 ■ Benefits: Smaller, faster, and eventually smarter –Less duplication: (1 GC, 1 JIT), Heap object sharing –Fast Startup: JVM is already running and warm when starting apps ■ Required: No code Changes!
  • Part 3 Demo (of a demo)
  • Launch your application ■ Opt-in to multitenancy by adding –Xmt
  • Register with javad daemon ■ JVM will locate/start daemon automatically locate
  • Create a new tenant ■ New tenant created inside the javad daemon locate Tenant1
  • Create a 2nd tenant ■ New tenant created inside the javad daemon locate Tenant1
  • Create a 2nd tenant ■ New tenant created inside the javad daemon locate Tenant1 Tenant2 One copy of common code
 lives in the javad process.
 
 Most runtime structures
 are shared.
  • Providing data isolation ■ ■ What if … the JVM knew about tenants and provided each one with a different view of static variables? Meet the @TenantScope annotation. Tenant1 …
 LocaleSettings.setDefaultLocale(
 LocaleSettings.UK ); … Tenant2 …
 LocaleSettings.setDefaultLocale(
 LocaleSettings.US ); … ■ @TenantScope Semantics: Static variable values are stored per-tenant ■ Each tenant has their own LocaleSettings.defaultLocale ■ Now many tenants can share a single LocaleSettings class
  • Did I say ‘no code changes?’ ■ ■ @TenantScope markup gets added automatically as classes are loaded Tenants see dedicated middleware – but behind the curtains classes (and JIT’ed code) are actually shared Application
 Changes merge
  • If it’s invisible - why have @TenantScope? Allows middleware to opt out ■ Opportunities for even more density. ■ ! ■ Basic operations on Tenants available to the middleware –Data Isolation –Resource Management (more in this in a minute) ! ■ Ability for the middleware to differentiate between Tenants –Which one is causing the problem? ! ■ Querying the state of Tenants –How much free memory do you have?
  • Part 4 Hey, aren’t we done yet?
  • Dealing with bad behaviour http://bit.ly/ficwkl images from http://www.rra.memberlodge.org/Neighbourhood-Watch-Reporting
  • Shared environments need resource ctl The closer your neighbours the better your controls must be ■ Multitenant JDK provides controls on ■ –CPU time –Heap size –Thread count –File IO: read b/w, write b/w –Socket IO: read b/w, write b/w !
  • Resource Control Ergonomics ■ Simple command-line switches for new resources – -Xlimit:cpu=10-30 // 10% minimum CPU, 30% max // 30% max CPU – -Xlimit:cpu=30 –-Xlimit:netIO=20M // Max bandwidth of 20 Mbps ■ Existing options get mapped for free – -Xms8m –Xmx64m ■ // Initial 8M heap, 64M max Plus some JMX beans to see how much of each resource you are using – i.e. understand how your code uses resources by wrapping in a tenant
  • Building on JSR 284 JSR-284 Resource Consumption Mgmt API ■ Throttling at Java layer for portability ■ Or, leveraging OS WLM directly for efficiency (Linux & AIX) – Note: many WLMs tend to like processes, not groups of threads Tenant Tenant Tenant Tenant Tenant JVM Resource Management JSR 284 API JVM Memory CPU GC 
 (Heap Mgmt) Thread File I/O Socket I/O CPU File I/O Socket I/O Resource Throttle Layer OS Level Resources Management OS resources Resource native API OS Thread Handler OS Workload Manager (WLM) Socket Hardware resources CPU: XXX GHZ Memory: XXX GB DISK: XXXKB/S Network: XXXKB/S
  • CPU Throttling Round 1239s 1452s 1390s 1094s 1122s 1139s 1123s 6 1244s 1134s Average 1243s 1212s cpu throttling in jvm controller 90.00 60.00 67.50 cpu% 80.00 cpu% 1167s 5 cpu throttling in os controller 1267s 4 Accuracy 1362s 3 • Duration comparison: Linux AMD64, run a CPU-intensive app with 10 threads with 100% CPU quota, each thread doing the same Fibonacci calculation, benchmark the duration • Accuracy comparison: Linux AMD64, run two CPU-intensive apps each doing the same Fibonacci calculation, but with different CPU quota: 60% vs 30%, benchmark the accuracy JVM as controller 2 Benchmark setting OS as controller 1 Duration 40.00 20.00 45.00 60% throttling 30% throttling 22.50 0.00 0.00 00:0000:3001:0001:3002:0002:3003:0003:3004:0004:3005:0005:3006:0006:3007:0007:3008:0008:3009:0009:30 00:0100:3101:0101:3102:0102:3103:0103:3104:0104:3105:0105:3106:0106:3107:0107:3108:0108:3109:0109:31 00:0200:3201:0201:3202:0202:3203:0203:3204:0204:3205:0205:3206:0206:3207:0207:3208:0208:3209:0209:32 00:0300:3301:0301:3302:0302:3303:0303:3304:0304:3305:0305:3306:0306:3307:0307:3308:0308:3309:0309:33 00:0400:3401:0401:3402:0402:3403:0403:3404:0404:3405:0405:3406:0406:3407:0407:3408:0408:3409:0409:34 00:0500:3501:0501:3502:0502:3503:0503:3504:0504:3505:0505:3506:0506:3507:0507:3508:0508:3509:0509:35 00:0600:3601:0601:3602:0602:3603:0603:3604:0604:3605:0605:3606:0606:3607:0607:3608:0608:3609:0609:36 00:0700:3701:0701:3702:0702:3703:0703:3704:0704:3705:0705:3706:0706:3707:0707:3708:0708:3709:0709:37 00:0800:3801:0801:3802:0802:3803:0803:3804:0804:3805:0805:3806:0806:3807:0807:3808:0808:3809:0809:38 00:0900:3901:0901:3902:0902:3903:0903:3904:0904:3905:0905:3906:0906:3907:0907:3908:0908:3909:0909:39 00:1000:4001:1001:4002:1002:4003:1003:4004:1004:4005:1005:4006:1006:4007:1007:4008:1008:4009:1009:40 00:1100:4101:1101:4102:1102:4103:1103:4104:1104:4105:1105:4106:1106:4107:1107:4108:1108:4109:1109:41 00:1200:4201:1201:4202:1202:4203:1203:4204:1204:4205:1205:4206:1206:4207:1207:4208:1208:4209:1209:42 00:1300:4301:1301:4302:1302:4303:1303:4304:1304:4305:1305:4306:1306:4307:1307:4308:1308:4309:1309:43 00:1400:4401:1401:4402:1402:4403:1403:4404:1404:4405:1405:4406:1406:4407:1407:4408:1408:4409:1409:44 00:1500:4501:1501:4502:1502:4503:1503:4504:1504:4505:1505:4506:1506:4507:1507:4508:1508:4509:1509:45 00:1600:4601:1601:4602:1602:4603:1603:4604:1604:4605:1605:4606:1606:4607:1607:4608:1608:4609:1609:46 00:1700:4701:1701:4702:1702:4703:1703:4704:1704:4705:1705:4706:1706:4707:1707:4708:1708:4709:1709:47 00:1800:4801:1801:4802:1802:4803:1803:4804:1804:4805:1805:4806:1806:4807:1807:4808:1808:4809:1809:48 00:1900:4901:1901:4902:1902:4903:1903:4904:1904:4905:1905:4906:1906:4907:1907:4908:1908:4909:1909:49 00:2000:5001:2001:5002:2002:5003:2003:5004:2004:5005:2005:5006:2006:5007:2007:5008:2008:5009:2009:50 00:2100:5101:2101:5102:2102:5103:2103:5104:2104:5105:2105:5106:2106:5107:2107:5108:2108:5109:2109:51 00:2200:5201:2201:5202:2202:5203:2203:5204:2204:5205:2205:5206:2206:5207:2207:5208:2208:5209:2209:52 00:2300:5301:2301:5302:2302:5303:2303:5304:2304:5305:2305:5306:2306:5307:2307:5308:2308:5309:2309:53 00:2400:5401:2401:5402:2402:5403:2403:5404:2404:5405:2405:5406:2406:5407:2407:5408:2408:5409:2409:54 00:2500:5501:2501:5502:2502:5503:2503:5504:2504:5505:2505:5506:2506:5507:2507:5508:2508:5509:2509:55 00:2600:5601:2601:5602:2602:5603:2603:5604:2604:5605:2605:5606:2606:5607:2607:5608:2608:5609:2609:56 00:2700:5701:2701:5702:2702:5703:2703:5704:2704:5705:2705:5706:2706:5707:2707:5708:2708:5709:2709:57 00:2800:5801:2801:5802:2802:5803:2803:5804:2804:5805:2805:5806:2806:5807:2807:5808:2808:5809:2809:58 00:2900:5901:2901:5902:2902:5903:2903:5904:2904:5905:2905:5906:2906:5907:2907:5908:2908:5909:29 00:0000:4201:2402:0602:4803:3004:1204:5405:3606:1807:0007:4208:2409:0609:48 00:0100:4301:2502:0702:4903:3104:1304:5505:3706:1907:0107:4308:2509:0709:49 00:0200:4401:2602:0802:5003:3204:1404:5605:3806:2007:0207:4408:2609:0809:50 00:0300:4501:2702:0902:5103:3304:1504:5705:3906:2107:0307:4508:2709:0909:51 00:0400:4601:2802:1002:5203:3404:1604:5805:4006:2207:0407:4608:2809:1009:52 00:0500:4701:2902:1102:5303:3504:1704:5905:4106:2307:0507:4708:2909:1109:53 00:0600:4801:3002:1202:5403:3604:1805:0005:4206:2407:0607:4808:3009:1209:54 00:0700:4901:3102:1302:5503:3704:1905:0105:4306:2507:0707:4908:3109:1309:55 00:0800:5001:3202:1402:5603:3804:2005:0205:4406:2607:0807:5008:3209:1409:56 00:0900:5101:3302:1502:5703:3904:2105:0305:4506:2707:0907:5108:3309:1509:57 00:1000:5201:3402:1602:5803:4004:2205:0405:4606:2807:1007:5208:3409:1609:58 00:1100:5301:3502:1702:5903:4104:2305:0505:4706:2907:1107:5308:3509:17 00:1200:5401:3602:1803:0003:4204:2405:0605:4806:3007:1207:5408:3609:18 00:1300:5501:3702:1903:0103:4304:2505:0705:4906:3107:1307:5508:3709:19 00:1400:5601:3802:2003:0203:4404:2605:0805:5006:3207:1407:5608:3809:20 00:1500:5701:3902:2103:0303:4504:2705:0905:5106:3307:1507:5708:3909:21 00:1600:5801:4002:2203:0403:4604:2805:1005:5206:3407:1607:5808:4009:22 00:1700:5901:4102:2303:0503:4704:2905:1105:5306:3507:1707:5908:4109:23 00:1801:0001:4202:2403:0603:4804:3005:1205:5406:3607:1808:0008:4209:24 00:1901:0101:4302:2503:0703:4904:3105:1305:5506:3707:1908:0108:4309:25 00:2001:0201:4402:2603:0803:5004:3205:1405:5606:3807:2008:0208:4409:26 00:2101:0301:4502:2703:0903:5104:3305:1505:5706:3907:2108:0308:4509:27 00:2201:0401:4602:2803:1003:5204:3405:1605:5806:4007:2208:0408:4609:28 00:2301:0501:4702:2903:1103:5304:3505:1705:5906:4107:2308:0508:4709:29 00:2401:0601:4802:3003:1203:5404:3605:1806:0006:4207:2408:0608:4809:30 00:2501:0701:4902:3103:1303:5504:3705:1906:0106:4307:2508:0708:4909:31 00:2601:0801:5002:3203:1403:5604:3805:2006:0206:4407:2608:0808:5009:32 00:2701:0901:5102:3303:1503:5704:3905:2106:0306:4507:2708:0908:5109:33 00:2801:1001:5202:3403:1603:5804:4005:2206:0406:4607:2808:1008:5209:34 00:2901:1101:5302:3503:1703:5904:4105:2306:0506:4707:2908:1108:5309:35 00:3001:1201:5402:3603:1804:0004:4205:2406:0606:4807:3008:1208:5409:36 00:3101:1301:5502:3703:1904:0104:4305:2506:0706:4907:3108:1308:5509:37 00:3201:1401:5602:3803:2004:0204:4405:2606:0806:5007:3208:1408:5609:38 00:3301:1501:5702:3903:2104:0304:4505:2706:0906:5107:3308:1508:5709:39 00:3401:1601:5802:4003:2204:0404:4605:2806:1006:5207:3408:1608:5809:40 00:3501:1701:5902:4103:2304:0504:4705:2906:1106:5307:3508:1708:5909:41 00:3601:1802:0002:4203:2404:0604:4805:3006:1206:5407:3608:1809:0009:42 00:3701:1902:0102:4303:2504:0704:4905:3106:1306:5507:3708:1909:0109:43 00:3801:2002:0202:4403:2604:0804:5005:3206:1406:5607:3808:2009:0209:44 00:3901:2102:0302:4503:2704:0904:5105:3306:1506:5707:3908:2109:0309:45 00:4001:2202:0402:4603:2804:1004:5205:3406:1606:5807:4008:2209:0409:46 00:4101:2302:0502:4703:2904:1104:5305:3506:1706:5907:4108:2309:0509:47 time time os throttling jvm throttling Result: JVM control achieves comparable performance, but less accuracy.
  • Part 5 Performance so far
  • Current Performance Data ■ Environment: Measure standard benchmarks in a 1 GB + 1 core VirtualBox guest – Advantage: Easy to control, highly reproducible ■ Methodology: Add applications until the system swaps, then it’s ‘full’ – More applications is better – Per tenant cost is amount of RAM / # tenants
  • Standard JVM 20 15 10 5 0 instances Standard JVM Hand tuned Standard JVM MT support
  • Best hand-tuned JVM config 60 45 30 15 0 instances Standard JVM Hand tuned Standard JVM MT support
  • Simple out of the box -Xmt 300 4MB
 tenant 225 150 75 0 instances Standard JVM Hand tuned Standard JVM MT support
  • Part 6 Wrap up
  • This is still ‘experimental’ ■ Focus to date has been ‘zero application changes’ – We can do even better with tenant-aware middleware ■ API’s used to provide isolation & throttling are available to stack products – JSR-284 (Resource Management) – JSR-121 (Isolates) – @TenantScope fields ■ Java language and frameworks (EclipseLink) are evolving to have first-class multitenant support ■ Stay tuned for progress: watch the IBM Java 8 beta program
  • This is still ‘experimental’ Simplifying the software stack by removing all extraneous pieces makes better use of hardware (and people who run it). ! Multitenancy can make us more efficient: –Trades isolation for footprint and agility –JVM support makes multitenancy safer and easier –Measuring resource usage and load patterns is critical –Multitenant JDK primitives give us room for future growth
  • Conclusion Now that you’ve completed this session, you are able to:
 – Understand what multitenancy is and what it’s good for • Per-tenant costs measured in single-digit MB are possible
 – Describe challenges of multitenant Java deployments • Hard for VM guys, should be easy for you • Choreography of load / deployment is up to you
 – Understand new JDK features to convert existing applications into multitenant deployments • Are we on the right track? Could you use this in your business? Thank you - any questions?