© 2013 IBM Corporation
Runtime Innovation:
NextGen Ninja Hacking of the JVM
Ryan Sciampacone – IBM Managed Runtime Archite...
© 2013 IBM Corporation2
Important Disclaimers
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR
INFORMATIONAL...
© 2013 IBM Corporation3
Introduction to the speaker
■  16 years experience developing and deploying Java SDKs
■  Recent wo...
© 2013 IBM Corporation4
What should you get from this talk?
■  JVM proving to be a fertile ecosystem for languages
■  Plen...
© 2013 IBM Corporation5
The runtime isn’t boring!
© 2013 IBM Corporation6
Packed Objects
© 2013 IBM Corporation7
Problem? What problem?
■  JNI just isn’t a great way to marshal data
■  Locality in Java can matte...
© 2013 IBM Corporation8
What are we trying to solve?
Simple enough…
Hash
Array
Entry
(object)
(object)
Object header
Objec...
© 2013 IBM Corporation9
What are we trying to solve?
Simple enough…
■  Header overhead
Hash
Array
Entry
(object)
(object)
...
© 2013 IBM Corporation10
What are we trying to solve?
Simple enough…
■  Header overhead
■  Pointer chasing
Hash
Array
Entr...
© 2013 IBM Corporation11
What are we trying to solve?
Simple enough…
■  Header overhead
■  Pointer chasing
■  Locality
Has...
© 2013 IBM Corporation12
What are we trying to solve?
int
int
Object header
Fields / data
Java Native
…
int[] d
anObject
F...
© 2013 IBM Corporation13
Ok so we have some criteria…
■  Ability to do away with headers
■  Ability to bring multiple obje...
© 2013 IBM Corporation14
Packed Objects: Under the covers
int y
int x
aPoint
Object header
Object field / data
© 2013 IBM Corporation15
Packed Objects: Under the covers
int y
int x
aPoint
Object header
Object field / data
offset
targ...
© 2013 IBM Corporation16
Packed Objects: Under the covers
int y
int x
aPoint
Object header
Object field / data
offset
targ...
© 2013 IBM Corporation17
Packed Objects: In Practice
int y
int x
aPoint
Object field / data
int y
int x
aPointPoint e
Poin...
© 2013 IBM Corporation18
Packed Objects: In Practice
int y
int x
aPoint
Object header
Object field / data
int y
int x
aPoi...
© 2013 IBM Corporation19
Packed Objects: In Practice
int y
int x
aPoint
Object header
Object field / data
int y
int x
aPoi...
© 2013 IBM Corporation20
Packed Objects: In Practice
int y
int x
aPoint
Object header
Object field / data
int y
int x
aPoi...
© 2013 IBM Corporation21
Packed Objects: In Practice
int y
int x
aPoint
Object header
Object field / data
int y
int x
aPoi...
© 2013 IBM Corporation22
Packed Objects: In Practice
int y
int x
aPoint
Object header
Object field / data
int y
int x
aPoi...
© 2013 IBM Corporation23
Packed Objects: In Practice
int y
int x
aPoint
Object header
Object field / data
int y
int x
aPoi...
© 2013 IBM Corporation24
Packed Objects: In Practice with Native Access
int y
int x
Object header
Struct field / data
Java...
© 2013 IBM Corporation25
Packed Objects: In Practice with Native Access
int y
int x
Object header
Struct field / data
offs...
© 2013 IBM Corporation26
Packed Objects: In Practice with Native Access
int y
int x
Object header
Struct field / data
offs...
© 2013 IBM Corporation27
Lets Build Something in C!
■  Nested substructures
■  Compact representation
■  Alignment aspects...
© 2013 IBM Corporation28
Let’s Build the Same “Something” in Java!
■  Headers
■  No locality
■  Alignment
class Address {
...
© 2013 IBM Corporation29
What does the Java code look like under the covers?
■  From a code point of view, this isn’t terr...
© 2013 IBM Corporation30
What if we did this with Packed Objects?
■  The Java code is pretty clean… and a pretty good resu...
© 2013 IBM Corporation31
Ok, what about the code under the covers?
■  Bytecodes don’t change… JIT code is pretty good too!...
© 2013 IBM Corporation32
What about native access?
Java Native
…
anObject
How do we implement this normally?
port
addr
por...
© 2013 IBM Corporation33
JNI implementation
■  Usual “stash pointers in long types” tricks
■  JNI costs tend to be high
© 2013 IBM Corporation34
DirectByteBuffer implementation
■  No extra JNI to write (this is good)
■  Keeping your indices s...
© 2013 IBM Corporation35
Unsafe implementation
■  You shouldn’t be here
■  Still playing the indices game
© 2013 IBM Corporation36
PackedObject answer
■  Looks like natural Java code
■  Foregoes JNI
■  Same type capable of on-he...
© 2013 IBM Corporation37
Active work and next steps
■  Experimenting with this now
■  Yes, there are security aspects to b...
© 2013 IBM Corporation38
Multitenancy
© 2013 IBM Corporation39
Just what do you mean by “multitenancy”?
With a multitenant architecture, a software application ...
© 2013 IBM Corporation40
S5. Shared
Application
JDK Support for Spectrum of Sharing / Multitenancy (Level 1-5)
S1. No Shar...
© 2013 IBM Corporation41
= LApplication
Changes
© 2013 IBM Corporation42
Hardware Virtualization
■  Hypervisors run multiple applications side-by-side safely
■  Advantage...
© 2013 IBM Corporation43
Hardware Virtualization
■  Hypervisors JVMs can run multiple applications side-by-side safely
■  ...
© 2013 IBM Corporation44
Multitenancy: Low (or no) barrier to entry
■  Multitenancy is all about reducing duplication by t...
© 2013 IBM Corporation45
JVM: Separating State
■  Static variables are a problem for sharing
■  Consider use of System.out...
© 2013 IBM Corporation46
JVM: Separating State
getstatic does 2 things
1.  Triggers class initialization on first contact
...
© 2013 IBM Corporation47
JVM: Standard Static Fields
getstatic <cpIndex>
0: if constantPool[cpIndex] is resolved then {
1:...
© 2013 IBM Corporation48
JVM: Isolated Static Fields
getstatic <cpIndex>
0: if constantPool[cpIndex] is resolved then {
1:...
© 2013 IBM Corporation49
JVM: Separating State
■  There are other initialization triggers too
– putstatic
– invokestatic
–...
© 2013 IBM Corporation50
More that just JVM State…
■  Throttling of resources
– Threads, GC, sockets, files (IO in general...
© 2013 IBM Corporation51
Other Thoughts – Native Libraries and shared state
■  Use separate processes to manage different ...
© 2013 IBM Corporation52
Heap Memory
(not Garbage Collection!)
© 2013 IBM Corporation53
Scaling Garbage Collection
■  Existing Garbage Collection (GC) technology defers expensive costs
...
© 2013 IBM Corporation54
Challenge: Segregate Memory by Differentiators
Heap
Varying memory
characteristics and
requiremen...
© 2013 IBM Corporation55
Memory Hierarchy and Performance
■  Increased core and HW thread counts are not silver bullets to...
© 2013 IBM Corporation56
Let’s look at transferring data
Heap
A
B
C
© 2013 IBM Corporation57
Let’s look at transferring data
Heap
A
B
C
Remote
Transfer
© 2013 IBM Corporation58
Let’s look at transferring data
Heap
A
B
C
Remote
Transfer
© 2013 IBM Corporation59
Let’s look at transferring data
Heap
A
B
C
Remote
Transfer
© 2013 IBM Corporation60
PackedObjects could help…
Heap
A
Remote
Transfer
C
B
© 2013 IBM Corporation61
PackedObjects could help…
Heap
A
B
C
Remote
Transfer
A
Packed
C
B
© 2013 IBM Corporation62
PackedObjects could help…
Heap
A
B
C
Remote
Transfer
A
Packed
B
C
A
Packed
C
B
© 2013 IBM Corporation63
Making the data transfer easier…
Heap
Remote
Transfer
© 2013 IBM Corporation64
Making the data transfer easier…
Heap
Remote
Transfer
Specialized Heap Area
© 2013 IBM Corporation65
Making the data transfer easier…
Heap
B
Remote
Transfer
A
B
C
A
C
Specialized Heap Area
© 2013 IBM Corporation66
Making the data transfer easier…
Heap
B
Remote
Transfer
A
B
C
A
C
Specialized Heap Area
© 2013 IBM Corporation67
Making the data transfer seamless
Heap
Remote
Transfer
Specialized Heap Area
© 2013 IBM Corporation68
B
C
A
B
C
A
Making the data transfer seamless
Heap
Remote
Transfer
B
C
A
Specialized Heap Area
© 2013 IBM Corporation69
B
C
A
B
C
A
Making the data transfer seamless
Heap
Remote
Transfer
B
C
A
Packed
Specialized Heap ...
© 2013 IBM Corporation70
Tip of the Iceberg
■  Just scratching the surface of what is possible
■  GC technology has an eff...
© 2013 IBM Corporation71
Virtualization
© 2013 IBM Corporation72
Finding your way in The Cloud
■  Constant pressure to reduce cost and deliver more services for a...
© 2013 IBM Corporation73
The Layers of Liars
■  Your application could end up querying 3 different layers to get a (hopefu...
© 2013 IBM Corporation74
The Layers of Liars
Hypervisor
Hardware
App
OS
App
Image source (Wikipedia Commons)
CPU: https://...
© 2013 IBM Corporation75
The Layers of Liars
Hypervisor
Hardware
App
OS
App
Image source (Wikipedia Commons)
CPU: https://...
© 2013 IBM Corporation76
The Layers of Liars
Hypervisor
Hardware
App
OS
App App
OS
Image source (Wikipedia Commons)
CPU: h...
© 2013 IBM Corporation77
The Layers of Liars
■  You don’t even have a constant picture of the universe
Hypervisor
Hardware...
© 2013 IBM Corporation78
Knowing what you don’t know
(What does this even mean anymore?)
© 2013 IBM Corporation79
Interfaces… more importantly behavior
■  Provide interfaces to the “Layers of liars” to enable mi...
© 2013 IBM Corporation80
…And after all that
© 2013 IBM Corporation81
Tenant
Nimble Reactions to External Events
Hypervisor
Hardware
OS
Application
Tenant
© 2013 IBM Corporation82
Tenant
Nimble Reactions to External Events
Hypervisor
Hardware
OS
Application
Tenant
© 2013 IBM Corporation83
TenantTenant
Nimble Reactions to External Events
Hypervisor
Hardware
OS
ApplicationApp
OS
Tenant
© 2013 IBM Corporation84
TenantTenant Tenant Tenant
Nimble Reactions to External Events
Hypervisor
Hardware
OS
Application...
© 2013 IBM Corporation85
TenantTenant Tenant Tenant
Nimble Reactions to External Events
Hypervisor
Hardware
OS
Application...
© 2013 IBM Corporation86
Questions?
© 2013 IBM Corporation87
References
■  Get Products and Technologies:
– IBM Java Runtimes and SDKs:
•  https://www.ibm.com...
© 2013 IBM Corporation88
Copyright and Trademarks
© IBM Corporation 2012. All Rights Reserved.
IBM, the IBM logo, and ibm....
Upcoming SlideShare
Loading in …5
×

Runtime Innovation - Nextgen Ninja Hacking of the JVM, by Ryan Sciampacone

2,001 views

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,001
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
35
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Runtime Innovation - Nextgen Ninja Hacking of the JVM, by Ryan Sciampacone

  1. 1. © 2013 IBM Corporation Runtime Innovation: NextGen Ninja Hacking of the JVM Ryan Sciampacone – IBM Managed Runtime Architect 14th June 2013
  2. 2. © 2013 IBM Corporation2 Important Disclaimers THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES. ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE. IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE. IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF: - CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS
  3. 3. © 2013 IBM Corporation3 Introduction to the speaker ■  16 years experience developing and deploying Java SDKs ■  Recent work focus: ■  Managed Runtime Architecture ■  Java Virtual Machine improvements ■  Multi-tenancy technology ■  Native data access and heap density ■  Footprint and performance ■  Garbage Collection ■  Scalability and pause time reduction ■  Advanced GC technology ■  My contact information: – Ryan_Sciampacone@ca.ibm.com
  4. 4. © 2013 IBM Corporation4 What should you get from this talk? ■  JVM proving to be a fertile ecosystem for languages ■  Plenty of opportunity to innovate in other spaces ■  Runtime is the gateway to this innovation ■  Largely ignored the last few years, but this is where the core inventions can occur
  5. 5. © 2013 IBM Corporation5 The runtime isn’t boring!
  6. 6. © 2013 IBM Corporation6 Packed Objects
  7. 7. © 2013 IBM Corporation7 Problem? What problem? ■  JNI just isn’t a great way to marshal data ■  Locality in Java can matter (e.g., JEP 142) ■  Existing native and data placement stories aren’t very good ■  In many cases, legacy systems exist – the interop is just terrible ■  So we want something that integrates well with the Java language and helps us…
  8. 8. © 2013 IBM Corporation8 What are we trying to solve? Simple enough… Hash Array Entry (object) (object) Object header Object field / data table key value
  9. 9. © 2013 IBM Corporation9 What are we trying to solve? Simple enough… ■  Header overhead Hash Array Entry (object) (object) Object header Object field / data table key value
  10. 10. © 2013 IBM Corporation10 What are we trying to solve? Simple enough… ■  Header overhead ■  Pointer chasing Hash Array Entry (object) (object) Object header Object field / data table key value
  11. 11. © 2013 IBM Corporation11 What are we trying to solve? Simple enough… ■  Header overhead ■  Pointer chasing ■  Locality Hash Array Entry (object) (object) Object header Object field / data table key value
  12. 12. © 2013 IBM Corporation12 What are we trying to solve? int int Object header Fields / data Java Native … int[] d anObject Fighting the Java/Native interface int int int …
  13. 13. © 2013 IBM Corporation13 Ok so we have some criteria… ■  Ability to do away with headers ■  Ability to bring multiple objects close together ■  On heap / off heap seamless referencing of data ■  This actually sounds a lot like C structure types ■  Packed Objects! struct Address { char[4] addr; short port; } struct Header { struct Address src; struct Address dst; } struct Header port addr Address src Address dst port addr
  14. 14. © 2013 IBM Corporation14 Packed Objects: Under the covers int y int x aPoint Object header Object field / data
  15. 15. © 2013 IBM Corporation15 Packed Objects: Under the covers int y int x aPoint Object header Object field / data offset target aPackedPoint int y int x
  16. 16. © 2013 IBM Corporation16 Packed Objects: Under the covers int y int x aPoint Object header Object field / data offset target aPackedPoint int y int x
  17. 17. © 2013 IBM Corporation17 Packed Objects: In Practice int y int x aPoint Object field / data int y int x aPointPoint e Point s aLine Object header
  18. 18. © 2013 IBM Corporation18 Packed Objects: In Practice int y int x aPoint Object header Object field / data int y int x aPointPoint e Point s aLine aPackedLine int y int x int y int x offset target
  19. 19. © 2013 IBM Corporation19 Packed Objects: In Practice int y int x aPoint Object header Object field / data int y int x aPointPoint e Point s aLine int y int x int y int x aPackedPoint s aPackedPoint e aPackedLine offset target
  20. 20. © 2013 IBM Corporation20 Packed Objects: In Practice int y int x aPoint Object header Object field / data int y int x aPointPoint e Point s aLine int y int x int y int x aPackedPoint s aPackedPoint e @Packed final class PackedPoint extends PackedObject { int x; int y; } @Packed final class PackedLine extends PackedObject { PackedPoint s; PackedPoint e; } aPackedLine offset target
  21. 21. © 2013 IBM Corporation21 Packed Objects: In Practice int y int x aPoint Object header Object field / data int y int x aPointPoint e Point s aLine int y int x int y int x aPackedLine offset target
  22. 22. © 2013 IBM Corporation22 Packed Objects: In Practice int y int x aPoint Object header Object field / data int y int x aPointPoint e Point s aLine int y int x int y int x aPackedLine.e aPackedLine offset target
  23. 23. © 2013 IBM Corporation23 Packed Objects: In Practice int y int x aPoint Object header Object field / data int y int x aPointPoint e Point s aLine int y int x int y int x aPackedLine.e offset target aPackedPoint aPackedLine offset target
  24. 24. © 2013 IBM Corporation24 Packed Objects: In Practice with Native Access int y int x Object header Struct field / data Java Native struct Point { int x; int y; }
  25. 25. © 2013 IBM Corporation25 Packed Objects: In Practice with Native Access int y int x Object header Struct field / data offset target aPackedPoint Java Native struct Point { int x; int y; } @Packed final class PackedPoint extends PackedObject { int x; int y; }
  26. 26. © 2013 IBM Corporation26 Packed Objects: In Practice with Native Access int y int x Object header Struct field / data offset target aPackedPoint Java Native struct Point { int x; int y; } Ø @Packed final class PackedPoint extends PackedObject { int x; int y; }
  27. 27. © 2013 IBM Corporation27 Lets Build Something in C! ■  Nested substructures ■  Compact representation ■  Alignment aspects struct Address { char[4] addr; short port; } struct Header { struct Address src; struct Address dst; } struct Header port addr Address src Address dst port addr
  28. 28. © 2013 IBM Corporation28 Let’s Build the Same “Something” in Java! ■  Headers ■  No locality ■  Alignment class Address { byte[] addr; short port; } class Header { Address src; Address dst; } Address port addr addr Header Address dst Address src Address port addr addr byte[] byte[]
  29. 29. © 2013 IBM Corporation29 What does the Java code look like under the covers? ■  From a code point of view, this isn’t terrible… Bytecodes: aload1 getfield Header.dest LAddress; getfield Address.addr [B iconst0 baload bipush 192 ificmpeq ... JIT (32 bit): mov EBX, dword ptr -4[ECX] // load temp1 mov EBX, dword ptr 8[EBX] // load dest mov EBX, dword ptr 4[EBX] // load addr movsx EDI, byte ptr 8[EBX] // array[0] cmp EDI, 192
  30. 30. © 2013 IBM Corporation30 What if we did this with Packed Objects? ■  The Java code is pretty clean… and a pretty good result! @Packed final class Address extends PackedObject { PackedByte[[4]] addr; short port; } @Packed final class PacketHeader extends PackedObject { Address src; Address dest; } port addr Address src Address dst port addr offset target aPackedHeader
  31. 31. © 2013 IBM Corporation31 Ok, what about the code under the covers? ■  Bytecodes don’t change… JIT code is pretty good too! JIT (32 bit): mov EBX, dword ptr -4[ECX] // load temp1 mov EAX, dword ptr 4[EBX] // load target mov EDX, dword ptr 8[EBX] // load offset lea EBX, dword ptr [EAX + EDX] movsx EDI, byte ptr 8[EBX] // array[0] cmp EDI, 192 Bytecodes: aload1 getfield PackedHeader.dest LAddress; getfield Address.addr [B iconst0 baload bipush 192 ificmpeq ...
  32. 32. © 2013 IBM Corporation32 What about native access? Java Native … anObject How do we implement this normally? port addr port addr
  33. 33. © 2013 IBM Corporation33 JNI implementation ■  Usual “stash pointers in long types” tricks ■  JNI costs tend to be high
  34. 34. © 2013 IBM Corporation34 DirectByteBuffer implementation ■  No extra JNI to write (this is good) ■  Keeping your indices straight is never fun
  35. 35. © 2013 IBM Corporation35 Unsafe implementation ■  You shouldn’t be here ■  Still playing the indices game
  36. 36. © 2013 IBM Corporation36 PackedObject answer ■  Looks like natural Java code ■  Foregoes JNI ■  Same type capable of on-heap representation port addr port addr offset target aPackedHeader Ø
  37. 37. © 2013 IBM Corporation37 Active work and next steps ■  Experimenting with this now ■  Yes, there are security aspects to be aware of here ■  This is potentially part of a larger look at Java / Platform interop ■  Not specifically viewed as a cure to GC problems ■  This forms the basis for many other solutions to existing problems…
  38. 38. © 2013 IBM Corporation38 Multitenancy
  39. 39. © 2013 IBM Corporation39 Just what do you mean by “multitenancy”? With a multitenant architecture, a software application is designed to virtually partition its data and configuration, and each client organization works with a customized virtual application instance. ■  Working Definition – A single instance of a software application that serves multiple customers à Each customer is a tenant. – Tenants can customize some parts of the application (look and feel) but not the code. – Infrastructure usually opaque à opportunity for provider Why? Cost Savings: As compared to single-tenant deployment model
  40. 40. © 2013 IBM Corporation40 S5. Shared Application JDK Support for Spectrum of Sharing / Multitenancy (Level 1-5) S1. No Sharing Infrastructure Middleware Application Tenant OS Middleware Application Tenant OS Infrastructure Middleware Application Tenant OS Middleware Application Tenant OS Infrastructure Data Center floorData Center floor Middleware Application Tenant Middleware Application Tenant Infrastructure Data Center floor OS Application Tenant Application Tenant Infrastructure Data Center floor OS Middleware Application data Tenant Tenant Infrastructure Data Center floor OS Middleware Application S2. Shared Hardware S3. Shared Operating System S4. Shared Middleware Application data Application data Application data Application data Application data Application data Application data Application data Application data Sharing servers storage, networks in a data center Hypervisors (e.g. KVM, VMWare) are used to virtualize the hardware Multiple applications sharing the same middleware Sharing the same application Multiple copies of middleware in a single operating system Application Changes Application Changes ? Application Changes
  41. 41. © 2013 IBM Corporation41 = LApplication Changes
  42. 42. © 2013 IBM Corporation42 Hardware Virtualization ■  Hypervisors run multiple applications side-by-side safely ■  Advantages – Capture idle CPU cycles – Automatic de-duplication (RAM) – Ability to meter and shift resource toward demand – No need to change tenant applications Hypervisor Hardware tenant tenant tenant tenant
  43. 43. © 2013 IBM Corporation43 Hardware Virtualization ■  Hypervisors JVMs can run multiple applications side-by-side safely ■  Advantages – Capture idle CPU cycles – Automatic de-duplication (ability to share Java artifacts) – Ability to meter and shift resource toward demand – No need to change tenant applications Hypervsisor Hardware tenant tenant tenant tenant Java VM Operating System
  44. 44. © 2013 IBM Corporation44 Multitenancy: Low (or no) barrier to entry ■  Multitenancy is all about reducing duplication by transparently sharing a JVM – 1 GC, 1 JIT, shared heap objects – plus: JVM-enforced resource constraints to meter and limit consumption ■  Ergonomics: Opt-in to multitenancy with a single flag: -Xmt (multitenancy) – no application changes required javad Tenant1 Tenant2 One copy of common code + data lives in the javad process.
  45. 45. © 2013 IBM Corporation45 JVM: Separating State ■  Static variables are a problem for sharing ■  Consider use of System.out in code we want to share below
  46. 46. © 2013 IBM Corporation46 JVM: Separating State getstatic does 2 things 1.  Triggers class initialization on first contact –  Danger: Each ‘tenant’ needs to do this 2.  Resolves a name (out) to a storage location and reads from it –  Danger: Each tenant needs dedicated storage
  47. 47. © 2013 IBM Corporation47 JVM: Standard Static Fields getstatic <cpIndex> 0: if constantPool[cpIndex] is resolved then { 1: fetch field address from constantPool 2: read field value from class 3: return field value 4: } else { 5: initialize class 6: determine address of field 7: store field address into constantPool 8: goto 1 9: }
  48. 48. © 2013 IBM Corporation48 JVM: Isolated Static Fields getstatic <cpIndex> 0: if constantPool[cpIndex] is resolved then { 1: fetch field address index and class from constantPool 2: fetch tenant->data[class->initIndex] 3: if (!initialized) { 4: initialize class; 5: } 6: read field value from tenant->data[index] 7: return field value 8: } else { 9: initialize class 10: determine address index of field 11: store field address index into constantPool 12: goto 1 13: }
  49. 49. © 2013 IBM Corporation49 JVM: Separating State ■  There are other initialization triggers too – putstatic – invokestatic – instanceof – new ■  Some of these warrant special handling as they are heavily used (invokestatic) and initialization checks are expensive. – Separate stack frame build from opcode
  50. 50. © 2013 IBM Corporation50 More that just JVM State… ■  Throttling of resources – Threads, GC, sockets, files (IO in general), native memory ■  Past and existing examples do exist! – Commercial / In house custom solutions – JSR 181 Isolates / 284 Resource Management ■  Security is of course huge
  51. 51. © 2013 IBM Corporation51 Other Thoughts – Native Libraries and shared state ■  Use separate processes to manage different state ■  Each process now holds the context ■  Challenges: Latency Tenant1 Tenant2 JVM Proxy Library Shared Library Shared Library Tenant1 Tenant2 JVM Shared Library Shared State! ■  Native libraries contain state that may not be shareable across tenants
  52. 52. © 2013 IBM Corporation52 Heap Memory (not Garbage Collection!)
  53. 53. © 2013 IBM Corporation53 Scaling Garbage Collection ■  Existing Garbage Collection (GC) technology defers expensive costs Heap Local Collection Global Collection Heap ■  Global approach introduces linear scaling problem (100ms @ 1GB → 10s @ 100GB) ■  Goal: Scale using result-based, incremental (through partitioned heap) garbage collection ■  Target appropriate data sets based on environment factors ■  Key: Region Based Garbage Collection approach
  54. 54. © 2013 IBM Corporation54 Challenge: Segregate Memory by Differentiators Heap Varying memory characteristics and requirements per application JVM must dynamically recognize and adapt given platform resources  IaaS optimization + sharing (consolidate)  SSD Exploitation (memory efficiency)  Multi-core scalability (scale up)  Very Large Heap GC (low pause times)  Soft Real-Time ■  Objects with common properties •  Locality •  Usage frequency •  Lifetime •  Levels of “Read-only”ness ■  Optimize: Memory characteristics •  Page sharing status •  Memory speed aware •  Flash / SSD exploitation ■  “Results based” incremental operations •  Productive GC every cycle •  Localized garbage collect Custom  Placement improves cache performance  Allocation efficiency (thoughtput)  Reduced size working set (paging)
  55. 55. © 2013 IBM Corporation55 Memory Hierarchy and Performance ■  Increased core and HW thread counts are not silver bullets to scaling in a memory intensive environment such as Java ■  Pressure from increased cores / HW threads ■  Cache hierarchy does not scale Cache Hierarchy BUS Vs. Logical Processors Memory MemoryMemory Memory CPU CPU CPUCPU ■  Memory speed varies based on execution context ■  Magnitude of variance related to CPU socket count ■  Uneven application performance over time •  Data access can be fast or slow (or both) •  Behavior can be variable run to run due to data placement •  Threads scheduling can be unpredictable •  Same deployments on different hardware cause unpredictable performance shifts
  56. 56. © 2013 IBM Corporation56 Let’s look at transferring data Heap A B C
  57. 57. © 2013 IBM Corporation57 Let’s look at transferring data Heap A B C Remote Transfer
  58. 58. © 2013 IBM Corporation58 Let’s look at transferring data Heap A B C Remote Transfer
  59. 59. © 2013 IBM Corporation59 Let’s look at transferring data Heap A B C Remote Transfer
  60. 60. © 2013 IBM Corporation60 PackedObjects could help… Heap A Remote Transfer C B
  61. 61. © 2013 IBM Corporation61 PackedObjects could help… Heap A B C Remote Transfer A Packed C B
  62. 62. © 2013 IBM Corporation62 PackedObjects could help… Heap A B C Remote Transfer A Packed B C A Packed C B
  63. 63. © 2013 IBM Corporation63 Making the data transfer easier… Heap Remote Transfer
  64. 64. © 2013 IBM Corporation64 Making the data transfer easier… Heap Remote Transfer Specialized Heap Area
  65. 65. © 2013 IBM Corporation65 Making the data transfer easier… Heap B Remote Transfer A B C A C Specialized Heap Area
  66. 66. © 2013 IBM Corporation66 Making the data transfer easier… Heap B Remote Transfer A B C A C Specialized Heap Area
  67. 67. © 2013 IBM Corporation67 Making the data transfer seamless Heap Remote Transfer Specialized Heap Area
  68. 68. © 2013 IBM Corporation68 B C A B C A Making the data transfer seamless Heap Remote Transfer B C A Specialized Heap Area
  69. 69. © 2013 IBM Corporation69 B C A B C A Making the data transfer seamless Heap Remote Transfer B C A Packed Specialized Heap Area offset target Packed offset target
  70. 70. © 2013 IBM Corporation70 Tip of the Iceberg ■  Just scratching the surface of what is possible ■  GC technology has an effect on what is possible – But this should be invisible to the application and seamless in the language ■  Innovation in the heap space is not limited to GC
  71. 71. © 2013 IBM Corporation71 Virtualization
  72. 72. © 2013 IBM Corporation72 Finding your way in The Cloud ■  Constant pressure to reduce cost and deliver more services for a given cost ■  Maximize the available resources (Hardware) ■  CPU and Memory are typically the biggest – Other critical ones as well, including I/O ■  Plenty of solutions – Cloud and Multitenancy just to name a few ■  And so by and large we’re in a Virtualization scenario
  73. 73. © 2013 IBM Corporation73 The Layers of Liars ■  Your application could end up querying 3 different layers to get a (hopefully) correct picture Hypervisor Hardware OS API! availableProcessors() [no accuracy / understanding] Thinks it knows but doesn’t (being lied to) Knows and controls, but can change its mind App
  74. 74. © 2013 IBM Corporation74 The Layers of Liars Hypervisor Hardware App OS App Image source (Wikipedia Commons) CPU: https://en.wikipedia.org/wiki/Central_processing_unit DRAM: http://en.wikipedia.org/wiki/Dynamic_random-access_memory RAM CPU
  75. 75. © 2013 IBM Corporation75 The Layers of Liars Hypervisor Hardware App OS App Image source (Wikipedia Commons) CPU: https://en.wikipedia.org/wiki/Central_processing_unit DRAM: http://en.wikipedia.org/wiki/Dynamic_random-access_memory RAM CPU
  76. 76. © 2013 IBM Corporation76 The Layers of Liars Hypervisor Hardware App OS App App OS Image source (Wikipedia Commons) CPU: https://en.wikipedia.org/wiki/Central_processing_unit DRAM: http://en.wikipedia.org/wiki/Dynamic_random-access_memory RAM CPU
  77. 77. © 2013 IBM Corporation77 The Layers of Liars ■  You don’t even have a constant picture of the universe Hypervisor Hardware App OS App App OS Image source (Wikipedia Commons) CPU: https://en.wikipedia.org/wiki/Central_processing_unit DRAM: http://en.wikipedia.org/wiki/Dynamic_random-access_memory RAM CPU
  78. 78. © 2013 IBM Corporation78 Knowing what you don’t know (What does this even mean anymore?)
  79. 79. © 2013 IBM Corporation79 Interfaces… more importantly behavior ■  Provide interfaces to the “Layers of liars” to enable middleware “smarts” ■  Runtime reads and reacts (in co-operation with application / other runtimes) Hypervisor Hardware App OS Hypervisor Hardware App OS App App OS Adjust –Xmx Adjust –Xgcthreads Adjust thread pool counts …
  80. 80. © 2013 IBM Corporation80 …And after all that
  81. 81. © 2013 IBM Corporation81 Tenant Nimble Reactions to External Events Hypervisor Hardware OS Application Tenant
  82. 82. © 2013 IBM Corporation82 Tenant Nimble Reactions to External Events Hypervisor Hardware OS Application Tenant
  83. 83. © 2013 IBM Corporation83 TenantTenant Nimble Reactions to External Events Hypervisor Hardware OS ApplicationApp OS Tenant
  84. 84. © 2013 IBM Corporation84 TenantTenant Tenant Tenant Nimble Reactions to External Events Hypervisor Hardware OS ApplicationApp OS Tenant Hypervisor Hardware OS Application
  85. 85. © 2013 IBM Corporation85 TenantTenant Tenant Tenant Nimble Reactions to External Events Hypervisor Hardware OS ApplicationApp OS Tenant Hypervisor Hardware OS Application Tenant RDMA + Packed Objects
  86. 86. © 2013 IBM Corporation86 Questions?
  87. 87. © 2013 IBM Corporation87 References ■  Get Products and Technologies: – IBM Java Runtimes and SDKs: •  https://www.ibm.com/developerworks/java/jdk/ – IBM Monitoring and Diagnostic Tools for Java: •  https://www.ibm.com/developerworks/java/jdk/tools/ ■  Learn: – IBM Java InfoCenter: •  http://publib.boulder.ibm.com/infocenter/java7sdk/v7r0/index.jsp ■  Discuss: – IBM Java Runtimes and SDKs Forum: •  http://www.ibm.com/developerworks/forums/forum.jspa?forumID=367&start=0
  88. 88. © 2013 IBM Corporation88 Copyright and Trademarks © IBM Corporation 2012. All Rights Reserved. IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., and registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web – see the IBM “Copyright and trademark information” page at URL: www.ibm.com/legal/copytrade.shtml

×