JavaOne 2013: Introduction to PackedObjects

1,837 views

Published on

Introduction to PackedObjects
JavaOne 2013 CON5758
Ryan A. Sciampacone, Senior Software Developer, IBM JTC
Abstract: In Java the layout of objects is abstracted away from the application, leaving Java inherently challenged by concerns such as (1) interoperation with native data structures, (2) the dense packing of Java objects, and (3) cache conflicts and false sharing. With PackedObjects, there is a proposal for a new explicit object model that enables direct binding of native data structures, dense packing of Java objects to improve the performance of operations such as serialization, and precise object layout to allow for finer-grained control over locality and the development of a high-performance concurrent library. Learn more in this session.

Published in: Technology, Business
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,837
On SlideShare
0
From Embeds
0
Number of Embeds
76
Actions
Shares
0
Downloads
40
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

JavaOne 2013: Introduction to PackedObjects

  1. 1. © 2013 IBM Corporation Ryan A. Sciampacone – Managed Runtime Architect 25 September 2013 An Introduction to Packed Objects
  2. 2. © 2013 IBM Corporation Important Disclaimers THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES. ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE. IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE. IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF: - CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS 2
  3. 3. © 2013 IBM Corporation Your Speaker: Ryan A. Sciampacone • Runtime Architect @ IBM (JTC) • Interpreters • Garbage Collection • Platform interfacing / optimization • Hardware exploitation (E) ryan_sciampacone@ca.ibm.com (T) @rsciampacone • Visit the IBM Booth #5112 to meet other IBM Developers 3
  4. 4. © 2013 IBM Corporation PROBLEM SPACE What are we trying to solve here? 4
  5. 5. © 2013 IBM Corporation Problem? What problem? • JNI just isn’t a great way to marshal data • Locality in Java can matter (e.g., JEP 142) • Existing native and data placement stories aren’t very good • In many cases, legacy systems exist – the interop is just terrible • So we want something that integrates well with the Java language and helps us… 5
  6. 6. © 2013 IBM Corporation Native Access Fighting the Java/Native interface 6 int int Java Native … int[] d anObject int int int … Object header Object field / data
  7. 7. © 2013 IBM Corporation Everything is an Object 7 Hash Array Entry (object) (object) table key value Object header Object field / data
  8. 8. © 2013 IBM Corporation Everything is an Object 8 Hash Array Entry (object) (object) table key value Object header Object field / data
  9. 9. © 2013 IBM Corporation Everything is an Object 9 Hash Array Entry (object) (object) table key value Object header Object field / data
  10. 10. © 2013 IBM Corporation Everything is an Object 10 Hash Array Entry (object) (object) table key value Object header Object field / data
  11. 11. © 2013 IBM Corporation Object Internals • Field ordering has performance implications • JVM can potentially reorder your fields for you 11 ….“Hot” fields Object header Object field / data
  12. 12. © 2013 IBM Corporation Establishing Goals • On heap / off heap seamless referencing of data • Ability to do away with headers • Ability to bring related objects close together 12 struct Address { char[4] addr; short port; } struct Header { struct Address src; struct Address dst; } struct Header port addr Address src Address dstport addr Packed Objects!
  13. 13. © 2013 IBM Corporation THE BASICS From the ground up, let’s start with… 13
  14. 14. © 2013 IBM Corporation Packed Objects: Under the covers 14 int y int x aPoint Object header Object field / data
  15. 15. © 2013 IBM Corporation Packed Objects: Under the covers 15 int y int x aPoint offset target aPackedPoint int y int x Object header Object field / data
  16. 16. © 2013 IBM Corporation Packed Objects: Under the covers 16 int y int x aPoint offset target aPackedPoint int y int x Object header Object field / data
  17. 17. © 2013 IBM Corporation Packed Objects: In Practice 17 int y int x aPoint int y int x aPointPoint e Point s aLine Object header Object field / data
  18. 18. © 2013 IBM Corporation Packed Objects: In Practice 18 int y int x aPoint int y int x aPointPoint e Point s aLine int y int x int y int x aPackedLine offset target Object header Object field / data
  19. 19. © 2013 IBM Corporation Packed Objects: In Practice 19 int y int x aPoint int y int x aPointPoint e Point s aLine int y int x int y int x aPackedLine offset target Object header Object field / data aPackedPoint s aPackedPoint e
  20. 20. © 2013 IBM Corporation Packed Objects: In Practice 20 int y int x aPoint int y int x aPointPoint e Point s aLine int y int x int y int x aPackedLine offset target Object header Object field / data @Packed final class PackedPoint extends PackedObject { int x; int y; } @Packed final class PackedLine extends PackedObject { PackedPoint s; PackedPoint e; } aPackedPoint s aPackedPoint e
  21. 21. © 2013 IBM Corporation Packed Objects: In Practice 21 int y int x aPoint int y int x aPointPoint e Point s aLine int y int x int y int x aPackedLine offset target Object header Object field / data
  22. 22. © 2013 IBM Corporation Packed Objects: In Practice 22 int y int x aPoint int y int x aPointPoint e Point s aLine int y int x int y int x aPackedLine offset target Object header Object field / data aPackedLine.e
  23. 23. © 2013 IBM Corporation Packed Objects: In Practice 23 int y int x aPoint int y int x aPointPoint e Point s aLine int y int x int y int x offset target aPackedPoint aPackedLine offset target aPackedLine.e Object header Object field / data
  24. 24. © 2013 IBM Corporation Packed Objects: In Practice with Arrays 24 int y int x int y int x offset target aPackedPoint anArrayOfPackedPoints offset target int y int x int y int x …. Object header Object field / data
  25. 25. © 2013 IBM Corporation Packed Objects: In Practice with Native Access 25 int y int x Java Native struct Point { int x; int y; } Object header Object field / data
  26. 26. © 2013 IBM Corporation Packed Objects: In Practice with Native Access 26 int y int x offset target aPackedPoint Java Native @Packed final class PackedPoint extends PackedObject { int x; int y; } struct Point { int x; int y; } Object header Object field / data
  27. 27. © 2013 IBM Corporation Packed Objects: In Practice with Native Access 27 int y int x offset target aPackedPoint Java Native Ø @Packed final class PackedPoint extends PackedObject { int x; int y; } struct Point { int x; int y; } Object header Object field / data
  28. 28. © 2013 IBM Corporation ADVANTAGES So what do we get? 28
  29. 29. © 2013 IBM Corporation Lets Build Something in C! • Nested substructures • Compact • Alignment 29 struct Address { char[4] addr; short port; } struct Header { struct Address src; struct Address dst; } struct Header port addr Address src Address dstport addr
  30. 30. © 2013 IBM Corporation Let’s Build the Same “Something” in Java! • Headers • Locality • Alignment 30 class Address { byte[] addr; short port; } class Header { Address src; Address dst; } Address port addr addr Header Address dst Address src Address port addr addr byte[] byte[]
  31. 31. © 2013 IBM Corporation Java code: What’s it look like under the covers? • From a code point of view, this isn’t terrible… 31 Bytecodes: aload1 getfield Header.dest LAddress; getfield Address.addr [B iconst0 baload bipush 192 ificmpeq ... JIT (32 bit): mov EBX, dword ptr -4[ECX] // load temp1 mov EBX, dword ptr 8[EBX] // load dest mov EBX, dword ptr 4[EBX] // load addr movsx EDI, byte ptr 8[EBX] // array[0] cmp EDI, 192
  32. 32. © 2013 IBM Corporation What if we did this with Packed Objects? 32 @Packed final class Address extends PackedObject { PackedByte[[4]] addr; short port; } @Packed final class PacketHeader extends PackedObject { Address src; Address dest; } port addr Address src Address dst port addr offset target aPackedHeader • The Java code is pretty clean… and a pretty good result!
  33. 33. © 2013 IBM Corporation Ok, what about the code under the covers? • Bytecodes don’t change… JIT code is pretty good too! 33 Bytecodes: aload1 getfield Header.dest LAddress; getfield Address.addr [B iconst0 baload bipush 192 ificmpeq ... JIT (32 bit): mov EBX, dword ptr -4[ECX] // load temp1 mov EAX, dword ptr 4[EBX] // load target mov EDX, dword ptr 8[EBX] // load offset lea EBX, dword ptr [EAX + EDX] movsx EDI, byte ptr 0[EBX] // array[0] cmp EDI, 192
  34. 34. © 2013 IBM Corporation What about native access? How do we implement this normally? 34 Java Native … anObject port addr port addr
  35. 35. © 2013 IBM Corporation JNI implementation • Usual “stash pointers in long types” tricks • JNI costs tend to be high 35
  36. 36. © 2013 IBM Corporation JNI implementation • Usual “stash pointers in long types” tricks • JNI costs tend to be high 36
  37. 37. © 2013 IBM Corporation Unsafe implementation • You shouldn’t be here • Keeping your indices straight is never fun 37
  38. 38. © 2013 IBM Corporation DirectByteBuffer implementation • No extra JNI to write (this is good) • Still playing the indices game 38
  39. 39. © 2013 IBM Corporation PackedObject answer • Looks like natural Java code • Foregoes JNI • Same type capable of on-heap representation 39 port addr port addr offset target aPackedHeader Ø
  40. 40. © 2013 IBM Corporation POSSIBILITIES With a little bit of imagination… 40
  41. 41. © 2013 IBM Corporation Let’s look at transferring data 41 Heap A B C
  42. 42. © 2013 IBM Corporation Let’s look at transferring data 42 Remote Transfer Heap A B C
  43. 43. © 2013 IBM Corporation Let’s look at transferring data 43 Heap A B C Remote Transfer
  44. 44. © 2013 IBM Corporation Let’s look at transferring data 44 Heap A B C Remote Transfer
  45. 45. © 2013 IBM Corporation PackedObjects could help… 45 Heap A B C Remote Transfer
  46. 46. © 2013 IBM Corporation PackedObjects could help… 46 B C A Packed Heap A B C Remote Transfer
  47. 47. © 2013 IBM Corporation Heap A B C PackedObjects could help… 47 Remote Transfer B C A Packed B C A Packed
  48. 48. © 2013 IBM Corporation Making the data transfer easier… 48 Remote Transfer Heap
  49. 49. © 2013 IBM Corporation Making the data transfer easier… 49 Specialized Heap Area Remote Transfer Heap
  50. 50. © 2013 IBM Corporation Making the data transfer easier… 50 B A B C A C Remote Transfer Heap Specialized Heap Area
  51. 51. © 2013 IBM Corporation Making the data transfer easier… 51 Remote Transfer Heap Specialized Heap Area B A B C A C
  52. 52. © 2013 IBM Corporation Making the data transfer seamless 52 Remote Transfer Heap Specialized Heap Area
  53. 53. © 2013 IBM Corporation53 B C A B C A B C A Remote Transfer Heap Specialized Heap Area Making the data transfer seamless
  54. 54. © 2013 IBM Corporation54 Packed offset target Packed offset target Remote Transfer Heap B C A B C A B C A Specialized Heap Area Making the data transfer seamless
  55. 55. © 2013 IBM Corporation IN PRACTICE Case study 55
  56. 56. © 2013 IBM Corporation Scenario and Measurement • PackedObjects and jVerbs • Benefits of combining two technologies • Scenario • Remote Procedure Call • Low latency RPC • Zero-copy / zero-serialization • PackedCache • PackedObjects in a remote cache • Work done by IBM Zurich Research Lab (Stuedi, Metzler, Trivedi) 56
  57. 57. © 2013 IBM Corporation Motivation: PackedObject transfer using jVerbs • PackedObject used in off heap scenario • Transfer native data directly using jVerbs • Zero-copy / Zero-marshalling or serialization (both ends) 57
  58. 58. © 2013 IBM Corporation Background on traditional RPC 58
  59. 59. © 2013 IBM Corporation RPC using PackedObjects and jVerbs 59
  60. 60. © 2013 IBM Corporation RPC Performance: Single Object • Simple PackedObject containing int, float and long • Signature: PackedObject foo(PackedObject obj) 60 RPCLatency
  61. 61. © 2013 IBM Corporation RPC Performance: Packed Array • Simple PackedObject containing int, float and long • Signature PackedObject[] foo(PackedObject[] array) 61 RPCLatency Packed Array size
  62. 62. © 2013 IBM Corporation On Demand Object Access • Avoid copying / serialization / CPU utilization • RDMA direct data placement 62
  63. 63. © 2013 IBM Corporation PackedObject Cache Performance • Robust with regard to the data size 63 RPCLatency Packed Array size
  64. 64. © 2013 IBM Corporation CHALLENGES How is this different in the context of Java? 64
  65. 65. © 2013 IBM Corporation Identity Crisis What does aPackedLine == anotherPackedLine mean?  The data is what really matters int y int x offset target aPackedLine Java Native Ø int y int x offset target anotherPackedLine Ø
  66. 66. © 2013 IBM Corporation Synchronization int y int x offset target aPackedLine Java Native Ø int y int x offset target anotherPackedLine Ø
  67. 67. © 2013 IBM Corporation Synchronization int y int x offset target aPackedLine Java Native Ø int y int x offset target anotherPackedLine Ø Thread T1: synchronized (aPackedLine) { aPackedLine.e.x = 10; } Thread T2: synchronized (anotherPackedLine) { anotherPackedLine.e.x = 13; } aPackedLine.e.x = 10 anotherPackedLine.e.x = 13
  68. 68. © 2013 IBM Corporation Finalization 68 int y int x int y int x offset target aPackedPoint aPackedLine offset target
  69. 69. © 2013 IBM Corporation Finalization 69 int y int x int y int x offset target aPackedPoint aPackedLine offset target
  70. 70. © 2013 IBM Corporation Finalization 70 int y int x int y int x offset target aPackedPoint aPackedLine offset target Finalization ?
  71. 71. © 2013 IBM Corporation Finalization 71 int y int x int y int x offset target aPackedPoint aPackedLine offset target Finalization ? Finalization ?
  72. 72. © 2013 IBM Corporation Finalization int y int x offset target aPackedLine Java Native Ø int y int x offset target anotherPackedLine Ø Finalization ?
  73. 73. © 2013 IBM Corporation Nested Data Structures 73 aPackedLine.s = aPackedPoint; int y int x offset target aPackedLine int y int x int y int x offset target aPackedPoint PackedPoint s PackedPoint e
  74. 74. © 2013 IBM Corporation Nested Data Structures • Base types do share the same assignment operator • Helps convey aPackedLine == anotherPackedLine as meaningless 74 aPackedLine.s := aPackedPoint; int y int x offset target aPackedLine int y int x int y int x offset target aPackedPoint PackedPoint s PackedPoint e
  75. 75. © 2013 IBM Corporation Field Initialization No no-argument constructor Implicitly instantiates PackedPoint objects for s & e fields @Packed final class PackedPoint extends PackedObject { int x; int y; PackedPoint(int x, int y) { … } } @Packed final class PackedLine extends PackedObject { PackedPoint s; PackedPoint e; PackedLine(int sx, int sy, int ex, int ey) { … } }
  76. 76. © 2013 IBM Corporation Field Initialization @Packed final class PackedPoint extends PackedObject { int x; int y; void init(int x, int y) { this.x = x; this.y = y; } PackedPoint(int x, int y) { … } } @Packed final class PackedLine extends PackedObject { PackedPoint s; PackedPoint e; PackedLine(int sx, int sy, int ex, int ey) { s.init(sx, sy); e.init(ex, ey); } }
  77. 77. © 2013 IBM Corporation ADVANCED Taking it to the next level… 77
  78. 78. © 2013 IBM Corporation Modeling Native Data Pointers 78 *right *left aBinaryTreeNode *right *left *right *left … … … … <data> <data> <data> Java offset target Ø Native struct BinaryTreeNode { struct BinaryTreeNode* left; struct BinaryTreeNode* right; // data }
  79. 79. © 2013 IBM Corporation Modeling Native Data Pointers 79 *right *left aBinaryTreeNode *right *left *right *left … … … … <data> <data> <data> Java offset target Ø Native aBinaryTreeNode.right ? struct BinaryTreeNode { struct BinaryTreeNode* left; struct BinaryTreeNode* right; // data }
  80. 80. © 2013 IBM Corporation Modeling Native Data Pointers 80 *right *left aBinaryTreeNode *right *left *right *left … … … … <data> <data> <data> Java offset target Ø Native aBinaryTreeNode.right 32b or 64b? ? struct BinaryTreeNode { struct BinaryTreeNode* left; struct BinaryTreeNode* right; // data }
  81. 81. © 2013 IBM Corporation Modeling Native Data Pointers • Annotation to mark a field as a native pointer (rather than a Java one) • Enhance getfield / putfield to recognize • Restrict for security reasons • Unmanaged pointers (no GC involvement) 81 @Packed class BinaryTreeNode extends PackedObject { @NativePointer BinaryTreeNode left; @NativePointer BinaryTreeNode right; // data }
  82. 82. © 2013 IBM Corporation Modeling Native Data Pointers 82 Java Native offset target Ø aBinaryTreeNode.right aBinaryTreeNode *right *left *right *left … <data> <data> … …
  83. 83. © 2013 IBM Corporation Modeling Native Data Pointers 83 Java Native offset target offset target Ø Ø aBinaryTreeNode.right aBinaryTreeNode *right *left *right *left … <data> <data> … …
  84. 84. © 2013 IBM Corporation Alignment 84 @Packed final class Address extends PackedObject { PackedByte[[4]] addr; short port; } @Packed final class PacketHeader extends PackedObject { Address src; Address dest; } port addr port addr offset target aPackedHeader
  85. 85. © 2013 IBM Corporation Alignment • Which is the correct default behavior? • How do you get the alternate if that’s what you want? 85 port addr port addr addr @Packed final class Address extends PackedObject { PackedByte[[4]] addr; short port; } @Packed final class PacketHeader extends PackedObject { Address src; Address dest; } port addr port addr offset target aPackedHeader offset target aPackedHeader
  86. 86. © 2013 IBM Corporation Alignment 86 class A { int i; short s; short padding; // align long l; } class A { int i; short s; @Align long l; }
  87. 87. © 2013 IBM Corporation Alignment • Padding isn’t quite right in the context of nested structures… 87 class A { int i; short s; short padding; // align long l; } class A { int i; short s; @Align long l; } @Packed final class Address extends PackedObject { PackedByte[[4]] addr; short port; } @Packed final class PacketHeader extends PackedObject { @Align Address src; @Align Address dest; }
  88. 88. © 2013 IBM Corporation Endian • Provide a field annotation @BigEndian (and @LittleEndian) 88 @Packed final class SimpleValue extends PackedObject { int value; } Java Native offset target Ø aSimpleValue DE 37 C4 FE
  89. 89. © 2013 IBM Corporation PARTING REMARKS And finally… 89
  90. 90. © 2013 IBM Corporation What next? • Not an IBM Special! • Working with Oracle on a JSR (Java9+) • Existing Betas now available • Visit ibm.com/java for downloads • Full disclosure: Helper methods / classes in place of proposed syntax • Your feedback is not only welcome, it’s required! 90
  91. 91. © 2013 IBM Corporation91
  92. 92. © 2013 IBM Corporation Copyright and Trademarks © IBM Corporation 2013. All Rights Reserved. IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., and registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web – see the IBM “Copyright and trademark information” page at URL: www.ibm.com/legal/copytrade.shtml 92

×