Your SlideShare is downloading. ×
Fuel
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Fuel

986
views

Published on

ESUG 2011, Edinburgh

ESUG 2011, Edinburgh

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
986
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
14
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Mariano Martinez Peck marianopeck@gmail.com http://marianopeck.wordpress.com/ 1Monday, August 22, 2011
  • 2. THANKS a lot!!! SummerTalk 2011 Student: Martin Dias Mentor: Mariano Martínez Peck 2Monday, August 22, 2011
  • 3. Object references 3Monday, August 22, 2011
  • 4. Object graph 4Monday, August 22, 2011
  • 5. An objet graph serializer. 5Monday, August 22, 2011
  • 6. Serialize Input: an object graph Output: stream of bytes 6Monday, August 22, 2011
  • 7. Materialize (deserialize) Input: stream of bytes Output: an object graph 7Monday, August 22, 2011
  • 8. Once serialized... Stream of bytes Database File Memory Socket 8Monday, August 22, 2011
  • 9. Fuel’s main goals Provide fast object serialization and materialization. Be flexible and easy to customize. Have a good OO design, well tested and benchmarked. No need of special support from the VM. Be a general purpose serializer. Allow tools to be built on top of Fuel. 9Monday, August 22, 2011
  • 10. Key features Fast serialization and materialization. Class reshape support. Serialization of any kind of object. Cycles support. Global objects references. Buffered writing. Support for some “hook methods”. 10Monday, August 22, 2011
  • 11. Key Characteristics Pickle format. Objects grouped in clusters. Analysis phase before writing. Stack over recursion. Two phases for writing instances and references. Iterative graph recreation. 11Monday, August 22, 2011
  • 12. Pickle format Invest more time in serialization so that objects can then be materialized much faster. 12Monday, August 22, 2011
  • 13. Grouping objects in clusters “Similar” objects (they share writing/loading information) aregrouped together in clusters. The most common case, yet not the only one, takes place when a class is a cluster for its instances. 13Monday, August 22, 2011
  • 14. 14Monday, August 22, 2011
  • 15. Each jar has a specific type of element 14Monday, August 22, 2011
  • 16. Each jar has a specific type of element Jars are in order 14Monday, August 22, 2011
  • 17. Label: - What’s inside? Each jar has - How much? a specific type of element Jars are in order 14Monday, August 22, 2011
  • 18. Label: - What’s inside? Each jar has - How much? a specific type of element Different sizes and different Jars are in amounts of order elements 14Monday, August 22, 2011
  • 19. s/jar/cluster Label: - What’s inside? Each jar has - How much? a specific type of element Different sizes and different Jars are in amounts of order elements 14Monday, August 22, 2011
  • 20. s/jar/cluster Label: - What’s inside?Each jar has - How much? a specific type of element Different sizes and different Jars are in amounts of order elements 14Monday, August 22, 2011
  • 21. s/jar/cluster Label: - What’s inside? - How much? Each cluster has a specific type of object Different sizes and different Jars are in amounts of order elements 14Monday, August 22, 2011
  • 22. s/jar/cluster Label: - What’s inside? - How much? Each cluster has a specific type of object Different sizes and differentJars are in amounts of order elements 14Monday, August 22, 2011
  • 23. s/jar/cluster Label: - What’s inside? - How much? Each cluster has a specific type of object Different sizes and different Clusters are amounts of in order elements 14Monday, August 22, 2011
  • 24. s/jar/cluster Label: - What’s inside? Each cluster - How much? has a specific type of object Different sizes and different Clusters are amounts of in order elements 14Monday, August 22, 2011
  • 25. s/jar/cluster Label: - Cluster ID Each cluster - Amount of has a specific objects type of object Different sizes and different Clusters are amounts of in order elements 14Monday, August 22, 2011
  • 26. s/jar/cluster Label: - Cluster ID Each cluster - Amount of has a specific objects type of object Different sizes and different Clusters are amounts of in order elements 14Monday, August 22, 2011
  • 27. s/jar/cluster Label: - Cluster ID Each cluster - Amount of has a specific objects type of object Different sizes and different Clusters are amounts of in order objects 14Monday, August 22, 2011
  • 28. Pickle format basic Stream 15Monday, August 22, 2011
  • 29. Pickle format basic Stream 15Monday, August 22, 2011
  • 30. Pickle format basic Stream 15Monday, August 22, 2011
  • 31. Pickle format basic Stream 15Monday, August 22, 2011
  • 32. Why the pickle format is so fast in materialization? Standard serializers Fuel pickle format 16Monday, August 22, 2011
  • 33. Why the pickle format is so fast in materialization? Standard serializers Fuel pickle format ive ive n rat tio urs tion ec za Ite iza R ial eri ali at er m at m 16Monday, August 22, 2011
  • 34. Pickle advantages Batch/Bulk/Iterative materialization. Efficient since types are stored and fetch only once. Fast because at materialization we know the size of everything. The generated stream is smaller. More next.... 17Monday, August 22, 2011
  • 35. There is no silver bullet... ation rializ e) Fa st se ickl (with out p Slow seri (wit aliza h pi tion ckle ) 18Monday, August 22, 2011
  • 36. Fuel requires Traversing the object graph. Mapping each object to a specific cluster. This is done in a phase before serialization called “Analysis”. 19Monday, August 22, 2011
  • 37. Analysis phase 1) Traverse (#fuelAccept: aVisitor) 2) Fill dictionary (#fuelSerializer) cluster1 aPoint ... cluster2 x y ... ... ... key (a cluster) value (a set) 20Monday, August 22, 2011
  • 38. Serialization cluster1 aPoint ... cluster2 x y ... ... ... key (a cluster) value (a set) Cluster ID serialize: anObject on: aStream materializeFrom: aStream A cluster defines how its objects are serialized and materialized. 21Monday, August 22, 2011
  • 39. Stack over recursion To traverse the object graph, Fuel uses a custom stack implementation rather than a recursion. 22Monday, August 22, 2011
  • 40. Basic steps Serialization 1. Analyze. 2. Serialize header. 3. Serialize instances. 4. Serialize references. 5. Serialize root. Materialization 1. Materialize header. 2. Materialize instances. 3. Materialize references. 4. Materialize root. 23Monday, August 22, 2011
  • 41. Fuel for far) (so software Moose export utility. SandstoneDB persistence. Pier kernel persistence. Newspeak language. Marea (my own research project!). 24Monday, August 22, 2011
  • 42. Future Work Continue efforts on performance optimization. Create a tool for loading class and trait packages. Support user-defined Singletons. Fast statistics/brief info extraction of a stored graph. Partial loading of a stored graph. 25Monday, August 22, 2011
  • 43. Future work 2 Enable to deploy serialization and materialization behavior independently. Support object replacement for serialization and materialization. Allow cycle detections to be disabled. Partial loading. 26Monday, August 22, 2011
  • 44. Serialization of primitive objects Memory based stream 27Monday, August 22, 2011
  • 45. Serialization of primitive objects Memory based stream StOMP SRP Fuel 27Monday, August 22, 2011
  • 46. Serialization of primitive objects Memory based stream File based stream StOMP SRP Fuel 27Monday, August 22, 2011
  • 47. Serialization of primitive objects Memory based stream File based stream StOMP Fuel SRP SRP Fuel 27 StOMPMonday, August 22, 2011
  • 48. Materialization of primitive objects 28Monday, August 22, 2011
  • 49. Materialization of primitive objects Fuel StOMP ImageSegment 28Monday, August 22, 2011
  • 50. Non primitive objects 29Monday, August 22, 2011
  • 51. Non primitive objects StOMP SRP Fuel 29Monday, August 22, 2011
  • 52. Non primitive objects StOMP SRP Fuel 29Monday, August 22, 2011
  • 53. Non primitive objects StOMP Fuel SRP ImageSegment Fuel 29 StOMPMonday, August 22, 2011
  • 54. Links Website: http://rmod.lille.inria.fr/web/pier/ software/Fuel Issue tracker: http://code.google.com/p/fuel Source repository: http://www.squeaksource.com/ Fuel Continuous integration server: https://pharo- ic.lille.inria.fr/hudson/job/Fuel/ 30Monday, August 22, 2011
  • 55. Conclusion for us Excellent performance without special support from VM and good OO design. 31Monday, August 22, 2011
  • 56. Conclusion for YOU Fuel is a vehicle. It is infrastructure. You can build cool stuff on top of it. 32Monday, August 22, 2011
  • 57. Thanks! Mariano Martinez Peck marianopeck@gmail.com http://marianopeck.wordpress.com/Monday, August 22, 2011
  • 58. Concrete example 34Monday, August 22, 2011
  • 59. 35Monday, August 22, 2011
  • 60. Analysis phase Analysis cluster1 aRectangle phase cluster2 anOrigin aCorner cluster3 10 20 30 40 key (a cluster) value (a IdentitySet) 20 36Monday, August 22, 2011
  • 61. cluster1 aRectangle Serialization cluster2 anOrigin aCorner instances cluster3 10 20 30 40 phaseMonday, August 22, 2011
  • 62. cluster1 aRectangle Serialization cluster2 anOrigin aCorner instances cluster3 10 20 30 40 phaseMonday, August 22, 2011
  • 63. cluster1 aRectangle Serialization cluster2 anOrigin aCorner instances cluster3 10 20 30 40 phase instance variable count instance variable names instance cluster ID 1 Rectangle 2 origi corne 1 count class nameMonday, August 22, 2011
  • 64. cluster1 aRectangle Serialization cluster2 anOrigin aCorner instances cluster3 10 20 30 40 phase instance variable count instance variable names instance cluster ID 1 Rectangle 2 origi corne 1 count class nameMonday, August 22, 2011
  • 65. cluster1 aRectangle Serialization cluster2 anOrigin aCorner instances cluster3 10 20 30 40 phase instance variable count instance variable names instance cluster ID 1 Rectangle 2 origi corne 1 count class name 1 Point 2 x y 2Monday, August 22, 2011
  • 66. cluster1 aRectangle Serialization cluster2 anOrigin aCorner instances cluster3 10 20 30 40 phase instance variable count instance variable names instance cluster ID 1 Rectangle 2 origi corne 1 count class name 1 Point 2 x y 2 54 10 20 30 40Monday, August 22, 2011
  • 67. cluster1 aRectangle Serialization cluster2 anOrigin aCorner references cluster3 10 20 30 40 phase instancesIndex aRectangle 20 40 aCorner anOrigi 10 30(IdentityDictionary) 1 5 7 3 2 4 6 54 10 20 30 40 instances referencesMonday, August 22, 2011
  • 68. cluster1 aRectangle Serialization cluster2 anOrigin aCorner references cluster3 10 20 30 40 phase instancesIndex aRectangle 20 40 aCorner anOrigi 10 30(IdentityDictionary) 1 5 7 3 2 4 6 54 10 20 30 40 instances 2 3 referencesMonday, August 22, 2011
  • 69. cluster1 aRectangle Serialization cluster2 anOrigin aCorner references cluster3 10 20 30 40 phase instancesIndex aRectangle 20 40 aCorner anOrigi 10 30(IdentityDictionary) 1 5 7 3 2 4 6 54 10 20 30 40 instances 2 3 references 4 5 6 7Monday, August 22, 2011
  • 70. cluster1 aRectangle Serialization cluster2 anOrigin aCorner references cluster3 10 20 30 40 phase instancesIndex aRectangle 20 40 aCorner anOrigi 10 30(IdentityDictionary) 1 5 7 3 2 4 6 54 10 20 30 40 instances 2 3 references 4 5 6 7Monday, August 22, 2011
  • 71. final stream { 1 Rectangle 2 origi corne 1 instances 1 Point 2 x y 2 54 10 20 30 40 references { 2 3 4 5 6 7Monday, August 22, 2011
  • 72. final stream objectsCounts header { 7 3 clustersCount { 1 Rectangle 2 origi corne 1 instances 1 Point 2 x y 2 54 10 20 30 40 references { 2 3 4 5 6 7 trailer { 1 rootMonday, August 22, 2011
  • 73. Materialization objectsCounts header { 7 3 clustersCount 1 Rectangle 2 origin corner 1 instances { 1 Point 2 x y 2 54 1 2 3 4 2 3 references { 4 5 6 7 trailer { 1 rootMonday, August 22, 2011
  • 74. 41Monday, August 22, 2011