EMF Compare 2.0: Scaling to Millions (updated)

4,405 views
4,597 views

Published on

Slides from the Eclipse Con North America 2013 presentation

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
4,405
On SlideShare
0
From Embeds
0
Number of Embeds
3,075
Actions
Shares
0
Downloads
27
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

EMF Compare 2.0: Scaling to Millions (updated)

  1. 1. EMF Compare 2Scaling to MillionsMikaël Barbero 26 March 2013
  2. 2. http://idealistpropaganda.blogspot.fr/2010/04/imaginary-foundations-make-difference.htmlEMF Compare 2.0 Released mid-se ptember 2012 Sponsore d by Ericsson under Polarsys and Modeling Platform Workin g Group (MPWG)
  3. 3. http://www.flickr.com/photos/9781230@N06/2270505202Goal: Performance Should re late to number of differenc es, not to the size of the modelBut also simpler API, better understanding of differences for end-users...
  4. 4. Brand New UI Better integration with Team/Compare
  5. 5. http://www.flickr.com/photos/viggum/2417430161/DemoBecause it worths a thousand words
  6. 6. How does it works? in a Nutshell
  7. 7. Match
  8. 8. Two Strategies
  9. 9. Two Strategies Technical ID XMI:id, resource URI + URI fragment, your own...
  10. 10. Two Strategies Technical ID Content based XMI:id, resource URI + URI fragment, Heuristics, String distance your own...
  11. 11. http://www.flickr.com/photos/brianjmatis/3209203494/Differencing
  12. 12. RationalizationLongest Common Subsequencehttp://www.youtube.com/watch?v=RUckZMzqUcw
  13. 13. Dependencies http://www.flickr.com/photos/qilin/2262821682/
  14. 14. Package 1 Addition of ‘Package 1’ within ‘something’State machine 1 requires the merge of Addition of ‘State Machine 1’ within ‘Package 1’ Requirements between differences
  15. 15. Equivalences http://www.bavarianfootballworks.com/2012/9/24/3384502/preview-fc-bayern-vs-vfl-wolfsburg
  16. 16. Class 1 Setting ‘Property 1’ to reference ‘Associtation 1’ Property 1 the merge is equivalent to ends ass. Association 1 Setting ‘Association 1’ to reference ‘Property 1’‘ass.’ and ‘ends’ are opposite references Equivalence between differences
  17. 17. http://www.theology21.com/2010/07/27/are-you-talking-to-me-the-art-of-misunderstanding-and-theological-double-talk/ Conflicts
  18. 18. Match DifferenceSummary Requirement Equivalence Conflict
  19. 19. How to scale to Millions? http://www.flickr.com/photos/somemixedstuff/2403249501/
  20. 20. A graph of EObjectsA «logical» EMF model
  21. 21. A C DA graph of EObjects B GIn Resources E F
  22. 22. A C DA graph of EObjects B GCross-resources links E F
  23. 23. A B CA graph of Resources D E G F
  24. 24. A B C Ancestor D E GA B C Left (local) F D EG A B C F D E G F Right (remote)
  25. 25. A B C Ancestor D E GA B C Left (local) F D EG A B C F D E G F Right (remote)
  26. 26. A B C Ancestor D E GA B C Left (local) F D EG A B C F D E G F Right (remote)
  27. 27. A B C To be LoadedG E D To be Unloaded Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  28. 28. AA B C To be LoadedG E D To be Unloaded Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  29. 29. A B C To be LoadedG E D To be Unloaded Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  30. 30. A B C To be LoadedG E D To be Unloaded Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  31. 31. A B C To be LoadedG E D To be Unloaded Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  32. 32. B GA B C To be LoadedG E D To be Unloaded Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  33. 33. B GA B C To be Loaded AG E D To be Unloaded A Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  34. 34. B GA B C To be Loaded AG E D To be Unloaded A Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  35. 35. GA B C To be Loaded AG E D To be Unloaded A Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  36. 36. GA B C To be Loaded AG E D To be Unloaded A Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  37. 37. GA B C To be Loaded AG E D To be Unloaded A Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  38. 38. G CA B C To be Loaded AG E D To be Unloaded A Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  39. 39. G CA B C To be Loaded A BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  40. 40. G CA B C To be Loaded A BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  41. 41. G CA B C To be Loaded BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  42. 42. G CA B C To be Loaded BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  43. 43. G CA B C To be Loaded BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  44. 44. G CA B C To be Loaded BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  45. 45. A B C To be Loaded BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  46. 46. A B C To be Loaded BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  47. 47. A B C To be Loaded BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  48. 48. E DA B C To be Loaded BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  49. 49. E DA B C To be Loaded B C GG E D To be Unloaded A B C G Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  50. 50. E DA B C To be Loaded B C GG E D To be Unloaded A B C G Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  51. 51. DA B C To be Loaded GG E D To be Unloaded A B C G Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  52. 52. DA B C To be Loaded GG E D To be Unloaded A B C G Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  53. 53. DA B C To be Loaded GG E D To be Unloaded A B C G Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  54. 54. DA B C To be Loaded GG E D To be Unloaded A B C G Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  55. 55. D FA B C To be Loaded G EG E D To be Unloaded A B C G E Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  56. 56. D FA B C To be Loaded G EG E D To be Unloaded A B C G E Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  57. 57. FA B C To be LoadedG E D To be Unloaded A B C G E Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  58. 58. FA B C To be LoadedG E D To be Unloaded A B C G E Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  59. 59. FA B C To be LoadedG E D To be Unloaded A B C G E Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  60. 60. FA B C To be LoadedG E D To be Unloaded A B C G E Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  61. 61. FA B C To be LoadedG E D To be Unloaded A B C G E Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  62. 62. A B C To be LoadedG E D To be Unloaded A B C G E D Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  63. 63. A B C To be LoadedG E D To be Unloaded A B C G E D Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  64. 64. A B C To be LoadedG E D To be Unloaded A B C G E D F Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  65. 65. A B C To be LoadedG E D To be Unloaded A B C G E D F Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  66. 66. A B C To be LoadedG E D To be Unloaded A B C G E D F Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  67. 67. A B C To be Loaded FG E D To be Unloaded A B C G E D F Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  68. 68. A B C To be LoadedG E D To be Unloaded A B C G E D F Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  69. 69. A B C To be LoadedG E D To be Unloaded A B C G E D F Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  70. 70. Why concurrent loading?The HDD is the bottleneck, right?
  71. 71. Scheduling is crucialBetween loading and unloading threads toavoid high-memory consumption (our goal)
  72. 72. A B C Ancestor D E GA B C Left (local) F D EG A B C F D E G F Right (remote)
  73. 73. A B C Ancestor D E GA B C Left (local) F D EG A B C F D E G F Right (remote)
  74. 74. A B C Ancestor D E GA B C Left (local) F D EG A B C F D E G F Right (remote)
  75. 75. Summary
  76. 76. Summary Build the graph of resources
  77. 77. Summary Build the graph of resources Concurrent resolve all
  78. 78. Summary Find which Build the graph of resources has resources changed Concurrent resolve all
  79. 79. Summary Find which Build the graph of resources has resources changed Concurrent Binary comparison resolve all of resources
  80. 80. Summary Find which Build the graph of Compare resource resources has resources with differences changed Concurrent Binary comparison resolve all of resources
  81. 81. Summary Find which Build the graph of Compare resource resources has resources with differences changed Match, Diff, etc. Concurrent Binary comparison without resolving resolve all of resources proxies
  82. 82. Kill two pigs with one (angry) bird http://extremehdwallpapers.com/wp-content/uploads/2012/08/angry-birds-wallpaper-hd-.jpg
  83. 83. http://www.flickr.com/photos/roberts87/2646193801/ Lightweight How light?
  84. 84. http://www.flickr.com/photos/themonnie/2495892146/ Fast How fast?
  85. 85. Number of fragments (resources) Number of EObjects Disk Usage (XMI files size)10000 10000K 1000M1000 1000K 100M 100 100K 10M 10 10K Small Medium Large Small Medium Large Small Medium Large Sample Models The numbers (x2 in 2-ways, x3 in 3-ways)
  86. 86. The promised millions Number of fragments (resources) Number of EObjects Disk Usage (XMI files size)10000 10000K 1000M1000 1000K 100M 100 100K 10M 10 10K Small Medium Large Small Medium Large Small Medium Large Sample Models The numbers (x2 in 2-ways, x3 in 3-ways)
  87. 87. Small Medium Large 120 90CPU Usage Time (sec) 60Some changes in 1 resource 30 1 10 100 number of changes
  88. 88. Small Medium Large 120 90CPU Usage Time (sec) 601 change in some fragments 30 1 5 10 20 number of changed fragments
  89. 89. 600 450 Max Heap (MiB)Memory Usage 300Max Heap (-Xmx512m) 150 0 Small Medium Large
  90. 90. http://www.flickr.com/photos/exploratorium/4689347618/in/set-72157624060408411 http://www.flickr.com/photos/exploratorium/4680503314/in/set-72157624060408411 http://www.flickr.com/photos/exploratorium/4689346802/in/set-72157624060408411 What’s Next?
  91. 91. Model Resolving Scope Narrowing Matching Differencing Post-Process 100 % 75 %Time distribution 50 % 25 % Small 0 % Nominal Large
  92. 92. Index Cross-Resources Links http://www.flickr.com/photos/manchesterlibrary/5425247171/
  93. 93. Eclipse Papyrus support http://www.flickr.com/photos/manchesterlibrary/5425247171/
  94. 94. Porting Eclipse Papyrus support
  95. 95. CA B C To be LoadedG E D To be Unloaded Already browsed F Loading Thread Unloading Thread Protect the Integrity of the Model When committing, updating...
  96. 96. http://nn.wikipedia.org/wiki/Fil:Postcards_and_magnifying_glass.jpg Generic Content Based Match Engine Decently Fast
  97. 97. http://cdn.memegenerator.net/instances/400x/20451927.jpg Merge Wizard Help to Resolve Conflict
  98. 98. http://www.space.com/17376-sharing-the-light-of-two-suns.html EMF Compare 2.1 With Kepler (Eclipse 4.3), June 2013
  99. 99. What you should remember New UI Blazing Fast and Lightweight More to come ;)
  100. 100. http://www.eclipse.org/emf/compare/http://www.slideshare.net/mikaelbarbero/

×