EMF Compare 2.0: Scaling to Millions (updated)
Upcoming SlideShare
Loading in...5
×
 

EMF Compare 2.0: Scaling to Millions (updated)

on

  • 2,082 views

Slides from the Eclipse Con North America 2013 presentation

Slides from the Eclipse Con North America 2013 presentation

Statistics

Views

Total Views
2,082
Views on SlideShare
779
Embed Views
1,303

Actions

Likes
0
Downloads
22
Comments
0

4 Embeds 1,303

http://modeling-languages.com 1294
http://modelinglang.staging.wpengine.com 6
http://translate.googleusercontent.com 2
http://plus.url.google.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

EMF Compare 2.0: Scaling to Millions (updated) EMF Compare 2.0: Scaling to Millions (updated) Presentation Transcript

  • EMF Compare 2Scaling to MillionsMikaël Barbero 26 March 2013
  • http://idealistpropaganda.blogspot.fr/2010/04/imaginary-foundations-make-difference.htmlEMF Compare 2.0 Released mid-se ptember 2012 Sponsore d by Ericsson under Polarsys and Modeling Platform Workin g Group (MPWG)
  • http://www.flickr.com/photos/9781230@N06/2270505202Goal: Performance Should re late to number of differenc es, not to the size of the modelBut also simpler API, better understanding of differences for end-users...
  • Brand New UI Better integration with Team/Compare
  • http://www.flickr.com/photos/viggum/2417430161/DemoBecause it worths a thousand words
  • How does it works? in a Nutshell
  • Match
  • Two Strategies
  • Two Strategies Technical ID XMI:id, resource URI + URI fragment, your own...
  • Two Strategies Technical ID Content based XMI:id, resource URI + URI fragment, Heuristics, String distance your own...
  • http://www.flickr.com/photos/brianjmatis/3209203494/Differencing
  • RationalizationLongest Common Subsequencehttp://www.youtube.com/watch?v=RUckZMzqUcw
  • Dependencies http://www.flickr.com/photos/qilin/2262821682/
  • Package 1 Addition of ‘Package 1’ within ‘something’State machine 1 requires the merge of Addition of ‘State Machine 1’ within ‘Package 1’ Requirements between differences
  • Equivalences http://www.bavarianfootballworks.com/2012/9/24/3384502/preview-fc-bayern-vs-vfl-wolfsburg
  • Class 1 Setting ‘Property 1’ to reference ‘Associtation 1’ Property 1 the merge is equivalent to ends ass. Association 1 Setting ‘Association 1’ to reference ‘Property 1’‘ass.’ and ‘ends’ are opposite references Equivalence between differences
  • http://www.theology21.com/2010/07/27/are-you-talking-to-me-the-art-of-misunderstanding-and-theological-double-talk/ Conflicts
  • Match DifferenceSummary Requirement Equivalence Conflict
  • How to scale to Millions? http://www.flickr.com/photos/somemixedstuff/2403249501/
  • A graph of EObjectsA «logical» EMF model
  • A C DA graph of EObjects B GIn Resources E F
  • A C DA graph of EObjects B GCross-resources links E F
  • A B CA graph of Resources D E G F
  • A B C Ancestor D E GA B C Left (local) F D EG A B C F D E G F Right (remote)
  • A B C Ancestor D E GA B C Left (local) F D EG A B C F D E G F Right (remote)
  • A B C Ancestor D E GA B C Left (local) F D EG A B C F D E G F Right (remote)
  • A B C To be LoadedG E D To be Unloaded Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • AA B C To be LoadedG E D To be Unloaded Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • A B C To be LoadedG E D To be Unloaded Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • A B C To be LoadedG E D To be Unloaded Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • A B C To be LoadedG E D To be Unloaded Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • B GA B C To be LoadedG E D To be Unloaded Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • B GA B C To be Loaded AG E D To be Unloaded A Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • B GA B C To be Loaded AG E D To be Unloaded A Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • GA B C To be Loaded AG E D To be Unloaded A Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • GA B C To be Loaded AG E D To be Unloaded A Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • GA B C To be Loaded AG E D To be Unloaded A Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • G CA B C To be Loaded AG E D To be Unloaded A Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • G CA B C To be Loaded A BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • G CA B C To be Loaded A BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • G CA B C To be Loaded BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • G CA B C To be Loaded BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • G CA B C To be Loaded BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • G CA B C To be Loaded BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • A B C To be Loaded BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • A B C To be Loaded BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • A B C To be Loaded BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • E DA B C To be Loaded BG E D To be Unloaded A B Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • E DA B C To be Loaded B C GG E D To be Unloaded A B C G Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • E DA B C To be Loaded B C GG E D To be Unloaded A B C G Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • DA B C To be Loaded GG E D To be Unloaded A B C G Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • DA B C To be Loaded GG E D To be Unloaded A B C G Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • DA B C To be Loaded GG E D To be Unloaded A B C G Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • DA B C To be Loaded GG E D To be Unloaded A B C G Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • D FA B C To be Loaded G EG E D To be Unloaded A B C G E Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • D FA B C To be Loaded G EG E D To be Unloaded A B C G E Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • FA B C To be LoadedG E D To be Unloaded A B C G E Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • FA B C To be LoadedG E D To be Unloaded A B C G E Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • FA B C To be LoadedG E D To be Unloaded A B C G E Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • FA B C To be LoadedG E D To be Unloaded A B C G E Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • FA B C To be LoadedG E D To be Unloaded A B C G E Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • A B C To be LoadedG E D To be Unloaded A B C G E D Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • A B C To be LoadedG E D To be Unloaded A B C G E D Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • A B C To be LoadedG E D To be Unloaded A B C G E D F Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • A B C To be LoadedG E D To be Unloaded A B C G E D F Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • A B C To be LoadedG E D To be Unloaded A B C G E D F Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • A B C To be Loaded FG E D To be Unloaded A B C G E D F Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • A B C To be LoadedG E D To be Unloaded A B C G E D F Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • A B C To be LoadedG E D To be Unloaded A B C G E D F Already browsed F Loading Thread Unloading Thread Concurrent Resolve All Multi-Threaded
  • Why concurrent loading?The HDD is the bottleneck, right?
  • Scheduling is crucialBetween loading and unloading threads toavoid high-memory consumption (our goal)
  • A B C Ancestor D E GA B C Left (local) F D EG A B C F D E G F Right (remote)
  • A B C Ancestor D E GA B C Left (local) F D EG A B C F D E G F Right (remote)
  • A B C Ancestor D E GA B C Left (local) F D EG A B C F D E G F Right (remote)
  • Summary
  • Summary Build the graph of resources
  • Summary Build the graph of resources Concurrent resolve all
  • Summary Find which Build the graph of resources has resources changed Concurrent resolve all
  • Summary Find which Build the graph of resources has resources changed Concurrent Binary comparison resolve all of resources
  • Summary Find which Build the graph of Compare resource resources has resources with differences changed Concurrent Binary comparison resolve all of resources
  • Summary Find which Build the graph of Compare resource resources has resources with differences changed Match, Diff, etc. Concurrent Binary comparison without resolving resolve all of resources proxies
  • Kill two pigs with one (angry) bird http://extremehdwallpapers.com/wp-content/uploads/2012/08/angry-birds-wallpaper-hd-.jpg
  • http://www.flickr.com/photos/roberts87/2646193801/ Lightweight How light?
  • http://www.flickr.com/photos/themonnie/2495892146/ Fast How fast?
  • Number of fragments (resources) Number of EObjects Disk Usage (XMI files size)10000 10000K 1000M1000 1000K 100M 100 100K 10M 10 10K Small Medium Large Small Medium Large Small Medium Large Sample Models The numbers (x2 in 2-ways, x3 in 3-ways)
  • The promised millions Number of fragments (resources) Number of EObjects Disk Usage (XMI files size)10000 10000K 1000M1000 1000K 100M 100 100K 10M 10 10K Small Medium Large Small Medium Large Small Medium Large Sample Models The numbers (x2 in 2-ways, x3 in 3-ways)
  • Small Medium Large 120 90CPU Usage Time (sec) 60Some changes in 1 resource 30 1 10 100 number of changes
  • Small Medium Large 120 90CPU Usage Time (sec) 601 change in some fragments 30 1 5 10 20 number of changed fragments
  • 600 450 Max Heap (MiB)Memory Usage 300Max Heap (-Xmx512m) 150 0 Small Medium Large
  • http://www.flickr.com/photos/exploratorium/4689347618/in/set-72157624060408411 http://www.flickr.com/photos/exploratorium/4680503314/in/set-72157624060408411 http://www.flickr.com/photos/exploratorium/4689346802/in/set-72157624060408411 What’s Next?
  • Model Resolving Scope Narrowing Matching Differencing Post-Process 100 % 75 %Time distribution 50 % 25 % Small 0 % Nominal Large
  • Index Cross-Resources Links http://www.flickr.com/photos/manchesterlibrary/5425247171/
  • Eclipse Papyrus support http://www.flickr.com/photos/manchesterlibrary/5425247171/
  • Porting Eclipse Papyrus support
  • CA B C To be LoadedG E D To be Unloaded Already browsed F Loading Thread Unloading Thread Protect the Integrity of the Model When committing, updating...
  • http://nn.wikipedia.org/wiki/Fil:Postcards_and_magnifying_glass.jpg Generic Content Based Match Engine Decently Fast
  • http://cdn.memegenerator.net/instances/400x/20451927.jpg Merge Wizard Help to Resolve Conflict
  • http://www.space.com/17376-sharing-the-light-of-two-suns.html EMF Compare 2.1 With Kepler (Eclipse 4.3), June 2013
  • What you should remember New UI Blazing Fast and Lightweight More to come ;)
  • http://www.eclipse.org/emf/compare/http://www.slideshare.net/mikaelbarbero/