Performance and Extensibility with EMF

Getting the Most out of Your Models Performance and Extensibility with EMF Dave Steinberg, IBM Marcelo Paternostro, IBM Kenn Hussey, Committerati Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 1

Exercise Prerequisites • USB drives are being passed around – Separate images for 64-bit and 32-bit platforms – Copy the contents to a temporary location and pass it on • Extract platform-appropriate eclipse-SDK-3.6M6-*.zip to some location – Do not nest deeply on Windows • Launch Eclipse and use Help > Install New Software... to install the modeling features – Add a local site, and specify the modeling-projects/ subfolder of the temporary location – Install all the features from the site – Accept installation of unsigned software – Restart the workbench • Import all of the projects from emf-tutorial-workspace.zip Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 2

Agenda • Introduction • Performance – Generated code – Resources – Exercise • Extensibility – Validation delegates – Setting delegates – Invocation delegates – Exercise • Q&A Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 3

As you already know... • EMF is the foundation for modeling and data integration at Eclipse • Ecore models describe types of objects, their attributes, operations, and relationships – Extensible import support including UML, XML Schema, and annotated Java interfaces • Merging generator enables mixing of generated and hand-written code • Runtime framework provides notiﬁcation, reﬂective API, dynamic EMF, XML persistence, validation... Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 4

Balancing Performance Speed Memory Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 6

Balancing Performance Speed Memory Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 7

Measuring Performance • Measuring performance in Java is tricky • Speed – What are you measuring? (inlining, JIT, garbage collection, etc.) – Precision was a big problem for small operations, until Java 5 added System.nanoTime() • Memory – What are you measuring? (deep size, shared objects, static ﬁelds, etc.) – No way to estimate the size of a given object until Java 5 added Instrumentation.getObjectSize() • Comparisons are useful, single measurements not so much Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 8

EObject Overhead public class EObjectImpl ... 32 bytes { protected int eFlags; // deliver, dynamic, proxy protected BasicEList<Adapter> eAdapters; EAdapterList protected InternalEObject eContainer; ≥ 32 bytes protected int eContainerFeatureID; protected EPropertiesHolder eProperties; ... } protected static class EPropertiesHolderImpl ... 32 bytes { protected EClass eClass; protected Resource.Internal eResource; protected Object[] eSettings; EContentsEList protected URI eProxyURI; ≥ 24 bytes protected EList<EObject> eContents; protected EList<EObject> eCrossReferences; ... ECrossReferenceEList } ≥ 24 bytes A fully ﬂuffed up (but empty) instance has 144 bytes of overhead! Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 10

Alternative EObject Implementations • DynamicEObjectImpl moves eSettings and eClass out of properties holder • FlatEObjectImpl moves eProxyURI, eContents, and eCrossReferences out of properties holder Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 11

“EMF Ultra Slim Diet” • MinimalEObjectImpl uses a different approach: – Dynamically sized array (or single object) for all overhead – Does not cache contents or cross references – Adapters stored as array or simply delegated to container public class MinimalEObjectImpl ... { private int eFlags; private Object eStorage; ... public static class Container extends MinimalEObjectImpl { protected InternalEObject eContainer; ... } } • Overhead of empty, ﬂuffed up MinimalEObjectImpl.Container instance is just 24 bytes Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 12

Modeled Features public class ProjectImpl extends EObjectImpl implements Project { ... protected ProjectPhase phase = PHASE_EDEFAULT; protected boolean ipClean = IP_CLEAN_EDEFAULT; 4 bytes protected boolean inSimultaneousRelease = IN_SIMULTANEOUS_RELEASE_EDEFAULT; ... 1 byte public ProjectPhase getPhase() { return phase; } public boolean isIpClean() { return ipClean; } ... } Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 13

Boolean Flags & Packed Enums public class ProjectImpl extends EObjectImpl implements Project { ... protected int flags = 0; protected static final int PHASE_EFLAG_OFFSET = 0; protected static final int PHASE_EFLAG = 0x3 << PHASE_EFLAG_OFFSET; private static final ProjectPhase[] PHASE_EFLAG_VALUES = ProjectPhase.values(); protected static final int IP_CLEAN_EFLAG = 1 << 2; protected static final int IN_SIMULTANEOUS_RELEASE_EFLAG = 1 << 3; ... public ProjectPhase getPhase() { return PHASE_EFLAG_VALUES[ (flags & PHASE_EFLAG) >>> PHASE_EFLAG_OFFSET]; } public boolean isIpClean() { return (flags & IP_CLEAN_EFLAG) != 0; } ... } Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 14

Boolean Flags & Packed Enums • Unset states also recorded in flags field • Reduces size by up to 87.5% for booleans • Can leverage protected eFlags field in EObjectImpl (not MinimalEObjectImpl), but 8 bits must be reserved Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 15

Virtual Delegation public class ProjectImpl extends EObjectImpl implements Project { ... protected Object[] eVirtualValues; protected int eVirtualIndexBits0; protected Object[] eVirtualValues() { return eVirtualValues; } ... public ProjectPhase getPhase() { return (ProjectPhase)eVirtualGet( FoundationPackage.PROJECT__PHASE, PHASE_EDEFAULT); } ... } Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 16

Virtual Delegation • Appropriate only for very sparsely populated models with many features – measure before adopting! Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 17

Reﬂective Method Dispatch public class TopLevelProjectImpl extends ProjectImpl implements TopLevelProject { ... public Object eGet(int featureID, boolean resolve, boolean coreType) { switch (featureID) { case FoundationPackage.TOP_LEVEL_PROJECT__PROJECTS: return getProjects(); case FoundationPackage.TOP_LEVEL_PROJECT__PMC_MEMBERS: return getPmcMembers(); } return super.eGet(featureID, resolve, coreType); } ... } Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 18

O(1) Reﬂective Methods public class TopLevelProjectImpl extends ProjectImpl implements TopLevelProject { ... public Object eGet(int featureID, boolean resolve, boolean coreType) { switch (featureID) { case FoundationPackage.TOP_LEVEL_PROJECT__PHASE: return getPhase(); case FoundationPackage.TOP_LEVEL_PROJECT__IP_CLEAN: return isIpClean(); case FoundationPackage.TOP_LEVEL_PROJECT__IN_SIMULTANEOUS_RELEASE: return isInSimultaneousRelease(); case FoundationPackage.TOP_LEVEL_PROJECT__PROJECTS: return getProjects(); case FoundationPackage.TOP_LEVEL_PROJECT__PMC_MEMBERS: return getPmcMembers(); } return eDynamicGet(featureID, resolve, coreType); } ... } Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 19

O(1) Reﬂective Methods • Performance improvement possible only for deep inheritance hierarchies • Trade-offs are memory impact of duplicated code (not heap) and cross-model inheritance fragility (need to regenerate) • Note: default reﬂective dispatch is not binary compatible Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 20

Identifying EObjects in Resources • Fragment paths – The default, a slash-delimited path computed based on containment within the resource – For multi-valued features, segments can be based simply on an index or on values of one or more “key” attributes • Intrinsic IDs – Used when an EObject's EClass deﬁnes an “ID” attribute and that attribute is set • Extrinsic IDs (XML resources only) – Used only when an EObject is explicitly assigned an ID – Particular resource implementations may do this automatically Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 22

Extrinsic IDs • EObject-Extrinisic ID associations are recorded in two maps (one for each direction) XMLResource xmlResource = ... Project project = ... xmlResource.setID(project, "EMF-Project-123"); xmlResource.getContents().add(project); assert project == xmlResource.getEObject("EMF-Project-123"); assert "EMF-Project-123" == xmlResource.getID(project); • These maps are maintained by the XMLResource and should not be manipulated directly Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 23

Extrinsic IDs • Serialized via xmi:id or xsi:id attributes • UUIDs can be automatically assigned by the resource public class MyXMLResourceImpl extends XMLResourceImpl { @Override protected boolean useUUIDs() { return true; } } • Note: a single UUID consumes 88 bytes! Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 24

Intrinsic IDs • An EObject's intrinsic IDs is part of its state – The string value of the EAttribute that is deﬁned to be the ID of the EClass Resource resource = ... Project emf = (Project)resource.getEObject("EMF"); assert FoundationPackage.Literals.PROJECT__NAME.isID(); assert "EMF".equals(project.getName()); • Finding an EObject by intrinsic ID involves walking over the entire resource Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 25

“Secret” Caches • It is possible to improve performance by using some lesser known caches: – Intrinsic ID to EObject (in Resource) – URI to Resource (in ResourceSet) • Both caches – can be pre-populated – are lazily populated as elements are retrieved Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 26

Intrinsic ID to EObject Cache Resource resource = ... Map<String, EObject> cache = new HashMap<String, EObject>(); ((ResourceImpl)resource).setIntrinsicIDToEObjectMap(cache); Project project = ... project.setName("EMF"); assert FoundationPackage.Literals.PROJECT__NAME.isID(); resource.getContents().add(project); • Heads up: – EObjects that are removed from the resource are automatically removed from the cache – EObjects with non-null intrinsic IDs that are added to the resource are automatically added to the cache – The cache may become inconsistent (e.g. if an EObject's intrinsic ID is changed after it has been added to the resource) Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 27

URI to Resource Cache • Obtaining the resource for a URI can be expensive, especially if it involves URI normalization ResourceSet resourceSet = ... Map<URI, Resource> cache = new HashMap<URI, Resource>(); ((ResourceSetImpl)resourceSet).setURIResourceMap(cache); • Heads up: – Resources that are removed from the resource set are automatically removed from the cache – The cache may become inconsistent (e.g. if a resource's URI is changed) – URIs in the cache (key values) don't include fragments Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 28

Intrinsic ID Performance Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 29

Large Resource Performance Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 30

Cross-Resource Containment • Cross-resource containment allows an object hierarchy to be persisted across multiple resources – eObject.eResource() may be different from eObject.eContainer().eResource() • Side beneﬁt: facilitates ﬁner-grained partial loading Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 31

Cross-Resource Containment EClass TopLevelProjectEClass = ... EClass projectEClass = ... EReference projects = EcoreFactory.eINSTANCE.createEReference(); Dynamic projects.setName("projects"); Model projects.setUpperBound(ETypedElement.UNBOUNDED_MULTIPLICITY); projects.setEType(projectEClass); projects.setContainment(true); projects.setResolveProxies(true); topLevelProjectEClass.getEStructuralFeatures().add(projects); Generated Model Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 32

Binary Resources • EMF also supports a high-performance proprietary binary serialization format – Numeric IDs for EObjects, metadata, URIs – String lengths speciﬁed, no delimiters – Signature to recognize corruption – Versioned to allow for future format changes Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 33

Binary Resources Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 34

Binary Resource Resource.Factory binaryResourceFactory = new Resource.Factory() { public Resource createResource(URI uri) { return new BinaryResourceImpl(uri); } }; ResourceSet resourceSet = new ResourceSetImpl(); resourceSet.getResourceFactoryRegistry(). getExtensionToFactoryMap().put("dat", binaryResourceFactory); Resource resource = resourceSet.createResource( URI.createFileURI("projects.dat")); Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 35

Save and Load Options • Resource implementations offer a set of options that can be used to tweak load and save behavior • Options passed to the resource’s save and load methods as entries in a map – Key: the constant that represents the option – Value: appropriately typed for the option Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 36

Save and Load Options XMLResource xmlResource = ... Map<String, Object> options = new HashMap<String, Object>(); options.put(Resource.OPTION_ZIP, true); options.put(XMLResource.OPTION_USE_PARSER_POOL, new XMLParserPoolImpl()); options.put(XMLResource.OPTION_USE_CACHED_LOOKUP_TABLE, new ArrayList<Object>()); xmlResource.save(options); • Options are usually declared in – resource interface (e.g. XMLResource) – resource implementation class (e.g. BinaryResourceImpl) • Get used to opening the EMF source code or Javadoc to read about them Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 37

Save and Load Options Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 38

A Nod to Scalability: CDO • Connected Data Objects (CDO): an EMF component offering central persistence for models and instances – Multiple clients can have views on and transactions against a single remote repository – Clients may modify the shared object graph concurrently – Other clients are immediately notiﬁed about modiﬁcations • Supports CDO-targeted generated models, dynamic models, and “legacy” models • The way forward for truly scalable EMF – Pluggable DB storage (Derby, MySQL, HSQLDB, etc.) – Full demand loading and unloading of objects, with partial collection loading and adaptable pre-fetching Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 39

Exercise • Have Fun! – The goal for the next few minutes is to let you experiment with EMF – Use the knowledge you've learned during the tutorial so far: • Try saving and loading resources using different options • See the difference between different types of resources (XMI and binary for example) • Re-generate the model using different genmodel settings Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 41

Sample Model Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 42

Cheat Sheet: Test Harness • In project org.eclipsecon.foundation.test: – ModelFactory • Creates instances of the Foundation model • The number of instances can be controlled by constructor arguments – MemoryTester • Outputs the size of the instances of the model • Requires the “-javaagent” JVM argument – ResourceTester • Base class for classes that time saving and loading resources Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 43

Cheat Sheet: Resource Options • org.eclipse.emf.ecore.resource.Resource – OPTION_ZIP – OPTION_SAVE_ONLY_IF_CHANGED • OPTION_SAVE_ONLY_IF_CHANGED_MEMORY_BUFFER • OPTION_SAVE_ONLY_IF_CHANGED_FILE_BUFFER • org.eclipse.emf.ecore.resource.impl.BinaryResourceImpl – OPTION_BUFFER_CAPACITY Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 44

Cheat Sheet: XML Resource • Load – OPTION_DEFER_IDREF_RESOLUTION – OPTION_USE_PARSER_POOL – OPTION_USE_XML_NAME_TO_FEATURE_MAP – OPTION_USE_DEPRECATED_METHODS • Save – OPTION_FLUSH_THRESHOLD – OPTION_CONFIGURATION_CACHE – OPTION_FORMATTED – OPTION_USE_FILE_BUFFER – OPTION_USE_CACHED_LOOKUP_TABLE Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 45

Extensibility • Many enhancements have been made to EMF over the past few years to make the framework more extensible • As of the M4 milestone of the Helios release, EMF has been made even more extensible with the addition of support for validation, feature setting, and operation invocation delegation • We'll consider each of these new mechanisms in turn and look at how they can be implemented and used for our sample model Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 48

Validation Delegation • The EMF Validation Framework previously supported evaluation of invariants and constraints, deﬁned as methods and implemented in Java, via a validator that is invoked at important moments by an application or at a user’s discretion • In order to delegate evaluation of invariants and constraints to an external engine, the framework was enhanced by introducing the concept of a validation delegate Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 50

Implementing Validation Delegates • Deﬁne a validation delegate by implementing the ValidationDelegate interface • Register the validation delegate, either programmatically or via an extension Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 51

Using Validation Delegates • Reference the validation delegate via an annotation on an Ecore package • Define the behaviors of invariants and/or constraints via expressions in annotations on Ecore operations and/or classifiers • Evaluate invariants and/or constraints, either statically via generated code or dynamically via EMF reflection Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 52

Feature Setting Delegation • EMF previously supported computation of features, deﬁned as methods and implemented in Java, via an API that is invoked at important moments by an application • In order to delegate computation of a feature’s value to an external engine, the framework was enhanced by exposing the previously existing concept of a setting delegate Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 54

Implementing Setting Delegates • Deﬁne a setting delegate by implementing the SettingDelegate interface • Deﬁne a setting delegate factory by implementing the Factory interface • Register the setting delegate factory, either programmatically or via an extension Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 55

Using Setting Delegates • Reference the setting delegate factory via an annotation on an Ecore package • Deﬁne the values of features via expressions in annotations on Ecore structural features • Compute the values of features, either statically via generated code or dynamically via EMF reﬂection Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 56

Operation Invocation Delegation • EMF previously supported execution of operations, deﬁned as methods and implemented in Java, via an API that is invoked at important moments by an application • In order to delegate execution of an operation’s behavior to an external engine, the framework has been enhanced by introducing the concept of an invocation delegate Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 58

Implementing Invocation Delegates • Deﬁne an invocation delegate by implementing the InvocationDelegate interface • Deﬁne an invocation delegate factory by implementing the Factory interface • Register the invocation delegate factory, either programmatically or via an extension Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 59

Using Invocation Delegates • Reference the invocation delegate factory via an annotation on an Ecore package • Deﬁne the values of operations via expressions in annotations on Ecore operations • Execute the behaviors of operations, either statically via generated code or dynamically via EMF reﬂection Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 60

Resources • Online help – http://help.eclipse.org/ganymede/index.jsp?nav=/14 • Website – http://www.eclipse.org/emf • Downloads • Wiki • FAQ • Newsgroup • Documentation • Book – Eclipse Modeling Framework • Second Edition – http://safari.awprofessional.com/9780321331885 Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 65

Legal Notices Copyright © IBM Corp. and Committerati Consulting Corporation, 2010. All rights reserved. This presentation and the source code in it are made available under the EPL, v1.0. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Eclipse and the Eclipse logo are trademarks of Eclipse Foundation, Inc. IBM and the IBM logo are trademarks or registered trademarks of IBM Corporation, in the United States, other countries or both. UML and XMI are registered trademarks of the Object Management Group. HTTP, Metadata, and XML are trademarks of the World Wide Web Consortium; marks of W3C are registered and held by its host institutions MIT, ERCIM, and Keio. Other company, product, or service names may be trademarks or service marks of others. THE INFORMATION DISCUSSED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION, IT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, AND IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, SUCH INFORMATION. ANY INFORMATION CONCERNING IBM'S PRODUCT PLANS OR STRATEGY IS SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0 66

http://www.eclipsecon.org	185
http://www.slideshare.net	41
http://webcache.googleusercontent.com	4
http://www.lmodules.com	1

Performance and Extensibility with EMF

by Kenn Hussey

Accessibility

Categories

Upload Details

Usage Rights

4 Embeds 231

Statistics