Using ATL/EMFTVM for import/export of medical data - #sda2014
•Download as PPTX, PDF•
0 likes•640 views
Most datacentric software must deal with some form of import/export of their internal data model to an external data format. In many cases, this external data format is some sort of standard format, or otherwise dictated by external sources, and does not map one-on-one to the internal data model. This was also the case for CareConnect, HealthConnect/Corilus‘ latest Electronic Medical Record software. CareConnect must be able to import/export its data from/to SUMEHR, PMF, and GPSMF documents. On top of this comes that CareConnect’s internal data model consists of some 300 classes, which means there are a lot of mappings to define. To deal with the size and complexity of this scenario, we decided to use a specialised language: the ATL transformation language in combination with the EMF Transformation Virtual Machine: EMFTVM is a new runtime for ATL, which adds a number of performanceenhancing features that make it more suitable for use within a Java application. The declarative, rulebased nature of ATL allowed us to write more concise code as well as distribute the workload of writing the ATL transformation code over multiple developers. This significantly increased our ability to deal with complexity.
Recommended
More Related Content
Similar to Using ATL/EMFTVM for import/export of medical data - #sda2014
Similar to Using ATL/EMFTVM for import/export of medical data - #sda2014 (20)
Recently uploaded
Recently uploaded (20)
Using ATL/EMFTVM for import/export of medical data - #sda2014
- 1. Using ATL/EMFTVM for import/export of medical data 8-Oct-2014, Dennis Wagelaar
- 2. 2
- 4. Import/export a common programming scenario SuMEHR GPSMF PMF Corilus XML SuMEHR GPSMF PMF Corilus XML 4
- 5. Corilus XML as pivot model SuMEHR GPSMF PMF Corilus XML 5
- 6. Corilus XML as pivot model SuMEHR GPSMF PMF Corilus XML 6
- 7. Import/export simplified Corilus XML Corilus XML 7
- 8. Why ATL? (ATL Transformation Language) Domain-specific language for transformation More expressive than mapping frameworks embedded in Java, e.g. Dozer Less verbose for transformations than general-purpose languages Uses OCL standard for expressions Uses EMF for data representation Closely related to plain Java objects Enriched with additional concepts, e.g. containment and associated properties 8
- 9. Why EMFTVM? (EMF Transformation Virtual Machine) Enhanced for “online” use (performance) Reuse pre-loaded transformations for multiple executions JIT compiler translates to Java bytecode Adaptive matching algorithm adds performance Improved modularity Supports multiple rule inheritance across different modules Supports module import across different source languages 9
- 10. Why EMFTVM? (EMF Transformation Virtual Machine) 10 https://wiki.eclipse.org/ATL/EMFTVM
- 11. EMF vs. POJOs (JPA) Uses Uses EMF Models Plain Old Java Objects 11
- 12. EMF vs. POJOs This is what we have... (JPA) Uses Plain Old Java Objects 12
- 13. From POJOs to EMF MoDisco and EMiFy Java Model Reverse Engineer Plain Old Java Objects 13
- 14. From POJOs to EMF MoDisco and EMiFy Java Model Ecore Model EMiFy.atl 14
- 15. From POJOs to EMF MoDisco and EMiFy Ecore Model Generate Model EMF Java Objects 15
- 16. From POJOs to EMF MoDisco and EMiFy Java Model Ecore Model EMiFy.atl Reverse Engineer Generate Model EMF Java Objects 16
- 17. Using ATL 17
- 18. Using ATL 18
- 19. Using ATL Implicit tracing 19
- 20. Using ATL Implicit tracing 20
- 21. Using ATL Collaboration 21 Java developer Java developer Common: 178 lines of ATL code Import: 7283 lines of ATL code Export: 4617 lines of ATL code .NET developer Java developer 4906 lines of Java import/export code
- 22. Runtime performance Medium-large patient file (2277 record entries) 22 03-10-2014 | 13:57:19,223 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.rs.ImportResource | Import partial document 504 03-10-2014 | 13:57:19,307 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Start full import for ... 03-10-2014 | 13:57:20,006 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Starting transformation... 03-10-2014 | 13:57:21,099 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.emftvm.impl.AbstractEMFTVMTransformer | Timing data: Loading finished at 0,000036 seconds (duration: 0,000036 seconds) Matching finished at 0,285989 seconds (duration: 0,285953 seconds) Applying finished at 1,091511 seconds (duration: 0,805522 seconds) Post-applying finished at 1,091712 seconds (duration: 0,000201 seconds) Recursive stage finished at 1,091751 seconds (duration: 0,000039 seconds) Execution finished at 1,091915 seconds (duration: 0,000203 seconds) 03-10-2014 | 13:57:21,118 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 1 Patient instances 03-10-2014 | 13:57:21,155 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 3 ExternalUser instances 03-10-2014 | 13:57:21,156 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 1 HealthInsurance instances 03-10-2014 | 13:57:21,161 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 138 Contact instances 03-10-2014 | 13:57:21,282 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 33 Episode instances 03-10-2014 | 13:57:21,316 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 33 HealthApproach instances 03-10-2014 | 13:57:21,349 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 553 SubContact instances 03-10-2014 | 13:57:21,925 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 2 ContactPerson instances 03-10-2014 | 13:57:23,240 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 1 Vaccination instances 03-10-2014 | 13:57:23,255 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 134 NormalMedicationEntry instances 03-10-2014 | 13:57:23,649 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 4 MagistralMedicationEntry instances 03-10-2014 | 13:57:23,673 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 138 FreeTextPosology instances 03-10-2014 | 13:57:24,041 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 224 SOEPRule instances 03-10-2014 | 13:57:24,473 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 78 Letter instances 03-10-2014 | 13:57:25,416 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 30 LabResult instances 03-10-2014 | 13:57:25,495 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 776 LabResultEntry instances 03-10-2014 | 13:57:28,533 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 51 ParameterGroup instances 03-10-2014 | 13:57:28,657 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 72 ParameterGroupEntry instances 03-10-2014 | 13:57:28,992 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 3 SickLeave instances 03-10-2014 | 13:57:29,023 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.services.impl.CorilusXMLImportServiceImpl | Processing 2 Note instances 03-10-2014 | 13:57:33,992 | INFO | http-0.0.0.0-8080-4 | be.healthconnect.emr.server.rs.ImportResource | Finished import partial document 504 in 14.768414891 XML loading 0,7 sec Transformation 1,1 sec Hibernate storage 12,9 sec
- 23. Conclusion We tackled a complex and common programming scenario such as import/export by breaking it up in three ways: 23 Use specialised language for translating between domain model and pivot model Use pivot model for import/ export => only support a single import/export format Use regular Java to handle file I/O and database interaction XML
- 24. 24 Questions? http://www.healthconnect.be/ http://www.corilus.be/ http://www.careconnect.be/ Questions ?
Editor's Notes
- Most data-centric software must deal with some form of import/export of their internal data model to an external data format. In many cases, this external data format is some sort of standard format, or otherwise dictated by external sources, and does not map one-on-one to the internal data model.
- This was also the case for CareConnect, HealthConnect/Corilus‘ latest Electronic Medical Recordsoftware. CareConnect must be able to import/export its data from/to SUMEHR, PMF, and GPSMF documents. On top of this comes that CareConnect’s internal data model consists of some 300 classes, which means there are a lot of mappings to define.
- EMFTVM performance is roughly 80% better than the default ATL EMF-specific VM. EMFTVM has a JIT-compiler that improves performance of complex code blocks. It also allows for reuse of a pre-loaded VM instance (when invoking from Java), which is useful when invoking the same transformation on different models many times over. Finally, it uses an adaptive rule matching algorithm that configures itself against the metamodels and transformation modules used on the first run of the VM. The EcoreUtil.Copier entry is the standard Java implementation for copying Ecore models, and forms the baseline ("it doesn't get faster than this"). On the following lines one can see the evolution in performance of the various ATL VMs.
- Note that the MoDisco-EMiFy-EMF round-trip scenario for generating the Ecore model and EMF reflective methods was performed for each change to our domain model, and takes about 10 minutes each time. Most of this time was taken up by running MoDisco and the EMF code generator. This takes up extra time from the developer who manages the domain model changes, besides the regular Java code changes and SQL migration code. The positive side of this picture is that the Ecore domain model can also be used reflectively to do dry-run transformations outside of the application codebase. This allows the ATL developers to test each change in isolation.
- ATL is a mapping language, which applies its mapping rules top-down for each model element it can find, and uses a two-pass compiler approach to “weave” the elements generated by different rules back together. This frees the programmer from dealing with model traversal and rule execution order: this is taken care of by ATL. Instead, the programmer can focus on defining the mapping between specific input and output elements. To give you an idea, here’s what the ATL rule for mapping lab results looks like.
- Helpers can be used to encapsulate complex navigations. Helper attributes also improve performance, because their values are cached. They function as a sort of query index over the input data.
- The references LabResultEntry instances from the LabResult rule are transformed here. This rule takes care of title entries...
- ...and this rule takes care of the actual lab result values.
- The implicit tracing mechanism has allowed us to distribute the transformation writing over three to four developers. One of these programmers was trained in .NET instead of Java, which turned out not to be a barrier for writing ATL code. Because we also chose to separate our transformation code from our file I/O and database interaction code, another developer could work on the file I/O and database interaction in Java. Finally, the Corilus XML conversion service for SUMEHR, PMF, and GPSMF was taken care of by a separate development team at Corilus HQ, and was based on existing code. The critical path in the development pipeline was therefore made up by the ATL code, which is exactly the kind of workload that can easily be dispersed over multiple developers.