EclipseCon Europe 2012 - Big Models

  • 939 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
939
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. EclipseCon Europe 2012 Challenges in the Automotive Industry: EMF Model Repositories - Scalability vs Performance Or The “Big Model” Problem Christof Hammel Andreas Rytina Cross Divisional Group - Software, Methods and Tools1 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 2. EclipseCon Europe 2012 Outline Part I  Automotive  Domain  Software Development  Software Architecture & Use Cases  Development Environment  The “Big Model” Problem Part II  Model  Structure  Technology  CDO  Scalability & Performance  Conclusion Cross Divisional Group - Software, Methods and Tools2 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 3. EclipseCon Europe 2012 Automotive Domain Case Study: Domain & Products: Powertrain Engine Control Units Our Customer (OEM) Products:5500 employees worldwide400 projects with all OEMs Passenger Car Heavy Duty20 Mio ECUs / year Off Highway Cross Divisional Group - Software, Methods and Tools3 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 4. EclipseCon Europe 2012 Our Embedded Product-Software 500.000 LOC ~500Mio LOC (> than 1st Space-Shuttle)  Product-Line development with high variability  1.500 variation points (pre compile) / “Meta” project  Automotive SW Component-Model (e.g. BlueArX, Autosar) .xml .c .xml .c  Interface consistency mainly determined by XML Component-Model data  Interfaces by Name-Refs (no class-path)   ~ 1.000 components (= PlugIn) / project Cross Divisional Group - Software, Methods and Tools4 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 5. EclipseCon Europe 2012 Desktop Software Domain PlugIn Development Project Integration (ww) Distributed teams develop independent Integration A B C A B C Target Target Target Target PlugIn-Teams use same target Cross Divisional Group - Software, Methods and Tools5 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 6. EclipseCon Europe 2012 Automotive Software Domain Component Development Project Integration 2.0.0 C 2.0.0 1.0.0 C C 2.0.0 Component Selection B B B A 1.0.0 2.0.0 1.0.0 2.0.0 A 1.0.0 A Cross Divisional Group - Software, Methods and Tools6 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 7. EclipseCon Europe 2012 Our Software Architecture (Powertrain) Cross Divisional Group - Software, Methods and Tools7 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 8. EclipseCon Europe 2012 Our Use Cases Use-Case 1: Validation  Check interface consistency across complete project: < 1min ~30 Interfaces ... ... ... ... 1.000 components ( PlugIns)  30.000 Traverse, Search, Check & Logging operations < 1min  Fast data cache  Complete project data in memory as Object-Model (Ecore) Use-Case 2: Diff/Merge  Updates require comparison to Reference/Predecessor-Proj.  Multiple projects loaded in IDE Cross Divisional Group - Software, Methods and Tools8 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 9. EclipseCon Europe 2012 Our Development Environment IDE Applications / Use-Cases: • View, Navigate, Search, Edit • Build (Processing) • Validation • Diff/Merge BDOM-Model (EMF based) Loader / Serializer Workspace Version-Control, SCM File-Server Meta-DB Comp-Baselines Cross Divisional Group - Software, Methods and Tools9 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 10. EclipseCon Europe 2012 Our BDOM-Model  Based on EMF  A few numbers:  900 entities (meta classes)  2.800 attributes and  2.700 references  ~ 800MB heap consumption / project  Windows XP (32Bit):  Max. 4GB address space  2GB User-Space  1GB for other tools (Outlook,...)  1GB heap size remaining for: Java classes, EMF domain model  Only 1 medium size project can be loaded into IDE  Big or multiple projects exceeds 32Bit Java systems  The Big Model Problem !!! Cross Divisional Group - Software, Methods and Tools10 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 11. EclipseCon Europe 2012 Model Structure – Variant Management extends PL<CLASS> P<CLASS> implements PLIConditional instance of instance of +sysCond: PLExpression Product Models a : P.. Product Line Model  The BlueArX Meta Model is seperated into a Product Line and a : PL.. b : P.. a Product specific part b : PL.. c : PL.. d : P..  Each meta class which is System configurable in the product line Condition a : P.. has a +sysCond attribute d : PL.. configuration b : P.. c : P.. Cross Divisional Group - Software, Methods and Tools11 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 12. EclipseCon Europe 2012 Model Structure – Name Based References <source> <target>NameRef <target> * 1 1 *  Handle incomplete data structures  Easy loading, adding and deleting  Deleting data structures from model without damaging the model consistency Conditional Name Based References Cond<target>NameRef <source> <target>NameRef <target> +sysCond: PLExpression 1 * 1 * 1  Makes it possible to relocate complete sub trees in the architecture tree Cross Divisional Group - Software, Methods and Tools12 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 13. EclipseCon Europe 2012 Premises and Technology Selection  Large number of applications have been implemented on top of BDOM  API must be stable (protection of investiment, quality)  Premise: Solution must not disrupt existing applications  Technology Selection:  Teneo: Relational DB. Not selected because SW-Engineering data is a graph model  Mongo/EMF  EMFStore  CDO : At the time of evaluation, CDO was most visible in the Eclipse community. Cross Divisional Group - Software, Methods and Tools13 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 14. EclipseCon Europe 2012 Solution with CDO – the components Net4j JVMConnector Client Server TxPool RafStore ReadOnlyView +Accessor *.arxml *pavast.xml Builder Loader Random BDOM Access File Cache Processor Cross Divisional Group - Software, Methods and Tools14 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 15. EclipseCon Europe 2012 Solution – write complete model to store Net4j JVMConnector Client Server TxPool RafStore *.arxml *pavast.xml ReadOnlyView +Accessor … Builder 2. Directly fill random access file Loader Random Loader Access File Cache Processor Cross Divisional Group - Software, Methods and Tools15 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 16. EclipseCon Europe 2012 Solution – query a model instance Net4j JVMConnector • View • Validation Client Server • Editing • Processing TxPool Existing API RafStore ReadOnlyView +Accessor *.arxml *pavast.xml 3. get raf pointer from index 4. load element from raf Builder Loader Random Loader Access File Cache Processor Cross Divisional Group - Software, Methods and Tools16 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 17. EclipseCon Europe 2012 Solution – edit and save a model instance Net4j • View JVMConnector • Validation Client Server • Editing 10. generate raf pointer 11. insert raf pointer at the index end • Processing 12. write new & changed elements Existing API TxPool RafStore ReadOnlyView +Accessor 6. serialize *.arxml *pavast.xml 3. get raf pointer from index Builder 4. load element from raf Loader Random Loader Access File Cache Processor Cross Divisional Group - Software, Methods and Tools17 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 18. EclipseCon Europe 2012 Reduce memory consumption  The memory consumption can be reduced after the the model building step when the model is written to the database. But due to the cdo overhead the model building step consumes about 50% more memory.  Building a big model… (vmargs: -Xmx3g -XX:MaxPermSize=256m) …w/o CDO …w/ CDO building writing to store Cross Divisional Group - Software, Methods and Tools18 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 19. EclipseCon Europe 2012 Don´t lose too much performance...  This couldn„t be achived for very query intensive use cases, i.e. model validations, content assist,…. Some measurements with EMF Validation Framework based constraints: (64bit JVM, vmargs: -Xmx3g -XX:MaxPermSize=256m) Use-Case w/o CDO w/ CDO BlueArX validation with Validation time: 2 min Validation time: 12 min a small model ~ factor 6 performance decrease Heap-Peak: 450MB Heap-Peak: 742MB BlueArX validation with Validation time 1:30 min Validation time: 23 min a medium size model ~ factor 15 performance decrease Heap-Peak: 1.1GB Heap-Peak: 2.2GB BlueArX validation with Validation time: 2 min n.a. a big model Heap-Peak: 1.9GB OutOfMemoryError with –Xmx3g  (~ 4GB estimated) Cross Divisional Group - Software, Methods and Tools19 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
  • 20. EclipseCon Europe 2012 Conclusion  Other Design Options:  Manually Optimizing all queries  Replace EMF by Memory-Mapped-Files  3- or 2-tier architecture, using powerful servers for model queries  High effort (at least 1-2 years)  Developer‟s PC Infrastructure:  32Bit are gradually going extinct  64Bit/Win7 will be rolled-out at the time other design options would be implemented.  No benefit in investing in a different tool architecture  Efforts to be invested to improve speed performance:  E.g. EMF Indexing  Evaluate / Define standard solution for Automotive (e.g. Inc-Query)  Coordination in the Eclipse Auto-IWG Cross Divisional Group - Software, Methods and Tools20 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.