EMF-­‐INCQUERY:                                                      Efficient	  Model	  Queries	  for	                     ...
Overview§   IntroducAon§   EMF-­‐INCQUERY	  overview§   Performance	  benchmarking§   Roadmap	  and	  ideas	  for	  th...
A	  key	  problem	  of	  MDE§ Scalability	  vs.	  modeling	  tools    o Issues	  encountered	  by	  several	  industrial	...
Focus:	  model	  queries§ Model	  queries:	     “a	  piece	  of	  code	  that	  retrieves	  a	  given	  set	  of	  the	  ...
EMF-­‐INCQUERY	  Overview§ A	  model	  query	  engine	  for	  batch	  and	  live	  queriesLanguage                       ...
Benefits	  of	  EMF-­‐INCQUERY§ Makes	  on-­‐the-­‐fly	  well-­‐formedness	  validaAon,	  view	     maintenance,	  …	  feas...
Architecture                              Input	  =	  Model	  contents	  EMF	  instance	             +	  EMF	  noAficaAons ...
Architecture                                          Input	  =	  Model	  contents	             EMF	  instance	           ...
IncQuery	  Base	  Indexers§ Stand-­‐alone	  core	  library	  with	  minimal	  dependencies§ Provides	  high	  performanc...
DEMO INCQUERY	  PaOern	  Language§ School	  metamodel	  in	  EMF
INCQUERY	  PaOern	  Language
INCQUERY	  PaOern	  Language       Expressive	  declaraAve	  query	  language	         by	  graph	  paOerns          Captu...
Feature	  highlights§ State-­‐of-­‐the-­‐art	  development	  tools	  based	  on	  Xtext2     o Content	  assist	  (code	 ...
INCQUERY	  Development	  Tools
INCQUERY	  Development	  Tools    PaOern	  Editor                              Query	  Explorer
INCQUERY	  Development	  Tools    PaOern	  Editor                              Query	  Explorer
INCQUERY	  Development	  Tools                                          • Works	  with	  most	  EMF-­‐                    ...
IntegraAon	  opAons§ IncQuery	  Java	  API   o Generic	  (on-­‐the-­‐fly	  queries)   o Generated	  code	  (opLmized	  for...
IncQuery	  Databinding1.	  Model	                             3.a	  ResultModificaAon                                      ...
Work	  in	  progress:	  IncQuery	  Viewers
Work	  in	  progress:	  IncQuery	  Viewers             1.	  Define	  “edges”	               and	  “nodes”	  of	  an	       ...
Work	  in	  progress:	  IncQuery	  Viewers                                                   2.	  Create	                 ...
PERFORMANCE	  BENCHMARK     Instantaneous	  result	  retrieval?
Benchmarking	  scenario§ In-­‐memory	  models   o Embedded	  railways	  control	  sodware	  system	  design	  domain	    ...
Batch	  operaAons
Batch	  operaAonsNeo4j	  scales	  somewhat	   beOer	  than	  top	  EMF	              tools                                ...
Batch	  operaAonsNeo4j	  scales	  somewhat	   beOer	  than	  top	  EMF	              tools                                ...
Real	  strength:	  on-­‐the-­‐fly	  model	  processing
Real	  strength:	  on-­‐the-­‐fly	  model	  processing                                   • IncQuery	  provides	  near	     ...
Real	  strength:	  on-­‐the-­‐fly	  model	  processing                                       • IncQuery	  provides	  near	 ...
Memory	  usage§ Depends	  on	  “query	  complexity”	  and	  model	  size/   contents§ For	  typical	  query/model	  comb...
Known	  applicaAons§ Well-­‐formedness	  validaAon	  of	  very	  large,	  industrial	     AUTOSAR	  models	  (AutomoAve	 ...
Development	  agenda§ Our	  current	  plan   o IncQuery	  Viewers   o Dynamic	  EMF	  support   o Xcore	  integraAon	  (q...
Other	  ideas	  on	  the	  table§ ParLal	  support	  for	  OCL§ Dedicated	  support	  for	  Complex	  Event	  Processing...
Final	  points§ EMF-­‐INCQUERY	  has	  been	  successfully	  migrated	  to	     Eclipse.org	  hUp://eclipse.org/incquery§...
Acknowledgements§ IncQuery	  Development	  Team    o Gábor	  Bergmann,	  Ábel	  Hegedüs,	  Ákos	  Horváth,	  András	  Ökr...
Upcoming SlideShare
Loading in...5

EMF-IncQuery 0.7 Presentation for Itemis


Published on

Presentation on the brand new EMF-IncQuery 0.7, for Itemis.

Published in: Technology
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

EMF-IncQuery 0.7 Presentation for Itemis

  1. 1. EMF-­‐INCQUERY: Efficient  Model  Queries  for   EMF  Tools   Tamás  Szabó,  István  Ráth  Budapest  University  of  Technology  and  Economics IncQuery  Labs  Ltd.Fault  Tolerant  Systems  Research  Group
  2. 2. Overview§ IntroducAon§ EMF-­‐INCQUERY  overview§ Performance  benchmarking§ Roadmap  and  ideas  for  the  future
  3. 3. A  key  problem  of  MDE§ Scalability  vs.  modeling  tools o Issues  encountered  by  several  industrial  partners  using   tools  based  on  the  Eclipse  Modeling  Framework  (EMF) o Modeling  scenarios  can  get  really  complex  really  quickly • Instance  models  of  size  1-­‐2M  and  beyond • Performance  issues  with  model  transformaLons  and  code   generators  have  an  adverse  effect  on  everyday  development  tasks§ Scalability? Recognized  e.g.  by  AUTOSAR  tool  vendors hOp://wiki.eclipse.org/Auto_IWG_WP2 o Complex (meta)models o Large  instance  models o Complex  query  and  manipulaLon  scenarios
  4. 4. Focus:  model  queries§ Model  queries:   “a  piece  of  code  that  retrieves  a  given  set  of  the  model”§ Queries  are  at  the  heart  of  MDE o Every  model  access/read  is  a  (simple)  query o More  complex:  Views,  content  providers o Most  complex:  Model  transformaAons,  code  generators,  …§ Query  performance  (=  the  speed  of  content  retrieval)  is  crucial o First  vs.  consecuAve  vs.  throughput o Query  result  vs.  query  contents§ Core  problems 1. Slow  query  engines   2. Considerable  programming  effort  necessary  to  get  complex   queries  right
  5. 5. EMF-­‐INCQUERY
  6. 6. EMF-­‐INCQUERY  Overview§ A  model  query  engine  for  batch  and  live  queriesLanguage ExecuAon Tools• Expressive   • Incremental • On-­‐the-­‐fly  well-­‐graph  paOern-­‐based   evaluaAon  based  on  the   formedness  validaAonquery  language Rete  algorithm • Incremental  model  • Focus  on   • Instantaneous   processing  reusability:  query   response  for  complex   (synchronizaAon,  ...)libraries,  recursive   queries  over  large   • View  maintenancepaOern  calls models • ...§ Origins o 10+  years  of  R&D  experience  in  model  queries  &  transformaAons  
  7. 7. Benefits  of  EMF-­‐INCQUERY§ Makes  on-­‐the-­‐fly  well-­‐formedness  validaAon,  view   maintenance,  …  feasible  over  really  large  instance   models§ Simplifies  wriAng  really  complex  queries o Graph  paOern  language o Highly  reusable  à query  libraries§ Easy-­‐to-­‐integrate  into  exisAng  apps o Works  with  any  EMF  domain  metamodel o Integrates  through  Eclipse  standards  à works  with   many  EMF-­‐based  apps  out-­‐of-­‐the-­‐box
  8. 8. Architecture Input  =  Model  contents  EMF  instance   +  EMF  noAficaAons Rete  network EMF-­‐INCQUERY  Base   Input   Intermedia Output Change   PaOern  matcher   Query  engine Output  =  Query  results  +  Query  result  deltas (con<nuously  updated  as  the  model  changes)
  9. 9. Architecture Input  =  Model  contents   EMF  instance   +  EMF  noAficaAonsModel  ModificaAon Rete  network EMF-­‐INCQUERY  Base   Input   Intermedia Output Change   PaOern  matcher   Query  engine Output  =  Query  results  +  Query  result  deltas (con<nuously  updated  as  the  model  changes)
  10. 10. IncQuery  Base  Indexers§ Stand-­‐alone  core  library  with  minimal  dependencies§ Provides  high  performance  indexers o Typeàall  instances o EdgeàSource,Target o NodeàIncoming/Outgoing  edges o AUribute  valueàhost§ Supports  Incremental  TransiLve  Closure o Efficiently  compute  &  maintain  reachability  regions  along  EReference   navigaLon  paths§ Indexer  iniLalizaLon o According  to  visitor  strategy o Can  be • Generic  (all  known  Ecore  types  are  indexed)  [“wildcard  mode”]   • Or  filtered  according  to  a  set  of  types  [memory  opLmizaLon  for  generated   query  code]
  11. 11. DEMO INCQUERY  PaOern  Language§ School  metamodel  in  EMF
  12. 12. INCQUERY  PaOern  Language
  13. 13. INCQUERY  PaOern  Language Expressive  declaraAve  query  language   by  graph  paOerns Capture  local  +  global  queries ComposiAonality  +  Reusabilility   Recursion,  NegaAon
  14. 14. Feature  highlights§ State-­‐of-­‐the-­‐art  development  tools  based  on  Xtext2 o Content  assist  (code  compleAon),  validaAon,  outline,  … o Advanced  (syntacAc  and  semanAc)  validaAon§ Unique  query  language  features o Recursive  queries  and  transiAve  closure o Inference-­‐supported  type  system  and  expressions  based  on  Xbase  and  Ecore o PaOern  composiAon  and  reuse§ RunLme o Build  and  execute  queries  on-­‐the-­‐fly o Strong  generaAve  bindings  to  Java§ Extension  APIs o Query-­‐based  derived  features  with  noAficaAons o ValidaAon  Engine  for  efficient  on-­‐the-­‐fly  validaAon o Event-­‐driven  Virtual  Machine  for  transformaAons
  15. 15. INCQUERY  Development  Tools
  16. 16. INCQUERY  Development  Tools PaOern  Editor Query  Explorer
  17. 17. INCQUERY  Development  Tools PaOern  Editor Query  Explorer
  18. 18. INCQUERY  Development  Tools • Works  with  most  EMF-­‐ based  editors  out-­‐of-­‐ the-­‐box • Reveals  matches  as   selecAon PaOern  EditorQueries  are  applied  &  updates  on-­‐the-­‐fly Query  Explorer
  19. 19. IntegraAon  opAons§ IncQuery  Java  API o Generic  (on-­‐the-­‐fly  queries) o Generated  code  (opLmized  for  performance)§ Query-­‐based  derived  features o Edges  and  aUribute  values  computed  and  maintained  by   IncQuery o Transparent  integraLon  through  genmodeled  EMF o Unique  feature:  Efficient  and  automated  noLficaLon   support§ Add-­‐on  APIs o IncQuery  ValidaLon  (Demo) o IncQuery  Databinding
  20. 20. IncQuery  Databinding1.  Model   3.a  ResultModificaAon 4.a  Add/Remove  items Update Query (list,  table,  tree…) Match  Set UI  Model Query Component 2.  Change NoAficaAon Match 3.b  Match 4.b  Update  Items (Property) (list,  table,  tree  item, Update  label,  …) JFace  Databinding
  21. 21. Work  in  progress:  IncQuery  Viewers
  22. 22. Work  in  progress:  IncQuery  Viewers 1.  Define  “edges”   and  “nodes”  of  an   overlay  graph  as   queries
  23. 23. Work  in  progress:  IncQuery  Viewers 2.  Create   automaAcally   updaAng  JFace   viewers  (List,  Tree,   1.  Define  “edges”   Table,  Zest  Graph)   and  “nodes”  of  an   without  manual   overlay  graph  as   coding queries
  24. 24. PERFORMANCE  BENCHMARK Instantaneous  result  retrieval?
  25. 25. Benchmarking  scenario§ In-­‐memory  models o Embedded  railways  control  sodware  system  design  domain   (complex  queries) o Instance  models  up  to  2.7M  EObjects  and  11.1M   EReferences§ Scenario:  model  transformaLon  combined  with  well-­‐ formedness  validaLon o Batch:  loading  models  +  execuLng  queries o On-­‐the-­‐fly:  transforming  models  +  re-­‐execuLng  queries   instantaneously§ Tools o Eclipse  tools  (OCL,  OCL  Impact  Analyzer,  INCQUERY,  EMF-­‐Java   manually  opLmized) o Neo4j  in-­‐memory  graph  database  (for  comparison)
  26. 26. Batch  operaAons
  27. 27. Batch  operaAonsNeo4j  scales  somewhat   beOer  than  top  EMF   tools …  unless  you  augment   EMF  with  INCQUERY
  28. 28. Batch  operaAonsNeo4j  scales  somewhat   beOer  than  top  EMF   tools Note  the  logarithmic   scales:  7x  faster! …  unless  you  augment   EMF  with  INCQUERY
  29. 29. Real  strength:  on-­‐the-­‐fly  model  processing
  30. 30. Real  strength:  on-­‐the-­‐fly  model  processing • IncQuery  provides  near   zero  response  Ames  for   up  to  14M  elements
  31. 31. Real  strength:  on-­‐the-­‐fly  model  processing • IncQuery  provides  near   zero  response  Ames  for   up  to  14M  elements CharacterisAc   difference
  32. 32. Memory  usage§ Depends  on  “query  complexity”  and  model  size/ contents§ For  typical  query/model  combinaAons,  memory   overhead  is  linear o (1.1-­‐2)  x  (heap  required  for  the  model  itself) o IncQuery’s  built-­‐in  semanAc  validators  and  the  Rete   Visualizer  aid  you  in  resolving  problemaAc  query/ model  combinaAons
  34. 34. Known  applicaAons§ Well-­‐formedness  validaAon  of  very  large,  industrial   AUTOSAR  models  (AutomoAve  OEM)§ Incremental  dependency  analysis  over  sotware   infrastructure  graphs  (tool  used  in  CERN)§ Queries,  views  and  transformaAons  for  model-­‐ driven  tool  development  in  the  avionics  domain   (aircrat  manufacturer)§ Model-­‐driven  refactoring  of  large  source  code   models  (CollaboraAve  naAonal  project  in  Hungary)
  35. 35. Development  agenda§ Our  current  plan o IncQuery  Viewers o Dynamic  EMF  support o Xcore  integraAon  (query-­‐based  derived  features) o Query  performance  opAmizaAon o Improve  support  for  stereotyped  UML  models§ ARTop/Sphinx  integraAon o Incremental  validaAon  using  “hybrid  queries” • Support  parAally  materialized  models
  36. 36. Other  ideas  on  the  table§ ParLal  support  for  OCL§ Dedicated  support  for  Complex  Event  Processing  over   live  EMF  models o hUp://incquery.net/incquery/demos/jnect o Think  incremental  queries  over  the  E4  Modeled  UI…§ Develop  a  high  performance  and  pragmaLc  model   transformaLon  engine o Build  the  language  on  Xbase  and  IQPL o …and  the  runLme  on  IncQuery§ Design  space  exploraLon o OpLmize  instance  models  towards  a  complex  design  goal
  37. 37. Final  points§ EMF-­‐INCQUERY  has  been  successfully  migrated  to   Eclipse.org  hUp://eclipse.org/incquery§ EMF-­‐INCQUERY  0.7  milestone  builds  are  available   immediately  from  the  Marketplace o Blog  and  other  resources:  hOp://incquery.net   o Lots  of  documentaAon:  examples,  demos,  tutorials§ Release  0.7  is  tentaAvely  scheduled  shortly  ater   the  Eclipse  Kepler  Launch  (beginning  of  July)§ Your  contribu<ons  (feedback,  forum  posts,  ideas,   bugzillas,  patches)  are  very  welcome!
  38. 38. Acknowledgements§ IncQuery  Development  Team o Gábor  Bergmann,  Ábel  Hegedüs,  Ákos  Horváth,  András  Ökrös,   Zoltán  Ujhelyi,  István  Ráth§ AddiLonal  contributors o Márk  CzoUer,  Tamás  Szabó§ Performance  benchmarking o Benedek  Izsó,  Zoltán  Szatmári§ TesLng,  examples o Asla  Csicsely,  Tamás  Csurgó,  Dániel  Kávássy,  Gábor  Szárnyas,   Tamás  Tóth,  Ádám  Dudás,  Oszkár  Semeráth,  Zoltán  Balogh§ IncQuery  Project  Lead:  István  Ráth§ Senior  advisor:  Dániel  Varró
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.