Xtext beyond the defaults - how to tackle performance problems
Report
Holger SchillFollow
Oct. 28, 2016•0 likes•2,349 views
Xtext beyond the defaults - how to tackle performance problems
Oct. 28, 2016•0 likes•2,349 views
Download to read offline
Report
Engineering
Nearly every Eclipse user is aware what Xtext is and what it is useful for. It is always a pleasure to see in which kind of areas it is used. One reason is clearly, that it is really easy to build a first working example in no time. The grammar alone is sufficient and the rest is done by a code generator. Xtext will generate stub classes, that are ready to fill in individual logic for different areas. By doing that most of the things just work, because Xtext comes with a lot of defaults that suits in most of the cases. But as projects get bigger and the amount of models grow, the defaults might not scale and you might need to tweak your language to make working smooth again. Here it becomes more complicated and you need to understand what’s going on under the covers. In this session I will go through typical problems and solutions, that we have seen in customer projects. Sometimes it’s a tradeoff to tackle those problems, but it’s important to understand the implications. For some typical examples I will go into details and explain what’s going on under the covers. After the session attendees should have an idea how to tweak their own languages and why it might be worth the effort to migrate to a newer version of Xtext.
Holger SchillFollow
Recommended
Using Xcore with XtextHolger Schill
Kafka to the Maxka - (Kafka Performance Tuning)DataWorks Summit
Terraform Modules RestructuredDoiT International
The Pushdown of Everything by Stephan Kessler and Santiago MolaSpark Summit
Hadoop benchmark: Evaluating Cloudera, Hortonworks, and MapRDouglas Bernardini
Cost-based Query Optimization in Apache Phoenix using Apache CalciteJulian Hyde
More Related Content
What's hot
Building Data Intensive Analytic Application on Top of Delta LakesDatabricks
Running Kafka as a Native Binary Using GraalVM with Ozan GünalpHostedbyConfluent
엘라스틱서치, 로그스태시, 키바나종민 김
Extending the Xbase TypesystemSebastian Zarnekow
Multiple Flat Files(CSV) to Target Table in ODI12c(12.2.1.0.0)Darshankumar Prajapati
AWSKRUG DS - 데이터 엔지니어가 실무에서 맞닥뜨리는 문제들Woong Seok Kang
Viewers also liked
From Stairway to Heaven onto the Highway to Hell with XtextKarsten Thoms
Serializing EMF models with Xtextmeysholdt
Building Your Own DSL with XtextGlobalLogic Ukraine
Code Generation idioms with XtendHolger Schill
Introduction to XbaseHolger Schill
The Xtext Grammar LanguageDr. Jan Köhnlein
Similar to Xtext beyond the defaults - how to tackle performance problems
Language Server Protocol - Why the Hype?mikaelbarbero
Introduction to MongoDBJustin Smestad
Streams of information - Chicago crystal language monthly meetupBrian Cardiff
Write code that writes code! A beginner's guide to Annotation Processing - Ja...DroidConTLV
Write code that writes code!Jason Feinstein
Writing a REST Interconnection Library in SwiftPablo Villar
Recently uploaded
NCL PREVIOUS YEAR QUESTIONS.pdfmining novel
Unit 4 Food packaging materials and testing.pptxDr P DINESHKUMAR
Work Pattern Analysis with and without Site-specific Information in a Manufac...Kurata Takeshi
DBMSKaranSingh274675
Bricks.pptxSubhamSharma20947
UNIT III PRINCIPLES OF ILLUMINATION karthi keyan
Xtext beyond the defaults - how to tackle performance problems
- 1. Xtext beyond the defaults How to tackle performance problems
- 2. Who is that guy?
- 7. Xtext defaults are a good start! But one size does not fit all!
- 8. Xtext defaults are a good start! But one size does not fit all! Amount of DSLs increases
- 9. Xtext defaults are a good start! But one size does not fit all! Amount of DSLs increases Files get bigger
- 10. Xtext defaults are a good start! But one size does not fit all! Amount of DSLs increases Files get bigger Amount of files increases
- 11. Xtext defaults are a good start! But one size does not fit all! Amount of DSLs increases Files get bigger Amount of files increases Many cross references
- 12. Xtext defaults are a good start! But one size does not fit all! Amount of DSLs increases Files get bigger Amount of files increases Many cross references Maybe transitive cross references…
- 13. Look what we have build!
- 14. Look what we have build! More than 20 DSLs…
- 15. Look what we have build! More than 20 DSLs… and 200.000 lines in one file…
- 16. Look what we have build! More than 20 DSLs… and 200.000 lines in one file… and we can check in broken models…
- 17. Look what we have build! More than 20 DSLs… and 200.000 lines in one file… and we can check in broken models… and we generate the missing stuff…
- 18. Look what we have build! More than 20 DSLs… and 200.000 lines in one file… and we can check in broken models… and we generate the missing stuff… to make them valid again!
- 19. Look what we have build! More than 20 DSLs… and 200.000 lines in one file… We are on Xtext 2.6.2 and can’t update! and we can check in broken models… and we generate the missing stuff… to make them valid again!
- 20. Look what we have build! More than 20 DSLs… and 200.000 lines in one file… We are on Xtext 2.6.2 and can’t update! and we can check in broken models… and we generate the missing stuff… to make them valid again! Could you please back-port the changes?
- 21. Everything is sooo slow!!! Xtext sucks!!!
- 22. Finally think about improving the situation!
- 23. Update to newest Xtext Version 2.10.x
- 24. Update to newest Xtext Version 2.10.x Xtext 2.11 released early 2017
- 25. Why should I update???
- 27. Because…we killed bugs… Let workarounds not last forever!
- 28. …Speed…
- 30. …new features…
- 33. …let’s work together. It was never easier to contribute! We are on G ithub :-D
- 35. Most problems happen in… …Index and scoping… …Memory consumption. Validations… … Complex generators…
- 36. I need Xbase in every language…
- 37. I need Xbase in every language… … because my language is complicated…
- 38. I need Xbase in every language… … because my language is complicated… … no I can’t explain why…
- 39. I need Xbase in every language… … because my language is complicated… … no I can’t explain why… … but there is this small corner case…
- 40. Is a FAST validation really fast…
- 41. Is a FAST validation really fast… … it’s triggered every time I stop typing
- 42. Scoping is sooo complicated…
- 43. Scoping is sooo complicated… … I heard that in nearly every project…
- 44. Scoping is sooo complicated… … I heard that in nearly every project… …and people stop improving things when it work’s for the first time…
- 45. Scoping is sooo complicated… … I heard that in nearly every project… …and people stop improving things when it work’s for the first time… … code is not touched any more because who knows…
- 46. Scoping is sooo complicated… … I heard that in nearly every project… …and people stop improving things when it work’s for the first time… … code is not touched any more because who knows… … it worked before you touched it!
- 47. What’s the problem? LocalScope GlobalScope ImportScopeTypeScope Caches everywhere The Index Lazylinking EcoreUtils.resolveAll should do the trick! DerivedState ContainerState BatchLinkableResource
- 48. Scoping API scope_Element_feature(Element element ,EReference ref)
- 49. Scoping API scope_Element_feature(Element element ,EReference ref)
- 50. Scoping API scope_Element_feature(Element element ,EReference ref)
- 51. Scoping API scope_Element_feature(Element element ,EReference ref)
- 52. Scoping API scope_Element_feature(Element element ,EReference ref) Called by the PolymorphicDispatcher…
- 53. Scoping API scope_Element_feature(Element element ,EReference ref) Called by the PolymorphicDispatcher… …in a reflective way…
- 54. Scoping API scope_Element_feature(Element element ,EReference ref) Called by the PolymorphicDispatcher… and if grammar changed it might not get called any more… …in a reflective way…
- 55. Scoping API scope_Element_feature(Element element ,EReference ref) Called by the PolymorphicDispatcher… and if grammar changed it might not get called any more… …in a reflective way… …BECAUSE IT’S SLOW!
- 56. Scoping API scope_Element_feature(Element element ,EReference ref) Called by the PolymorphicDispatcher… and if grammar changed it might not get called any more… …in a reflective way… …BECAUSE IT’S SLOW! Implement getScope(EObject context, EReference ref) if(context instanceof Element && ref == MyPackage.Literals.ELEMENT_FEATURE) up to 10 %
- 57. Scoping API scope_Element_feature(Element element ,EReference ref) Called by the PolymorphicDispatcher… and if grammar changed it might not get called any more… …in a reflective way… …BECAUSE IT’S SLOW! Implement getScope(EObject context, EReference ref) if(context instanceof Element && ref == MyPackage.Literals.ELEMENT_FEATURE) up to 10 %Will not work with Xbase anyway…
- 58. Use caches in the ScopeProvider
- 59. Use caches in the ScopeProvider For each cross reference the ScopeProvider gets asked…
- 60. Use caches in the ScopeProvider …and it calculates the very same stuff over and over again… For each cross reference the ScopeProvider gets asked…
- 61. Use caches in the ScopeProvider …and it calculates the very same stuff over and over again… For each cross reference the ScopeProvider gets asked… …cache the calculated stuff and invalidate it when necessary!
- 62. Use caches in the ScopeProvider …and it calculates the very same stuff over and over again… For each cross reference the ScopeProvider gets asked… …cache the calculated stuff and invalidate it when necessary! The cache of the GlobalScope should be the index and nothing else!
- 63. Use the force of the Index
- 64. Use the force of the Index Holds all lightweight representations of referenceable elements and their resources
- 65. Use the force of the Index Holds all lightweight representations of referenceable elements and their resources It’s a HashMap… It’s super fast since Xtext 2.x and improved further
- 66. Holds all lightweight representations of referenceable elements and their resources Use the force of the Index It’s a HashMap… It’s super fast since Xtext 2.x and improved further It’s YOUR Index - you can put data in…!
- 67. Index - how to feed it
- 68. Index - how to feed it 1. Builder picks up DSL files and loads them one by one
- 69. Index - how to feed it 1. Builder picks up DSL files and loads them one by one 2. Creates ResourceDescriptions through ResourceDescriptionManager
- 70. Index - how to feed it 1. Builder picks up DSL files and loads them one by one 2. Creates ResourceDescriptions through ResourceDescriptionManager 3. ResourceDescriptionManager calls ResourceDescriptionsStrategy to create EObjectDescriptions
- 71. Index - how to feed it 1. Builder picks up DSL files and loads them one by one 2. Creates ResourceDescriptions through ResourceDescriptionManager 3. ResourceDescriptionManager calls ResourceDescriptionsStrategy to create EObjectDescriptions You have to bind a custom impl for the ResourceDescriptionsStrategy in the RuntimeModule
- 72. Index - how to feed it 1. Builder picks up DSL files and loads them one by one 2. Creates ResourceDescriptions through ResourceDescriptionManager 3. ResourceDescriptionManager calls ResourceDescriptionsStrategy to create EObjectDescriptions You have to bind a custom impl for the ResourceDescriptionsStrategy in the RuntimeModule The default creates a EObjectDescription for EVERY Element that has a name…
- 73. IEObjectDescription create(String simpleName, EObject element, Map<String, String> userData) Information hiding and userData
- 74. IEObjectDescription create(String simpleName, EObject element, Map<String, String> userData) Information hiding and userData
- 75. IEObjectDescription create(String simpleName, EObject element, Map<String, String> userData) Information hiding and userData
- 76. IEObjectDescription create(String simpleName, EObject element, Map<String, String> userData) Information hiding and userData Save memory & time
- 77. UserData - what for?
- 78. UserData - what for? Validation Don’t load the world to get informations - put them in the index
- 79. UserData - what for? Validation Don’t load the world to get informations - put them in the index Scoping Don’t resolve proxies for simple informations
- 80. Reference non Xtext models
- 81. Reference non Xtext models We have our own GlobalScopeProvider that does the trick…
- 82. Reference non Xtext models We have our own GlobalScopeProvider that does the trick… …we load everything and the GlobalScope contains the corresponding EObjectDescriptions…
- 83. Reference non Xtext models We have our own GlobalScopeProvider that does the trick… …we load everything and the GlobalScope contains the corresponding EObjectDescriptions… …but it’s slow and we cannot navigate those elements in the editor… You can do better…
- 84. Reuse the Xtext infrastructure
- 85. Reuse the Xtext infrastructure Bind a ResourceServiceProvider… …so that you can put the elements in the index and make them navigable!
- 86. Reuse the Xtext infrastructure Bind a ResourceServiceProvider… …so that you can put the elements in the index and make them navigable! GenericResourceServiceProvider Use It will put everything that has a name in the index… … introduce your own ResourceDescriptionStrategy!
- 87. But the builder picks them up… and it takes ages!
- 88. Put the index in the jar
- 89. Put the index in the jar Why calculating the index of a static set of files?
- 90. Put the index in the jar The index is serialisable… …put it in the jar and load it. Why calculating the index of a static set of files?
- 91. Index in Standalone It’s sooo slow…
- 92. Index in Standalone It’s sooo slow… …of course I have my custom Standalone impl…
- 93. Index in Standalone ResourceSetGlobalScopeProvider Long time the was the default.
- 94. Index in Standalone ResourceSetGlobalScopeProvider Long time the was the default. Today we can do better but you need to use it and know that there is something new… For Maven- and Gradleplugin it’s done.
- 95. Index in Standalone Fill the index and install it on the ResourceSet! Load each resource and ask the ResourceDescriptionManager to create a ResourceDescription…
- 96. Index in Standalone Fill the index and install it on the ResourceSet! Load each resource and ask the ResourceDescriptionManager to create a ResourceDescription… Install it on the ResourceSet
- 97. Make big things run again and save your ass! Up to 80% more faster in 10 minutes!
- 98. Codegenerators They are sooo slow…
- 99. Codegenerators They are sooo slow… … of course I am using mwe2 to invoke it…
- 100. Codegenerators They are sooo slow… … of course I am using mwe2 to invoke it… … even in the IDE!
- 102. Incremental generators The builder triggers the generator for each resource…
- 103. Incremental generators The builder triggers the generator for each resource… …when a file changes the builder computes the impact and retriggers the generator for each affected file!
- 104. Incremental generators The builder triggers the generator for each resource… …when a file changes the builder computes the impact and retriggers the generator for each affected file! What about m:n scenarios? Solveable with a little effort - but it is worth the work!
- 105. Incremental generators The builder triggers the generator for each resource… …when a file changes the builder computes the impact and retriggers the generator for each affected file! What about m:n scenarios? Solveable with a little effort - but it is worth the work! There is a ParallelBuilderParticipant… …you have to bind it for your language… … make sure all proxies are already resolved!
- 106. Memory consumption
- 107. Memory consumption DisabledClusteringPolicy Do not unload resource…
- 108. Memory consumption DisabledClusteringPolicy DynamicResourceClusteringPolicy Do not unload resource… Unload resources when there is no free memory… …can be configured.
- 109. The NodeModel is huge Stores the syntax tree with text and positions…
- 110. The NodeModel is huge Stores the syntax tree with text and positions… …is used to get the position of a specific element or feature…
- 111. The NodeModel is huge Stores the syntax tree with text and positions… …is used to get the position of a specific element or feature… …produces a lot of Strings in memory…
- 112. The NodeModel is huge Stores the syntax tree with text and positions… …is used to get the position of a specific element or feature… …produces a lot of Strings in memory… …and during the builder runs you do not really need it.
- 113. Switch off the NodeModel
- 114. Switch off the NodeModel The parser creates it …
- 115. Switch off the NodeModel The parser creates it … …it is used to install proxies - so you have to do that differently…
- 116. Switch off the NodeModel The parser creates it … …it is used to install proxies - so you have to do that differently… You want to have it when the editor is used…
- 117. Switch off the NodeModel The parser creates it … …it is used to install proxies - so you have to do that differently… You want to have it when the editor is used… … the load options should be an indicator…
- 118. Switch off the NodeModel The parser creates it … …it is used to install proxies - so you have to do that differently… You want to have it when the editor is used… … the load options should be an indicator… …what are the implications?
- 119. Implications You cannot use the NodeModel in validations and scoping any more…
- 120. Implications You cannot use the NodeModel in validations and scoping any more… …Issues are marked at position 0,0… …an EditorCallback should revalidate the content.
- 121. Implications You cannot use the NodeModel in validations and scoping any more… …Issues are marked at position 0,0… …an EditorCallback should revalidate the content. THIS IS NOT A COMMON THING! DO IT ONLY WHEN IT IS REALLY NECESSARY!
- 122. Order makes a difference
- 123. Order makes a difference Files might belong together…
- 124. Order makes a difference Files might belong together… …and might reference each other.
- 125. Order makes a difference Files might belong together… …and might reference each other. Putting them in an logical order for the builder…
- 126. Order makes a difference Files might belong together… …and might reference each other. Putting them in an logical order for the builder… … and they do not have be loaded more than once.
- 127. Order makes a difference Files might belong together… …and might reference each other. Putting them in an logical order for the builder… … and they do not have be loaded more than once. Sometimes it makes sense to not unload them at all.
- 128. Now you know!
- 129. Professional Support