Your SlideShare is downloading. ×
Generative Software Development. Overview and Examples
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Generative Software Development. Overview and Examples

1,359

Published on

Slides from a guest lecture I gave at TU/E in October 2005.

Slides from a guest lecture I gave at TU/E in October 2005.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,359
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
22
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Generative Software Development overview and examples Eelco Visser visser@acm.org Universiteit Utrecht, Faculty of Science,Department of Information and Computing Sciences, Center for Software Technology October 27, 2005
  • 2. Generative Software Development Generative software development is a system-family approach, which focuses on automating the creation of system-family members: a given system can be automatically generated from a specification written in one or more domain-specific languages. K. Czarnecki. Overview of Generative Software Development. In J.-P. Bantre et al. (Eds.): Unconventional Programming Paradigms (UPP) 2004, Mont Saint-Michel, France, LNCS 3566, pp. 313-328, 2005
  • 3. Outline • Software system families • Commonality and variability • Domain engineering and application engineering • Feature modeling • Domain-specific languages • Examples • Generative programming • Problem space • Solution space • Mapping • Realizing generative software development • Overview of techniques • Example: assimilating embedded DSLs • Risks • Summary
  • 4. Software System Families Current research and practical experience suggest that achieving significant progress with respect to software reuse requires a paradigm shift towards modeling and developing software system families rather than individual systems. Czarnecki 2005
  • 5. Examples of Software System Families • Compilers • program representation, parsing, code generation, ... • Web applications • processing http requests, session management • E-commerce systems • products, user accounts, payment • Office applications • text processing • Web browsers • html rendering • TV software
  • 6. System Family Engineering System-family engineering (product-line engineering) seeks to exploit the commonalities among systems from a given problem domain while managing the variabilities among them in a systematic way. Czarnecki 2005
  • 7. Commonality and Variability • E-commerce systems • Commonalities: products, user accounts, payment, ... • Variabilities: types of products, shipment methods, ... • Web applications • Commonalities: processing http requests, session management • Variabilities: application workflow, page layout, ... • Web browsers • Commonalities: render HTML • Variabilities: plugins for email, chatting, ...
  • 8. Commonality and Variability • Compilers • Commonalities: program representation, parsing, code generation, ... • Variabilities: syntax and semantics of language • Office applications • Commonalities: text processing • Variabilities: spellchecking, grammarchecking, ... • TV software • Commonalities: ? • Variabilities: ?
  • 9. System Family Engineering Process system family engineering = domain engineering + application engineering reusable assets Domain engineering Application engineering Domain analysis new requirements System requirements Domain design System derivation Domain implementation System tailoring
  • 10. Domain Engineering = Development for Reuse Development of reusable assets • Reusable components, generic architectures, documentation Software engineering process for a class of systems • Domain analysis • scope of family • identify commonalities and variabilities • Domain design • common architecture • plan creation of systems from reusable assets • Domain implementation • implementation of reusable assets
  • 11. Feature Modeling A method and notation to elicit and represent common and variable features of the systems in a system family • And: all subfeatures must be selected • Alternative: only one subfeature can be selected • Or: one or more can be selected • Mandatory: features that are required • Optional: features that are optional Additional information • constraints • binding times • default attribute values, default features • stakeholders, priorities, etc.
  • 12. Feature Modeling: Example e-shop payment taxCalculation shipping creditCard debitCard electronicCheque addressResolution country postalCode streetAddress
  • 13. Feature Modeling: Alternative Notations • Feature diagrams can be expressed as context-free grammars • Feature selection is sentence generated by the grammar e-shop : payment taxCalculation [shipping] payment : paymentMethod+ paymentMethod : creditCard | debitCard | electronicCheque taxCalculation : addressResolution addressResolution : country | postalCode | streetAddres • Expressivity of feature diagrams is restricted • Features and other constraints can be expressed by means of propositional formulae See also D. Batory. Feature Models, Grammars, and Propositional Formulas. In H. Obbink and K. Pohl (Eds.) SPLC 2005. LNCS 3714, pp. 7–20, 2005
  • 14. Application Engineering = Development with Reuse Build concrete applications using reusable assets • Requirements = delta of generic system requirements • System derivation: manual or automated construction of system from reusable assets Feedback to domain engineering • Refine and adapt requirements • Tailor reusable assets to application
  • 15. Features vs Software Modules Concrete features • map to individual components Crosscutting features • can be implemented using aspect-oriented techniques • Examples: logging, synchronization Abstract features • Also known as: non-functional requirements • map to configuration of components and/or aspects Grouping features • Also known as: variation points • map to common interface and plug-compatible components
  • 16. Generative Software Development Generative software development is a system-family approach, which focuses on automating the creation of system-family members: a given system can be automatically generated from a specification written in one or more domain-specific languages. Czarnecki 2005
  • 17. General Purpose Languages Expressivity of general purpose languages is restricted to • while(cond) { x = foo(y, z); } • variable binding: storing values • function calls: computing values • control-flow: combining and choosing computations This is a rather powerful and generic approach • any computation can be expressed (Turing completeness) • modularization facilities allow further abstraction But it does not match application domains very well
  • 18. Advantages of Domain-Specific Languages • Domain-specific abstractions • pre-defined abstractions for concepts in application domain • Domain-specific concrete syntax • natural notation for a domain • Domain-specific error checking • find and report errors based on domain knowledge and notation • Domain-specific optimizations • opportunities for optimizations based on domain knowledge • Domain-specific tool support • tools can be aware of domain
  • 19. Domain-Specific Languages: Examples Successful domain-specific languages exist for many domains • Numerical computation • Database queries • Text search and analysis • Symbolic computation • Document transformation • Workflow • ...
  • 20. Domain-Specific Languages: Arithmetic x = c * y + z • Notation and semantics based on long tradition in mathematics • General purpose language notation would require x = plus(times(c, y), z) x = z.plus(y.times(c)) x = c.times(y).plus(z) • Compiler knows a lot about arithmetic • optimizations (partial evaluation, strength reduction, ...) • automatic coercions (int to real) • exceptions
  • 21. Domain-Specific Languages: A = c * B + C
  • 22. Domain-Specific Languages: Matrix Computation A = c * B + C • Now the elements of the assignment are matrices • The expression ‘means’ for i = lb to ub do A[i] = c * B[i] + C[i]
  • 23. Domain-Specific Languages: Matrix Computation A = c * B + C • Now the elements of the assignment are matrices • The expression ‘means’ for i = lb to ub do A[i] = c * B[i] + C[i] • General-purpose languages (C, C++, Java, ...) don’t understand matrices • ‘Mathematical’ languages such as Matlab and Octave do • Matrix notation allows • concise programs • compiler may apply optimizations
  • 24. Domain-Specific Languages: Database Queries SELECT NAME FROM ORDERS, INCLUDES, SUPPLIES WHERE CUST = ’Chaudron’ AND ORDERS.O# = INCLUDUES.O# AND INCLUDES.ITEM = SUPPLIES.ITEM • SQL: notation for querying relational databases • Abstracts from physical data structures of database • Based on small set of relational algebra operators • Allows efficient implementation of queries • Queries can be checked against database schema
  • 25. Domain-Specific Languages: Regular Expressions ^([a-z]+)/([A-Z][^/]+)/(.*) • A regular expression describes or matches a set of strings, according to certain syntax rules • Used by many text editors and utilities to search and manipulate bodies of text based on certain patterns. • Many programming languages support regular expressions for string manipulation. • Notation abstracts from underlying finite automata
  • 26. Domain-Specific Languages: Context-free Grammars <postal-address> ::= <name-part> <street-address> <zip-part> <personal-part> ::= <first-name> | <initial> "." <name-part> ::= <personal-part> <last-name> [<jr-part>] <EOL> | <personal-part> <name-part> <street-address> ::= [<apt>] <house-num> <street-name> <EOL> <zip-part> ::= <town-name> "," <state-code> <ZIP-code> <EOL> • Describes syntax of a formal language • Used to derive parsers that automatically recognize texts • Writing parsers manually is complex and error-prone
  • 27. Domain-Specific Languages: Rewrite Systems |[ if 0 then s1 else s2 ]| -> |[ s2 ]| |[ i + j ]| -> |[ k ]| where <add>(i,j) => k • Rewrite rule replaces a matching term with another term • Exhaustive application of rules reduces term to normal form • Applications: symbolic computation, program transformation,
  • 28. Domain-Specific Languages: Path Queries /bookstore/book[price>35]/title • XPATH path queries address parts of an XML document • used in document transformation with XSLT
  • 29. Domain-Specific Languages: Workflow
  • 30. Domain-Specific Languages: Observations • Domain-specific languages are everywhere • DSL are sucessful when they cover a well-defined domain and are based on solid foundations • DSLs typically do not cover entire systems, but rather well-defined parts/aspects of systems
  • 31. Generative Software Development Generative software development is a system-family approach, which focuses on automating the creation of system-family members: a given system can be automatically generated from a specification written in one or more domain-specific languages. Czarnecki 2005
  • 32. Generative Programming: Mapping Problems to Solutions mapping Problem space Solution space Problem Space • Set of domain specific abstractions • Used to specify desired system family member Solution Space • Implementation oriented abstractions • Can be instantiated to implement domain abstractions
  • 33. Problem Space and Solution Space: Example Problem Space • Regular expressions Solution Space • Finite automata • Expressed as transition table in code Mapping • Lexical analyzer generator (flex)
  • 34. Mapping: Configuration View Problem space • domain-specific concepts and their features • constraints: illegal feature combinations, default settings, default dependencies Specification • selection of features for a specific system Solution space • implementation components, composed to create system Mapping • feature configuration mapped to component configuration • mapping defined by construction rules and optimizations Separation of spaces allows freedom of implementation
  • 35. Configuration Example: Apache’s httpd.conf Software composition LoadModule file_cache_module modules/mod_file_cache.so LoadModule mem_cache_module modules/mod_mem_cache.so LoadModule cgi_module modules/mod_cgi.so Configuration settings <Directory /home/*/public_html> AllowOverride FileInfo AuthConfig Limit Options MultiViews Indexes SymLinksIfOwnerMatch IncludesNoExec <Limit GET POST OPTIONS> Order allow,deny Allow from all </Limit> <LimitExcept GET POST OPTIONS> Order deny,allow Deny from all </LimitExcept> </Directory> Binding-time: configuration is processed at start-up time
  • 36. Mapping: Transformational View Problem space • domain concepts captured by domain-specific language Specification • DSL program Solution space • implementation language • components/libraries (‘run-time system’) Mapping • transformation from DSL to implementation language
  • 37. Configuration vs Transformation • Two views on mappings provide different perspectives • Feature configuration defines a DSL with components as implementation language
  • 38. Combining Mappings chaining multiple problem spaces multiple solution spaces alternative problem spaces alternative solution spaces
  • 39. Combining Mappings language text search definition regexp automaton embed DFA SDF sdf2pp sdf2sig pgen codegen pp table signature SLR(0) table parse code sglr
  • 40. Realizing Generative Software Development Requires techniques/technologies for creating • Domain-specific languages • Mappings • Solution components
  • 41. Realizing Domain-Specific Languages Domain-specific languages come in many shapes and forms • Textual languages • Embedded in a programming language (MetaBorg) • Graphical languages • Wizards, GUIs
  • 42. Realizing Mappings Product line configurators • map configuration file + components to system Generators, implemented using • Template and frame processors • Transformation systems • Multi-staged programming • Program specialization • Built-in meta-programming facility Interpreters • run-time interpretation of DSL program
  • 43. Realizing Solution Space • Functions or classes • Generic components • Component models (JavaBeans, ActiveX, Corba) • Aspect-oriented programming (AspectJ)
  • 44. Combining Techniques Generative software development can be realized using various combinations of techniques Example scenario: • Textual embedded language • ‘Assimilation’ by program transformation • Library for basic functionality of target programs Example scenario: • Graphical model language • Graph transformations to map to aspect-oriented programs • Aspects ‘inject’ code from model into base application
  • 45. Relation To Other Fields Problem Space Configuration Knowledge Solution Space System Families, Product Lines Feature modeling, feature interactions Components ! Generators/Transformers ¨ Architectures Domain-specific languages © Aspect-oriented programming ¨ § Aspect-oriented modeling ¦ ¥ ¥ ¤£ ¢ Generic programming ¢ ¢ ¡   60
  • 46. Assimilation of Embedded Languages Embedding domain-specific languages • Extend language with syntax for application domain Examples • Java + Swing Userinterface Language: Expressions reflect hierarchy of Swing GUI objects • Java + RegExps: Compile-time syntactic checks of regular expressions, string rewrite rules • Java + Java: Java programs that generate Java programs Assimilation • Meld embedded DSL code with host code • Often requires local-to-global transformations
  • 47. Assimilation of Embedded Languages Embedding domain-specific languages • Extend language with syntax for application domain Concrete syntax for objects Examples syntactic abstractions for libraries (APIs) • Java + Swing Userinterface Language: Expressions reflect transformations to assimilate (and optimize) hierarchy of Swing GUI objects • Java + RegExps: Compile-time syntactic checks of regular expressions, string rewrite rules • Java + Java: Java programs that generate Java programs Assimilation • Meld embedded DSL code with host code • Often requires local-to-global transformations
  • 48. Assimilation of Embedded Languages Embedding domain-specific languages • Extend language with syntax for application domain local to global transformation Examples generation of event handler class • Java + Swing Userinterface regexp compiles caching of Language: Expressions reflect hierarchy of Swing GUI objects adding class variable declarations • Java + RegExps: Compile-time syntactic checks of regular expressions, string rewrite rules • Java + Java: Java programs that generate Java programs Assimilation • Meld embedded DSL code with host code • Often requires local-to-global transformations
  • 49. Example: Embedding Regular Expressions re := str | alpha | re1 |re2 | re1 re2 | re* | re+ e := /re/ e | /re;f / x t := regexp regexps /re/e tests whether regular expression re matches value of e
  • 50. Example: Embedding Regular Expressions re := str | alpha | re1 |re2 | re1 re2 | re* | re+ e := /re/ e | /re;f / x t := regexp regexps /re/e tests whether regular expression re matches value of e function isEmail(x : string) : int begin return / alpha+ @ (alpha+ .)+ alpha+ / x; end
  • 51. Example: Embedding Regular Expressions re := str | alpha | re1 |re2 | re1 re2 | re* | re+ e := /re/ e | /re;f / x t := regexp regexps /re/e tests whether regular expression re matches value of e function isEmail(x : string) : int begin return / alpha+ @ (alpha+ .)+ alpha+ / x; end /re;f/e tests if regular expression re matches a prefix of e, and the suffix satisfies predicate f
  • 52. Assimilating Regular Expression Matching function match(x : string) : int / (a | b) c / x; Desugar = ReMch + ReStr + ReAlt + ReSeq + ReKle + once-AddDef
  • 53. Assimilating Regular Expression Matching function match(x : string) : int / (a | b) c / x; Rewrite rule L : t1 - t2 where s replace fragment matching t1 with t2 Desugar = ReMch + ReStr + ReAlt + ReSeq + ReKle + once-AddDef ReMch : |[ /re/ e ]| - |[ begin var x : string; x := e; return /re; isEmpty/ x; end ]| where new = x
  • 54. Assimilating Regular Expression Matching function match(x : string) : int begin var z : string; z := x; return / (a | b) c; isEmpty / z; end Desugar = ReMch + ReStr + ReAlt + ReSeq + ReKle + once-AddDef ReMch : |[ /re/ e ]| - |[ begin var x : string; x := e; return /re; isEmpty/ x; end ]| where new = x
  • 55. Assimilating Regular Expression Matching function match(x : string) : int begin var z : string; z := x; return / (a | b) c; isEmpty / z; end ReSeq : |[ /re1 re2 ; f/ x ]| - |[ /re1 ; g/ x ]| where new = g ; add-def(||[ function g(a : string) : int /re2; f/ a; ]|) add-def(|def) = rules( AddDef :+ |[ def* ]| - |[ def def* ]| )
  • 56. Assimilating Regular Expression Matching Dynamic rules rules( L : t1 - t2 ) defines new rewrite rule inherits variable bindings from definition context function match(x : string) : int begin var z : string; z := x; return / (a | b) c; isEmpty / z; end AddDef : |[ def* ]| - |[ function k(a : string) : int /c; isEmpty/ a; def* ]| ReSeq : |[ /re1 re2 ; f/ x ]| - |[ /re1 ; g/ x ]| where new = g ; add-def(||[ function g(a : string) : int /re2; f/ a; ]|) add-def(|def) = rules( AddDef :+ |[ def* ]| - |[ def def* ]| )
  • 57. Assimilating Regular Expression Matching function match(x : string) : int begin var z : string; z := x; return / (a | b) ; k / z; end AddDef : |[ def* ]| - |[ function k(a : string) : int /c; isEmpty/ a; def* ]| ReSeq : |[ /re1 re2 ; f/ x ]| - |[ /re1 ; g/ x ]| where new = g ; add-def(||[ function g(a : string) : int /re2; f/ a; ]|) add-def(|def) = rules( AddDef :+ |[ def* ]| - |[ def def* ]| )
  • 58. Assimilating Regular Expression Matching function match(x : string) : int begin var z : string; z := x; return / (a | b) ; k / z; end ReAlt : |[ / (re1 | re2) ; f / x ]| - |[ (/re1;f/x) | (/re2;f/x) ]|
  • 59. Assimilating Regular Expression Matching function match(x : string) : int begin var z : string; z := x; return (/ a ; k / z) | (/ b ; k / z); end ReAlt : |[ / (re1 | re2) ; f / x ]| - |[ (/re1;f/x) | (/re2;f/x) ]|
  • 60. Assimilating Regular Expression Matching function match(x : string) : int begin var z : string; z := x; return (/ a ; k / z) | (/ b ; k / z); end ReStr : |[ /str ; f/ x ]| - |[ isPref(str, x) f(getSuf(str, x)) ]|
  • 61. Assimilating Regular Expression Matching function match(x : string) : int begin var z : string; z := x; return (isPref(a,z) k(getSuf(a,z))) | (isPref(b,z) k(getSuf(b,z))) end ReStr : |[ /str ; f/ x ]| - |[ isPref(str, x) f(getSuf(str, x)) ]|
  • 62. Assimilating Regular Expression Matching function match(x : string) : int begin var z : string; z := x; return (isPref(a,z) k(getSuf(a,z))) | (isPref(b,z) k(getSuf(b,z))) endAddDef : |[ def* ]| - |[ function k(a : string) : int /c; isEmpty/ a; def* ]| Desugar = ReMch + ReStr + ReAlt + ReSeq + ReKle + once-AddDef
  • 63. Assimilating Regular Expression Matching function k(a : string) : int begin return /c; isEmpty/ a; end function match(x : string) : int begin var z : string; z := x return (isPref(a,z) k(getSuf(a,z))) | (isPref(b,z) k(getSuf(b,z))) endAddDef : |[ def* ]| - |[ function k(a : string) : int /c; isEmpty/ a; def* ]| Desugar = ReMch + ReStr + ReAlt + ReSeq + ReKle + once-AddDef
  • 64. Assimilating Regular Expression Matching function k(a : string) : int begin return /c; isEmpty/ a; end function match(x : string) : int begin var z : string; z := x return (isPref(a,z) k(getSuf(a,z))) | (isPref(b,z) k(getSuf(b,z))) end ReStr : |[ /str ; f/ x ]| - |[ isPref(str, x) f(getSuf(str, x)) ]|
  • 65. Assimilating Regular Expression Matching function k(a : string) : int begin return isPref(c, a) isEmpty(getSuf(c, a)); end function match(x : string) : int begin var z : string; z := x return (isPref(a,z) k(getSuf(a,z))) | (isPref(b,z) k(getSuf(b,z))) end ReStr : |[ /str ; f/ x ]| - |[ isPref(str, x) f(getSuf(str, x)) ]|
  • 66. Risks of Domain-Specific Languages • Design and implementation effort • implementing complete toolset for language is major effort • Documentation • tutorials, reference manual, implementation documentation • Training • need programmers that are skilled in using the language • Maintenance • of the language • of DSL programs • How does it fit in the existing work flow • tool chain • DSLs need abstraction/modularity features too • DSL programs become large; reuse of parts needed
  • 67. Conclusion Summary • Generative software development is encapsulation of domain knowledge in generators and target components • Domain-specific languages are the interface to this knowledge • DSL programs can be seen as a configuration for the ‘solution space’ components Challenges • Finding good formalizations and encapsulations of new application domains • A platform for cheap construction of DSLs and generators • Integration of generators in workflow/toolchain
  • 68. References • K. Czarnecki. Overview of Generative Software Development. In J.-P. Bantre et al. (Eds.): Unconventional Programming Paradigms (UPP) 2004, Mont Saint-Michel, France, LNCS 3566, pp. 313-328, 2005 • D. Batory. Feature Models, Grammars, and Propositional Formulas. In H. Obbink and K. Pohl (Eds.) SPLC 2005. LNCS 3714, pp. 7–20, 2005 • M. Bravenboer and E. Visser. Concrete Syntax for Objects. Domain-Specific Language Embedding and Assimilation without Restrictions. In D. C. Schmidt, editor, Proceedings of the 19th ACM SIGPLAN Conference on Object-Oriented Programing, Systems, Languages, and Applications (OOPSLA’04), pages 365–383, Vancouver, Canada, October 2004. ACM Press. • E. Visser. Transformations for Abstractions. In J. Krinke and G. Antoniol, editors, Fifth IEEE International Workshop on Source Code Analysis and Manipulation (SCAM’05), Budapest, Hungary, October 2005. IEEE Computer Society Press. (Keynote paper)
  • 69. Further Studying At Master Program Software Technology; Universiteit Utrecht • Seminar on Software Generation and Configuration • Master course on Program Transformation

×