Software LanguageDesign & Engineering      Eelco Visser     http://eelcovisser.org                              Microsoft ...
Software EngineeringProblem                                                         MachineDomain   bridging the gap betwe...
High-Level LanguagesProblem                                                 HLL                     MachineDomain         ...
Domain-Specific LanguagesProblem                   DSL                 HLL               MachineDomain      domain-specific ...
Software Language Design & Engineeringenable software engineers toeffectively design, implement, and apply domain-specific ...
Research: Software Language Engineering automatically derive efficient,scalable, incremental compiler +   usable IDE from h...
The Spoofax Language Workbenchcreating full featured IDEs for domain-specific languages
Research: Software Language Designsystematically design domain-specific software languages   with optimal tradeoff between ...
Software Language Design Case Studies                                               Mobl: client-side stateful web applica...
Two Talks   Declaratively Programming   the Mobile Web with MoblThe Spoofax Language Workbench
Declaratively Programmingthe Mobile Web with Mobl* Zef Hemel                             Eelco Visser  http://zef.me      ...
Divergence in Mobile Platforms Objective-C            Java                  J2ME/C++HTML/Javascript         Java          ...
Convergence in Mobile PlatformWebkit browser   Webkit browserWebkit browser   Webkit browser
The Universal Userinterface Engine
Mobile Web Architecture
Rich Applications                                      Audio    WebDatabases                              Full-screen supp...
Native Applications            Address book                Camera               Compass                 File IO           ...
Software Engineering with JavaScriptannotated HTML                imperative Javascript MVC, No Integration, No Abstractio...
Late Failures in Web Applications               Zef Hemel, Danny M. Groenewegen, Lennart C. L. Kats, Eelco Visser.        ...
typed                declarative             http://www.mobl-lang.orgintegrated                   concise
Web Application with Touch
Portable Applications
Mobl Architecture
tipcalculator.moblapplication tipcalculatorimport mobl::ui::genericscreen root() {  var amount = 20  var percentage = 10  ...
Model-View Pattern
Task Manager
Data Modelentity Task {  name     : String (searchable)  done     : Bool  due      : DateTime  category : Category (invers...
Logicentity Task {  ...  function postpone(days : Num) {    this.due = DateTime.create(      this.due.getFullYear(),      ...
Reactive User Interfacesscreen root() {  var phrase = ""  header("Tasks") {    button("Add", onclick={ addTask(); })  }  s...
Navigationscreen root() { {         addTask()  var phrase = ""       t = Task()  header("Add") {  header("Tasks") {    but...
Continuationsscreen root() {  button("Ask", onclick={     alert("Hello " + prompt("First name") + " "                     ...
Reusable Controlsscreen root() {  tabSet([("One", tab1),          ("Two", tab2)],    defaultTab="One")}control tab1() {  h...
Tab Set: Higher-Order Controlcontrol tabSet(tabs : [(String,Control)], activeTab : String) {  list((tabName, tabControl) i...
User Interface Idiom: Master Detailscreen root() {  header("Tasks")  masterDetail(    Task.all() order by due desc,    tas...
Master Detail: Higher-Order Controlcontrol masterDetail(items : Collection<?>,                     masterItem : Control1<?...
Adaptive Layout
applications
Mobl Applications
GR8 Conference Program
mPodder
Kenhttp://itsneh.com/ken/
Mobl Summary❖ Linguistic integration  ★ data model, user interface, styling, application logic❖ Domain abstractions  ★ red...
Mobl IDE: Static Cross-Concern Consistency Checking        Constructed on Spoofax Language Workbench
The Spoofax Language Workbench* Rules for Declarative Specification of Languages and IDEs      Lennart Kats                ...
Research: Software Language Engineering Automatically derive efficient,scalable, incremental compiler +   usable IDE from h...
Language Definitions for CompilersSyntax definition★ concrete syntax★ abstract syntaxStatic semantics★ error checking★ name ...
Editor Services for Modern IDEsSyntactic Editor Services★ syntax checking★ bracket matching★ syntax highlighting★ code fol...
Syntax Definition with SDF
Declarative Syntax Definitionlexical syntax  [a-zA-Z_$][a-zA-Z0-9_]* -> IDcontext-free syntax  STRING                      ...
An Interactive Language Workbenchdevelop and use language in same environment
Debugging Ambiguities
Declarative Disambiguationcontext-free priorities  Exp "." ID -> LimitedExp> Exp "." ID "(" {NamedExp ","}* ")" -> Limited...
Parsing after Disambiguation
Natural and Flexible Error Recovery*permissive grammar + backtracking + region discovery [*OOPSLA & SLE 2009]
Transformation with Stratego
Syntactic Normalization and Desugaringrules  normalize :    FunctionNoReturnType(manno*, name, farg*, stat*) ->    Functio...
CPS Transformcps-lift-exprs :  [Return(e)|stats] ->     <concat>[stats2, [Return(e2)|<cps-lift-expressions> stats]]  where...
Type Analysiseq-type-of :  String(_) -> SimpleType(QId("mobl", "String"))eq-type-of :  FieldAccess(e, x) -> t  where Prope...
Type Checkingconstraint-error :  t@SimpleType(_) -> (t, $[Type is not defined: [<pp-mobl-type> t]])  where not(<lookup-typ...
Semantic Editor Serviceseditor-analyze:  (ast, path, project-path) -> (ast2, errors, warnings, notes)  with // ...editor-c...
Editor Services Bindingsmodule MoBL-Buildersimports MoBL-Builders.generatedbuilders  provider   :   include/mobl.jar  obse...
Integrated Language Definition Testing*Includes testing of the editor services [* OOPSLA 2011]
Software Language Design & Engineering           Linguistic abstraction:       capture software knowledge in         domai...
behind the scenes
synchronous programmingvar results = Task.all().list();for(var i = 0; i < results.length; i++) {   alert(results[i].name);}
render page                 timequery database and process    results      ...
render page                                  time                 query databasebrowser freeze    and process             ...
render page send query                time     ...process query    result     ...
asynchronous programmingTask.all.list(function(results) {    for(var i = 0; i < results.length; i++) {       alert(results...
Task.all().list(function(results) {  alert("Hello, ");});                      breaks sequentialalert("world!");          ...
Task.all().list(function(results) {  // make changes  ...  persistence.flush(function() {    alert("Changes saved!");  });...
continuation-passing style     transformation
function displayLocationAndReturn() : Coordinates {  var position = mobl::location::getPosition();  log("Lat: " + position...
function displayLocationAndReturn() : Coordinates {  var position = mobl::location::getPosition();  log("Lat: " + position...
function displayLocationAndReturn() : Coordinates {  var position = mobl::location::getPosition();  log("Lat: " + position...
Software Language Design & Engineering: Mobl & Spoofax
Upcoming SlideShare
Loading in …5
×

Software Language Design & Engineering: Mobl & Spoofax

1,642 views

Published on

Slides for talk at Microsoft Research about Mobl and Spoofax projects.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,642
On SlideShare
0
From Embeds
0
Number of Embeds
362
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Software Language Design & Engineering: Mobl & Spoofax

  1. 1. Software LanguageDesign & Engineering Eelco Visser http://eelcovisser.org Microsoft Research, Redmond October 20, 2011
  2. 2. Software EngineeringProblem MachineDomain bridging the gap between problem domain and solution domain
  3. 3. High-Level LanguagesProblem HLL MachineDomain "A programming language is low level when its programs require attention to the irrelevant." Alan J. Perlis (1982) HLLs reduce gap
  4. 4. Domain-Specific LanguagesProblem DSL HLL MachineDomain domain-specific languages support more specialization
  5. 5. Software Language Design & Engineeringenable software engineers toeffectively design, implement, and apply domain-specific software languages
  6. 6. Research: Software Language Engineering automatically derive efficient,scalable, incremental compiler + usable IDE from high-level, declarativelanguage definition
  7. 7. The Spoofax Language Workbenchcreating full featured IDEs for domain-specific languages
  8. 8. Research: Software Language Designsystematically design domain-specific software languages with optimal tradeoff between expressivity, completeness, portability, coverage, and maintainability
  9. 9. Software Language Design Case Studies Mobl: client-side stateful web applicationsWebDSL: server-side restful web applications
  10. 10. Two Talks Declaratively Programming the Mobile Web with MoblThe Spoofax Language Workbench
  11. 11. Declaratively Programmingthe Mobile Web with Mobl* Zef Hemel Eelco Visser http://zef.me http://eelcovisser.org * To appear: SPLASH/OOPSLA, October 2011, Portland
  12. 12. Divergence in Mobile Platforms Objective-C Java J2ME/C++HTML/Javascript Java .NET Native Applications not Portable
  13. 13. Convergence in Mobile PlatformWebkit browser Webkit browserWebkit browser Webkit browser
  14. 14. The Universal Userinterface Engine
  15. 15. Mobile Web Architecture
  16. 16. Rich Applications Audio WebDatabases Full-screen supportLocation information (GPS) Offline support Canvas Accelerator support Multi-touch
  17. 17. Native Applications Address book Camera Compass File IO Notifications
  18. 18. Software Engineering with JavaScriptannotated HTML imperative Javascript MVC, No Integration, No Abstraction, Accidental Complexity
  19. 19. Late Failures in Web Applications Zef Hemel, Danny M. Groenewegen, Lennart C. L. Kats, Eelco Visser. Static consistency checking of web applications with WebDSL. Journal of Symbolic Computation, 46(2):150-182, 2011.
  20. 20. typed declarative http://www.mobl-lang.orgintegrated concise
  21. 21. Web Application with Touch
  22. 22. Portable Applications
  23. 23. Mobl Architecture
  24. 24. tipcalculator.moblapplication tipcalculatorimport mobl::ui::genericscreen root() { var amount = 20 var percentage = 10 header("Tip calculator") group { item { numField(amount, label="amount") } item { numField(percentage, label="percentage") } item { "$" label(Math.round(amount * (1 + percentage/100))) } }}
  25. 25. Model-View Pattern
  26. 26. Task Manager
  27. 27. Data Modelentity Task { name : String (searchable) done : Bool due : DateTime category : Category (inverse: tasks) tags : Collection<Tag> (inverse: tasks)}entity Category { name : String tasks : Collection<Task> (inverse: category)}entity Tag { name : String tasks : Collection<Task> (inverse: tags)} HTML5 Data Persistence
  28. 28. Logicentity Task { ... function postpone(days : Num) { this.due = DateTime.create( this.due.getFullYear(), this.due.getMonth(), this.due.getDate() + days); } function import(user : String, pw : String) { var tasksJSON = httpRequest("/export?user="+ user + "&pw=" + pw); foreach(t in tasksJSON) { add(Task.fromSelectJSON(t)); } }} statically typed: catch errors early
  29. 29. Reactive User Interfacesscreen root() { var phrase = "" header("Tasks") { button("Add", onclick={ addTask(); }) } searchBox(phrase) group { list(t in Task.search(phrase) limit 20){ item { checkBox(t.done, label=t.name) } } }}
  30. 30. Navigationscreen root() { { addTask() var phrase = "" t = Task() header("Add") { header("Tasks") { button("Done", onclick={ button("Add", onclick={ addTask(); }) } add(t); searchBox(phrase) screen return; group { }) } list(t in Task.search(phrase) limit 20){ textField(t.name) item { datePicker(t.due) checkBox(t.done, label=t.name)} } } }}
  31. 31. Continuationsscreen root() { button("Ask", onclick={ alert("Hello " + prompt("First name") + " " + prompt("Last name")); })}screen prompt(question : String) : String { var answer = "" header(question) { button("Done", onclick={ screen return answer; }) } textField(answer)}
  32. 32. Reusable Controlsscreen root() { tabSet([("One", tab1), ("Two", tab2)], defaultTab="One")}control tab1() { header("Tab 1") label("This is tab 1")}control tab2() { header("Tab 2") label("This is tab 2")} User Interface Idiom: Tab
  33. 33. Tab Set: Higher-Order Controlcontrol tabSet(tabs : [(String,Control)], activeTab : String) { list((tabName, tabControl) in tabs) { block(onclick={ activeTab = tabName; }, style=activeTab==tabName ? activeTabButton : inactiveTabButton) { label(tabName) } } list((tabName, tabControl) in tabs) { block(activeTab==tabName ? visibleTab : invisibleTab) { tabControl() } }} increase coverage: developers can create abstractions
  34. 34. User Interface Idiom: Master Detailscreen root() { header("Tasks") masterDetail( Task.all() order by due desc, taskItem, taskDetail)}control taskItem(t : Task) { checkBox(t.done, label=t.name)}control taskDetail(t : Task) { textField(t.name) datePicker(t.due)}
  35. 35. Master Detail: Higher-Order Controlcontrol masterDetail(items : Collection<?>, masterItem : Control1<?>, detail : Control1<?>){ group { list(it in items) { item(onclick={ detailScreen(it,detail); }) { masterItem(it) } } }}screen detailScreen(it : ?, detail : Control1<?>) { header("Detail") { backButton() } detail(it)}
  36. 36. Adaptive Layout
  37. 37. applications
  38. 38. Mobl Applications
  39. 39. GR8 Conference Program
  40. 40. mPodder
  41. 41. Kenhttp://itsneh.com/ken/
  42. 42. Mobl Summary❖ Linguistic integration ★ data model, user interface, styling, application logic❖ Domain abstractions ★ reduce accidental complexity, platform details❖ User-defined abstractions ★ data binding ★ reactive programming ★ reusable controls❖ Model-View ★ automate the controller
  43. 43. Mobl IDE: Static Cross-Concern Consistency Checking Constructed on Spoofax Language Workbench
  44. 44. The Spoofax Language Workbench* Rules for Declarative Specification of Languages and IDEs Lennart Kats Eelco Visser http://www.lclnet.nl http://eelcovisser.org * OOPSLA 2010
  45. 45. Research: Software Language Engineering Automatically derive efficient,scalable, incremental compiler + usable IDE from high-level, declarativelanguage definition
  46. 46. Language Definitions for CompilersSyntax definition★ concrete syntax★ abstract syntaxStatic semantics★ error checking★ name resolution★ type analysisModel-to-model transformation★ express constructs in core languageCode generation★ translate core language models to implementation
  47. 47. Editor Services for Modern IDEsSyntactic Editor Services★ syntax checking★ bracket matching★ syntax highlighting★ code folding★ outline viewSemantic Editor Services★ error checking★ reference resolving★ hover help★ code completion★ refactoring
  48. 48. Syntax Definition with SDF
  49. 49. Declarative Syntax Definitionlexical syntax [a-zA-Z_$][a-zA-Z0-9_]* -> IDcontext-free syntax STRING -> LimitedSetExp {cons("String")} NUMBER -> LimitedSetExp {cons("Num")} LimitedSetExp -> Exp QId -> LimitedExp {cons("Var")} "(" Exp ")" -> LimitedExp {cons("Brackets")} Exp BoolMethodId Exp -> Exp {cons("BinMethodCall"), left} Exp CompareMethodId Exp -> Exp {cons("BinMethodCall"), left} Exp TermOperatorMethodId Exp -> Exp {cons("BinMethodCall"), left} Exp OperatorMethodId Exp -> Exp {cons("BinMethodCall"), left}character-level grammars: integration of lexical and context-free syntax
  50. 50. An Interactive Language Workbenchdevelop and use language in same environment
  51. 51. Debugging Ambiguities
  52. 52. Declarative Disambiguationcontext-free priorities Exp "." ID -> LimitedExp> Exp "." ID "(" {NamedExp ","}* ")" -> LimitedExp> Exp TermOperatorMethodId Exp -> Exp> Exp OperatorMethodId Exp -> Exp> Exp CompareMethodId Exp -> Exp> Exp BoolMethodId Exp -> Exp> "!" Exp -> Exp> Exp "?" Exp ":" Exp -> Exp> LimitedExp Filter+ -> Exp
  53. 53. Parsing after Disambiguation
  54. 54. Natural and Flexible Error Recovery*permissive grammar + backtracking + region discovery [*OOPSLA & SLE 2009]
  55. 55. Transformation with Stratego
  56. 56. Syntactic Normalization and Desugaringrules normalize : FunctionNoReturnType(manno*, name, farg*, stat*) -> Function(manno*, name, farg*, SimpleType(QId("mobl", "void")), stat*) normalize : IfNoElse(e, block) -> If(e, block, Block([])) desugar : ForInferred(lvalue, e, elem*) -> For(lvalue, t, e, elem*) where GenericType(_, [t]) := <type-of> e
  57. 57. CPS Transformcps-lift-exprs : [Return(e)|stats] -> <concat>[stats2, [Return(e2)|<cps-lift-expressions> stats]] where not(<is-sync> e) with {| Exp : stats2 := <cps-lift-expression> e ; e2 := <Exp> |}
  58. 58. Type Analysiseq-type-of : String(_) -> SimpleType(QId("mobl", "String"))eq-type-of : FieldAccess(e, x) -> t where Property(_, _, t, _) := <lookup-property> (<type-of> e, x)
  59. 59. Type Checkingconstraint-error : t@SimpleType(_) -> (t, $[Type is not defined: [<pp-mobl-type> t]]) where not(<lookup-type> t)constraint-error : t@FieldAccess(e, x) -> (t, $[Property [x] not defined]) where <type-of> e where not(type-of) origin tracking
  60. 60. Semantic Editor Serviceseditor-analyze: (ast, path, project-path) -> (ast2, errors, warnings, notes) with // ...editor-complete-proposal : SimpleType(COMPLETION(_)) -> proposals where all-types := <get-all-types>; proposals := <map(type-name-to-proposal); flatten-list> all-typeseditor-hover: (t@SimpleType(_), position, ast, path, project-path) -> <get-doc> <lookup-type> t2 where t2 := <lookup-node> (position, ast)editor-resolve: (t@SimpleType(qid), position, ast, path, project-path) -> target where target := <ensure-origin(lookup-type|qid)> t
  61. 61. Editor Services Bindingsmodule MoBL-Buildersimports MoBL-Builders.generatedbuilders provider : include/mobl.jar observer : editor-analyze on save : generate-artifacts builder : "Show ATerm" = generate-aterm (openeditor) (realtime) (meta) builder : "Format code" = format-code (openeditor) (realtime) builder : "Desugar" = editor-desugar (openeditor) (realtime) (meta) Connect transformations to IDE
  62. 62. Integrated Language Definition Testing*Includes testing of the editor services [* OOPSLA 2011]
  63. 63. Software Language Design & Engineering Linguistic abstraction: capture software knowledge in domain-specific languages http://spoofax.orgseparation of concerns + linguistic integration http://mobl-lang.org cross concern consistency checking http://webdsl.org early detection of failures http://researchr.org Language workbench: DSL design and http://eelcovisser.org implementation with less effort than traditional language engineering http://researchr.org/search/publication/mobl+spoofax+webdsl
  64. 64. behind the scenes
  65. 65. synchronous programmingvar results = Task.all().list();for(var i = 0; i < results.length; i++) { alert(results[i].name);}
  66. 66. render page timequery database and process results ...
  67. 67. render page time query databasebrowser freeze and process results ...
  68. 68. render page send query time ...process query result ...
  69. 69. asynchronous programmingTask.all.list(function(results) { for(var i = 0; i < results.length; i++) { alert(results[i].name); } });...
  70. 70. Task.all().list(function(results) { alert("Hello, ");}); breaks sequentialalert("world!"); execution assumption
  71. 71. Task.all().list(function(results) { // make changes ... persistence.flush(function() { alert("Changes saved!"); });});
  72. 72. continuation-passing style transformation
  73. 73. function displayLocationAndReturn() : Coordinates { var position = mobl::location::getPosition(); log("Lat: " + position.latitude); log("Long: " + position.longitude); return position;}function displayLocationAndReturn(callback) { mobl.location.getPosition(function(position) { console.log("Lat: " + position.latitude); console.log("Long: " + position.longitude); callback(position); });};
  74. 74. function displayLocationAndReturn() : Coordinates { var position = mobl::location::getPosition(); log("Lat: " + position.latitude); log("Long: " + position.longitude); return position;}function displayLocationAndReturn(callback) { mobl.location.getPosition(function(position) { console.log("Lat: " + position.latitude); console.log("Long: " + position.longitude); callback(position); });};
  75. 75. function displayLocationAndReturn() : Coordinates { var position = mobl::location::getPosition(); log("Lat: " + position.latitude); log("Long: " + position.longitude); return position;}function displayLocationAndReturn(callback) { mobl.location.getPosition(function(position) { console.log("Lat: " + position.latitude); console.log("Long: " + position.longitude); callback(position); });};

×