Whittle Modeling Wizards 2012


Published on

Researchers in model-driven development (MDD) should be intimately familiar with how MDD is used in industry. If they are not, there is a danger that new methods and tools are developed without proper consideration for the way that MDD developers actually work and think. A thorough understanding of current MDD industrial practice can inform research problems and ensure that research solutions are actually adopted.

This talk will describe results from a year long study, which applied methods from social science to understand how MDD is actually used in industry. Based on a survey of over 400 MDD practitioners, in-depth interviews with 22 industry professionals from 17 different companies, and a small number of on-site observational studies, the talk will discuss the current state-of-practice in industrial use of MDD, and will offer some insights on current research gaps.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • MODELS 2001, Toronto
  • Examples to show the diversity of how models/MDD is currently being used in industryBanking example: effort to define an industry-wide standard that will unambiguously and precisely define a set of domain concepts for financial applications (e.g., model for account transfer). Clearly, a massive, industry-wide undertaking. Limited code generation – since essentially trying to extract business rules currently hidden in Java code.
  • Need some anecdotes here
  • Up to here: estimated 25 minutes
  • Up to here: 32 minutes
  • Up to here: 42 minutes
  • Mainly due to lack of resources
  • Mainly due to lack of resources
  • Business driver
  • Ground-up
  • The motivation behind the start of the MDE/reuse effort was that engineers were developing for their own designs for different vehicles/vehicle types – which led to huge complexity in terms of manpower
  • Before the effort, specs were being produced and then outsourced for writing code – which inevitably led to lack of consistency between spec and code
  • Although this was a centralized effort coming from management, it was implemented ground-up not as big bang
  • Business driver: the software engineers didn’t exist (or were too expensive)
  • Motivation for building an in-house code generator
  • 2 side effects of obsessing about code generation: (i) cost arguments wont make sense, because holistic effects not taken into account; (ii) abstractions will be too low level
  • All new software/hardware from scratch; hardware guys were applying new technology for performance improvements
  • Not ground up
  • No control since tool too big/not understandable/didn’t do what they wanted
  • No control since tool too big/not understandable/didn’t do what they wanted
  • Gains offset elsewhere
  • That is, in the model – thousands of features that were really hard to add in the modeling language and then figure out how the code would be generated
  • Disclaimer image
  • Snowball image
  • Disclaimer image
  • Disclaimer image
  • Disclaimer image
  • Comment on how some managers prefer agile because it is (perceived to better) support decentralised management
  • Pragmatism prevails!
  • Inconclusive but illustrates that UML is not universally accepted
  • Inconclusive but illustrates that UML is not universally accepted
  • Inconclusive but illustrates that UML is not universally accepted
  • Whittle Modeling Wizards 2012

    1. 1. The Truth about Model-Driven Development in Industry – and Why Researchers Should Care Jon Whittle joint work with John Hutchinson, Mark Rouncefield School of Computing & Communications Lancaster University, UK j.n.whittle@lancaster.ac.uk
    2. 2. 2001 AD
    3. 3. Im sorry, Dave. Im afraid I cant do that.
    4. 4. model-driven architecture aka. model-driven development model-driven engineering model-based software development model-based design model-integrated computing domain-specific modeling
    5. 5. productivity
    6. 6. maintainability
    7. 7. portability
    8. 8. interoperability
    9. 9. all is well with the world
    10. 10. except for…those darned naysayers
    11. 11. 2012 AD
    12. 12. who was right?
    13. 13. talk outline• Project EAMDE – Social/organizational/technical factors• Discoveries – Some observations – Some lessons – Some tips
    14. 14. Project EAMDE• Widely-distributed questionnaire on how MDD is used – 35 questions pertaining to MDD application – 449 responses• In-depth interviews with MDD practitioners – 22 interviews; 17 different companies – 150,000 words of transcribed data – >360 years of cumulative experience• On-site ethnographic studies (ongoing) – 2 done (by Steinar Kristoffersen); more planned
    15. 15. What EAMDE is not• an attempt to quantify the penetration of MDD in industry• a study on UML – deliberately broad view of MD*• an attempt to evangelise or promote MDD – interested in failure as much as success
    16. 16. Diversity
    17. 17. Examples of MDD
    18. 18. Examples of MDD“the broader the domain you try and cover, theless the productivity increase… withDSM, you’re not looking at building a modelinglanguage for embedded applications … you’renot looking at building one for mobileapplications… mobile embedded, or evenmobile phones or even Brand X mobilephones, but a particular product family of BrandX mobile phones…”
    19. 19. First discovery:a lot of MDD successis hidden
    20. 20. Which modeling languages do you use? (tick all that apply) UML BPMN Vendor In-house SysML Matlab/ DSL DSL Simulink
    21. 21. DSLs favored over general-purpose modeling • mostly, companies write their own code generators for very specific tasks – In contrast, companies often ditch commercial tools because they cannot modify them the way they want or because they “don‟t do everything” – Multiple references to the fact that off-the-shelf tools could have killed an MDD effort • Generation of whole systems is not widespread
    22. 22. Second discovery:code generation is a red herring
    23. 23. Code generation“I guess at the end of the day, this dream of codegeneration from models doesn’t exist – I meaneverything ends up being done by hand becauseeither we don’t trust code generators or they justdon’t generate the code we need… it’s actuallyimpossible to get in non-functional requirementsinto code generators – it’s too difficult”
    24. 24. Offsetting gains withoutrealising it• “sometimes the code generated makes it necessary to use a larger CPU which costs more money than the efficient code of an experienced programmer”• 8x more expensive to certify generated code
    25. 25. Don‟t obsess about productivity • 65-100% code generated • Figures on productivity gains differ – 20-800% gain – 27% loss
    26. 26. Third discovery:the real benefits ofMDD are holistic
    27. 27. So if not productivity, then what? • The real benefits of MDD are quality, architecture, reuse • “whenever you name any single advantage…you can always achieve the same advantage with another approach…” – “it tends to be that a model-driven approach is more likely to have a well articulated design and architecture”
    28. 28. Example• An organisation finds itself developing lots of little (modeling) languages over time: – “we were generating 70% of the system off these little XML languages… we would try to separate out the pieces that were generateable and the pieces that weren’t… it motivated us to have better separation of concerns”
    29. 29. Fourth discovery:MDD must enable new things, not justspeed up old things
    30. 30. Doing things faster and better is not enough• If the status quo works (or is perceived to work), there will be insufficient buy-in to change – MDD should be sold not based on how it can do things (slightly) better, but in terms of how it can fix things that are broken – “software is no longer a bottleneck”
    31. 31. People/Organizations
    32. 32. The Psychology of MDD
    33. 33. Architects love MDD
    34. 34. The code guru hates it
    35. 35. as does the hobbyist developer
    36. 36. It‟s bad news for offshoring
    37. 37. Middle-managers are usually the bottleneck
    38. 38. The MDD guru is likelyto be a developer anddomain expert
    39. 39. MDD works best in companies that are not in the software business
    40. 40. Current Business PracticeDoesn‟t Support MDD
    41. 41. Business Practice Doesn‟tSupport MDD• “lead architects need to understand that just because their models are simple, they weren’t going to be put out of a job…”• Developers are defined by their expertise for a technology not a domain; so it is not in their interests to innovate
    42. 42. MDD Works Best in Non-Software Companies• domain experts already model – “they already have an established way to design in Powerpoint” – “I think they are more open than a company that has very long years of experience in software development”
    43. 43. The Good, The Bad, and The Ugly
    44. 44. The Good
    45. 45. • mature domain• non-software context• ground-up effort• real business driver
    46. 46. “we put it directly in the line of theproduct” “we are not allowed to fail”
    47. 47. “the benefit was not only because weintroduced model-driven design” “we also started a re-use group”“if you didn’t introduce model-drivendesign, the reuse initiative would havefailed”
    48. 48. “at this moment, embedded software isnot bottleneck in any project”
    49. 49. “56% of all our code is from re-used building blocks, but in the beginning it was only 5 or 10%”“normally, decisions are made as low inthe organisation as possible
    50. 50. The Good • critical path • non-software context • real business driver • MDD an enabler for reuse
    51. 51. The Bad -Ass
    52. 52. “the same electrical design in a full sizetruck as in a Cadillac”
    53. 53. “somebody would be writing the spec” “they were actually outsourcing”
    54. 54. “there was an order of magnitudedifference in terms of number of peopleinvolved from generation to generation”
    55. 55. “you couldn’t find a computer scientist ifyou went on a search party”
    56. 56. “vendors try to push out-of-the-box codegeneration” “very, very challenging to do… so we wrote our own code generator”
    57. 57. “if we did modeling just for code generation” “you’ll likely not get the right abstraction”
    58. 58. The Bad -Ass • mature domain • non-software context • ground-up effort • real business driver
    59. 59. The Ugly
    60. 60. “it was a totally new concept of switching system”
    61. 61. “he made this huge decision ” “50 developers all using this CASE tool”
    62. 62. “if there is a problem, they just contact the[CASE tool] engineers.. That was kind oftheir strategy”
    63. 63. “and suddenly the tool doesn’t do somethingexpected” “they try to contact the vendor but they don’t really know what’s going on”
    64. 64. “they couldn’t optimize the generated code” “so the way they had to do it was asking the hardware guys to have more hard disc, more memory, because of the tool”
    65. 65. “it was a nightmare to them to add all thefeatures”
    66. 66. The Ugly • immature domain • top-down effort • no real business driver • lack of control
    67. 67. What would 449 MDDpractitioners say?
    68. 68. Disclaimers
    69. 69. Snowball sampling
    70. 70. Not everyone answered every question
    71. 71. We encouraged only those with actual MDE experienceto participate…
    72. 72. …and discouraged those with opinion but no experience
    73. 73. Aims of the Survey
    74. 74. Real use of MDE in practice
    75. 75. Unraveling points of contention related to MDE
    76. 76. Technical factors• Code will write itself!• BUT needs to be integrated with legacy systems
    77. 77. People factors• Easier to train new hires• BUT new hires require more training
    78. 78. Organisational factors• Encourages organisation- wide standards• BUT requires heavy- handed, rigorous management
    79. 79. Key findings
    80. 80. The “Community” of Respondents
    81. 81. Do you consider MDE to be a good thing? 84%
    82. 82. • Significant software engineering experience – Over 1000 years total experience – 70% involved in software development for 5 years or more
    83. 83. • Employed in a range of different roles – Developer 17% – Modeler 21% – Team leader 19% – Project manager 16% – Domain expert 4% – Researcher 23% – Architect 7%
    84. 84. • Good spread of size of company – Roughly a third each had: • 1-100 employees • 100-1000 • >1000• A significant number of employees are not engaged in software development
    85. 85. Experience with MDE• Initial exploration: 10%• Prototyping: 10%• First major project: 18%• ~ 1/3: extensive experience of MDE on many projects and/or over many years
    86. 86. What are models used for?“Do not use” percentages for MDE activities
    87. 87. Which modeling languages do you use? UML BPMN Vendor In-house SysML Matlab/ DSL DSL Simulink
    88. 88. Use of multiple languages• 62% of those using custom DSLs also use UML• Almost all users of SysML and BPMN also use UML• UML is the most popular „single use‟ language – 38% of all respondents• UML used in combination with just about every combination of modeling languages – 14% of UML users combine with vendor DSL – 6% with both custom and vendor DSL
    89. 89. Which diagrams are used? 19 different diagram types are used regularly
    90. 90. Points of contention – paired questions
    91. 91. Effect of MDE on Training Costs Q: Does using MDE allow you to employ developers with less software engineering experience? 46% agree, 34% disagree Q: Does using MDE require you to carry out significant extra training in modeling? 74% agree, 9% disagree
    92. 92. Benefits of code generation Q: Is your use of code generation an important aspect of your MDE productivity gains? 75% agree, 10% disagree Q: Is integrating generated code into your existing projects a significant problem? 36% agree, 40% disagree
    93. 93. Changes on model or code? Q: Do you mainly make updates on the model rather than the code? 70% agree, 15% disagree Q: Do you spend a lot of time synchronizing the model and the code? 35% agree, 45% disagree
    94. 94. MDE and Agility Q: Does MDE make you faster at implementing new requirements? 75% agree, 12% disagree Q: Does MDE prevent you from responding to business opportunities? 32% agree, 38% disagree
    95. 95. Complexity of UML Q: Is UML too complex? 44% agree, 32% disagree Q: Is UML powerful enough? 52% agree, 31% disagree
    96. 96. Promoting understanding Q: Does your use of MDE lead to better understanding between stakeholders? 66% agree, 16% disagree Q: Does your use of MDE result in unexpected confusion and/or misunderstandings between stakeholders? 25% agree, 49% disagree
    97. 97. Tooling Q: Are MDE tools too expensive? 45% agree Q: Do organizations attempt to deploy MDE using inappropriate and/or cheap tools? 55% agree
    98. 98. Tools used ATL, Acceleo, Agile4R, AndroMDA, ANTLR, AppleXCode, ArgoUML, Artisan Studio, ASCET, ASF+SDF, AToM3, BluAge, Borland Together, BridgePoint, Eclipse (EMF, GMF, M2T), EnterpriseArchitect, Rhapsody, Rational Technical Developer, RationalRose, IdealXML, Influx, Intalio, Intelligent Software Over 100 toolsFactory, iUML, Kermeta, Lisp, LTS, M2Flex, MagicDraw, Matlab/Simulink, Maven, Mendix, MetaEdit+, MetaSketch, MiaGeneration, Microsoft DSL Tools, Microsoft VisualStudio, Visio, Omnigraffle, MicroTool Objectif, MID InnovatorObject, ModelBus, ModelPro, MOFScript, NexJ MDEFramework, Obeo, Objecteering, OlivaNova, OpenArchitectureWare, OpenEmbeDD, OptimalJ, OracleCase, OWLtoAspectJ, Papyrus, Popkin SA andBPWIN, Poseidon, PowerDesigner, Protégé, QVT-Operational (EclipseImplementation), Rascal, Rational SoftwareArchitect, Scheme, SketchiXML, starUML, Stratego, TAYLOR, TelelogicTau, Tibco Business Studio, Together Architect, Topcased, T-VEC, Umbrello, UModel, URDAD, VisualParadigm, VisualState, WEBRATIO, WebSphere IntegrationDeveloper, IBM XDE
    99. 99. Key findings• Productivity gains from code generation tend to outweigh losses from integration with existing code• MDE allows for faster turn-around on new requirements – But there is a significant risk that it may prevent organizations from responding to new business opportunities
    100. 100. Key findings• MDE increases overall training costs• UML is not yet universally accepted as the modeling language of choice – Indeed, DSLs are far more prevalent than anticipated
    101. 101. Education, education, education
    102. 102. • Are we teaching modeling the wrong way? – bottom-up versus top-down – combine teaching modeling & compilers/code optimizationoptimization
    103. 103. Top 10 Tips for Practitioners
    104. 104. 1Keep Domains Tight and Narrow
    105. 105. 2Target well-documented, well- understood domains
    106. 106. 3Put MDD on the Critical Path
    107. 107. 4MDD works best when driven from the ground-up
    108. 108. 5Be Careful About Gains Offset Elsewhere
    109. 109. 6Don‟t Obsess About Code Generation
    110. 110. 7Not everyone can think abstractly
    111. 111. 8Most projects fail at scale-up
    112. 112. 9Match tools/processes to the way people think; not the other way around
    113. 113. 10OK, there were only 9…
    114. 114. Bedtime reading…• Model Driven Engineering Practices in Industry, ICSE 2011• Empirical Assessment of MDE in Industry, ICSE 2011• Mismatches between Industry Practice and Teaching of Model-Driven Software Development. MODELS 2011 Educators‟ Symp.• A Survey of Practitioners’ Use of MDE (ask me for a copy)
    115. 115. Why should researchers care?
    116. 116. Match tools/processes to the way people think; not the other way around
    117. 117. Datasethttp://www.comp.lancs.ac.uk/~eamde/site/content_qres.php These Slideshttp://www.slideshare.net/jonathw/whittle-modeling-wizards-2012