Researchers in model-driven development (MDD) should be intimately familiar with how MDD is used in industry. If they are not, there is a danger that new methods and tools are developed without proper consideration for the way that MDD developers actually work and think. A thorough understanding of current MDD industrial practice can inform research problems and ensure that research solutions are actually adopted. This talk will describe results from a year long study, which applied methods from social science to understand how MDD is actually used in industry. Based on a survey of over 400 MDD practitioners, in-depth interviews with 22 industry professionals from 17 different companies, and a small number of on-site observational studies, the talk will discuss the current state-of-practice in industrial use of MDD, and will offer some insights on current research gaps.

1. The Truth about Model-Driven Development
in Industry
– and Why Researchers Should Care
Jon Whittle
joint work with John Hutchinson, Mark Rouncefield
School of Computing & Communications
Lancaster University, UK
j.n.whittle@lancaster.ac.uk

2. 2001 AD

4. I'm sorry, Dave. I'm afraid I can't do that.

6. model-driven architecture
aka.
model-driven development
model-driven engineering
model-based software development
model-based design
model-integrated computing
domain-specific modeling

7. productivity

8. maintainability

9. portability

10. interoperability

11. all is well with the world

12. except for…
those darned naysayers

14. 2012 AD

15. who was right?

16. talk outline
• Project EAMDE
– Social/organizational/technical
factors
• Discoveries
– Some observations
– Some lessons
– Some tips

17. Project EAMDE
• Widely-distributed questionnaire on how MDD is
used
– 35 questions pertaining to MDD application
– 449 responses
• In-depth interviews with MDD practitioners
– 22 interviews; 17 different companies
– 150,000 words of transcribed data
– >360 years of cumulative experience
• On-site ethnographic studies (ongoing)
– 2 done (by Steinar Kristoffersen); more planned

18. What EAMDE is not
• an attempt to quantify the penetration of MDD in
industry
• a study on UML
– deliberately broad view of MD*
• an attempt to evangelise or promote MDD
– interested in failure as much as success

19. Diversity

20. Examples of MDD

21. Examples of MDD
“the broader the domain you try and cover, the
less the productivity increase… with
DSM, you’re not looking at building a modeling
language for embedded applications … you’re
not looking at building one for mobile
applications… mobile embedded, or even
mobile phones or even Brand X mobile
phones, but a particular product family of Brand
X mobile phones…”

22. First discovery:
a lot of MDD success
is hidden

23. Which modeling languages do you use?
(tick all that apply)
UML BPMN Vendor In-house SysML Matlab/
DSL DSL Simulink

24. DSLs favored over general-
purpose modeling
• mostly, companies write their own code
generators for very specific tasks
– In contrast, companies often ditch commercial tools
because they cannot modify them the way they want
or because they “don‟t do everything”
– Multiple references to the fact that off-the-shelf tools
could have killed an MDD effort
• Generation of whole systems is not widespread

25. Second discovery:
code generation is a red herring

26. Code generation
“I guess at the end of the day, this dream of code
generation from models doesn’t exist – I mean
everything ends up being done by hand because
either we don’t trust code generators or they just
don’t generate the code we need… it’s actually
impossible to get in non-functional requirements
into code generators – it’s too difficult”

27. Offsetting gains without
realising it
• “sometimes the code generated makes it
necessary to use a larger CPU which costs more
money than the efficient code of an experienced
programmer”
• 8x more expensive to certify generated code

28. Don‟t obsess about productivity
• 65-100% code
generated
• Figures on
productivity gains
differ
– 20-800% gain
– 27% loss

29. Third discovery:
the real benefits of
MDD are holistic

30. So if not productivity, then what?
• The real benefits of MDD are
quality, architecture, reuse
• “whenever you name any single
advantage…you can always achieve the same
advantage with another approach…”
– “it tends to be that a model-driven approach is more
likely to have a well articulated design and
architecture”

31. Example
• An organisation finds itself developing lots of
little (modeling) languages over time:
– “we were generating 70% of the system off these little
XML languages… we would try to separate out the
pieces that were generateable and the pieces that
weren’t… it motivated us to have better separation of
concerns”

32. Fourth discovery:
MDD must enable new things, not just
speed up old things

33. Doing things faster and better
is not enough
• If the status quo works (or is perceived to
work), there will be insufficient buy-in to change
– MDD should be sold not based on how it can do
things (slightly) better, but in terms of how it can fix
things that are broken
– “software is no longer a bottleneck”

34. People/Organizations

35. The Psychology of MDD

36. Architects love MDD

37. The code guru hates it

38. as does the hobbyist developer

39. It‟s bad news for offshoring

40. Middle-managers
are usually the
bottleneck

41. The MDD guru is likely
to be a developer and
domain expert

42. MDD works best in companies
that are not in the
software business

43. Current Business Practice
Doesn‟t Support MDD

44. Business Practice Doesn‟t
Support MDD
• “lead architects need to understand that just
because their models are simple, they weren’t
going to be put out of a job…”
• Developers are defined by their expertise for a
technology not a domain; so it is not in their
interests to innovate

45. MDD Works Best in Non-
Software Companies
• domain experts already model
– “they already have an established way to design in
Powerpoint”
– “I think they are more open than a company that has very
long years of experience in software development”

46. The Good, The Bad, and The Ugly

47. The Good

48. • mature domain
• non-software context
• ground-up effort
• real business driver

49. “we put it directly in the line of the
product”
“we are not allowed to fail”

50. “the benefit was not only because we
introduced model-driven design”
“we also started a re-use group”
“if you didn’t introduce model-driven
design, the reuse initiative would have
failed”

51. “at this moment, embedded software is
not bottleneck in any project”

52. “56% of all our code is from re-used
building blocks, but in the beginning it was
only 5 or 10%”
“normally, decisions are made as low in
the organisation as possible

53. The Good
• critical path
• non-software context
• real business driver
• MDD an enabler for reuse

54. The Bad -Ass

56. “the same electrical design in a full size
truck as in a Cadillac”

57. “somebody would be writing the spec”
“they were actually outsourcing”

58. “there was an order of magnitude
difference in terms of number of people
involved from generation to generation”

59. “you couldn’t find a computer scientist if
you went on a search party”

60. “vendors try to push out-of-the-box code
generation”
“very, very challenging to do… so we wrote
our own code generator”

61. “if we did modeling just for code generation”
“you’ll likely not get the right abstraction”

62. The Bad -Ass
• mature domain
• non-software context
• ground-up effort
• real business driver

63. The Ugly

65. “it was a totally new concept of switching system”

66. “he made this huge decision ”
“50 developers all using this CASE tool”

67. “if there is a problem, they just contact the
[CASE tool] engineers.. That was kind of
their strategy”

68. “and suddenly the tool doesn’t do something
expected”
“they try to contact the vendor but they don’t
really know what’s going on”

69. “they couldn’t optimize the generated code”
“so the way they had to do it was asking the
hardware guys to have more hard disc, more
memory, because of the tool”

70. “it was a nightmare to them to add all the
features”

71. The Ugly
• immature domain
• top-down effort
• no real business driver
• lack of control

72. What would 449 MDD
practitioners say?

73. Disclaimers

74. Snowball sampling

75. Not everyone answered every question

76. We encouraged only those with actual MDE experience
to participate…

77. …and discouraged those with opinion but no experience

78. Aims of the Survey

79. Real use of MDE in practice

80. Unraveling points of contention related to MDE

81. Technical factors
• Code will write itself!
• BUT needs to be
integrated with legacy
systems

82. People factors
• Easier to train new hires
• BUT new hires require
more training

83. Organisational factors
• Encourages organisation-
wide standards
• BUT requires heavy-
handed, rigorous
management

84. Key findings

85. The “Community” of Respondents

86. Do you consider MDE to be a good thing?
84%

87. • Significant software engineering experience
– Over 1000 years total experience
– 70% involved in software development for 5 years or
more

88. • Employed in a range of different roles
– Developer 17%
– Modeler 21%
– Team leader 19%
– Project manager 16%
– Domain expert 4%
– Researcher 23%
– Architect 7%

89. • Good spread of size of company
– Roughly a third each had:
• 1-100 employees
• 100-1000
• >1000
• A significant number of employees are not
engaged in software development

90. Experience with MDE
• Initial exploration: 10%
• Prototyping: 10%
• First major project: 18%
• ~ 1/3: extensive experience of MDE on many
projects and/or over many years

91. What are models used for?
“Do not use” percentages for MDE activities

92. Which modeling languages do you use?
UML BPMN Vendor In-house SysML Matlab/
DSL DSL Simulink

93. Use of multiple languages
• 62% of those using custom DSLs also use UML
• Almost all users of SysML and BPMN also use
UML
• UML is the most popular „single use‟ language
– 38% of all respondents
• UML used in combination with just about every
combination of modeling languages
– 14% of UML users combine with vendor DSL
– 6% with both custom and vendor DSL

94. Which diagrams are used?
19 different diagram types are used regularly

95. Points of contention – paired questions

96. Effect of MDE on Training Costs
Q: Does using MDE allow you to employ
developers with less software engineering
experience?
46% agree, 34% disagree
Q: Does using MDE require you to carry out
significant extra training in modeling?
74% agree, 9% disagree

97. Benefits of code generation
Q: Is your use of code generation an important
aspect of your MDE productivity gains?
75% agree, 10% disagree
Q: Is integrating generated code into your existing
projects a significant problem?
36% agree, 40% disagree

98. Changes on model or code?
Q: Do you mainly make updates on the model
rather than the code?
70% agree, 15% disagree
Q: Do you spend a lot of time synchronizing the
model and the code?
35% agree, 45% disagree

99. MDE and Agility
Q: Does MDE make you faster at implementing
new requirements?
75% agree, 12% disagree
Q: Does MDE prevent you from responding to
business opportunities?
32% agree, 38% disagree

100. Complexity of UML
Q: Is UML too complex?
44% agree, 32% disagree
Q: Is UML powerful enough?
52% agree, 31% disagree

101. Promoting understanding
Q: Does your use of MDE lead to better
understanding between stakeholders?
66% agree, 16% disagree
Q: Does your use of MDE result in unexpected
confusion and/or misunderstandings between
stakeholders?
25% agree, 49% disagree

102. Tooling
Q: Are MDE tools too expensive?
45% agree
Q: Do organizations attempt to deploy MDE using
inappropriate and/or cheap tools?
55% agree

103. Tools used
ATL, Acceleo, Agile4R, AndroMDA, ANTLR, Apple
XCode, ArgoUML, Artisan Studio, ASCET, ASF+SDF, AToM3, Blu
Age, Borland Together, BridgePoint, Eclipse (EMF, GMF, M2T), Enterprise
Architect, Rhapsody, Rational Technical Developer, Rational
Rose, IdealXML, Influx, Intalio, Intelligent Software Over 100 tools
Factory, iUML, Kermeta, Lisp, LTS, M2Flex, Magic
Draw, Matlab/Simulink, Maven, Mendix, MetaEdit+, MetaSketch, MiaGen
eration, Microsoft DSL Tools, Microsoft Visual
Studio, Visio, Omnigraffle, MicroTool Objectif, MID Innovator
Object, ModelBus, ModelPro, MOFScript, NexJ MDE
Framework, Obeo, Objecteering, Oliva
Nova, OpenArchitectureWare, OpenEmbeDD, OptimalJ, Oracle
Case, OWLtoAspectJ, Papyrus, Popkin SA and
BPWIN, Poseidon, PowerDesigner, Protégé, QVT-Operational (Eclipse
Implementation), Rascal, Rational Software
Architect, Scheme, SketchiXML, starUML, Stratego, TAYLOR, Telelogic
Tau, Tibco Business Studio, Together Architect, Topcased, T-
VEC, Umbrello, UModel, URDAD, Visual
Paradigm, VisualState, WEBRATIO, WebSphere Integration
Developer, IBM XDE

104. Key findings
• Productivity gains from code generation tend to
outweigh losses from integration with existing
code
• MDE allows for faster turn-around on new
requirements
– But there is a significant risk that it may prevent
organizations from responding to new business
opportunities

105. Key findings
• MDE increases overall training costs
• UML is not yet universally accepted as the
modeling language of choice
– Indeed, DSLs are far more prevalent than anticipated

106. Education, education, education

107. • Are we teaching modeling the
wrong way?
– bottom-up versus top-down
– combine teaching modeling
& compilers/code
optimizationoptimization

108. Top 10 Tips for Practitioners

109. 1
Keep Domains Tight and Narrow

110. 2
Target well-documented, well-
understood domains

111. 3
Put MDD on the Critical Path

112. 4
MDD works best when driven
from the ground-up

113. 5
Be Careful About Gains
Offset Elsewhere

114. 6
Don‟t Obsess About Code Generation

115. 7
Not everyone can think abstractly

116. 8
Most projects fail at scale-up

117. 9
Match tools/processes to the
way people think; not the
other way around

118. 10
OK, there were only 9…

119. Bedtime reading…
• Model Driven Engineering Practices in
Industry, ICSE 2011
• Empirical Assessment of MDE in
Industry, ICSE 2011
• Mismatches between Industry Practice and
Teaching of Model-Driven Software
Development. MODELS 2011 Educators‟
Symp.
• A Survey of Practitioners’ Use of MDE (ask
me for a copy)

121. Why should researchers care?

122. Match tools/processes to the
way people think; not the
other way around

123. Dataset
http://www.comp.lancs.ac.uk/~eamde/site/content_qres.php
These Slides
http://www.slideshare.net/jonathw/whittle-modeling-wizards-2012