2. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
2
Research Problem
Software development project disasters and lost of billions due to
poor project implementations (Nash, 2000)
Only a third of the more than 13,500 projects evaluated in 2003
were successful (Standish Group)
Half of the software development projects over budget
(Larkowski, 2003)
Project size success rate: large 28%, medium 25% and small
20% (Gartner, 2011)
Croatian e-government projects (6 billion kunas)
Author’s experience of not seeing planned reuse while
working on projects (travel, manufacturing, pharmacy,
telecomunications, finacial, insurance, banking, utility) except the
world class software companies
3. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
3
Research Motivation
Developers of today’s business applications are using
J2EE, Spring Framework, Hibernate, iBatis, JPA, Top
Link, The IBM Insurance Application Architecture, Unisys
3D Blueprints, Eclipse 4 Applicatin model, and etc. as
application models
Most of the named models represents inter-
organizational reuse (? not intra-organizational)
Level of abstraction - most of the time is low or high
(missing a common business application level, e.g.
mobile phone, cars, insurance, banking, etc.)
4. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
4
Research Issues (RI)
1. Lack of approaches and solutions explicitly
supporting business application’s reference
architecture at the appropriate level of
abstraction in software product line
engineering
2. Weak work division between client/server and
application/domain infrastructure developers
3. Lack of reference architecture quality metrics and it’s
product stability predicting techniques in
software product line engineering
5. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
5
Research Questions (RQ)
1. What are the required characteristics (features,
capabilities, variability points, optimal structure, scope,
etc.) of an effective Business Application Reference
Architecture that organizations can expect from, in
order to be used as a template for concrete
architectures for a family of business software systems
to accelerate delivery through the reuse of a solution
based on software product lines approach?
2. How can we build the reference architecture with the
required characteristics?
3. Is the referenced architecture providing required
characteristics efective and usefull in practice?
6. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
6
Research Questions - Addressed
RQ1 will be addressed by analyzing state-of-the-art of
business application software architecture based on software
product lines and defining it’s required characteristics.
Feedback from the experienced developers will increase
confidence on the defined characteristics (mandatory, optional,
importance, relevance, etc.).
RQ2 will be addressed by designing and implementing the
reference architecture solution that provides the required
characteristics.
RQ3 will be addressed by applying the developed solution in
practical case study and by collecting the data for analysis.
Test for: usefulness (utility, usability), effectiveness by asking
3 groups of 10 experienced business application
developers/architects to reply on questionnaire (s).
7. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
7
Research Objectives (RO1)
1. Define and evaluate required characteristics of
Business Application Reference Architecture based
on Software Product Line approach.
a) Analyze existing research on reference architecture in
SPL for business applications and identify key issues
b) Analyze existing reference architecture or
programming models used by developers of business
applications
c) Define the required characteristics for reference
architecture
d) Analyze the opinion of experienced developers
regarding the importance of the defined required
characteristics
8. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
8
Research Objectives (RO2)
2. The purpose of this research is to create the
Business Applications Reference Architecture
(SPL Platform Framework, Components, Tools,
Documentation) based on Software Product Line
approach by using: Java 7+, Eclipse 4+, MD, CB, AO,
Patterns (Dependency Injection,...), as a template for
concrete architectures for a family of business software
systems (level of abstraction), to accelerate delivery
through the reuse of an “effective” solution.
9. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
9
Research Objectives (RO3)
3. Validate the developed reference architecture (RA) and
its effectiveness and usefuleness in practice.
a) Access the usefulness (utility and usability) of the
RA
b) Access the effectiveness of the RA
c) Apply the RA in case study(s) in different domains(?)
d) Analyze the usefuleness and effectiveness of the of
the RA based on qustionariee from 3 groups of 10
experienced developers
10. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
10
Research Approach
The world is satisfied with words.
Few appreciate the things beneath.
Blaise Pascal
11. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
11
Research Methodology - Design
Science
Design Science Research in Information Systems
Research (Alan R. Hevner, 2004)
History key distinction: natulal science and “science of
the artificial” (Simon, 1969)
All or part of the phenomenon may be created as
opposed to naturally occurring
Design scientists create IS artifacts, behavioral
scientists create IS theories
Design Science Research (DSR) types of artifacts:
constructs, models, methods, and instantiations
12. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
12
Research Methodology (DSR)
Design Science Researchers work on understanding,
explaining, and improving information systems
DSR addresses important unsolved problems in unique
or innovative ways or solve problem in more effective or
efficient ways
Design science is an iterative, problem solving process
which seeks to extend the boundaries of human and
organizational capabilities by creating new and
innovative artifacts and by evaluating it’s effectiveness
Risk management for the design science research
framework, six areas of potential risk (Jan Pries-Heje)
13. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
13
Research Methodology (DSR)
Robert W. Lucky, IEEE Spectrum, July 2009General design cycle framework (GDC),Vaishnavi and Kuechler (2004)
14. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
14
General Design Cycle Framework
(GDC)
Awareness of Problem: the problem identified in the
current research is the lack of effective solutions
explicitly supporting business application’s reference
architecture at the appropriate level of abstraction
in software product line engineering,…
Suggestion: Analyze existing research on reference
architecture by using : DSR pattern Industry and Practice
Awareness and/or technique of: a systematic literature
review [Kitchenham, 2004].
Analyze existing reference architecture, define the
required characteristics for reference architecture,
analyze the opinion of experienced developers, survey
based on questionnaires [Yin, 2004].
15. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
15
General Design Cycle Framework
(GDC)
Development: design and create artifact - the Business
Applications Reference Architecture (SPL Reference
Architecture, Components, Aspects, Eclipse plug/in,
Documentation, Test cases)
Evaluation: evaluate the artifact using empirical methods
“to determine how well an artifact works” (Hevner et al.,
2004). Case study(s) (“research-in-the-typical“) for
collecting data – predicting stability, controlled
experiment (“research-in-the-small “, Wohlin 2000,
initial usability test), DSR patterns: Using Metrics and
Benchmarking, survey for usabiliy (Perceived Usefulness
and Ease of Use, Davis, 1989) and effectiveness
questionaire (http://hcibib.org/perlman/question.html)
16. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
16
General Design Cycle Framework
(GDC)
Conclusion: Findings, Knowledge gained,
Further work, Metrics (proposed), case-based
reasoning for predicting Mainainability:Stability
Limitations: it is possible that the evaluation of
effectiveness may not be general enough since it
will be applied in one case study, but we will try to
use the solution in multiple products of the same
product line, and in multiple version of the
products (longitudinal study) to collect more data
for analysis
18. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Architecture – we can not buy!
An architecture is important since ≈80% of effort in
systems occurs after deployment
Systems can be built from large, externally developed
components that are tied together via architecture.
An architecture is an abstraction: enables a one-to-many
mapping (one architecture, many systems).
Architecture is the basis for product (system)
commonality.
Entire software product lines can share a single
architecture.
18
19. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Why Software Product Line
19
Frank van der Linden, 2010.
21. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Software Product Lines
A software product line is a set of software-intensive
systems sharing a common, managed set of features
that satisfy the specific needs of a particular market
segment or mission and that are developed from a
common set of core assets in a prescribed way
(L. N. Paul Clement, 2001)
Standard Practice (planed) that proposes a proactive
reuse, stimulating interchangeable components to
construct high-quality products faster and cheaper.
NOKIA, CelsiusTech, Motorola, Hewlett Packard,
Market Maker…
21
22. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Software Product Lines – core
assets
requirements and requirements analysis
domain model
software architecture and design
performance engineering
documentation
test plans, test cases, and data
people: their knowledge and skills
processes, methods, and tools
budgets, schedules, and work plans
components
22
24. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
SPL Scope, Commonality,
Variability, Products
24
25. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
25
Not using planed approach to
reuse - traditional
Product
1
Product
2
Product
3
Product
4
Product
5
26. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Software Product Line approach
26
Managed Central
Asset Repository
Product
1
Product
2
Product
3
Product
4
Product
5
30. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
30
Software Product Line
Architecture (SPLA)
A product-line architecture is an abstraction: it is a
specification of the high level structures of a family of
applications. These structures reveal complementary
facets of an architecture (static structure, dynamic
structure, etc… ) and contain elements like
components, connections, data, processes [Bass et
al, 1998].
A product-line architecture has to capture both
architectural commonalities and variability's of a
family of applications (components, topology,
dynamics and physical environment, etc.)
31. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Benefits of a Software Production
Line
Economy of Scale from Automated Production
Increase in the scope of product diversity
Increase in the scale of different products effectively delivered and maintained
Cost Savings from Efficiency and Productivity
Increase in productivity and efficiency
Reduction in per-product development cost and overhead
Higher profit margins
Faster Profits from Faster Time to Market
Reduction in time-to-market for new and updated products
Increased agility to react to new opportunities and changing market conditions
Better Products from Better Quality
Increase in customer-perceived product quality
Reduction in defect density
Improved risk management
31
32. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
32
Examples of Domains
Compilers for programming languages
Consumer electronics
Electronic commerce
Video games
Business applications
Basic/Standard/“Pro”
We can subdivide, too:
Avionics systems
Boeing Jets
Boeing 747-400
Software Architecture: Foundations, Theory, and Practice; Richard N. Taylor, Nenad Medvidovic, and Eric M. Dashofy; (C) 2008 John Wiley & Sons, Inc. Reprinted with permission.
33. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
33
Capturing Product Line
Architectures
Common: features common to all products
A: features specific to product A
B: features specific to product B
Product A = Common + A
Product B = Common + B
Software Architecture: Foundations, Theory, and Practice; Richard N. Taylor, Nenad Medvidovic, and Eric M. Dashofy; (C) 2008 John Wiley & Sons, Inc. Reprinted with permission.
34. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Product Line Architectures
A product-line architecture has to meet three
fundamental requirements:
It has to drive the architectural design of new
applications in the product-line;
It has to facilitate the reuse of components at the
product-line level;
It has to permit various analyses in order to
assess the impact (cost, performance, etc… )
of specific requirements for the development
of new applications in the product-line.
34
39. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Evaluation – Dependent Variables
(ISO/IEC9126-2001)
39
Maintainability
Portability
Reusability
Testability
Time To Market
Cost and
Benefits
Projected life
time
Targeted Market
Integration with
Legacy System
Roll back
Schedule
End User’s view
Developer’s view
Business
Community
view
Performance
Availability
Usability
Security
A list of quality attributes exists in
ISO/IEC 9126-2001 Information Technology – Software Product Quality
40. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
40
Proposed structural model and
dependencies - Metrics
41. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Evaluation by DSR Using Metrics
(Maintainability: Stability)
41
Zdravko Rosko CECIIS (C) 2013
42. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Evaluation by DSR Using Metrics
(Maintainability: Stability)
42
Zdravko Rosko CECIIS (C) 2013
44. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Evaluation by DSR Using Metrics
(Maintainability: Stability)
44
D3 D4 D5 PR
P1 4 3 3 0,60
P2 4 3 0 0,43
P3 4 0 0 0,00
Total 12 6 3 0,43
Measure type Measure name
Size Number of Platform/environment (language) class
dependencies (D1)
Number of Platform/external components class
dependencies (D2)
Number of Product/Platform class dependencies
(D3)
Number of Product/environment (language) class
dependencies (D4)
Number of Product/external components class
dependencies (D5)
Complexity Platform framework responsibility (PR)
Zdravko Rosko CECIIS (C) 2013
45. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Questionnaire to 3 groups of 10
developers (usability)
45
Method Research Question Questionnaire Questions
PUEU
What is the usefulness
perceived by users
(developrs/architects) of the
tools (reference architecture RA
and helper plug-in tool)?
Using RA in my job would enable me to accomplish tasks more quickly
Using RA would improve my job performance
Using RA in my job would increase my productivity
Using RA would enhance my effectiveness on the job
Using RA would make it easier to do my job
I would find RA useful in my job
-2: strongly disagree, -1: disagree, 0: undecided, 1: agree, 2: strongly agree
46. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Questionnaire to 3 groups of 10
developers (effectiveness)
46
Can you program the same functionality with ?
Less programming code (eg. Java, XML, CSS)
Less needed knowledge about the required APIs
Less dependencies on external componentes (eg. Jdbc, JSP, Reporting)
Less dependencies on Java platform (rt.jar)
Less time for developing
Less time for testing (having the size of code in mind)
Less documentation needed for somebody new to take maintenance over
Less dependencies on specific tools to help while developing
Less complicated design (simpler UML diagrams)
Likert scale (7): 1 - not possible at all
2 - not as far as I know
3 - may be but not sure
4 - not sure
5 - it is possible
6 - it is possible as far as I know
7 - sure it is possible
47. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Conclusion – Contribution of this
Thesis
Systematic analysis of the state-of-the-art in business
applications reference architecture based on software product
line approach
Definition and evalutaion of required characteristics for the
reference architecture
Reference architecture solution and tools (Eclipse plug-in) to
help development of applications based on the architecture
Case study of the effectivenes and usefulness of the solution
Metrics for reference architecture “responsibility” used at the
“early stage” of the product line, case-base reasoning
technique to predict the product stability, feature models, and
entities structural model
47
49. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Research approach
49
Research Question Research Approach Validation
What are the required characteristics
(features capabilities, variability points,
optimal structure, metrics, scope, etc.) of
an effective Business Application
Reference Architecture that
organizations can expect from, in order
to be used as a template for concrete
architectures for a family of business
software systems to accelerate delivery
through the reuse of a solution based on
software product lines approach?
Analyzing state-of-the-art
of business application
software architecture
based on software product
lines.
Systematic
literature review (reference
architecture, software product line,
research) and review of existing
reference architectures
Define cahracteristics and
revise them based on
experineced
deveopers/architects
opinion
Survey among
developers/architects
How can we build the reference
architecture with the required
characteristics?
Iteratively designing and
implementing the reference
architecture solution that
provides the required
characteristics
Describe approach
and reference architecture and show
the way they
provide the required characteristics
Is the referenced architecture providing
required characteristics efective and
usefull in practice?
Show that the reference
architecture is effective
and usable in practice
Usability and effectiveness test
Use the reference
architecture in practice and
collect the data about its
usage, i.e., the metrics for
maintainability:stability,
usage, performance, etc.
Case study
50. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Evaluation – 3 Groups (10+ people
– developers/architects)
Group A (2-3 years)
Group B (4-5 years)
Group C (6+ years)
50
52. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Initial Usability test: Variables in
an Controlled Experiment (Wohlin)
52
1. Design
Standard design: (1) One independent variable with two values;
(2) One independent variable with two values, paired design; (3)
One independent variable with more than two values; (4) More
than one independent variable
2. Operation
Commit participants, Prepare instrumentation, Execution
3. Analysis and interpretation
Descriptive statistics, outliers (scatter-plots), hypothesis testing
(parametric: t-test, paired t-test, ANOVA), non-parametric
(normally distributed?); internal validity; external validity
53. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Feedback from Expert - Survey
Required characteristics for Reference Architecture(RA)
Parallel to survey we design and develop RA
Questions to participants:
1. Year of experience (development, architecture, OO,
…)
2. Importance of capabilities (-2 totally irrelevant ,-1
unimportant, 1- important, 2- very important)
3. What other characteristics do you propose as
important part of RA
53
54. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Case Study
Sample over variables (not being manipulated as experiment)
but that represent the typical situation
Harder to generalize (but having more products, and periods:
2012, 2013, 2014 will empower the generalization)
Robert K. Yin:
6 data sources (documents, interviews, participant
observations, archival records, direct observations,
physical artifacts)
3 principles (? Multiple sources of evidence, create case
study data base, maintain a chain of evidence)
Comparing the results of using two approaches (defect rate,
productivity, stability)
54
55. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Answer to Etnographic
55
No. Feature Name Feature Implementation
C L I E N T
1 Clent Type RCP, Swing, GWT, SWT, JSP,...
2 Client Communication REST, JMS, RMI, HTTP, IIOP, LOCAL,...
3 Logging Log4j, Xyz,...
4 Session Xyz session,...
5 Reporting HTML, PDF, Jasper, Excel,...
6 Caching Client RAM, Cookie,...
7 Security LDAP, Xyz Security,...
8 Language English, Xyz Language,...
9 Data Validation Rule Engine, Xyz Validation,...
10 UI Components Component 1-N,...
S E R V E R
11 Logging Log4j, Xyz,...
12 Server Communication REST, JMS, RMI, HTTP, IIOP, LOCAL,...
13 Transaction JTA, EJB, Xyz,...
14 Business Object POJO, EJB,...
15 Data Access Object JDBC, CICS, JMS, iSeries,...
16 Data Source Connection Application Server, Apache Pool, Xyz Pool,...
17 Session Container, Xyz Session,...
18 Caching Server RAM, File,Container, Data Base,...
19 Security LDAP, Xyz Security,...
20 Language English, Xyz Language,...
21 Data Validation Rule Engine, Xyz Validation,...
22 Business Component Component 1-N,...
58. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
Domain Specific Software
Architecture
58
Domain Specific Software Architecture is a set of
principal design decisions composed of:
1) A reference architecture which describes a general
computational framework for a significant domain of
applications
2) A component library of reusable chunks of domain
expertise
3) An application configuration method for selecting
and configuring components within the architecture
to meet particular application requirements
59. INTERNATIONAL DOCTORAL SEMINAR 2013, May 13-15, 2013, Dubrovnik, Croatia
59
Capturing Product Line
Architectures
There must be an up-
front (and on-going)
investment in developing
a reusable architecture
that can be instantiated
for each product.
Editor's Notes
World class: IBM, Nortel, Algorithmics, Juniper networks
Reference architecture, Structure, Metrics and Case-base reasoning for predicting stability