2nd Business and System Conference 2013
Riga (Latvia), November 2013

Patterns in Business Analysis and
Enterprise Modelin...
Motivation
Business analysis and solution design projects
• are highly specialized for the specific enterprise under consi...
Patterns
Patterns in Computer Science and Enterprise Modeling:
• Initial definition:
Christopher Alexander: “Each pattern ...
Business Value of IT
Kinds of approaches for evaluating the business value of IT
• Process-oriented approaches
• Perceived...
Perceived Values – DeLone and McLean (1992 + 2006)
Information can be captured at different levels
• the technical level,
...
Multi-perspective approach - Zee (2002)
A ‘dashboard’ of relevant performance indicators for IT
Three levels
 Business Va...
Evaluation Strategy for Patterns: Origin
Evaluation strategy: integrating two different lines of work
1.

pattern use as p...
Overview
Motivation
Background
• Patterns for Capturing Organizational Knowledge
• Evaluating the Business Value of IT
Eva...
Information Demand Patterns

Information Demand Patterns (Excerpt)
• Change Administrator
• Process Verifier
• Material Sp...
Problem The pattern addresses the general problem of submitting proposals of
unnecessary low quality, which basically is w...
Information Demand
The information demand of the role responsible for a proposal consists of:
• The work program or policy...
Effects

Economic
effect

Time
efficiency

Quality
effect

Motivation

Learning
Experience

Customer

Work Program

low

V...
Selected Results - Organisational Perspective
•

All respondents agree
– Very much / much with the tasks / responsibilitie...
Selected Results – Individual Perspective
•

The respondents agree
– All agree very much / much with the tasks / responsib...
Balanced Scorecard
Approach: Balanced Scorecard (Kaplan & Norton):
A measurement-based strategic management system, which ...
Evaluation Results (2)
•

•

Quality of Product Documentation
– Number of design rules:
C3: from 95 to 160; C2 from 25 to ...
Pattern Use for Business Analysis
Knowledge areas according to BABOK:
• Business Analysis Planning and Monitoring
• Enterp...
Pattern Use for Business Analysis (3)
Knowledge areas with potential pattern use:
• Enterprise Analysis
– identify the pot...
Thank you for your attention!
Questions?

39

19
Upcoming SlideShare
Loading in …5
×

'Patterns in Business Analysis and Enterprise Modeling: How to evaluate their value? by Kurt Sandkuhl, GE

496 views

Published on

Published in: Business, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
496
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
8
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

'Patterns in Business Analysis and Enterprise Modeling: How to evaluate their value? by Kurt Sandkuhl, GE

  1. 1. 2nd Business and System Conference 2013 Riga (Latvia), November 2013 Patterns in Business Analysis and Enterprise Modeling: How to evaluate their value? Kurt Sandkuhl The University of Rostock, Germany Overview Motivation Background • Patterns for Capturing Organizational Knowledge • Evaluating the Business Value of IT Evaluation Strategy for Patterns Pattern Examples • Information Demand Patterns • Task Patterns Pattern Use in Business Analysis Summary and Future Work 12.05.2011 © 2011 The UNIVERSITY OF ROSTOCK | FACULTY FOR COMPUTER SCIENCE AND ELECTRICAL ENGINEERING 2 1
  2. 2. Motivation Business analysis and solution design projects • are highly specialized for the specific enterprise under consideration, i.e. project is unique • different enterprises in the same area often show similarities • potential to reuse at least some of the knowledge from earlier business analysis projects in later projects Different techniques for supporting reuse in analysis and solution design • high abstraction level: reference models/frameworks (COBIT, TOGAF, ITIL) • Lower abstraction level: patterns • Methods: means of capturing proven knowledge as support for action (ARIS, OneMethodology, 4EM) How to evaluated the “value” of patterns for business analysis process and the solution developed? Overview Motivation Background • Patterns for Capturing Organizational Knowledge • Evaluating the Business Value of IT Evaluation Strategy for Patterns Pattern Examples • Information Demand Patterns • Task Patterns Pattern Use in Business Analysis Summary and Future Work 12.05.2011 © 2011 The UNIVERSITY OF ROSTOCK | FACULTY FOR COMPUTER SCIENCE AND ELECTRICAL ENGINEERING 4 2
  3. 3. Patterns Patterns in Computer Science and Enterprise Modeling: • Initial definition: Christopher Alexander: “Each pattern describes a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice” • Pattern type examples: – Analysis Patterns (Fowler) – Design Patterns (Gamma et al.) – Architecture Patterns (Buschmann et al.) – Workflow Patterns (van der Aalst et al.) – Knowledge Patterns (Clark et al.) – and many more … 5 Organizational Knowledge Patterns An organizational knowledge pattern is a formalization of knowledge for a recurring organizational task abstracting from organization-specific aspects, which is of value for an organizational actor and an asset for an organization. Characteristics • OKP address recurring organizational tasks and at the same time abstracting from a specific organization. They represent organizational knowledge, not individual knowledge • OKP are expressed in a formalized way, which requires a formal language or at least a structured representation • OKP are an asset of the organization, i.e. are not only a resource as such but capture knowledge about the resource’s use • An OKP is of value for an organizational actor in its original form and / or its adaptation for a specific organization. 3
  4. 4. Business Value of IT Kinds of approaches for evaluating the business value of IT • Process-oriented approaches • Perceived value approaches • Project-focused approaches • Multi-perspective approaches based on performance indicators Process Oriented approaches (Mooney et al., 1995) Focus: value created by process improvements in different dimensions Potential IT Business Value Metrics (Mooney et al. 1995) 4
  5. 5. Perceived Values – DeLone and McLean (1992 + 2006) Information can be captured at different levels • the technical level, • the semantic level and • the effectiveness level Identification of six dimensions of IS success: Project-based Approach - Parker (1998) • • • • Identification of stakeholders and risk factors Financial and strategic objectives of the business; add the technical dimensions of a project Definition of evaluation scales Negotiation of the importance of each evaluation criterion via a weighting factor (CSC 1997) 5
  6. 6. Multi-perspective approach - Zee (2002) A ‘dashboard’ of relevant performance indicators for IT Three levels  Business Value of IT (business management level)  Effectiveness of IT (IT management level)  Effectiveness and Efficiency of IT supply (IT supply management level) BVIT is measurable on three dimensions  Financial performance  Business performance  Strategic performance Overview Motivation Background • Patterns for Capturing Organizational Knowledge • Evaluating the Business Value of IT Evaluation Strategy for Patterns Pattern Examples • Information Demand Patterns • Task Patterns Pattern Use in Business Analysis Summary and Future Work 12.05.2011 © 2011 The UNIVERSITY OF ROSTOCK | FACULTY FOR COMPUTER SCIENCE AND ELECTRICAL ENGINEERING 12 6
  7. 7. Evaluation Strategy for Patterns: Origin Evaluation strategy: integrating two different lines of work 1. pattern use as part of solutions – evaluation of the solution, not the elicitation process of the pattern type and the actual pattern – patterns were evaluated from the economic perspective using the balanced scorecard approach 2. pattern type elicitation and actual pattern development – evaluation strategies from naturalistic inquiry • alternating phases of evaluation in theory and practice • Internal and external validation 13 Evaluation Strategy: Characteristics Four evaluation stages • Assessment of the patterns by experts from the field • Evaluation by the pattern users: perception of the pattern utility • Case studies. Observation of real-world cases by researchers • Measurement using scorecards: long-term way of assessing Four perspectives • the pattern concept from the perspective of an individual • the pattern concept from the perspective of an organization • the actual pattern from the perspective of an individual • the actual pattern from the perspective of an organization 14 7
  8. 8. Overview Motivation Background • Patterns for Capturing Organizational Knowledge • Evaluating the Business Value of IT Evaluation Strategy for Patterns Pattern Examples • Information Demand Patterns • Task Patterns Pattern Use in Business Analysis Summary and Future Work 12.05.2011 © 2011 The UNIVERSITY OF ROSTOCK | FACULTY FOR COMPUTER SCIENCE AND ELECTRICAL ENGINEERING 15 Evaluation Strategy: Status for Example Patterns Pattern: Individual perspective Pattern: organizational perspective Concept: Individual perspective Concept: Organizational perspective Assessment: Expert Reviews IDP: [25] and [26] IDP: [24] for collaborative engineering TP: [32] IDP: [25] and [26] IDP: [27] for change management TP: [30] Perception: User Evaluation IDP: subject of section 4.2 Observation: Case Studies IDP: [24] and [27] Measurement: Scorecard IDP: Not performed yet TP: summarized in section 5.2 IDP: Information Demand Patterns IDP: [28 TP: [31] TP: TaskPatterns 16 8
  9. 9. Information Demand Patterns Information Demand Patterns (Excerpt) • Change Administrator • Process Verifier • Material Specification Responsible • Responsible for Quote Preparation • Responsible for Proposal Writing • Branding Responsible 18 Example: Proposal Writing Responsible Context • Research and development projects at universities or research institutes to a large extent are based on project funding. Preparing successful proposals for funding is a task that requires experience and accurate information about the published call, funding conditions, economic parameters, etc., which usually originate from different information sources and actors. The pattern describes the information demand typically experienced by the role responsible for preparing a proposal for funding. • The pattern is supposed to be useful for any type of organization aiming at submitting proposals for project funding to funding agencies. 19 9
  10. 10. Problem The pattern addresses the general problem of submitting proposals of unnecessary low quality, which basically is wasting resources at the preparing organization or reducing the probability and volume of grants. This includes the following problems, which were often observed when preparing proposals for funding: • Submitted proposals fail to meet the formal requirements published in the call, which usually leads to a rejection. Examples are missing attachments, missing signature, wrong number or different kind of partners. • Submitted proposals contain an invalid or incorrect budget. Typical mistakes are to include non-eligible costs, use the wrong cost rates or don’t include applicable surcharges • Proposals ask for funding of activities, which already are being funded in other projects of the same fund giver • Proposals are submitted to late, either due to deadline changes or misunderstandings 20 Tasks and Responsibility The tasks of the role responsible for a proposal include • The preparation of the a competitive, complete and high quality technical content of the proposal • The preparation of an accurate, complete and consistent formal part of the proposal • Partner management, i.e. coordination of all contributions from participating partners • On-time submission of the proposal including all attachments, forms and signatures 21 10
  11. 11. Information Demand The information demand of the role responsible for a proposal consists of: • The work program or policy documents guiding the funding from the fund giver • General conditions from the fund giver stating eligibility of costs, preconditions to apply for funding or evaluation procedure • The complete call information, including supplementary material like the guide for proposers, schedule of information meetings, forms to be submitted, etc. • Economic parameters from the own organization, including personnel cost rates, surcharges, administrative overheads, investment and facility costs to be included, VAT rates, etc. • Earlier funded projects at the fund giver in the subject field of the call under consideration • Plans of the competitors regarding submissions in the same call to the fund giver 22 Quality Criteria General importance Work Program Call Content Accurate high high as early as possible complete high high decisive decisive high decisive General Conditions high high high Economic Parameter high decisive Earlier Funded Projects high Nice to have Nice to have Nice to have Nice to have Nice to have Nice to have Competitor plans Nice to have high high Nice to have 11
  12. 12. Effects Economic effect Time efficiency Quality effect Motivation Learning Experience Customer Work Program low Very high Very high moderate moderate moderate Call Content low Very high Very high moderate moderate moderate General Conditions moderate Very high moderate low low moderate Economic Parameter Very high low Very high low low low Earlier Funded Projects low moderate Very high moderate moderate low Competitor plans low moderate moderate low moderate low 24 Perception – Organisational Perspective • • Method: – Questionnaire, 5 point Likert scale – Demand pattern ”proposal responsible” Participants – 4 different organisations from 3 different countries – 2 research institutes, 2 universities – Vice-director / Assoc. Dean for Research – 25 – 100 proposals / year – 20+ employees involved 25 12
  13. 13. Selected Results - Organisational Perspective • All respondents agree – Very much / much with the tasks / responsibilities – Very much / much with the quality criteria – that the described effects are valuable / very valuable (exception: timeline, motivation) – that all elements of the pattern are needed – Could imagine to use the demand pattern for • training of new employees • Training of current employees • Basis or best practice description • Improvement of best practices 26 Perception – Individual Perspective • • Method: – Questionnaire, 5 point Likert scale – Demand pattern ”proposal responsible” Participants – 3 PhD students (less than 2 proposals) – 4 researchers (4-6 proposals) – 1 Head of research group (10-12 proposals) – 2 senior researchers (more than 20 proposals) – from • 4 different organisations from 3 different countries • 2 research institutes, 2 universities 27 13
  14. 14. Selected Results – Individual Perspective • The respondents agree – All agree very much / much with the tasks / responsibilities – 70% find that the information demand, quality criteria, effects and task & responsibility descriptions are valuable / very valuable – 90% find that all elements of the pattern are needed – Difficult to estimate how much effort the overall pattern or the different parts of the information demand pattern would save – 3 propose the same additional part: ”background information from program manager at the fund giver” 28 Task Pattern Example 30 14
  15. 15. Balanced Scorecard Approach: Balanced Scorecard (Kaplan & Norton): A measurement-based strategic management system, which provides a method of aligning business activities to the strategy, and monitoring performance of strategic goals over time. Implementation of a scorecard in each MAPPER use case and on project level Financial Customer Internal Business Process Learning and Growth 31 Evaluation Results – Use and Quality of Task Patterns • • • • Number of Best Practices – number of best practice descriptions increased significantly – C1: 7; C2: 5; C3: 6; C4: 9 Quality of Best Practices – average level of detail of the best practice descriptions improved – intensity of use of best practices at significantly C2 and C4, – accuracy of the best practice descriptions – training new employees within C4 Updating best practice descriptions – The time needed to update best practice descriptions decreased – at C3 22 days to 14 days; at C4 6 to 4 months; stable at C1 and C2 Conclusion: clearly positive evaluation of task patterns 32 15
  16. 16. Evaluation Results (2) • • Quality of Product Documentation – Number of design rules: C3: from 95 to 160; C2 from 25 to 27; C4: no changes – refinement level improved at C4 from 3 refinements to 4; Cycle Time, Time to market – indicators provided by C4 • material specification task: from 4 months to 3 months (reduced by 25%) – indicators provided by C1 • average length of the target setting process: from 14 to 12 months • average time for assessing solutions for a customer problem: reduced by 50%. 33 Overview Motivation Background • Patterns for Capturing Organizational Knowledge • Evaluating the Business Value of IT Evaluation Strategy for Patterns Pattern Examples • Information Demand Patterns • Task Patterns Pattern Use in Business Analysis Summary and Future Work 12.05.2011 © 2011 The UNIVERSITY OF ROSTOCK | FACULTY FOR COMPUTER SCIENCE AND ELECTRICAL ENGINEERING 34 16
  17. 17. Pattern Use for Business Analysis Knowledge areas according to BABOK: • Business Analysis Planning and Monitoring • Enterprise Analysis • Elicitation • Requirements Analysis • Solution Assessment and Validation • Requirements Management and Communication 35 Pattern Use for Business Analysis (2) IDP and TP do not offer support: • Business Analysis Planning and Monitoring • Enterprise Analysis • Elicitation • Requirements Analysis • Solution Assessment and Validation • Requirements Management and Communication Patterns do not offer method support, but provide solution elements! 36 17
  18. 18. Pattern Use for Business Analysis (3) Knowledge areas with potential pattern use: • Enterprise Analysis – identify the potential source for shortcomings and operational problems in an enterprise • Elicitation – patterns propose a solution which has to be configured or instantiated for the company under consideration • Solution Assessment and Validation – knowledge patterns can be used as a best practice reference 37 Summary & Future Work The intention of our work is to contribute to evaluating the value of patterns and improve patterns for use in enterprise modeling and business analysis Contributions so far • (1) a proposal for how to evaluate the maturity and value of patterns • (2) two actual pattern types and results regarding their evaluation • (3) a discussion of pattern use in the context of business analysis Future work • Continue evaluation of more pattern types • Further investigate the use in business analysis 18
  19. 19. Thank you for your attention! Questions? 39 19

×