Mastering Business Analytics with SAP, 2015 - Melbourne, Australia. This presentation provide a detailed list of measures across the information value chain that organisations can use to determine the success of their BI functions and programs.

1. How to measure the
value of BI
MARITZA CURRY

2. For the past three years
analytics and business intelligence
has been the
top CIO technology
priority
Main Message
2
$217
million
BI hardware & software industry
has grown from $9.3 billion (2009)
to $13.1 billion (2013)
$17 billion
Worldwide Business
Intelligence revenue
forecasted for 2016
40% of business executives base
their major decisions on judgment
instead of on business analytics
Fewer than 30% of BI initiatives will
succeed in aligning analytics with
enterprise business drivers
Failure rates for BI projects are in the
30-90% range
The BI software and analytics industry in Middle East
and Africa totalled $217 million in 2013. This showed a
growth of 11% from 2012.
Sources: 1,2,3,4,5,6,7,8,9

3. Key points to take
home
What I’ll Cover
3
THE CURRENT STATE OF BI
IMPACT MEASUREMENT4
RESEARCH: MEASURING THE
IMPACT OF BI7
Defining the problem
Limitations of current
models/frameworks
5
6
Research design
Delphi Panel results
per measure group
Information Value
Chain
BI Critical Success
Factors
8
9
10
11
13
14
RESEARCH RESULTS12
The BI Performance
Measurement Framework

4. 4
THE CURRENT STATE OF
BI IMPACT MEASUREMENT
1

5. 5
Defining the problem
1 2
3 No clear definition of BI success
BI success is difficult to measure
Limited bodyof knowledge

6. Limitations of current models/frameworks
6Sources: 17,18, 19,20,21
1 Too conceptual
2 Scope too narrow
3 Scope too wide - complex
4 Limited to a single case
study

7. 7
RESEARCH: measuring
the impact of BI
2

8. Information value chain
8
Z
INFORMATIONCOLLECT
Z Z
STORE VALIDATE PRESENT/SHARE INTELLIGENCE KNOWLEDGE WISDOM
Z Z Z
DATA INTEGRATION INFORMATION
QUALITY
BI ENABLEMENT BUSINESS ENABLEMENT
BI OPERATIONAL EFFICIENCY
BI GOVERNANCE
BUSINESS STEWARDSHIP
GETTING DATA IN GETTING DATA OUT TURNING DATA INTO BUSINESS OUTCOMES
BI STRATEGY
Sources: 22

9. BI critical success factors
9Sources: 10,11,12,13,14

10. Research design
10
A Delphi study is a virtual panel of experts gathered to arrive at an answer
to a difficult question – Okoli & Pawlowski, 2004
Anonymity Feedback Experts Iteration Consensus
DELPHI QUESTION
What measures should be used
to determine the impact of the
Business Intelligence function of
an organisation?
DELPHI PANEL INVITEES
Sources: 15

11. 20
%
20
%
20
%
10
%
10
%
10
%
0
%
0
%
Delphi panel results per measure group
11

12. 12
RESEARCH RESULT: The BI
Performance Management
Framework
3

13. Data Integration
13
BI critical
success factor
Measure group Measure Measure source
Reduced
technical risk
Technical factors 1. Source data quality Hwang & Xu (2008)
2. Proper development
technology
Hwang & Xu (2008)
3. Adequate IS staff and
consultants
Hwang & Xu (2008)
4. Project management
(team work)
Hwang & Xu (2008)
Value chain
representation
Information
integration
5. % of operational source
systems represented in
Data Warehouse
Delphi panel (2015)

14. Information Quality
14
BI critical
success factor
Measure group Measure Measure source
Information
content quality
Relevance 1. Comprehensiveness Eppler (2001)
2. Completeness Doll and Torkzadeh (1988)
3. Accuracy Eppler (2001); Doll and
Torkzadeh (1988)
4. Clarity Eppler (2001); Doll and
Torkzadeh (1988)
5. Applicability Eppler (2001); Doll and
Torkzadeh (1988)
Soundness 6. Conciseness Eppler (2001)
7. Consistency Eppler (2001)
8. Correctness Eppler (2001)
9. Currency Eppler (2001)
Aggregate /
hierarchy measure
10. % Improvement in
information quality (per
month or per year)
Delphi panel (2015)

15. BI Enablement
15
BI critical
success factor
Measure group Measure Measure source
Information media
/ access quality
Process
optimisation
1. Format Doll and Torkzadeh (1988)
2. Convenience (ease-of-
use)
Eppler (2001); Doll and
Torkzadeh (1988)
3. Timeliness Eppler (2001); Doll and
Torkzadeh (1988)
4. Traceability Eppler (2001)
5. Interactivity Eppler (2001)
6. Reduced information
search time
Delphi panel (2015)
Infrastructure
reliability
7. Accessibility Eppler (2001)
8. Security Eppler (2001)
BI and analytical
capability
Self-service BI /
end user
enablement
9. Number of end user
created reports
Delphi panel (2015)
10. % Decrease in time spent
on reporting/dashboard
creation by IT BI team
Delphi panel (2015)
11. Number of business users
capable of doing self-
service reporting
Delphi panel (2015)
12. Number of business users
doing self-service
reporting
Delphi panel (2015)

16. Business Enablement
16
BI critical
success factor
Measure group Measure / instrument Measure source
Business
alignment and
relevance
Use of information
in business
processes
1. Improved business
processes (measured
using the BI the BIS
Success Model
questionnaire)
Hwang & Xu (2008);
Elbashir, Popovic, Hackney,
Coelho & Jacklic (2010)
Organisational
benefits
2. Number of business
outcomes
Delphi panel (2015)
3. The level of business
enablement of the BI
system (measured using
the BI Systems
Performance Impact
Framework survey)
Collier and Davern (2008)
Aggregate /
hierarchy measure
4. Strategic alignment Delphi panel (2015)

17. Business Stewardship
17
BI critical
success factor
Measure group Measure / instrument Measure source
Information /
knowledge
culture
System usage 1. BI portal usage (number of
hits) / number of active BI
users
Pirttimaki, Lonnqvist and
Karjaluoto (2006); Delphi
panel (2015)
2. Frequency of system usage Hou (2012)
3. Duration of system usage Hou (2012); Delphi panel
(2015)
4. Number of queries
generated by business
users
Delphi panel (2015)
5. % increase in BI users (per
month or per year)
Delphi panel (2015
6. Maturity of analytical
decision-making culture
(measured using the BIS
Success Model
questionnaire)
Popovic, Hackney, Coelho &
Jacklic (2010)
Aggregate /
hierarchy measures
7. Number of departments
involved in the BICC
Delphi panel (2015)
8. Information coverage Delphi panel (2015)
Executive buy-in Executive support 9. % of senior executives
actively using measures
from BI system outputs
Delphi panel (2015)
10. % of senior executives that
regularly engage with the
BI team to define and refine
important performance
measures and analytical
processes for the business
Delphi panel (2015)

18. BI Operational Efficiency
18
BI critical
success factor
Measure group Measure Measure source
Consistently
reliable, in-time
BI delivery
BI process 1. BI turnaround time Pirttimaki, Lonnqvist and
Karjaluoto (2006); Delphi
panel (2015)
2. Number of fulfilled
assignments
Pirttimaki, Lonnqvist and
Karjaluoto (2006)
Data reliability Infrastructure
reliability
3. System response time Lin, Tsai, Shiang, Kuo and
Tsai (2009); Eppler (2010)
Financial
sustainability of
the BI system
Financial
sustainability
4. Total annual BI cost =
maintenance cost +
personnel costs + hosting
costs
Ghilic-Micu, Stoica and
Mircea (2008)

19. BI Governance
19
BI critical
success factor
Measure group Measure Measure source
Joint BI and
business BI
portfolio
management
BI ROI 1. Organisational benefits – BI
project costs (hardware +
software + implementation
costs)
Hwang & Xu (2008), Ghilic-
Micu, Stoica and Mircea
(2008), Delphi panel (2015)
Implementation
success
2. Budget Hwang & Xu (2008), Yeoh
and Koronios (2010)
3. Time schedule Yeoh and Koronios (2010)

20. BI Strategy
20
BI critical
success factor
Measure group Measure / instrument Measure source
BI Maturity BI system 1. Maturity of BI system
(measured using the BIS
Success Model
questionnaire)
Popovic, Hackney, Coelho &
Jacklic (2010)
2. TDWI's Business
Intelligence Maturity Model
The Data Warehouse
Institute
3. Gartner’s Maturity Model
for Business Intelligence
and Performance
Management
Gartner
Analytical
capabilities
4. Maturity of organisation’s
analytical capabilities
(measured using the BIS
Success Model
questionnaire)
Popovic, Hackney, Coelho &
Jacklic (2010)
5. Gartner’s Maturity Model
for Business Intelligence
and Performance
Management
Gartner

21. BI User Satisfaction
21
BI critical
success factor
Measure group Measure / instrument Measure source
Overall
satisfaction of BI
users with the BI
function
Information quality 1. DWH end-user satisfaction
questionnaire
Chen, Soliman, Mao &
Frolick (2000)
2. End-user computing
satisfaction survey
Doll and Torkzadeh (1988)
BI enablement 3. End-user computing
satisfaction survey
Doll and Torkzadeh (1988)
BI operational
efficiency
4. DWH end-user satisfaction
questionnaire
Chen, Soliman, Mao &
Frolick (2000)
5. End-user computing
satisfaction survey
Doll and Torkzadeh (1988)
Business
enablement
6. DWH end-user satisfaction
questionnaire
Chen, Soliman, Mao &
Frolick (2000)

22. Key Points to Take Home
22
BI is
important.
Make sure
you can prove
it to the
Business, the
CIO, the CEO
and the Board.

24. E: Maritza.Curry@gmail.com
M: +27 083 460 5636
za.linkedin.com/in/maritzacurry/
@MaritzaCurry1

25. References
25
1. Gartner. 2011. Gartner Executive Programs Worldwide Survey of More Than 2,000 CIOs Identifies Cloud Computing as Top Technology Priority for CIOs in 2011. [Online] Available:
http://www.gartner.com/newsroom/id/1526414 Accessed: 23 September 2013.
2. Gartner. 2012. Gartner Executive Programs' Worldwide Survey of More Than 2,300 CIOs Shows Flat IT Budgets in 2012, but IT Organizations Must Deliver on Multiple Priorities. [Online] Available:
http://www.gartner.com/newsroom/id/1897514 Accessed: 23 September 2013.
3. Gartner. 2013. Gartner Executive Program Survey of More Than 2,000 CIOs Shows Digital Technologies Are Top Priorities in 2013. [Online] Available: http://www.gartner.com/newsroom/id/2304615
Accessed: 23 September 2013.
4. Gartner. 2013. Gartner Says Worldwide Business Intelligence, CPM and Analytic Applications/Performance Management Software Market Grew Seven Percent in 2012. [Online] Available:
http://www.gartner.com/newsroom/id/2507915 23 September 2013. Accessed: 23 September 2013.
5. Gartner. 2013. Gartner Says Worldwide Business Intelligence Software Revenue to Grow 7 Percent in 2013. [Online] Available: http://www.gartner.com/newsroom/id/2340216 Accessed: 14 June 2014.
6. Gartner. 2014. Gartner Says Middle East and Africa Business Intelligence and Analytics Software Market Grew 11 Percent in 2013. [Online] Available: http://www.gartner.com/newsroom/id/2769417
Accessed: 14 June 2014.
7. Gartner. 2013. Gartner says Business Analytics will be central for business reinvention. [Online] Available: http://www.gartner.com/newsroom/id/2510815 Accessed: 23 September 2013.
8. Williams, S. 2011. 5 Barriers to BI success and how to overcome them. Strategic Finance, July, 27-33.
9. Gartner. 2012a. Gartner Says Fewer Than 30 Percent of Business Intelligence initiatives Will Align Analytic Metrics Completely With Enterprise Business Drivers by 2014. [Online] Available:
http://www.gartner.com/newsroom/id/1891515 Accessed: 23 September 2013.
10. Watson, H.J. & Wixom, B.H. 2007. The current state of Business Intelligence. Computer, 40(9), 96-99.
11. Zeng, L., Xu, L., Shi, Z., Wang, M. & Wu, W. 2006. Techniques, process and enterprise solutions of Business Intelligence. Paper delivered at the IEEE Conference on Systems, Man and Cybernetics, Taipei,
Taiwan, October.
12. Adamala, S. & Cidrin, L. 2011. Key success factors in Business Intelligence. Journal of Intelligence Studies in Business, 1(1), 107-127.
13. Hwang, M.I. & Xu, H. 2008. A structural model of Data Warehousing success. The Journal of Computer Information Systems, 49(1), 48-56.
14. Yeoh, W., Koronios, A. & Gao, J. 2008. Managing the implementation of Business Intelligence systems: a critical success factors framework. International Journal of Enterprise Information Systems, 4(3),
79-94.
15. Okoli, C. & Pawlowski, S.D. 2004. The Delphi Method as a research tool: an example, design considerations and applications. Information & Management, 42(1), 15-29.
16. Theodore, J.G. The Delphi Method. Futures Research Methodology, 3, 1-31.
17. Popovic, A., Hackney, R., Coelho, P.S. & Jaklic, J. 2012. Towards business intelligence systems success: effects of maturity and culture on analytical decision making. Decision Support Systems, 54, 729-
739.
18. Popovic, A., Turk, T. & Jaklic, J. 2010. Conceptual model of business value of Business Intelligence systems. Management, 15(1), 5-30.
19. Hou, C. 2012. Examining the effect of user satisfaction on system usage and individual performance with business intelligence systems: an empirical study of Taiwan’s electronics industry. International
Journal of Information Management, 32, 560-573.
20. Chuah, M. & Wong, K. 2011. A review of business intelligence and its maturity models. African Journal of Business Management, 5(9), 3424-3428.
21. Hribar Rajteric, I. 2010. Overview of Business Intelligence maturity models. Management, 15(1), 47-67.
22. Bizannes, E. 2010. The information value chain and its network. [Online] Available: http://eliasbizannes.com/blog/2010/01/the-information-value-chain-and-its-network/ Accessed: 16 May
2015.