1. Agile BI: recommended by the experts,
but is it used in the real world?
Agile BI Research by Petr Podrouzek

2. Research questions
Q1: How often is agile development used in
business intelligence projects?
Q2: What are the risks which can lead to
unsuccessful implementation of agile
methodology in a BI project?
Q3: How to deal with these risk?

3. Research methodology I.
● Literature and journal review
● Interview with 28 agile BI practitioners; question asked:
I1 Do you know what agile BI is? If yes, how would you define it?
I2
On how many BI projects, which have utilized agile methodology, have
you participated on?
I3
Which agile methodology have you used the most while working on BI
projects?
I4
Please name three problems (based on your experience) which can cause
failure of an agile methodology implementation on a BI project.
I5
For the problems from question (4), please describe shortly, how have you
(or would you) address those issues.
I6
Does the initiative of using agile come from the business or from the IT (BI
team)?
I7 Do you have any preferred tools for agile BI management?
I8 Other follow up questions will arise during the interview.

4. Research methodology II.
● Once the responses were collected these were analysed
using root cause analysis.
● Ishikawa diagram with the following dimensions was
used for the root cause analysis:
D1
Environment – the conditions under which the agile BI project runs (e.g. culture
of the company or the team).
D2
Management – representing various levels of management which has an
influence on the project (e.g. operational day to day task assigning to the team
members).
D3
Measurements – concerns measurement of the performance of the agile
methodology implementation (e.g. deadlines being met as a way to measure
performance of the project management).
D4 People – people who are involved in the process (e. g. developers).
D5
Processes – processes supporting the agile software development (e. g. HR
process supporting further education of the developers).
D6
Tools – software tools (e.g. project management tools like JIRA) but also
methodologies (e. g. Pomodoro method for time management).

5. Basic definitions
① Business intelligence (BI) is “an umbrella term that includes the
applications, infrastructure and tools, and best practices that
enable access to and analysis of information to improve and
optimize decisions and performance” (Gartner.com 2013).
② Agile is a set of “software development methods based on
iterative and incremental development, where requirements and
solutions evolve through collaboration between self-organizing,
cross-functional teams.” (Wikipedia 2013).
③ Agile BI is understood differently by different authors:
● Hughes (2013) considers agile BI to be application of various styles of iterative
and incremental development tailored to challenges of data integration and
presentation.
● Hill et al. (2009) even more emphasizes the fact that the authors of the agile
methods were not BI or data-warehousing (DWH) experts so these methods
were not developed to support this kind of projects.
● BI Evelson et al. (2010). He sees agile BI as a technical concept, where
development tools support automatic meta data generated BI applications.

6. Why waterfall fails for BI
projects?
● DM Review magazine study the average deployment time
of a BI project was 17 months and the failure rate was
nearly 65%.
● Moss (2013) lists the following reasons:
● Waterfall methodology was developed in 1970's and in that time,
concept of data governance and data integration did not exist.
● Contrary to OLTP systems, where significant amount of the work is to
write code, this is not the case of BI projects.
● In general, waterfall does not take into account the complexity of data
integration and the fact that this is most of the work BI developer does.
● Hughes (2013) argues that data-warehousing project
usually aims to deliver multiple applications at once
serving different purposes compared to OLTP apps that
can be delivered one by one.

7. Question I1
● Question I1:
● Do you know what agile BI is? If yes, how would you define it?
● Sample answers:
● "An approach to the development of a BI solution that is evolutionary
(iterative and incremental) and collaborative in nature.” (Ambler 2014,
email interview, 15/1/2014)
● "It has to be BI solutions delivered using the core values and principles of
the agile manifesto.” (Corr 2014, email interview, 8/1/2014)
● “Agile BI is an iterative and spiral development approach that embraces
Agile principles but has been modified for the unique characteristics and
subtleties associated with BI projects.” (Gallo 2014, email interview,
6/1/2014)

8. Question I2
● Question I2:
● On how many BI projects, which have utilized agile methodology, have
you participated on?
● Answer:
62 % BI projects used
agile methodology

9. Question I3
● Question I3:
● Which agile methodology have you used the most while working on BI
projects?
● Answers:
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
11% 11% 11% 56% 11%
BEAM Disciplined Agile Delivery Extreme Scoping Scrum/Scrumban Spiral

10. Questions I4 and I5 I.
● Questions:
● I4: Please name three problems (based on your experience) which can
cause failure of an agile methodology implementation on a BI project.
● I5: For the problems from question (4), please describe shortly, how have
you (or would you) address those issues.
● Answers (Answers were analysed using Ishikawa diagram dimensions),
samples for each dimension are given:
● D1: “Traditional mindset. Too many data professionals are stuck in 1970s
thinking.” (Ambler 2014, email interview, 15/1/2014)
● D2: “Management won't commit to a firm feature set per sprint,
therefore good deadlines won't be set.” (Ellis 2014, email interview,
3/1/2014)
● D3: “Measurements of project progress and success is measured still
using traditional waterfall approach especially on completeness, where
Agile is more flexible in terms of “completeness” due to it’s nature of
using an iterative approach.” (Chu 2014, email interview, 14/1/2014)

11. Question I4 and I5 II.
● Answers (samples for the rest of the dimensions):
● D4: “Inexperienced project members concerning Scrum-know-how.”
(Schwarz 2014, email interview, 6/1/2014)
● D5: “Not starting agile. Leaving agile to the development/testing and not
applying it to analysis and design. It very difficult to go agile mid
project.” (Corr 2014, email interview, 8/1/2014)
● D6: “Heavy project administration.” (Gallo 2014, email interview,
6/1/2014)
● Answers (distribution by dimensions):
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
19% 26% 4% 26% 7% 19%
D1 D2 D3 D4 D5 D6

12. Question I6
● Question I6:
● Does the initiative of using agile come from the business or from the IT
(BI team)?
● Answers:
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
22% 56% 11% 11%
Business IT Inconclusive Did not answer

13. Question I7
● Question I7:
● Do you have any preferred tools for agile BI management?
● Answers:
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
33% 44% 22%
No specific tool Uses specific tool to support agile development Did not answer

14. Conclusion of the research I.
● Reminder of the research goals:
● Q1: How often is agile development used in business intelligence
projects?
● Q2: What are the risks which can lead to unsuccessful implementation of
agile methodology in a BI project?
● Q3: How to deal with these risk?
● Answer for Q1:
● Agile methodologies are used roughly on 62% of the BI projects.
● Answer for Q2 and Q3:
● Answers to Q2 and Q3 are given in the context of Ishikawa diagram.
● Dimensions D3 (Measurements) and D5 (Processes) were the least
common which means that the answers to Q2 and Q3 will be given in the
context of D1, D2, D4 and D6.
● For each dimension root cause of the risk is provided and also the risk
mitigation.

15. Conclusion of the research II.
Dimension Root cause (risk) Risk mitigation
D1
(Environment)
Prevailing traditional
mindset.
Continuously promote advantages of agile BI to the
management and mentor existing staff how to use agile BI. Also
make sure that the business has business representative in the
core (project management) team. The initiative needs to come
from the IT side; it cannot be expected that business will
promote agile BI.
D2
(Management)
Managing BI projects
using agile without
considering specifics of
such projects.
Always keep in mind that agile was designed by OLTP
developers; OLAP (BI/DWH) projects are different and can
become very complex due to data integration element. Either
choose agile methodology that has been designed specifically
for BI (Extreme Scoping, BEAM, DAD) or use generic agile (like
Scrum) but make sure it is applied in a way that respects BI
project specifics.
D4
(People)
Inadequate agile skills.
It cannot be assumed that when someone is a good BI
developer he will easily pick up agile approach and become
valuable member of agile driven team. Training and coaching is
required. The team that recently started using agile on a project
should be supported by an experienced agile practitioner and
observed for a while. The risk of reverting back to waterfall or
developing agile-waterfall mix will be present.
D6
(Tools)
Using tools and
methodologies which
are not compatible with
agile.
Always challenge tools and methodologies the team is used to
use and ask whether these are compatible with agile. Especially
in the area of data modelling such an analysis needs to be
performed (in regards to traditional Inmon or Kimball
approaches to designing DWH).

16. References and useful agile BI
literature
● Agilemanifesto.org, 2013. Manifesto for Agile Software Development. [online] Available at:
http://agilemanifesto.org/ [Accessed: 19 Dec 2013].
● Avison, D. E. and Torkzadeh, G. 2010. Information systems project management. New Delhi: Sage
Publications/Sage South Asia
● Brobst, S., McIntire M., Rado E., 2008. Agile Data Warehousing With Integrated Sandboxing. Business Intelligence
Journal, 13(1), pp. 13-22.
● Bullington, K., 2012. Learning to Fish. Quality Progress, 45(7), pp. 16-21.
● Corr, L. and Stagnitto, J., 2012. Agile data warehouse design. Leeds: Decisionone Press.
● DeSarra, P., 2012. BI Dashboards the Agile Way. Business Intelligence Journal, 17(4), pp. 8-16.
● Evelson, B., Karel, R. and Others, 2010. Agile BI Out Of The Box. Forrester Research, p. 1.
● Gartner.com, 2013. Gartner IT Glossary - Business Intelligence (BI). [online] Available at:
http://www.gartner.com/it-glossary/business-intelligence-bi/ [Accessed: 18 Dec 2013].
● Hill, J., Moss, L.T., Sorenson, C. and Weeks, W., 2009. Agile Development. Business Intelligence Journal, 14(2), pp.
53-60.
● Hughes, R, 2013. Agile data warehousing project management. Waltham, MA: Morgan Kaufmann.
● Moss, L, 2013. Extreme scoping. [S.l.]: Technics Pubns Llc.
● Perry, M.S., 2006. A Fish(bone) Tale. Quality Progress, 39(11), pp. 88.
● Van Decker, J.,E. and Sinnett, W.M., 2013. The CFO's Top Technology Imperatives. Financial Executive, 29(5), pp.
25-28.
● Wikipedia, 2013. Agile software development. [online] Available at:
http://en.wikipedia.org/wiki/Agile_software_development [Accessed: 18 Dec 2013].

17. Final comments
● Please note that this presentation is just a very
brief overview of the research.
● The full research paper can be downloaded from
my LinkedIn profile.