"We now do Agile BI too” is often heard in todays BI community. But can you really "create" agile in Business Intelligence projects? This presentation shows that Agile BI doesn't necessarily start with the introduction of an iterative project approach. An organisation is well advised to establish first the necessary foundations in regards to organisation, business and technology in order to become capable of an iterative, incremental project approach in the BI domain. In this session you learn which building blocks you need to consider. In addition you will see what a meaningful sequence to these building blocks is. Selected aspects like test automation, BI specific design patterns as well as the Disciplined Agile Framework will be explained in more and practical details.

1. Introducing Agile Business
Intelligence Sustainably:
Implement the Right Building
Blocks in the Right Order
Workshop Excercises
Raphael Branger, IT-Logix AG

3. ERP, CRM, Excel,
…
Weather data,
Demographic
data, Social
Media…
The BI strategy is rooted in the overall business strategy. The IT strategy provides
guidance regarding its implementation. On the other hand, BI can be seen as a
company in the company: You have a market (& your BI offerings) as well as
customers. Typically a fast «time to market» for a BI solution is critical to your
customers!
A BI strategy consists of a vision and
mission statement. It defines objectives
which are broken down in partial strategies
(organisation, data, applications).
Development speed & high agility is often
an objective – an agile mindset and values
are the strategic guidelines to implement
the strategy.
One needs BI-specific processes to develop new solutions as well as
operate, maintain and extend existing solutions. Agile processes like
Disciplined Agile are well suited to fullfil strategic objectives.
BI needs a dedicated organisation. Following an
agile path, you should organize resources around
teams, not projects.
Data from various sources is Extracted,
Tranformed and Loaded into a central place,
the data warehouse (DWH). During this
process data is technically cleansed,
semantically and often technically integrated.
History of records is preserved. The DWH
contains the facts (what is delivered by the
source).
Data marts put facts (from
the DWH) into a specific
context. The context
drives the application of
business rules valid for
this context. Data marts
provide a «semantic
layer» to translate
technical attribute names
into end user language.
BI Applications are the visible part of a BI system. A
BI application typically consists of two layers:
information products (like reports, dashboards or
data science based predictions) as well as an
underlaying server platform (providing features like
authentication & autorisation, scheduling, web &
mobile access etc.)
ABIsystemcanserveabroadrangeofusergroups.These
includeemployeesofthecompanyitselfbutaswellofpartners
likesuppliersexternaltothecompany.Sometimeseven
customerscanaccessandusetheBIsystem
From left to right data is refined to information and knowledge. On the left side we tend to find more technical metadata like «from which
source does this record come from». The more we get closer to the end user we’ll have more business metadata like the definition of a certain
measure. Process metadata gives us information about how the data was transported through the system, e.g. «when was the data
refreshed».
A data scientist works with data from various sources incl. the DWH or data marts. It employs techniques and
theories drawn from many fields within the broad areas of mathematics, statistics, information science, and
computer science, in particular from the subdomains of machine learning, classification, cluster analysis, data
mining, databases, and visualization (Source: Wikipedia)

5. EXERCISE 2 «AGILE BI BUILDING BLOCKS»

6. 7
IT-LOGIX AGILE BI BUILDING BLOCKS (V2.0 EN)
Agility
Amount of
upfront design needed
Basic
Infrastructure
Basic Patterns &
Standards
Agile Mindset & Organisation
AgileBasics&Mindset
Agile Infrastructure & Patterns
BI-specific
User Stories
Envision Cycle &
Inception Phase
No / Relative
Estimating
(Self Organizing)
Team
Workspace &
Co-Location
Identify & Apply
Design Patterns
Version Control
Test Automation
Test Driven
Development
Sandbox
Development
Infrastructure
Continuous
Integration
Push-Button
Deployments
Multi Layer
Refactoring
Heavy Iterative
Data Modeling
Gulf
Chasm
Agile Contracting
Scriptable BI/DWH
toolset
«Tweaked» Waterfall
«Timeboxed» Iterations
«Lean» Development
Lean & Fast
Processes
Legend
Processes
& Organi-
sation
Develop-
ment
Methods
Techno-
logies
Values &
Principles
© by Raphael Branger, IT-Logix AG, www.it-logix.ch
Data Lab
Sandboxes
Continuous Delivery
Data Modeling &
Metadata Mgt.
Data Warehouse
Automation
Basic Refactoring
Additive Iterative
Data Modeling
10
2
3
4
5
6
71
8
9
11
12
13
14
15
16
17
18

7. Individuals and interactions over processes and tools
Working BI solution over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
Source:
https://www.infoq.com/articles/modern-agile-intro
http://www.agilemanifesto.org

10. 600
750
450
Estimate:
“to judge tentatively or approximately
the value, worth, or significance of”
(Merriam-Webster)
An estimate is unbiased.
An estimate is subjective.
What is a good estimate?
If the estimate is +- 25% of the actual
value, for 75% of all cases.
(Source: Conte, Dunsmore, and Shen 1986, in: Steve McConnell, Software Estimation:
Demystifying the Black Art)
Story points help to estimate tasks / stories.
Planning poker helps with relative
estimation as it compares the effort
between several tasks or stories.
«Estimates do not directly add value to your
process, so we want to find ways to reduce
the estimation process or even stop it where
possible»
(Vasco Duarte)

11. Team Lead = “Facilitator”
Product Owner
“most important” decision maker
Business person who actively
participates throughout the project
Responsible for the product vision
of the BI solution
Prioritizes the tasks to maximize the
value for the stakeholder
Architecture Owner
Represents the enterprise IT
architecture
Ensures that the BI solution fits into
the governance
Source: http://www.disciplinedagiledelivery.com/roles-on-dad-teams/

13. Is the Envision – Explore approach feasible at all
in highly regulated environments with a lot of compliance requirements?
Trust is important to work exploratorily.
?
15
Source & Copyright: Peter Stevens, http://www.scrum-breakfast.com/

14. Source: DataViz for Dummies / http://bidashboardformula.com/

15. Model Driven Approach
Data Driven Approach
A definition by Dr. Barry Devlin:
“Data warehouse automation is the use of an integrated
set of tools and techniques that automate the design,
delivery and maintenance of data warehouses and
marts .”
automation addresses the old conundrum of
delivering consistent, quality data in the
timeframe demanded by modern business
needs
streamlining the overall process provides a
single repository of metadata and integrated
tooling to speed and simplify development
business and IT can truly collaborate with an
agile approach
http://www.mid.de/leistungen/tools/innovator
https://www.wherescape.com/products-services/our-software-products/

16. In theory it would be ideal if the content of a “release” is kept in one place.
How to deal with this idea in practice is highly dependent on the available tools and their own versioning capabilities.
Don’t forget there is a close link between versioning and deployment.
Tool Specific Versioning
18
Generic Versioning
DWA /
ETL Tool
BI Tool 1
BI Tool 2
Files, Scripts, …
Versioning Tool
Development Versioning Release VersioningExport to archive file
Development Environment Test Environment Production Environment
Versioning
Deployment

17. Source: Amber Debroux and Chris reed, “How to Build Trust in Your Data Warehouse”, in Business Intelligence Journal, Volume 20, Number 1, S. 50

18. Source: www.disciplinedagiledelivery.com/lifecycle/

19. Allocate a Data Story to a “Timeline Type”:
Discrete Event:
Customers purchases product in a store.
Recurring Event:
Customers generate revenue.
Evolving Event:
Customer orders product in the online-shop.
A Timeline Type can be mapped with a Design
Pattern for the corresponding fact table:
Discrete Event:
Transaction Fact Table
Recurring Event:
Periodic Snapshot Fact Table
Evolving Event:
Accumulating Snapshot Fact Table
Design Patterns from the different Fact Tables
can used to formulate Design Pattern for the
corresponding ETL (ELT) transformations…
Source: www.modelstorming.com/

20. Source: http://www.agiledata.org/essays/databaseRefactoring.html

21. Source: http://www.agilemodeling.com/essays/introductionToAM.htm

22. Source: http://agiledata.org/essays/tdd.html
Source: https://bigeval.com/en/data-warehouse-etl-testing/

24. Checkout what is
needed from the
Version Control
Development of
all DBs and
DB-Schemas
Development of
all physical
data models
Installation of all
needed stored
procedures
Installation of all
ETL-Modules
Configuration of
OLAP-Cube
(if needed)
Frontend
Installation
(Dashboards,
Reports, etc.)
Configure
DB-connection
Execute initial
Dataload
Build
OLAP-Cube
Execute and
automate testing

25. EXERCISE 3 «BI USER STORIES»

26. Exercise 3 «BI User Stories»
DWH
Gather together in teams of two to four
people.
Take the excercise sheets handed out.
FactEventParticipant
RegisterDate
EventID
ParticipantID
NoShow (Y/N)
(Count participants)
DimEvent
EventDate
Country
City
Venue Address
Location (Geo)
Max. Participants
DimDate_Register
DateValue
DimParticipant
Name
Member Category
Roundtable
Registration
System
(Web Service
or CSV export)
TDWI
Membership
System
(SQL Server)
Define at least three user stories. Remember the User
Story should be small enough to be implmented in 1
single day.
Timebox 10 minutes.
DWH
Automation
Tool
Feature 1

28. Feature (following the regular User Story schema):
As a TDWI Backoffice employee, I need to see the number of registered participants for a Roundtable
event so that I can organize the logistics for this event.
Connectivity Epic (following the FDD schema) (<action> the <result> <by|for|of|to> <object>)
Extract the event and participant data of the web based Roundtable Registration System to a CSV file.
Connectivity User Story (following the FDD schema):
Manually export the event and participant data for all events to a CSV file.
<write a couple of other stories>
Possible User Stories (Connectivity & Infrastructure)

29. Feature (following the regular User Story schema):
As a TDWI Backoffice employee, I need to see the number of registered participants for a Roundtable event
so that I can organize the logistics for this event.
DWH Epic (following the FDD schema) (<action> the <result> <by|for|of|to> <object>)
Model and load the event and participant data of the web based Roundtable Registration System to the DWH
and Data Mart.
DWH User Story (following the FDD schema):
Model and (full) load the event master data (without Location / Geo info, not historized) to DimEvent on the
DWH layer.
<write a couple of other stories>
Possible User Stories (DWH)

30. Feature (following the regular User Story schema):
As a TDWI Backoffice employee, I need to see the number of registered participants for a Roundtable
event so that I can organize the logistics for this event.
BI Application Epic (following the regular User Story schema)
As a TDWI Backoffice employee, I need a BI application to see the number of registered participants for
a Roundtable event so that I can organize the catering for this event.
BI Application User Story (following the regular User Story schema):
As a TDWI Backoffice employee I need to see the number of registered participants for the next
Roundtable in a selected location so that I can organize the catering for this event.
<write a couple of other stories>
Possible User Stories (BI Application)