Retour d'expérience sur un projet de Business Intelligence réalisé à l'EVAM selon une méthodologie Agile et avec un modèle de données Data Vault. Présentation faite lors du Swiss Data Forum du 24 novembre 2015 à Lausanne

1. BÂLE BERNE BRUGG DUSSELDORF FRANCFORT S.M. FRIBOURG E.BR. GENÈVE
HAMBOURG COPENHAGUE LAUSANNE MUNICH STUTTGART VIENNE ZURICH
Agile Business Intelligence @ Evam

2. Plan
• Introduction ( F. Kang à Birang)
• Pre-project (F. Kang à Birang & J-M. Delacrétaz)
• Agile project management (A. Martino)
• Agile architecture (E. Fidel)
• Data quality (A. Martino)
• EVAM Feedback (B. Albietz)

3. Introduction
Fabienne Kang à Birang – Business Analyst / Product owner

4. Introduction
• EVAM Presentation
• Project Sponsor
• Director
• Indicators
• 2013 – Existing B.I.

5. Pre-Project Phase
Fabienne Kang à Birang – Business Analyst / Product owner
Jean-Marc Delacrétaz – Developer

6. Pre-Project Phase
• Target
• Operational reporting
• Problems encountered @ EVAM
• Data interpretation
• Business rules errors
• Prerequisites
• Dictionary
• Population hierarchized

7. Preexisting B.I.
• 2013
• P.O.C. to introduce B.I. «philosophy»
• Chosen Tools
• ETL : Talend
• Reporting : Tibco JasperReport
• Weaknesses
• Lack of expertise & methodology
• Bad performances

8. Decision in August 2014
• Start from scratch
• With Trivadis Lausanne as a partner
• Tools
• Performances
• Architecture with « Best practices »

9. Agile Project
Management
Adriano Martino – Senior B.I. Consultant

10. Agility
We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:
• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

11. Organisation
• Evam• Evam
• Trivadis
• Evam• Trivadis
Scrum Master
Product
Owner
CustomerDeveloppers

12. Agile Objectives
• Deliver working software frequently
• Adapt to change

13. Scrum components overview
Sprint
Planning
Sprint
Backlog
Product
Backlog
Daily
Stand up
Sprint
2 to 4
weeks
Sprint
Review
Retrospective

14. Normal Process for a B.I. need
Business
Analysis
Design of the
model
Implementation
Unit Testing
Volume
testing
User
Acceptance
Testing
New
Need
Rework
Rework Rework
Rework
Deployment
to Validation
Deployment Production

15. Normal Process for a B.I. need

16. Agile Objectives
• Adapt to change
• Deliver working software frequently
• At regular intervals, the team reflects on how
to become more effective

17. Cadence
SCRUM
EVENT
DRIVEN
Sprint1 Sprint2 Sprint3 …
RetrospectiveReviewReleasePlanning1 2
1
3 4
2 3 4 1 2 3 4 1 2 3 4
1 2 2 2 2 213 42

18. Agile Objectives
• Adapt to change
• Deliver working software frequently
• At regular intervals, the team reflects on how
to become more effective
• Work close to business

19. Collaborative Workshops
Business
Need
analysis
Technical
analysis
Live dev
Prototyping
Live
testing

20. Agile B.I. Architecture
• Evolutive
• Easy change management
• Parallelisable development
• Business oriented
• Integration
• Possibility to automate generation
We choose Data Vault
Modelling

21. Agile
Architecture
Eddie Fidel – Senior B.I. Consultant

22. STAGING
DYNAMIC ETL
Enterprise
Data
Warehouse
With data vault
Modeling
Agile Bi Architecture
SOURCES
Virtualized
Data Marts

23. STAGING
DYNAMIC ETL
Enterprise
Data
Warehouse
With data vault
Modeling
Data Warehouse Layer
SOURCES
Virtualized
Data Marts
DYNAMIC ETL

24. What is Data Vault ?
• Data Modelling Method for Data Warehouses in Agile Environments
• Developed by Dan Linsted
• Suitable for
• DWH Core Layer
• Optimized for
• Agility / Integration /
Historization

25. Data Vault composition
• Decomposition of Source Data
• Split Data into Separate Parts
Hubs Business Entity
Links Relations
Satellites Contexts
Business Oriented

26. Data Vault composition
• Elements : Hub – Link – Sat
Customer
Sat
Sat
Sat
Customer Product
Sat
Sat
Sat
Product
Hub = List of Unique Business Keys
Link = List of Relationships, Associations
Satellites = Descriptive Data
Order
Sat
Sat
Sat
Order
Link

27. Avantages and challenges
• Standard ETL Rules to Load Data Vault
• Easy Extensibility of Data Vault Model
• Integration of Multiple Source Systems
• Traceability and Complete History
• High Number of Tables in Data Vault

28. What does the Data Vault generator do ?
• Tables
• Indexes
• Surrogate keys
• Foreign keys
• Partitions
• Loading process
• SCD1 / SCD2
• Loading audits
• Handling Errors

29. Generator value
29
Business spec
Technical spec
Development
Test
Deployment
Qualityassurance
Documentation
Simplify
Generator
Documentation
QS
Total Savings
 Fast and short implementation cycles
 Broad flexibility of change
 Auto-generated quality assured components
 Huge time and cost savings
On-going and recurrent with each
step of modification or enlargement!!!

30. STAGING
DYNAMIC ETL
Enterprise
Data
Warehouse
With data vault
Modeling
Dynamic ETL
SOURCES
Virtualized
Data Marts

31. Dynamic ETL for DWH
• Parallel Loading
• HUB
• LINK et SAT
• Dynamic call to loading procedures
• No deployment of ETL needed

32. STAGING
DYNAMIC ETL
Enterprise
Data
Warehouse
With data vault
Modeling
Dynamic ETL
SOURCES
Virtualized
Data Marts
DYNAMIC ETL

33. Data Mart
• Business Need Oriented
• Virtualized DM (materialized view)
• Can be regenerated from scratch
• Find value at a point in time
• Good perfomance
• Automatically regenerated (no deployment)

34. Data Quality
Adriano Martino – B.I. Consultant

35. Quality report
• Automated
• Daily execution
• Simple development
• Possible to send mail based on result
• Direction support to involve Business

36. EVAM Feedback
Bruno Albietz – I.T. Manager

37. Keys Learnings
• Show business value as early as possible and keep the ball rolling
• Project: December 2014 – June 2016
• Phased implementation: 1st output in June 2015, then regular outputs
on a monthly basis
• Be prepared to spend most of your time on data quality
• The lifeblood of B.I. projects

38. Keys Learnings
• Prepare knowledge transfer to your staff during the project
• Modelling, ETL, Reporting
• Good project management practice, from business requirements to
report development
• Increase user buy-in with Scrum
• Key users and management involved from day 1

39. Keys Learnings
• Learn to say “ No ”
• B.I. quality versus business process quality
• B.I. is also here to show process deficiencies, do not try to solve all
business issues within the B.I. project

40. Q & A