The document discusses best practices for data warehouse automation. It covers challenges organizations face with business intelligence (BI), how data warehouse (DWH) automation can help address these challenges, and the Centennium BI Ability Model for DWH automation. Case studies of successful DWH automation projects at Rotterdam and KAS BANK are provided. The presentation also outlines the Centennium Methodology (CDM) for DWH automation best practices and concludes with information about Centennium as an independent BI expertise organization.

1. Datawarehouse Automation
Best practices and customer cases
20 september 2012
Amsterdam
Erik Fransen
e.fransen@centennium.nl
@erikfransen

2. Agenda
• BI Challenges of today
• BI Ability Model & DWH Automation
• Best practices
• Cases: Rotterdam & KAS BANK
• Q&A
2 Best practices & customer cases

3. Centennium BI expertisehuis
• Independent knowledge partner
• Provides clients with the right skills, at the right time and
the right way to maximize BI results
• Consultancy, ad-interim support, project implementation
and training services
• We take or share responsibility for the execution and
management and support of BI and DWH projects
• We offer an extensive portfolio of courses and training
services: www.bi-opleidingen.nl
• We provide our customers with the knowledge and
practical insights required to be(come) self-sufficient in
maintaining and expanding their BI-environments
3 Best practices & customer cases

4. What are (y)our biggest BI-challenges today?
• Create more business value
• Empower users
• Lower the overall cost
• Deliver high quality BI products
• Reduce complexity
• Organize Business Intelligence to become more
effective
How can DWH automation contribute?
4 Best practices & customer cases

5. DWH Automation – in perspective
DEMAND
Ability to
Benefit
BI Ability
Ability to Model Ability to
Implement Specify
Ability to
Execute
SUPPLY
5 Best practices & customer cases

6. Automation – in perspective
• Supports your ability to execute
– Cuts complexity and resources
• Requires a strategy to change focus from
coding to (data) modeling
But….
• You’re the solution – Automation is your
instrument…
6 Best practices & customer cases

7. The solution?
• Automation vs. BI Professionals or….
• The tool is the tool, the methodology is the
methodology, etc…
• Team up to optimize our profession in helping
organizations to reach ‘infinity and beyond…’
Community
Professionals
Methodologies Automation tools
7 Best practices & customer cases

8. Back to the Challenges
• How does DWH automation help face the
challenges?
• Directly and indirectly.
• In short… DWH automation is great (!) if
handled by a skilled and knowledgeable
professional (you!)
8 Best practices & customer cases

9. CDM: set of best practices
• Best practice methodology for BI and DWH automation
• Create datawarehouse and BI products fast, with high and
constant quality and low cost
• CDM includes:
– Tooling (free, open source or licensed)
– Modeling paradigms like, DV, 3NF, DM
– Quality control mechanism
– Agile development
– Knowledge partnership, training, coaching
• Quality control mechanism: extensive checklists and
documentation
• Knowledge transfer by training, certification and learning
on-the-job
9 Best practices & customer cases

10. CDM
Knowledge Partnership
Structure Model Generate Present Benefit
Organize
10 Best practices & customer cases

11. Your datawarehouse is like a diner in a 3 star
restaurant…
Suppliers
Create
Select
cook Ingredients
Cook Serve Eat
book
Manage the kitchen
11 Best practices & customer cases

12. Knowledge partnership (suppliers)
• Goal: self-supporting clients
• On-the-job coaching, learning by doing
• Training
– BI fundamentals
– Data Vault certification
– Dimensional modelling
– Tool training (partners)
• Centennium supports it’s customers by
(management)
consultancy, assessments, projects, training and
sourcing
12 Best practices & customer cases

13. Structure (create cookbook)
• Establish a common vision on DWH and BI and
the role of automation
• Identify needs, pain, benefits, goals
• Create a roadmap: what and how?
• Develop teams, knowledge, new roles: prepare
organization for automation
• Design automation architecture
• Create short list tools
• Select agile approach
13 Best practices & customer cases

14. Model (select ingredients)
• Information & requirements analysis
– Business needs like KPIs, reports, cubes
– Identify semantic gaps, business rules
– MDM, MTM, Reference data
– Business keys
• Analyze & model source extraction to support
automation
• Model the staging, datawarehouse, data
marts, meta layers, cubes
• Select tool(s) for generation
– DWH Automation tools
– Combination of “classic” ETL tools and automation tools
• Should fit the structure!
14 Best practices & customer cases

15. Generate (Cook)
• Create system setup with respect to
automation architecture
• Develop test scenarios
– Generic testing of automation process
(completeness and correctness)
– Test compliance to architecture
• Generate components and objects
– Focus on understanding data, data modeling &
business rules, not on coding
• Test and implement
15 Best practices & customer cases

16. Present (serve)
• Deliver information products to users, fast and
of high quality
• Automation leverages BI self-service
– Model and generate cubes
– Model and generate business rules
– Adapt quickly to changing source data and
information needs
– Reduced technical complexity
– Speed up of the overall BI process
16 Best practices & customer cases

17. Benefit (eat)
• The business users can focus on creating value
adding information products
• Constant and predictable quality
• Short time to market
• IT can quickly adapt to changing business
needs/focus
• More time and resources for creating value-
adding BI
• In the end: lower cost, higher value
• But…
17 Best practices & customer cases

18. Organize (manage the kitchen)
• Automation may have high impact on existing
datawarehouse and BI teams
– Elimination of tools, data manipulation processes and
people… (cutting complexity and resources)
– Also: new roles like source analyst, data model
specialist, automation architect are introduced
– Resistance by traditional vendors and suppliers
• Automation can create high business value, but not
on it’s own
– A knowledgeable team is essential
• We believe in self service: organize knowledge
transfer a.s.a.p. and coach teams to be self
supporting
18 Best practices & customer cases

19. Rotterdam
• Challenge: service team as a central and flexible point for
management information delivery for employees and
partners of Rotterdam
• Structure:
– As a knowledge partner we combined Rotterdam’s knowledge on
Oracle eBS with DWH automation
– Automation architecture “forces” Rotterdam to comply to the
rules
• Model & generate
– DWH automation optimized for Oracle eBS
– Reusable methods for other eBS customers
• Benefit
– Cuts complexity and lowers licensing & consultancy fees
significantly
– DWH is now of strategic importance and acts as a central data hub
for Rotterdam
– Self service for users
19 Best practices & customer cases

20. • Challenge
– Deliver integrated information instead of stovepipe data
– Minimize development cost, maximize business value
– Make KAS BANK self supporting as much as possible
• Structure
– Create a shared view on datawarehousing and automation benefits and proof it
– Data Vault Methodology, near real-time and high volume
– Strong focus on team training and coaching of architects
• Model & generate
– Re-usable extraction of complex Oracle and Adabas data stores
– Business driven Data Vault and Dimensional models
– Generated data integration, data distribution, EDW code, without ETL tools
– Transparent and self supporting
• Benefit
– Significantly shorter time to market of information products
– 100% reliability, auditability and predictability
– Business is eager for more
– Next step: introduce self-service at business level
20 Best practices & customer cases

21. Automation – do’s
• Do’s
– Define an automation vision and strategy
• end-to-end or step-by-step
– Take your time and involve all stakeholders
– Explain concepts, (business) benefits and potential
risks
– Consider a two-step approach: pilot - project
– Align and train the development- and maintenance
teams
21 Best practices & customer cases

22. Automation –don ‘ts
• Don’ts
– Deviate from the Structure
– Generate or automate the DWH as a goal, not a
means to an end
– Underestimate the need for presenting and
benefiting from information…
The proof of the pudding is in the eating!
22 Best practices & customer cases

23. Experience with CDM and DWH Automation
 CDM is an evolving set of best practices
 Introducing additional modelling approaches
 We are partnering with DWH automation vendors
 Research topic: generating datawarehouse models directly from business process models
23 Best practices & customer cases

24. Next data vault certification seminar:
November 1-2, 2012 Amsterdam
www.data-vault.nl
www. geneseeacademy.com
25 october, the Hague
Crash Course Datawarehouse Automation
www.bi-opleidingen.nl
24 Best practices & customer cases

25. ABOUT CENTENNIUM
25 Best practices & customer cases

26. Centennium BI expertisehuis houses all the experts under one
roof, hereby offering all knowledge and expertise to address the
complex business intelligence issues facing our clients today
Facts and figures: Services overview:
• Founded: 1998 • Consultancy
• 45+ business intelligence consultants • Projects
• Resourcing
Core values: • Education
• Human Capital
• In close collaboration Some of our clients:
• Objective and Independent Woonbron, Albron, NZa, CAK, OBR, Vopak,
several Dutch
Expertise: Municipalities, Aegon, Nutreco, TNO, Genz
yme, Tata
• Business intelligence
Steel, KPN, DELTA, IKEA, Accell, TomTom,
• Strategic, tactic and operational KAS BANK, LeasePlan, Brabant Water
• Vision based on “effective BI”
26

27. Centennium BI expertisehuis
Lange Voorhout 43
2514 EC 's-Gravenhage
Telefoon 070 31 20 370
Fax 070 31 20 371
URL www.centennium.nl
www.bi-opleidingen.nl
www.data-vault.nl
27