Best Practices: Datawarehouse Automation Conference September 20, 2012 - Amsterdam


Published on

Published in: Technology, Business
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Best Practices: Datawarehouse Automation Conference September 20, 2012 - Amsterdam

  1. 1. Datawarehouse AutomationBest practices and customer cases20 september 2012AmsterdamErik
  2. 2. Agenda • BI Challenges of today • BI Ability Model & DWH Automation • Best practices • Cases: Rotterdam & KAS BANK • Q&A2 Best practices & customer cases
  3. 3. Centennium BI expertisehuis• Independent knowledge partner• Provides clients with the right skills, at the right time and the right way to maximize BI results• Consultancy, ad-interim support, project implementation and training services• We take or share responsibility for the execution and management and support of BI and DWH projects• We offer an extensive portfolio of courses and training services:• We provide our customers with the knowledge and practical insights required to be(come) self-sufficient in maintaining and expanding their BI-environments3 Best practices & customer cases
  4. 4. What are (y)our biggest BI-challenges today? • Create more business value • Empower users • Lower the overall cost • Deliver high quality BI products • Reduce complexity • Organize Business Intelligence to become more effective How can DWH automation contribute?4 Best practices & customer cases
  5. 5. DWH Automation – in perspectiveDEMAND Ability to Benefit BI Ability Ability to Model Ability to Implement Specify Ability to ExecuteSUPPLY 5 Best practices & customer cases
  6. 6. Automation – in perspective • Supports your ability to execute – Cuts complexity and resources • Requires a strategy to change focus from coding to (data) modeling But…. • You’re the solution – Automation is your instrument…6 Best practices & customer cases
  7. 7. The solution? • Automation vs. BI Professionals or…. • The tool is the tool, the methodology is the methodology, etc… • Team up to optimize our profession in helping organizations to reach ‘infinity and beyond…’ Community Professionals Methodologies Automation tools7 Best practices & customer cases
  8. 8. Back to the Challenges • How does DWH automation help face the challenges? • Directly and indirectly. • In short… DWH automation is great (!) if handled by a skilled and knowledgeable professional (you!)8 Best practices & customer cases
  9. 9. CDM: set of best practices • Best practice methodology for BI and DWH automation • Create datawarehouse and BI products fast, with high and constant quality and low cost • CDM includes: – Tooling (free, open source or licensed) – Modeling paradigms like, DV, 3NF, DM – Quality control mechanism – Agile development – Knowledge partnership, training, coaching • Quality control mechanism: extensive checklists and documentation • Knowledge transfer by training, certification and learning on-the-job9 Best practices & customer cases
  10. 10. CDM Knowledge Partnership Structure Model Generate Present Benefit Organize10 Best practices & customer cases
  11. 11. Your datawarehouse is like a diner in a 3 star restaurant… Suppliers Create Select cook Ingredients Cook Serve Eat book Manage the kitchen11 Best practices & customer cases
  12. 12. Knowledge partnership (suppliers)• Goal: self-supporting clients• On-the-job coaching, learning by doing• Training – BI fundamentals – Data Vault certification – Dimensional modelling – Tool training (partners)• Centennium supports it’s customers by (management) consultancy, assessments, projects, training and sourcing12 Best practices & customer cases
  13. 13. Structure (create cookbook) • Establish a common vision on DWH and BI and the role of automation • Identify needs, pain, benefits, goals • Create a roadmap: what and how? • Develop teams, knowledge, new roles: prepare organization for automation • Design automation architecture • Create short list tools • Select agile approach13 Best practices & customer cases
  14. 14. Model (select ingredients) • Information & requirements analysis – Business needs like KPIs, reports, cubes – Identify semantic gaps, business rules – MDM, MTM, Reference data – Business keys • Analyze & model source extraction to support automation • Model the staging, datawarehouse, data marts, meta layers, cubes • Select tool(s) for generation – DWH Automation tools – Combination of “classic” ETL tools and automation tools • Should fit the structure!14 Best practices & customer cases
  15. 15. Generate (Cook) • Create system setup with respect to automation architecture • Develop test scenarios – Generic testing of automation process (completeness and correctness) – Test compliance to architecture • Generate components and objects – Focus on understanding data, data modeling & business rules, not on coding • Test and implement15 Best practices & customer cases
  16. 16. Present (serve) • Deliver information products to users, fast and of high quality • Automation leverages BI self-service – Model and generate cubes – Model and generate business rules – Adapt quickly to changing source data and information needs – Reduced technical complexity – Speed up of the overall BI process16 Best practices & customer cases
  17. 17. Benefit (eat) • The business users can focus on creating value adding information products • Constant and predictable quality • Short time to market • IT can quickly adapt to changing business needs/focus • More time and resources for creating value- adding BI • In the end: lower cost, higher value • But…17 Best practices & customer cases
  18. 18. Organize (manage the kitchen) • Automation may have high impact on existing datawarehouse and BI teams – Elimination of tools, data manipulation processes and people… (cutting complexity and resources) – Also: new roles like source analyst, data model specialist, automation architect are introduced – Resistance by traditional vendors and suppliers • Automation can create high business value, but not on it’s own – A knowledgeable team is essential • We believe in self service: organize knowledge transfer a.s.a.p. and coach teams to be self supporting18 Best practices & customer cases
  19. 19. Rotterdam • Challenge: service team as a central and flexible point for management information delivery for employees and partners of Rotterdam • Structure: – As a knowledge partner we combined Rotterdam’s knowledge on Oracle eBS with DWH automation – Automation architecture “forces” Rotterdam to comply to the rules • Model & generate – DWH automation optimized for Oracle eBS – Reusable methods for other eBS customers • Benefit – Cuts complexity and lowers licensing & consultancy fees significantly – DWH is now of strategic importance and acts as a central data hub for Rotterdam – Self service for users19 Best practices & customer cases
  20. 20. • Challenge – Deliver integrated information instead of stovepipe data – Minimize development cost, maximize business value – Make KAS BANK self supporting as much as possible• Structure – Create a shared view on datawarehousing and automation benefits and proof it – Data Vault Methodology, near real-time and high volume – Strong focus on team training and coaching of architects• Model & generate – Re-usable extraction of complex Oracle and Adabas data stores – Business driven Data Vault and Dimensional models – Generated data integration, data distribution, EDW code, without ETL tools – Transparent and self supporting• Benefit – Significantly shorter time to market of information products – 100% reliability, auditability and predictability – Business is eager for more – Next step: introduce self-service at business level20 Best practices & customer cases
  21. 21. Automation – do’s • Do’s – Define an automation vision and strategy • end-to-end or step-by-step – Take your time and involve all stakeholders – Explain concepts, (business) benefits and potential risks – Consider a two-step approach: pilot - project – Align and train the development- and maintenance teams21 Best practices & customer cases
  22. 22. Automation –don ‘ts • Don’ts – Deviate from the Structure – Generate or automate the DWH as a goal, not a means to an end – Underestimate the need for presenting and benefiting from information… The proof of the pudding is in the eating!22 Best practices & customer cases
  23. 23. Experience with CDM and DWH Automation CDM is an evolving set of best practices Introducing additional modelling approaches We are partnering with DWH automation vendors Research topic: generating datawarehouse models directly from business process models 23 Best practices & customer cases
  24. 24. Next data vault certification seminar: November 1-2, 2012 Amsterdam www. 25 october, the Hague Crash Course Datawarehouse Automation Best practices & customer cases
  25. 25. ABOUT CENTENNIUM25 Best practices & customer cases
  26. 26. Centennium BI expertisehuis houses all the experts under oneroof, hereby offering all knowledge and expertise to address thecomplex business intelligence issues facing our clients todayFacts and figures: Services overview:• Founded: 1998 • Consultancy• 45+ business intelligence consultants • Projects • ResourcingCore values: • Education• Human Capital• In close collaboration Some of our clients:• Objective and Independent Woonbron, Albron, NZa, CAK, OBR, Vopak, several DutchExpertise: Municipalities, Aegon, Nutreco, TNO, Genz yme, Tata• Business intelligence Steel, KPN, DELTA, IKEA, Accell, TomTom,• Strategic, tactic and operational KAS BANK, LeasePlan, Brabant Water• Vision based on “effective BI” 26
  27. 27. Centennium BI expertisehuis Lange Voorhout 43 2514 EC s-Gravenhage Telefoon 070 31 20 370 Fax 070 31 20 371 URL