Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to get started with Agile BI

1,707 views

Published on

Overview of Agile BI with maturity model to assess your current state and determine steps to move forward

Published in: Technology, Business
  • Be the first to comment

How to get started with Agile BI

  1. 1. Agenda ◊ Agile BI Definition ◊ Assess Your Current State ◊ Then… – – – Select an Agile Methodology Have a Kickoff Inspect and Adapt
  2. 2. BI BI encompasses all aspects of a system needed to produce meaningful information to drive data driven decision making – – – – Data Processing (Cleansing, Transforming, Loading) Data Architecture and Warehousing Data Analysis and Visualization Tools System Architecture
  3. 3. Agile
  4. 4. Agile BI Applying an Agile mindset to business intelligence ◊ Using an iterative, incremental, evolutionary approach ◊ Focusing on value-driven development ◊ Delivering production quality applications ◊ Using barely sufficient processes ◊ Automating everything ◊ Collaborating with the customer ◊ Encouraging self-organizing and self managing teams
  5. 5. Why Agile? The people who need the data see the data You need different data? Sure! The data source changed? We’re on it! Business Intelligence, enough said See Visibility (plus lots of testing!) Source: http://www.versionone.com/Agile101/Agile-Software-Development-Benefits/
  6. 6. Agile BI Maturity Model Level 3 *All team members can independently complete any task from database design to report creation *It’s not about getting your job done it’s about getting the job done Level 2 *Increased collaboration Example: ETL developers work with data modelers to come up with a database design that balances the tradeoffs between reporting and loading *Decreased formality in interactions across skill sets Level 1 Team Roles / Skill Sets *Collaboration among people with the same skill set - Example: data modelers work with other data modelers *Official transitions and likely disagreement across skill sets Example: ETL developers are given source to target mappings when the data modelers complete the database design and are upset that the design is hard to load Technical Architecture Engineering Practices *Clear understanding of data’s business value *Clear understanding of the purpose for each component of the technical architecture *End-to-end use of optimal engineering practices *Team self-enforces usage through criteria for completing work *Active effort to clarify understanding of data’s business value *Streamlined architecture where possible *Process to deprecate unused components *Numerous (possibly) redundant layers (staging, ODS, EDW, data marts, etc.) *Inclusion of data with no clear business value *Lingering tables, reports, ETL scripts, with no known purpose *Some configuration management (SQL scripts to create all db objects are under CM, but not ETL and report information) *Some automation is in place (perhaps to promote new objects or code to another environment or to test ETL) *Building blocks of the system (db create scripts, ETL packages, report files, etc.) are not maintained in any central location nor are they under configuration management *Files are manually copied from one environment to another
  7. 7. Assess Your Current State ◊ How well is your team setup for collaboration and change?
  8. 8. Agile BI Maturity Model Level 3 *All team members can independently complete any task from database design to report creation *It’s not about getting your job done it’s about getting the job done Level 2 *Increased collaboration Example: ETL developers work with data modelers to come up with a database design that balances the tradeoffs between reporting and loading *Decreased formality in interactions across skill sets Level 1 Team Roles / Skill Sets *Collaboration among people with the same skill set - Example: data modelers work with other data modelers *Official transitions and likely disagreement across skill sets - Example: ETL developers are given source to target mappings when the data modelers complete the database design and are upset that the design is hard to load
  9. 9. Agile BI Maturity Model Level 3 Frequently and consistently support the team’s self organized culture - let the team define their own success criteria - don’t overly proscribe HOW things must be done, for example, a source to target mapping may be required, but let the team decide how that information gets captured Level 2 Determine where there are skill set gaps - obtain training to fill the gaps and/or - pair up team members to cross train each other Have informal lunch and learns to share knowledge and perspectives, building team communication and trust Level 1 Team Roles / Skill Sets Create a 5 – 7 person team with a mix of all the skill sets needed to take data from a source system and get it into the hands of end users that need it to make decisions
  10. 10. Assess Your Current State ◊ What is your current technical architecture? What aspects present the biggest challenges to incremental evolution and change?
  11. 11. Change Is… Avoidable Inevitable
  12. 12. Change Is… ◊ ◊ ◊ ◊ ◊ ◊ ◊ ◊ ◊ ◊ ◊ ◊ ◊ Grain of fact table New type 2 attribute Change from type 1 to type 2 Multi-purpose column or table Redundant data Tables with too many columns or rows “Smart” columns Complex ETL objects Large SQL modules Unconformed Dimensions Indiscriminate use of materialized views Underutilization of materialized views Overreliance on documentation
  13. 13. Agile BI Maturity Model Level 3 *Clear understanding of data’s business value *Clear understanding of the purpose for each component of the technical architecture Level 2 *Active effort to clarify understanding of data’s business value *Streamlined architecture where possible *Process to deprecate unused components Level 1 Technical Architecture *Numerous (possibly) redundant layers (staging, ODS, EDW, data marts, etc.) *Inclusion of data with no clear business value *Lingering tables, reports, ETL scripts, with no known purpose
  14. 14. Agile BI Maturity Model Level 3 Keep it up! Don’t let complexity creep in. If there is a choice between refactoring an existing column and adding a new one that is almost identical – choose refactoring! Level 2 *Create a central repository for the entire organization with clear, consistent definitions of all major business data *Get rid of things that are no longer being used Level 1 Technical Architecture *Identify redundancy – are multiple tables capturing very similar information? Do multiple databases get populated with minor transformations between them? Is there a way to combine or streamline things?
  15. 15. Assess Your Current State ◊ Do you follow technical practices that can enable agility?
  16. 16. Agile BI Maturity Model Level 3 *End-to-end use of optimal engineering practices *Team self-enforces usage through criteria for completing work Level 2 *Some configuration management (SQL scripts to create all db objects are under CM, but not ETL and report information) *Some automation is in place (perhaps to promote new objects or code to another environment or to test ETL) Level 1 Engineering Practices *Building blocks of the system (db create scripts, ETL packages, report files, etc.) are not maintained in any central location nor are they under configuration management *Files are manually copied from one environment to another
  17. 17. Agile BI Maturity Model Level 3 *Hold your team to adding automated tests for everything new and work to reduce the technical debt incurred for existing things that do not have automated testing Level 2 *Start creating automated tests. Create test data for an existing or new ETL process and have scripts that check the load for expected results. Add this for all ETL over time. Level 1 Engineering Practices *Start putting files into a configuration management system *Work out the kinks of deploying new and changed tables, ETL code, etc. to another environment
  18. 18. Select an Agile Methodology Scrum Kanban BDD RUP XP And so on…
  19. 19. Have a Kick Off PEANUTS © 2013 PEANUTS Worldwide LLC Image from http://whodeyfans.com/2011/08/01/tis-the-season-for-charlie-brown-kicking-the-football/
  20. 20. Inspect and Adapt
  21. 21. Upcoming Meetups ◊ Wednesday, December 11, 2013 – Anne Marie Macek, Senior Manager – eCommerce, Marriott – Ecommerce Data and Web Analytics ◊ Wednesday, January 15, 2014 – Eric Nelson, Managing Principal, Synaptitude – Facilitated discussion on up front data behavior understanding and cleansing using agile approach/methodologies ◊ What are you interested in hearing more about?
  22. 22. Contact Information

×