Considerations for Streamlining Your
             Business Intelligence Implementation
                                   ...
Brief Introduction - Endasol (www.endasol.com)
    Focus Areas
          Data Warehouses/ BI Efforts
                 Lead...
Streamlining BI Implementation - Topics

           Components of Business Intelligence

           Methodologies for Stre...
Components of BI Systems




© 2008 Endasol, Inc. All Rights Reserved.
BI System Integration & Information Layers
    Data Warehouse
          Data movement from Sources to Target Tables
      ...
BI System Integration Layer

         Pre-established Reports
                Routine reports designed & developed in adva...
BI System Intelligence Layer

        The value of the BI System is often not realized until deploying this
        Layer
...
BI System Intelligence Layer

        Advanced Analytics
               Exploratory Data Mining – seeking new insights thr...
Predictive Analytics; Extending the Value of Your Data Warehousing Investment,
               2007 TDWI Best Practices Rep...
Streamlining BI Implementation
      Employ methods that get to the higher value BI layers sooner:

             Establish...
Establish Fundamentals Early for Any Phase
    Implement a Data Governance Program Upfront
          Assign Data Stewards ...
Establish Fundamentals Early for Any Phase

    Set realistic expectations from both business & IT perspectives
          ...
Use of Packaged BI Solutions & Starter Models

       Packaged BI Solutions: 3rd party products fulfilling a slice of your...
Use of Packaged BI Solutions & Starter Models

         A Packaged BI Solution is typically not:
               A front li...
Endasol’s                                        Flow Diagram


                                                    premir...
Use of Packaged BI Solutions & Starter Models
    Advantages:
          Requirements/Vision already defined – you see what...
Consider Agile Development for BI - Background
    Waterfall Methodology – Traditional Approach
          From Wikipedia: ...
Consider Agile Development for BI
    Agile Development is gaining interest - Agile is now included in training
    classe...
Consider Agile Development for BI Implementation

       Biggest Issue with many DW/BI projects
             Changes durin...
Consider Agile Development for BI Implementation

       Advantages of Agile Development
             Design at the beginn...
Consider Agile Development for BI Implementation

       Disadvantages of Agile Development
             Budgeting difficu...
Hybrid Approach for BI Implementation

         Optimum Approach may be a Hybrid between Waterfall & Agile
               ...
Hybrid Approach for BI Implementation

           Establish Project                High Level Design                  Infr...
Streamlining Interactive Layer

       Choosing Reporting and Querying Tools
              Business Ease of Use

         ...
Streamlining Interactive Layer

           Give Business Early Access to Data
                 Standard reports should be ...
Streamlining Intelligence Layer - KPIs
       Restrict Key Performance Indicators (KPIs) to a small number (6 –
       10)...
Streamlining Intelligence Layer – Visualizing

          Choosing Technology
                Numerous products available w...
Streamlining Intelligence Layer – Visualizing
         Implementing Dashboards & Scorecards

               Keep it simple...
Streamlining Intelligence Layer – Data Mining

      Determine Data Mining or Advanced Analytical Methodology
      Early
...
Align Your BI System with Corporate Objectives
           Ultimately, your BI System should be aligned to corporate object...
Streamlining BI Implementation - Summary

         Align your BI System with Corporate Objectives

         Value of your ...
Streamlining BI Implementation - Summary

      Objective is to get to these value BI Layers as soon as possible:
        ...
Upcoming SlideShare
Loading in …5
×

Endasol Streamlining Bi Implementation, Euci March 2008

691 views
615 views

Published on

Presenation at March 2008 EUCI Conference. Bill Dagan and Susan Hoffman

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
691
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
31
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Endasol Streamlining Bi Implementation, Euci March 2008

  1. 1. Considerations for Streamlining Your Business Intelligence Implementation March 2008 © 2008 Endasol, Inc. All Rights Reserved.
  2. 2. Brief Introduction - Endasol (www.endasol.com) Focus Areas Data Warehouses/ BI Efforts Lead Roles in PJM Enterprise-Wide Data Warehouse prior to forming Endasol Called back to PJM to help develop a separate Market Monitoring Data Mart Just underway with helping to identify business requirements and early design for a major data mart at an ISO. Business Analysts for a couple different energy companies’ major data consolidation & reporting projects Developed Comprehensive Solution for Wholesale Power P&L + Other KPIs Systems Integration – ETRM, Scheduling, Settlements, Generator Management, Fuel Inventory ISO Operations & Interfaces - PJM, MISO, NYISO, ISO-NE Developed Streamlined Interfaces to many of the PJM eSuite Applications Direct Experience in estimating Major Settlement Line Items Business Process Development/ Refinement © 2008 Endasol, Inc. All Rights Reserved.
  3. 3. Streamlining BI Implementation - Topics Components of Business Intelligence Methodologies for Streamlining a BI Implementation Relationship to Business Process Management © 2008 Endasol, Inc. All Rights Reserved.
  4. 4. Components of BI Systems © 2008 Endasol, Inc. All Rights Reserved.
  5. 5. BI System Integration & Information Layers Data Warehouse Data movement from Sources to Target Tables Usually scheduled in batched ETL processes Staging Tables or Operational Data Store often an intermediate step Data Model promotes business user reporting & querying – Dimensional Data Propagation Data movement among Systems Usually in Real Time or near Real Time Use of Enterprise Application Integration (EAI) tools to publish & subscribe to data Data Warehouse can be a Subscriber Data Federation No data movement Pre-established Views are often constructed to relate data across systems More suitable for highly decentralized organizations A sensible approach may involve elements of each method © 2008 Endasol, Inc. All Rights Reserved.
  6. 6. BI System Integration Layer Pre-established Reports Routine reports designed & developed in advance – often scheduled Typically ‘locked down’ and controlled by a couple of individuals - often IT May build intermediate tables, views and/or calculated fields to feed reports Self Service Reports and Queries Business users should be able to do reporting & querying without the need for IT Business users do not always know exactly what they want until they start using the data in ways not previously possible Requires a sound foundation (typically dimensional data model), data dictionary or other metadata, and training OLAP Cubes OLAP provides interactive analysis and reporting using multidimensional views of data Users can slice, dice, drill-down and roll-up data – more than Excel pivots Cubes are often pre-built and refreshed right after batch ETL processes so that they are immediately available to users © 2008 Endasol, Inc. All Rights Reserved.
  7. 7. BI System Intelligence Layer The value of the BI System is often not realized until deploying this Layer Advanced Visualization, including Dashboards & Scorecards Enable rapid assessment of Business Metrics or Key Performance Indicators Dashboards display information in tables, charts, gauges, and other indicators Scorecards display metrics against pre-established thresholds grouped into red, yellow, green or numerical weightings Ultimately, should have the capability to drill down into information feeding the dashboard and scorecard items © 2008 Endasol, Inc. All Rights Reserved.
  8. 8. BI System Intelligence Layer Advanced Analytics Exploratory Data Mining – seeking new insights through examining the information with statistical analysis or other techniques Predictive Analytics – formulating rule-based models to predict what will happen under various scenarios with high degree of confidence Many BI efforts are abandoned before reaching this Layer Why? Because implementation of the other layers have been overly costly with mediocre results © 2008 Endasol, Inc. All Rights Reserved.
  9. 9. Predictive Analytics; Extending the Value of Your Data Warehousing Investment, 2007 TDWI Best Practices Report, Wayne W. Eckerson
  10. 10. Streamlining BI Implementation Employ methods that get to the higher value BI layers sooner: Establish Fundamentals Early Consider using a Packaged Solution or Starter Model Consider Agile Development Approach Consider Hybrid Approach Considerations for the Interactive Layer Considerations for the Intelligence Layer © 2008 Endasol, Inc. All Rights Reserved.
  11. 11. Establish Fundamentals Early for Any Phase Implement a Data Governance Program Upfront Assign Data Stewards from the Business responsible for quality and use of data Establish Monitoring and Methods for Correcting Issues Consider a Master Data Management Program Managing your Reference Data, such as customer, generators Ensuring consistency and dynamic mapping among systems Some examples of the need: How many systems or databases house customers (individuals or organizations) and how many different identifications, abbreviations, or alias’s are used? How are you tracking who owns whom? Frequently changes - if not careful, you may have excess credit exposure to entities or entities may be exercising unfair market power. Various technologies are available Ensuring consistency among systems regarding Master Data will streamline BI implementation, but not absolutely needed since the data warehouse will ultimately solve this issue for users of the data warehouse © 2008 Endasol, Inc. All Rights Reserved.
  12. 12. Establish Fundamentals Early for Any Phase Set realistic expectations from both business & IT perspectives Define precise & measurable objectives Ensure that the BI system will be scalable to accommodate other corporate initiatives in the future More importantly, align the project with overall corporate BI objectives Minimize scope creep Well-defined business requirements Establish Change Control policies upfront – know how changes will be processed and who needs to approve these If you don’t have the expertise in-house, then hire professionals Ensure they have experience in the phase you are embarking on Ensure those on the team have experience with your business Be involved – you still need to ‘own’ the program & business must be involved BI systems are not static – you will need skills to advance & maintain your system If not prepared to adopt these & other fundamentals, then don’t start! © 2008 Endasol, Inc. All Rights Reserved.
  13. 13. Use of Packaged BI Solutions & Starter Models Packaged BI Solutions: 3rd party products fulfilling a slice of your BI needs A Packaged BI Solution should: Provide the business with a clear definition of the end results – what you will get is visible from the start! Integrate and merge data across all appropriate systems Enable easy access to underlying data and the information it provides Serve as a starting point for other analysis and reports Build on an existing data warehouse as appropriate © 2008 Endasol, Inc. All Rights Reserved.
  14. 14. Use of Packaged BI Solutions & Starter Models A Packaged BI Solution is typically not: A front line application, such as an Energy Trade & Risk Management System (ETRM), or a Customer Information System (CIS) - too many try to elevate these transactional systems to be an enterprise data mart for reports & analysis Are there Packaged BI Solutions available in our industry? For wholesale power, including financial transactions, we have our TM PREMIR Solution There may be solutions available for the Retail, and Transmission/Distribution Sectors Consulting Companies focused in your business may offer at least Starter Models © 2008 Endasol, Inc. All Rights Reserved.
  15. 15. Endasol’s Flow Diagram premir Solutions TM Our present focus is the Wholesale Power & Financial Transactions Data Models and Reporting © 2008 Endasol, Inc. All Rights Reserved.
  16. 16. Use of Packaged BI Solutions & Starter Models Advantages: Requirements/Vision already defined – you see what the end result looks like Data Model provided and can be built upon Key Reports may be provided and others can be added KPI’s & Dashboard may be provided and can be expanded In any case, they can provide a head start enabling quicker deployment Potential Pitfalls: Don’t expect a packaged solution or even starter model to meet all of your objectives – expect that some customization will be needed Compatibility with existing data warehouse, data marts must be considered Compatibility of technology used – database, reporting tools – must be considered Capability to expand the solution to other areas of the enterprise must exist Conclusion: A packaged solution or starter model can significantly expedite your BI implementation, but do your homework first © 2008 Endasol, Inc. All Rights Reserved.
  17. 17. Consider Agile Development for BI - Background Waterfall Methodology – Traditional Approach From Wikipedia: “The waterfall model is a sequential software development model (a process for the creation of software) in which development is seen as flowing steadily downwards (like a waterfall) through the phases of requirements analysis, design, implementation, testing (validation), integration, and maintenance.” Often characterized by lengthy requirements, design and signoff before any development is allowed to begin. Agile Methodology – Currently gaining acceptance From Ralph Kimball: “…In general, it (Agile) refers to a group of methodologies, including Extreme Programming, SCRUM, Adaptive Software Development and others, which share a common focus on iterative development and minimizing risk by delivering new functionality in short timeframes, often measured in weeks.” Less structured with emphasis on deliverables, not documents & signoffs © 2008 Endasol, Inc. All Rights Reserved.
  18. 18. Consider Agile Development for BI Agile Development is gaining interest - Agile is now included in training classes and conferences Scrum is one structured methodology for Agile Development Product/Project broken into deliverables, called ‘Sprints’ of 2 to 4 weeks. Sprints established to bring priority, but achievable results within time frame. Sprints typically locked down with high resistance to changing scope once started. Product Owner - The Product Owner represents the voice of the client. He/She ensures that the Scrum Team works with the right things from a business perspective. ScrumMaster (or Facilitator) - Scrum is facilitated by a ScrumMaster, whose primary job is to remove impediments to the ability of the team to deliver the sprint goal. The ScrumMaster is not the leader of the team (as they are self-organizing) but acts as a buffer between the team and any distracting influences. Empowered teams without traditional Project Management. Scrum Team - A small team of 2+ people with cross-functional skills to do the actual work (designer, developer etc.). Daily meetings among players – IT & Business. Development proceeds immediately and documentation often lags. © 2008 Endasol, Inc. All Rights Reserved.
  19. 19. Consider Agile Development for BI Implementation Biggest Issue with many DW/BI projects Changes during the project! Source system changes Requirements Data quality issues – some sources are just not very good. Change in business sponsor / business owner Market changes ISO rule changes Subscription services Business Priorities © 2008 Endasol, Inc. All Rights Reserved.
  20. 20. Consider Agile Development for BI Implementation Advantages of Agile Development Design at the beginning of each sprint - Learn from each sprint Phased / iterative approach – Useable tools delivered quicker Determine next sprint based on priorities. Working closely with the business users during data cleansing and development of reports, cubes and dashboards Increase user satisfaction and ‘ownership’ Decrease ‘do-overs’ Building what the business needs overrides building just to meet documentation © 2008 Endasol, Inc. All Rights Reserved.
  21. 21. Consider Agile Development for BI Implementation Disadvantages of Agile Development Budgeting difficulties due to minimal upfront analysis & design Hardware needs Staffing – how long? When is the project considered finished? How does the present sprint fit into the solution? Some rework may be needed Have a risk of delivering isolated silos of data which meet individual releases, but don’t integrate well with each other New to the organization and not well understood by IT and Business. Users are confused about their role. Users may not have sufficient time. Minimize risk by using company standards & following corporate SDLC process Documentation – Agile methodology minimizes documentation. What about support and future enhancements? © 2008 Endasol, Inc. All Rights Reserved.
  22. 22. Hybrid Approach for BI Implementation Optimum Approach may be a Hybrid between Waterfall & Agile Take time upfront to develop overall BI plan/ roadmap Spend some time on how the overall BI solution fits into the enterprise picture Identify subject areas for potential phases Develop framework of data model Set standards Select hardware / software Break into focused phases Phase 1 may be 4 – 6 months due to infrastructure, overall requirements, design & documentation Focus mainly on current phase subject area but know how it fits with the other areas. Other phases may decrease in length as more and more subject areas are added. Take each phase all the way through user reports, OLAP cubes, dashboard & scorecard Include support with the delivery of each phase © 2008 Endasol, Inc. All Rights Reserved.
  23. 23. Hybrid Approach for BI Implementation Establish Project High Level Design Infrastructure •Subject Area Data •Hardware •BI System Objectives •Subject Area Relationships •Software/ Licenses •Business Requirements •High Level Data Model •Set Up Environments •Prioritize Phases •Set Standards •Project Plan Detailed Requirements Phase 1 Detailed Phase Design 2 Phase ETL & 3 Test Repeat Each Phase for Phase each may get 4 User Phase shorter Reports OLAP Cubes Dashboard/ Scorecard © 2008 Endasol, Inc. All Rights Reserved.
  24. 24. Streamlining Interactive Layer Choosing Reporting and Querying Tools Business Ease of Use Deploying a mixture of tools What about Excel? MS Excel will continue to be the tool of choice for many business users Even with well managed data warehouses, end users still resort to Excel But this may not be bad as long as the source is the data warehouse and not direct pulls from source systems How important is OLAP? © 2008 Endasol, Inc. All Rights Reserved.
  25. 25. Streamlining Interactive Layer Give Business Early Access to Data Standard reports should be pre-established using the corporate reporting tool & ‘Locked Down’ Underlying structure/ data model should be such that business users can easily understand Allow them to develop their own reports and analysis - Stress using corporate reporting tools, but be flexible as long as they are sourcing from the data mart © 2008 Endasol, Inc. All Rights Reserved.
  26. 26. Streamlining Intelligence Layer - KPIs Restrict Key Performance Indicators (KPIs) to a small number (6 – 10) specific to the organization’s goals Develop different set of KPIs for each business unit, sub-unit The business unit’s KPIs should relate to corporate KPIs The business unit’s KPIs should also include what’s important to bonuses Most effective use is often a mixture of financial & non-financial KPIs as in the Kaplan and Norton Balanced Scorecard approach KPIs should follow the SMART test: Specific, Measurable, Achievable (Actionable), Realistic, Timely Once implemented, continue to refine KPIs Revise as corporate and business unit objectives change Monitor who is accessing and how often –retire KPIs not being used Ensure individuals can easily access KPIs and understand what they are viewing © 2008 Endasol, Inc. All Rights Reserved.
  27. 27. Streamlining Intelligence Layer – Visualizing Choosing Technology Numerous products available with nice dashboard & scorecard features Some are limited to specific technologies, such as using MS Analysis Services (cubes) to drive dashboard components Most effective is ability to drill down – start with the picture & drill to underlying information Don’t need to go overboard on dashboard/ scorecard technology If can’t justify expense upfront, then consider publishing output of certain reports & OLAP such as graphs or charts in a web-portal – this can be automated Once base functionality is working, can always buy advanced technology later © 2008 Endasol, Inc. All Rights Reserved.
  28. 28. Streamlining Intelligence Layer – Visualizing Implementing Dashboards & Scorecards Keep it simple – don’t clutter the viewing screen Consider allowing viewers to construct their own ‘myBI’ page Some items may be required viewing such as important metrics Others may be selected from a list of available dashboard reports, graphs, charts Continue to monitor usage and effectiveness & continue to refine © 2008 Endasol, Inc. All Rights Reserved.
  29. 29. Streamlining Intelligence Layer – Data Mining Determine Data Mining or Advanced Analytical Methodology Early Exploratory – looking for new insights and/ or Predictive – developing models to predict behavior/ market opportunities Different methodologies are available for each focus Methodology may require very specific data model or structure Data model used for advanced analytics may conflict with that optimal for reporting and user-queries If you know this upfront, you may be able to build a hybrid model that can serve both purposes, or You may be able to implement a model structure that can be more easily transformed into that needed for the advanced analytic application © 2008 Endasol, Inc. All Rights Reserved.
  30. 30. Align Your BI System with Corporate Objectives Ultimately, your BI System should be aligned to corporate objectives and provide monitoring & feedback for continued alignment Corporate KPIs should trickle down to corresponding KPIs pertinent to individual Business Units Business Unit KPIs should link to corporate & adjusted as corporate objectives change The above is part of the broader initiative referred to as Business Performance Management (BPM), Corporate Performance Management (CPM), or Enterprise Performance Management (EPM) There are advancement in software tools which support implementation & monitoring © 2008 Endasol, Inc. All Rights Reserved.
  31. 31. Streamlining BI Implementation - Summary Align your BI System with Corporate Objectives Value of your BI System is often not realized until deployment of Visualization and Advanced Analytics Dashboards & Scorecards to track KPIs & other important measures Drill-down into information driving displays is an important capability Data Mining for discovering new insights Predictive Analytics to gain edge in market © 2008 Endasol, Inc. All Rights Reserved.
  32. 32. Streamlining BI Implementation - Summary Objective is to get to these value BI Layers as soon as possible: Use Packaged BI Solutions or Starter Models if available and if applicable Use a Hybrid Approach of Agile & traditional Waterfall Development Do some detailed requirements & design upfront Break Project into short phases of 1 – 6 months covering selected subject areas Each Phase should include all BI Layers - Data Model, User Reports, Cubes, Dashboard & Scorecard items Technology – decide what’s important to you from a budget and capability perspective, be flexible and don’t get stuck in analysis paralysis. © 2008 Endasol, Inc. All Rights Reserved.

×