Become BI Architect with 1KEY Agile BI Suite - OLAP

Slide 1: What is Business Intelligence? Business intelligence is a broad category of applications and technologies for gathering, providing access to, and analyzing data for the purpose of helping enterprise users make better business decisions. Ekta Pardhi

Slide 2: What is OLTP ?  Traditional RDBMS are used for OLTP  On-Line Transaction Processing  used for daily processing  detailed, up to date data  read/update a few records  isolation, recovery and integrity are critical

Slide 3: Data, Data everywhere yet ...  I can’t find the data I need  data is scattered over the network  many versions, subtle differences  I can’t get the data I need  need an expert to get the data  I can’t understand the data I found  available data poorly documented  I can’t use the data I found  results are unexpected  data needs to be transformed from one form to other

Slide 4: What is OLAP(On-line Analytical Processing) ?  Online Analytical Processing  dynamic synthesis, analysis and consolidation of large volumes of multi- dimensional data  normally implemented using specialized multi-dimensional DBMS  a method of visualizing and manipulating data with many inter- relationships  Support common analytical operations such as  consolidation  drill-down  slicing and dicing

Slide 5: Online Analytical Processing (OLAP) SQL 2000 It enables analysts, managers and executives to gain insight into data through fast, consistent, interactive access to a wide variety of possible views of information that has been transformed from Oracle raw data to reflect the real dimensionality of the enterprise as understood by the user. Product Access 1KEY OLAP Region SQL 2005 Time

Slide 6: OLAP Applications - multi-dimensional views of data  Core requirement of building a ‘realistic’ business model.  Provides basis for analytical processing through flexible access to corporate data.  The underlying database design that provides the multi-dimensional view of data should treat all dimensions equally.

Slide 7: Conceptual Data Model Multi-dimensional view of Data  Each dimension is described by a set of attributes; the attributes of a dimension may be related via hierarchy of relationships. Dimensions: Product, Region, Time Hierarchical summarization paths Product Region Time on W Industry Country Year gi Re S N Juice Product Cola Category Region Quarter Milk Cream Product City Month Week Toothpaste Soap 1 2 34 5 6 7 Month Office Day

Slide 8: OLAP Applications - support for complex calculations  Must provide a range of powerful computational methods such as that required by sales forecasting, which uses trend algorithms such as moving averages and percentage growth.  Mechanisms for implementing computational methods should be clear and non- procedural.

Slide 9: OLAP Applications – time intelligence  Key feature of almost any analytical application as performance is almost always judged over time.  Time hierarchy is not always used in same manner as other hierarchies.  Concepts such as year-to-date and period-over-period comparisons should be easily defined.

Slide 10: What is Data warehouse? A DW is a subject-oriented, integrated, time-varying, non-volatile collection of data that is used primarily in organizational decision making Ekta Pardhi

Slide 11: Graphically Data Warehouse Integrated Subject Oriented OLTP Systems Credit Risk Interest Rate Forex Risk Risk Code: XXXX Code: XX/XX.XX Interest Rate Risk Code: CXXYY Credit Risk Forex Risk Common Code for various source codes ALM Subject Area ALM Subject Area Non Volatile Time variant U Read S E R OLTP OLTP Write S Dec 98 OLAP U S Read 1995 1996 1997 1998 Data E Warehouse R S

Slide 12: Data Warehouse is not …  a product, an environment  a system, an architecture  an end to itself but is at the heart of the Business Intelligence (BI) infrastructure of the organization

Slide 13: OLTP/OLAP  OLTP (on-line transaction processing)  Major task of traditional relational DBMS  Day-to-day operations: purchasing, inventory, banking, manufacturing, payroll, registration, accounting, etc.  OLAP (on-line analytical processing)  Major task of data warehouse system  Data analysis and decision making  Distinct features (OLTP vs. OLAP)  User and system orientation: customer vs. market  Data contents: current, detailed vs. historical, consolidated  Database design: ER + application vs. star + subject  View: current, local vs. evolutionary, integrated  Access patterns: update vs. read-only but complex queries

Slide 14: OLTP vs. OLAP OLTP OLAP users clerk, IT professional knowledge worker function day to day operations decision support DB design application-oriented subject-oriented data current, up-to-date historical, detailed, flat relational summarized, multidimensional isolated integrated, consolidated usage repetitive ad-hoc access read/write lots of scans index/hash on prim. key unit of work short, simple transaction complex query # records accessed tens millions #users thousands hundreds DB size 100MB-GB 100GB-TB metric transaction throughput query throughput, response

Slide 15: Examples of OLAP applications in various functional areas

Slide 16: Nature of OLAP Analysis • Aggregation -- (total sales, percent-to-total) • Comparison -- Budget vs. Expenses • Ranking -- Top 10, quartile analysis • Access to detailed and aggregate data • Complex criteria specification • Visualization

Slide 17: 1KEY OLAP Related Terms • Dimension • Attribute • Hierarchy • Fact table • Measures • Model(cube) • Calculated member

Slide 18: Dimension A dimension table contains the specific name of each member of the dimension and Dimensions determine the contextual background for the facts

Slide 19: Dimension

Slide 20: Attribute Information about a specific dimension member

Slide 21: Attribute

Slide 22: Fact table Fact table contains values for one or more measures at the lowest level of detail for one or more dimensions

Slide 23: Fact Table

Slide 24: Measures A summarizable numerical value used to monitor business activity

Slide 25: Measure

Slide 26: Calculated member A mechanism for aggregating measures using formulas more complex than those stored in a cube

Slide 27: Hierarchies Think of the points of the data cube as partitioned along each dimension (at some level of granularity) • E.g., time dimension: partition according to days, weeks, months,… • E.g., cars dimension: partition by model, by color, both model and color, … • E.g., dealers dimension: partition by dealer, by city, by state, ...

Slide 28: Aggregation Hierarchies

Slide 29: Model(cube) A collection of one or more related measure groups and their associated dimensions

Slide 30: T date, custno, prodno, cityname, sales p i r m o e d f a c c t p u r s o t m o

Slide 31: Scheme Components of a Data Cube (3D) Measure Cell Time Housing Car 2000 1999 Edu. 200 1998 1997 Industry East West North South Region Dimension

Slide 32: 1KEY OLAP console

Slide 33: DW Modeling • Modeling data warehouses: dimensions & measures – Star schema: A fact table in the middle connected to a set of dimension tables • A star schema is a set of tables comprised of a single, central fact table surrounded by de-normalized dimensions. Each dimension is represented in a single table. Star schema implement dimensional data structures with de- normalized dimensions. – Snowflake schema: A refinement of star schema where some dimensional hierarchy is normalized into a set of smaller dimension tables, forming a shape similar to snowflake • A snowflake schema is a set of tables comprised of a single, central fact table surrounded by normalized dimension hierarchies. Each dimension level is represented in a table. Snowflake schema implement dimensional data structures with fully normalized dimensions.

Slide 34: Star Schema • A single fact table and for each dimension one dimension table • Does not capture hierarchies directly T date, custno, prodno, cityname, sales p i r m o e d f a f c a c t c p u t r s o t m o

Slide 35: Snowflake Schema „ Represent dimensional hierarchy directly by normalizing tables. „ Easy to maintain and saves storage T date, custno, prodno, cityname, sales p i r m o e d f a f c a c t c p u t r r s o e t m g o i o n

Slide 36: Components of a star schema Excellent for ad-hoc queries, but bad for online transaction processing

Slide 37: Star schema example

Slide 38: Star Schema – Example 1 Product Order ProdNo Fact Table ProdName OrderNo ProdDescr OrderDate Category OrderNo CategoryDescr SalespersonID UnitPrice CustomerNo QOH ProdNo DateKey Customer CityName CustomerNo Quantity CustomerName TotalPrice Date CustomerAddress City DateKey Date Month Year Salesperson SalespersonID SalespersonName City City Quota CityName State Country

Slide 39: Snowflake – Example 1

Slide 40: Star Schema – Example 2 time time_key item day item_key day_of_the_week Sales Fact Table item_name month brand quarter time_key type year supplier_type item_key branch_key branch location location_key branch_key location_key branch_name units_sold street branch_type city dollars_sold province_or_street country avg_sales Measure s

Slide 41: Snowflake Schema –Example time item time_key day item_key supplier day_of_the_week Sales Fact Table item_name supplier_key month brand supplier_type quarter time_key type year supplier_key item_key branch_key branch location location_key location_key branch_key units_sold street branch_name city_key city branch_type dollars_sold city_key avg_sales city province_or_street Measure country s

Slide 42: Features of a OLAP Reporting Solution • Must connect with an OLAP data source • Must allow the user to drill down into the data with flexibility and ease • Must provide a user-friendly visual interface that includes charts, because reporting is most effective when it is visual. • Must allow the user to save and retrieve custom reports.

Slide 43: Some variants

Slide 44: Metadata Metadata explains what data exists, where it is located and how to access it. The metadata is a core of a data logistics system, the infrastructure for DW and ultimately the intelligence system.

Slide 45: Data Mart • A Data mart is a collection of data specifically designed for the use of a department. • Data mart is nothing but a smaller, more focused data warehouse. • In many organizations it is useful to create data marts for specific business units that have very specific data analysis needs. • The data mart reflects the peculiar needs of the department. • A typical example of a departmental data mart in a Bank would be that of a Human Resource Department.

Slide 46: Enterprise Data Warehouse vs. Data Mart

Slide 47: DW and Data Mart: some other issues Characteristics Data Mart Enterprise DW Time to build Months Years Complexity to build Low to medium High Shared Shared in Dept. Shared across Update Frequently Less Frequently Type of user Subject focused Corporate and top executives

Slide 48: Implementation and Success Criteria • Do not try to implement the entire data warehouse at once • The project should break up the functionality to be delivered in different phases • You will not only deliver something tangible for your users, but you may also flush out issues that can be quickly corrected • Users are constantly knocking on your door • The buzz in the hallways mentions the data warehouse, or meetings make reference to it as the source of data • The data warehouse and OLAP becomes the heartbeat of the business, where decisions are made from the data intelligence it provides

Slide 49: Strengths of OLAP • It is a powerful visualization tool • It provides fast, interactive response times • It is good for analyzing time series • It can be useful to find some clusters and outliners

Slide 50: Data Warehousing and OLAP Key Benefits • Centralized Data Repository • Single Version of the Truth • Clean, Accurate Data • Organized, Timely Availability • Historical Detail • Shared and Accessible • Ad hoc Discovery & Analysis

Slide 51: Walkthrough MAIA OLAP

Slide 61: 1KEYOLAP Project Management • Data warehouse projects are significantly more dynamic and complicated than traditional systems projects – more change to: – Scope – Tool evaluation and performance – Cross-departmental issues, requirements – Cross-departmental data integration and usage – Staffing resources – Cultural issues • DW Project Manager must be more actively involved in all facets of project, understand all facets and issues – not all project managers will succeed as DW project managers

Slide 62: Thank You! Ekta Pardhi ekta@maia-intelligence.com