This is a copy of my portfolio to highlight the hands-on BI project work I completed in the Set Focus Masters Program.To increase the slide size for easier viewing, use the (+) zoom in control in the lower right of the textbox.
The Masters program was 10 weeks full time alternating between classroom work and projects. The class was 8 hours a day Monday through Friday, using live meeting, Polycom TV technology, and VMWare to tie the students and instructor together. We covered the 3 main tools in the Microsoft BI stack (SSIS, SSAS, and SSRS) and the MDX programming language. We spent a week on OLAP data modeling and T-SQL review. We also covered integration with SharePoint, Excel pivot tables, and the Performance Point Server reporting tool.
This is the table of contents showing the instruction and project areas covered in the program.
The next set of slides are examples of the data modeling I did to denormalize OLTP tables so they could be used as a data sources for an OLAP cube.On the left is the Set Focus weekly student survey completed by each student to evaluate the instructor. The 12 tables on the right show the underlying OLTP schema for the survey. Our project goal was to transform these to OLAP dimension and fact tables for use in a reporting Data Warehouse.
I used Visio to create the OLAP model of 2 fact and 5 dimension tables. The Management Studio windows on the right show the Integration Services extract SQL I wrote to create the 2 fact tables, each with a different level of granularity. As you can see from the diagram, the 2 fact tables share the same dimension tables with the exception of DimCategory which relates at a lower level of granularity (sharing Dim tables is a best practice in the Kimball data warehouse model).
On the left is the extract SQL I wrote to create 2 of the dimension tables. The DimTrackSessionCourse table has a natural hierarchy that’s been denormalized in the OLAP model (track name, session name, and weekly course). The hierarchy is used in the cube to navigate the level of data.All tables use identity columns for the primary key on the data load. The primary key of the dimension tables are used as foreign keys in the fact tables. Surrogate keys are left in place but not used to define the relationships.
This is a different OLAP data model I created for a personal stock market project. The main fact table contains 2 years of daily stock price information. On the right is the CTE-based SQL I wrote as part of the transform to create the derived columns. The second fact table is a “factless” bridge table to model the M:M relationship needed to create a stock watchlist feature. This is used in the OLAP cube to filter on small groups of stocks to watch more closely.
The next slides are samples of the Integration Services projects I worked on. On the left is a master package containing 8 child packages for regular inserts and updates from flat files to an RDBMS target. The 9th package (outside the sequence container and detailed on the right) performs routine DB maintenance tasks. Send mail tasks are defined to email the success or failure of each step.
This slide shows a package in the same project with a Foreach Loop Control Flow task container that reads excel CSV time sheet files in a directory and loads them to an RDBMS destination (in Data Flow task not shown). Inserted, updated, and invalid row counts are maintained using package variables. A Send Mail task is used to announce success or failure.
On the left are the details of an Execute SQL task I created for the student survey project. The SQL is used to create the DimDates dimension table common to most OLAP cubes. Start and end date parameters control the range of the date rows created. The table is highly denormalized to expose the natural date hierarchies.
This slide shows associated Control and Data Flow tasks for my Stock Market project. A Foreach Loop is used to move stock price files from the FTP to the load directory. A second Foreach Loop loads the files to the price fact table and to a processed directory when completed. Details of the Data Flow task are shown on the right. Derived columns are added and duplicate rows are redirected. A Lookup task is performed and rows with an unmatched ticker symbol are moved to an exception table for further investigation.
The next set of slides show samples of MDX code I wrote.The left query uses the descendants SELF_AND_BEFORE and order functions to display the product category and subcategory dollar sales in descending year 2005 order. Unit sales and 2005 quarters are also shown.The right query creates 4 new members to combine and consolidate customer regions to higher levels showing unit and dollar returns for year 2005.
The left queryuses the descendants function to show yearly, quarterly, and monthly dollar and unit sales by product family. The where clause slices the results to include only the Mid-Atlantic region for the year 2005.The right query compares the average price per unit by product for the latest month against the previous month and shows the increase as a percent. The filter function is used to limit the display to only products with a 10% increase. The last month in the cube must be populated with data for this query to work.
The left query uses the rank function to display dollar sales and sales rank by customer city for the last 4 quarters in the cube. The order function is used to sort the quarter columns in descending order.The right query uses the generate and topcount functions to display the top 3 customers by dollar returns in each product family. The results are ordered within each product family by dollar returns descending.
The next set of slides show samples and code from my Analysis Services Stock Market project.The YrHfQtrMthWkDay hierarchy of the DimDate dimension is highlighted. This hierarchy allows the date dimension to be aggregated and navigated from the year down to the week level (the weekend date determines to which month a week belongs). The SQL code used in SSIS to populate this hierarchy was shown in an earlier slide.
This slide shows the hierarchies defined for the DimETFMaster dimension table. The RegionFunds hierarchy in the middle is expanded to show the country and ticker symbols associated with each. The cube browser view on the right shows the price data by country and security for the most recent month (in the filter).
This slide shows objects that can be created in an SSAS cube. On the left is a named perspective to limit client visibility to specific areas when the client uses the perspective instead of the cube. In this case, the defined Actions are not visible when the named perspective is used.On the right are examples of using MDX expressions to create calculated measures and named sets. [MA Close 10Day] is a calculated measure that returns the nonempty 10-day moving average of the Close Price. [MostRecentMonth] is a named set that returns the last month in the cube that contains Close Price data.
Expressions can also be used to create Actions in the cube.The upper right expression creates a cube action to launch a web browser and display an associated stock chart when a ticker symbol is right-clicked in a report. In this case, the IJR ticker was selected which launched a chart with it’s symbol in StockCharts.com.
The next set of slides shows reports I created in Reporting Services.This report demonstrates use of cascading parameters to display only valid start and end dates for employee jobs. Parameters were initially created by SSAS, but I added a filter in the MDX code to filter and limit the results (where hours worked not equal to null).
This report uses KPI goals and status defined in the cube to display Returns% plus status and trend images. The status of returns less than 5% is green, 5% to 5.5% is yellow, and over 5.5% is red. The images are stored in a SharePoint site.
This report displays sales by product category as an exploded pie chart. The percentage values are calculated using an expression and displayed on the data points. The bar chart version on the lower left is the same data further broken out by year. It demonstrate the one dimensional data limit of the pie chart report.
This report displays a stacked bar chart with corresponding detail matrix. The matrix provides an easy way to display row and column totals.
The next 2 slides show my implementation of data driven email subscriptions for the Stock Market project. On the left is the SQL to create and populate the subscription table. Each of the 3 subscribers requests a different watchlist of ticker symbols and render format. On the right is the Report Manager page linking the data fields to the delivery settings and the successful run results. For ease of administration, the reports were scheduled using a shared calendar.
Here are the 3 data driven email subscription reports, each with different data and formats. The PDF version displays an embedded hyperlink in the email to the same report on the Report Server.
This is a Performance Point server gauge scorecard report displaying quarterly progress by region toward a KPI goal stored in the cube. The results are generated from inside a SharePoint folder.
This is an Excel dual-Y axis pivot table report also displayed from SharePoint. The bars show dollar sales for all products select from a multi-value parameter and the line shows dollar sales as a percent of parent. Dual-Y axis reports are now available in SSRS 2008 (but not in 2005).Thank you for viewing these slides.
William Canning Portfolio Annotated
Business Intelligence<br />1<br />William Canning<br />email@example.com<br />858-485-6560<br />
Portfolio Overview<br />2<br />This portfolio contains examples from over 400 hours of hands-on Microsoft Business Intelligence training in the Set Focus Masters Program.<br />It contains additional examples from BI work performed after graduation.<br />SQL Server and Microsoft technologies covered in the program included Data Modeling, Management Studio, T-SQL Programming, Integration Services, Analysis Services, MDX Programming, Reporting Services, Visio, Performance Point Server, and SharePoint Services.<br />
Contents<br />3<br /><ul><li>Data Modeling and T-SQL (pg 4)
Performance Point, Excel, and SharePoint (p 25)</li></li></ul><li>Data Modeling and T-SQL<br />Final Project <br />Transform weekly Student Survey relational data model to denormalized star schema for SSAS OLAP application<br />4<br />
Data Modeling and T-SQL<br />Final Project <br />Visio diagram of 2 fact tables and 5 shared dimension tables. All use identity columns for primary keys.<br />T-SQL to populate FactEvalScore (question level granularity) and FactEvalReturned tables (survey level granularity)<br />5<br />
Data Modeling and T-SQL<br />Final Project <br />Completeddata model for populated RDBMS staging database in Management Studio<br />T-SQL to populate DimTrackSessionCourse and DimCategory denormalized tables<br />6<br />
Data Modeling and T-SQL<br />Stock Market Project <br />Cube diagram of 2 fact tables and 3 dimension tables for price analysis. Majority of data in main fact table was created using SQL derived columns.<br />T-SQL common table expression with rownumber function used to obtain previous close price value<br />7<br />
SSIS – Integration Services<br />8<br />SSIS Project <br />Master package created to run 8 packages for loading flat file data into RDBMS tables.<br />Last package run performs database maintenance tasks including shrink DB, rebuild indexes, update statistics, and DB backup.<br />
SSIS – Integration Services<br />9<br />SSIS Project <br />Control flow loop reads directory of Excel timesheets and loads row values to RDBMS table.<br />Last task following loop is to email success or failure using Package variable row counts.<br />
SSIS – Integration Services<br />10<br />Final Project <br />Execute SQL control flow task populates DimDates table in RDBMS staging database for OLAP cube.<br />
SSIS – Integration Services<br />11<br />Stock Market Project <br />Control flow to move daily CSV stock price files to load directory and insert to RDBMS staging database.<br />Data flow task filters for valid ETF, Forex, or World Indices symbols and adds derived audit columns.<br />
MDX Programming<br />MDX Project <br />Display ordered sales data by quarters and year 2005 for all product categories<br />Create calculated regions in Customer dimension to aggregate financial results<br />12<br />
MDX Programming<br />MDX Project <br />Display ordered percent product price increases for most recent month versus previous month<br />Display 2005 mid-Atlantic sales data by product category and year, quarter, month time hierarchy<br />13<br />
MDX Programming<br />MDX Project <br />Display customer city dollar sales and rank for each of the most recent 4 quarters<br />Display an ordered list of the top 3 customers in each product family by dollar returns<br />14<br />
SSAS – Analysis Services<br />Stock Market Project<br />Date dimension user defined hierarchies with Year, Half, Quarter, Month, WeekEnd, and Day hierarchy values displayed<br />15<br />
SSAS – Analysis Services<br />Stock Market Project<br />ETF Master dimension user defined hierarchies with Region, Country Name, Ticker Fund hierarchy values displayed in cube browser<br />16<br />
SSAS – Analysis Services<br />Stock Market Project<br />Calculated member for close price 10-day moving average and named set for last month with data<br />Perspective limiting display of web-based cube actions<br />17<br />
SSAS – Analysis Services<br />Stock Market Project<br />Cube action expression used to launch stock chart web site by right clicking on ticker symbol in Excel pivot table report<br />18<br />
SSRS – Reporting Services<br />19<br />SSRS Project <br />Cascading parameters report with filter in MDX expression to select only valid date ranges<br />Report displays employee hours worked and labor dollars grouped by job and week in date range<br />
SSRS – Reporting Services<br />SSRS Project <br />KPI report from calculated measures with image symbols stored in SharePoint<br />Report displays Q3 2005 dollar returns percent status and trend for all product categories<br />20<br />
SSRS – Reporting Services<br />21<br />SSRS Project <br />Exploded pie chart includes calculated point labels for percentage sales by category on chart data tab<br />Similar report displayed in bar chart format with additional breakout by sales year<br />
SSRS – Reporting Services<br />SSRS Project <br />Stacked bar chart display of employee sales by year with supporting matrix report<br />Report totals created using column and row Subtotal feature of Reporting Services<br />22<br />
SSRS – Reporting Services<br />Stock Market Project<br />Create data driven subscription for parameterized stock daily price report with email delivery.<br />Create table to drive report, tie columns to report attributes, and run on a shared schedule.<br />23<br />
SSRS – Reporting Services<br />24<br />Stock Market Project<br />Data driven email subscription reports displayed with different contents and formats.<br />Reports are for different stock data and rendered with or without hyperlinks in Excel, MHTML, and PDF formats. <br />
Performance Point, Excel,and SharePoint<br />SharePoint Project <br />PPS half gauge scorecard based on KPI values and goals deployed to SharePoint Server<br />Report displays quarterly progress toward annual sales goal by region for 2005<br />25<br />
Performance Point, Excel, and SharePoint<br />SharePoint Project<br />Excel dual Y-axis bar and line pivot chart deployed to SharePoint Server<br />Report displays 24 months dollar sales and percent of parent for Electronic products<br />26<br />