This document contains examples from a portfolio of business intelligence projects including data modeling, SQL programming, SSIS, SSAS, SSRS, PPS, Excel Services, and SharePoint. It includes examples of relational and dimensional data models, SQL queries, SSIS packages for data integration and processing, an SSAS cube with calculations, KPIs and reports, Excel dashboards published to SharePoint using Excel Services, and reports and dashboards deployed to SharePoint.
An introductory session to DAX and common analytic patterns that we've built and used in enterprise environments. This session was originally presented at SQL Saturday Silicon Valley 2016.
James Colby Maddox Business Intellignece and Computer Science Portfoliocolbydaman
This portfolio covers the business intelligence course work I have completed at Set Focus, and some of the course work I have completed at Kennesaw State University
This QuickStart is for users new to PowerPivot or tabular model projects
authored in SQL Server Data Tools. It is meant to give you a quick and easy introduction
on how you can use Data Analysis Expressions (DAX) to solve a number of basic data
modeling and analytical problems.
An introductory session to DAX and common analytic patterns that we've built and used in enterprise environments. This session was originally presented at SQL Saturday Silicon Valley 2016.
James Colby Maddox Business Intellignece and Computer Science Portfoliocolbydaman
This portfolio covers the business intelligence course work I have completed at Set Focus, and some of the course work I have completed at Kennesaw State University
This QuickStart is for users new to PowerPivot or tabular model projects
authored in SQL Server Data Tools. It is meant to give you a quick and easy introduction
on how you can use Data Analysis Expressions (DAX) to solve a number of basic data
modeling and analytical problems.
A talk given by Julian Hyde at DataCouncil SF on April 18, 2019
How do you organize your data so that your users get the right answers at the right time? That question is a pretty good definition of data engineering — but it is also describes the purpose of every DBMS (database management system). And it’s not a coincidence that these are so similar.
This talk looks at the patterns that reoccur throughout data management — such as caching, partitioning, sorting, and derived data sets. As the speaker is the author of Apache Calcite, we first look at these patterns through the lens of Relational Algebra and DBMS architecture. But then we apply these patterns to the modern data pipeline, ETL and analytics. As a case study, we look at how Looker’s “derived tables” blur the line between ETL and caching, and leverage the power of cloud databases.
Data Exploration with Apache Drill: Day 2Charles Givre
Study after study shows that data scientists and analysts spend between 50% and 90% of their time preparing their data for analysis. Using Drill, you can dramatically reduce the time it takes to go from raw data to insight. This course will show you how.
The course material for this presentation are available at https://github.com/cgivre/data-exploration-with-apache-drill
MSCD650 Final Exam feedback FormMSCD650 Final Exam Grading For.docxgilpinleeanna
MSCD650 Final Exam feedback Form
MSCD650 Final Exam Grading Form
(Instructions follow the form)
Coding
55 Percent
Points Earned
Comments:
Trigger Code:
· Code meets requirements
· Code compiles cleanly
/15
Pre-Calculation Procedure Code
· Code meets requirements
· Code compiles cleanly
/15
PL/SQL Block Code
· Code meets requirements
· Code compiles cleanly
/15
Function Code
· Code meets requirements
· Code compiles cleanly
/10
/55
Unit Testing
35 Percent
Points Earned
Comments:
Unit Test for Trigger Code:
· All conditions are thoroughly tested
· The code runs successfully
· All data to prove test worked is displayed
/10
Unit Test for Procedure Code:
· All conditions are thoroughly tested
· The code runs successfully
· All data to prove test worked is displayed
· The tester can easily follow the path of the execution.
/10
.
Unit Test for PL/SQL Block Code:
· All conditions are thoroughly tested
· The code runs successfully
· All data to prove test worked is displayed
· The tester can easily follow the path of the execution.
/10
Unit Test for View/Function Code:
· All conditions are thoroughly tested
· The code runs successfully
· All data to prove test worked is displayed
· The tester can easily follow the path of the execution.
/5
/35
Documentation
10 Percent
Points Earned
Comments:
Presentation:
· The document is easy to read.
· The document is Professional in appearance
· It is easy for the reader to find what they are looking for.
/5
Documentation:
· Code is documented so that anyone who picks it up knows what it is doing.
/5
/10
Total 100
Percent
Points Earned
Comments:
Case Study
Overview of assignment
As a new ABC Consultant assigned to the XYZ Company, you have been asked to enhance the current system to include payroll processing. Although the current employee table has monthly salary and commission columns, it does not provide any means for storing employee deductions. You will add the tables necessary to store employee deductions. Next you will create a payroll pre-calculation program that will calculate the net pay for all the employees via a batch process (a stored procedure in a package, which will call other stored procedures within the package). Although this is not a complete payroll system, the unit test results must be accurate.
Next you will create two PL/SQL blocks for inserting and deleting rows from the employee deduction table. These PL/SQL blocks will be passed information from host or bind variables and a third PL/SQL block which will assign the variables defined in SQL*Plus (e.g. employee number, dollar amount and deduction name). Since the XYZ Company wants to track changes to the employee and employee deduction tables, you will create two database triggers that will update audit tables when rows are changed or deleted.
The XYZ Company also requires a view that will display specific employee information, plus the number of deductions for an employe ...
A talk given by Julian Hyde at DataCouncil SF on April 18, 2019
How do you organize your data so that your users get the right answers at the right time? That question is a pretty good definition of data engineering — but it is also describes the purpose of every DBMS (database management system). And it’s not a coincidence that these are so similar.
This talk looks at the patterns that reoccur throughout data management — such as caching, partitioning, sorting, and derived data sets. As the speaker is the author of Apache Calcite, we first look at these patterns through the lens of Relational Algebra and DBMS architecture. But then we apply these patterns to the modern data pipeline, ETL and analytics. As a case study, we look at how Looker’s “derived tables” blur the line between ETL and caching, and leverage the power of cloud databases.
Data Exploration with Apache Drill: Day 2Charles Givre
Study after study shows that data scientists and analysts spend between 50% and 90% of their time preparing their data for analysis. Using Drill, you can dramatically reduce the time it takes to go from raw data to insight. This course will show you how.
The course material for this presentation are available at https://github.com/cgivre/data-exploration-with-apache-drill
MSCD650 Final Exam feedback FormMSCD650 Final Exam Grading For.docxgilpinleeanna
MSCD650 Final Exam feedback Form
MSCD650 Final Exam Grading Form
(Instructions follow the form)
Coding
55 Percent
Points Earned
Comments:
Trigger Code:
· Code meets requirements
· Code compiles cleanly
/15
Pre-Calculation Procedure Code
· Code meets requirements
· Code compiles cleanly
/15
PL/SQL Block Code
· Code meets requirements
· Code compiles cleanly
/15
Function Code
· Code meets requirements
· Code compiles cleanly
/10
/55
Unit Testing
35 Percent
Points Earned
Comments:
Unit Test for Trigger Code:
· All conditions are thoroughly tested
· The code runs successfully
· All data to prove test worked is displayed
/10
Unit Test for Procedure Code:
· All conditions are thoroughly tested
· The code runs successfully
· All data to prove test worked is displayed
· The tester can easily follow the path of the execution.
/10
.
Unit Test for PL/SQL Block Code:
· All conditions are thoroughly tested
· The code runs successfully
· All data to prove test worked is displayed
· The tester can easily follow the path of the execution.
/10
Unit Test for View/Function Code:
· All conditions are thoroughly tested
· The code runs successfully
· All data to prove test worked is displayed
· The tester can easily follow the path of the execution.
/5
/35
Documentation
10 Percent
Points Earned
Comments:
Presentation:
· The document is easy to read.
· The document is Professional in appearance
· It is easy for the reader to find what they are looking for.
/5
Documentation:
· Code is documented so that anyone who picks it up knows what it is doing.
/5
/10
Total 100
Percent
Points Earned
Comments:
Case Study
Overview of assignment
As a new ABC Consultant assigned to the XYZ Company, you have been asked to enhance the current system to include payroll processing. Although the current employee table has monthly salary and commission columns, it does not provide any means for storing employee deductions. You will add the tables necessary to store employee deductions. Next you will create a payroll pre-calculation program that will calculate the net pay for all the employees via a batch process (a stored procedure in a package, which will call other stored procedures within the package). Although this is not a complete payroll system, the unit test results must be accurate.
Next you will create two PL/SQL blocks for inserting and deleting rows from the employee deduction table. These PL/SQL blocks will be passed information from host or bind variables and a third PL/SQL block which will assign the variables defined in SQL*Plus (e.g. employee number, dollar amount and deduction name). Since the XYZ Company wants to track changes to the employee and employee deduction tables, you will create two database triggers that will update audit tables when rows are changed or deleted.
The XYZ Company also requires a view that will display specific employee information, plus the number of deductions for an employe ...
2. Table of Contents This portfolio contains examples that were developed while participating in the SetFocus Microsoft Business Intelligence Masters Program.
4. Relational Physical Model of the database used in a team project. This database tracked evaluations submitted by students and was used to build a series of charts and reports that allowed instructors to see how they were performing over a variety of metrics.
5. Data Model of a Star Schema used to create a staging area to import data for a project. This project was used to track book sales by author, customer and region. Examples how it was used are found in the following section.
7. The queries in this section used the staging area shown in the previous section. This query returns all publishers with books that are currently out of stock, quantities still needed, and order status, ordered by greatest quantity needed first . SELECT books.ISBN, title as [Book Title], PUBLISHER, QUANTITYORDERED-QUANTITYDISPATCHED AS [Quantity Needed], orders.orderid, case when orderdate+3 < getdate() and quantityordered>quantitydispatched then 'Needs Review' else 'Standard Delay' end AS [STATUS] FROM BOOKS inner join ORDERITEMS on books.isbn=orderitems.isbn inner join orders on orderitems.orderid=orders.orderid WHERE QUANTITYORDERED>QUANTITYDISPATCHED and stock=0 group by books.ISBN, title, PUBLISHER, orderdate, orders.orderid, quantityordered, quantitydispatched order by [quantity needed] desc
8. This query determines the list of countries from which orders have been placed, the number of orders and order items that have been had from each country, the percentages of the total orders, and the percentages of total order items received from each country --Create temporary table select [Country], count(OrderNum) as [Total Orders], sum(QuantityOrdered) as [Total Items Ordered] into #C_Totals from (select right(address, charindex(' ',reverse(address))-1) as 'Country', customers.customerID , orders.orderID as 'OrderNum', orderItems.orderItemID , case when QuantityOrdered is null then 0 else QuantityOrdered end as QuantityOrdered from orders inner join orderItems on orders.orderID=orderItems.orderID right join customers on orders.customerID=customers.customerID group by QuantityOrdered, customers.customerID, address, orders.orderID, orderItems.OrderItemID ) as C_Orders group by [Country] with rollup --Query temporary table to extract, calculate and format data appropriately select [Country], [Total Orders], [Total Items Ordered], (convert(numeric(10,4),([Total Orders]/(select convert(numeric,[Total Orders]) from #C_Totals where [Country] is null)))*100) as [% of Total Orders] , (convert(numeric(10,4),([Total Items Ordered]/(select convert(numeric,[Total Items Ordered]) from #C_Totals where [Country] is null)))*100) as [% of Total Items Ordered] from #C_Totals where [Country] is not null group by [Country], [Total Orders], [Total Items Ordered] -- Should clean up afterwards drop table #C_Totals
10. This is the data flow and control flow for the Job Time Sheets package. This package imports multiple csv files doing checks to insure data integrity and writing rows that fail to an error log file for review. This is just one part of a project that brought in the employee and customer data of a construction company. The data structure created here was used in several subsequent BI projects.
11. This is the script that tracks how many records were processed for the Job Time Sheets package. It breaks them up into Inserted, Updated and Error categories. These totals are used in the email that is sent when processing is completed. ' Microsoft SQL Server Integration Services Script Task ' Write scripts using Microsoft Visual Basic ' The ScriptMain class is the entry point of the Script Task. Imports System Imports System.Data Imports System.Math Imports Microsoft.SqlServer.Dts.Runtime Public Class ScriptMain Public Sub Main() Dim InsertedRows As Integer = CInt(Dts.Variables("InsertedRows").Value) Dim ErrorRows As Integer = CInt(Dts.Variables("ErrorRows").Value) Dim RawDataRows As Integer = CInt(Dts.Variables("RawDataRows").Value) Dim UpdatedRows As Integer = CInt(Dts.Variables("UpdatedRows").Value) Dim InsertedRows_fel As Integer = CInt(Dts.Variables("InsertedRows_fel").Value) Dim ErrorRows_fel As Integer = CInt(Dts.Variables("ErrorRows_fel").Value) Dim RawDataRows_fel As Integer = CInt(Dts.Variables("RawDataRows_fel").Value) Dim UpdatedRows_fel As Integer = CInt(Dts.Variables("UpdatedRows_fel").Value) Dts.Variables("InsertedRows").Value = InsertedRows + InsertedRows_fel Dts.Variables("ErrorRows").Value = ErrorRows + ErrorRows_fel Dts.Variables("RawDataRows").Value = RawDataRows + RawDataRows_fel Dts.Variables("UpdatedRows").Value = UpdatedRows + UpdatedRows_fel Dts.TaskResult = Dts.Results.Success End Sub End Class
12. This is the data flow of a package that merged data from multiple tables together in order to create a fact table for a cube. This fact table was the key fact table used in the final BI project for our class.
14. This section uses the staging area that was created in the SSIS section. Here the data is deployed from the staging area into a cube and SSAS is used to create KPIs and Excel Reports.The AllWorks database deployed as a cube using SSAS
15. Browsing the AllWorks data cube to make sure the data is organized in the desired manner for purposes in which it will be used.
17. Creating Key Performance Indicators (KPIs) to show visually goals, trends and changes in tracked metrics
18. Use of a KPI in Excel to show Profits against projected goals.
19. Partitioning the cube to separate archival data from more frequently accessed data. Also Aggregations were setup here to further enhance storage and query performance.
21. This section uses the AllWorks cube created in the last section. The code created here was used in created calculations for use in KPIS.For 2005, show the job and the top three employees who worked the most hours. Show the jobs in job order, and within the job show the employees in hours worked order SELECT [Measures].[Hoursworked] ON COLUMNS, non empty Order(Generate([Job Master].[Description].[Description].members, ([Job Master].[Description].currentmember, TOPCOUNT([Employees].[Full Name].[Full Name].members, 3,[Measures].[Hoursworked]))), [Measures].[Hoursworked],DESC) on rows FROM Allworks WHERE[All Works Calendar].[Fy Year].&[2005]
22. Show Overhead by Overhead Category for currently selected quarter and the previous quarter, and also show the % of change between the two. WITH MEMBER [Measures].[Previous Qtr] AS ([Measures].[Weekly Over Head], ParallelPeriod ([All Works Calendar].[Fy Year - Fy Qtr].level ,1,[All Works Calendar].[Fy Year - Fy Qtr].currentmember) ) ,FORMAT_STRING = '$#,##0.00;;;0‘ MEMBER [Measures].[Current Qtr] AS ([Measures].[Weekly Over Head], [All Works Calendar].[Fy Year - Fy Qtr].currentmember) ,FORMAT_STRING = '$#,##0.00;;;0‘ MEMBER [Measures].[% Change] AS iif([Measures].[Previous Qtr], (([Measures].[Current Qtr] - [Measures].[Previous Qtr]) / [Measures].[Previous Qtr]),NULL), FORMAT_STRING = '0.00%;;;‘ SELECT {[Measures].[Current Qtr], [Measures].[Previous Qtr], [Measures].[% Change]} on columns, non empty [Overhead].[Description].members on rows FROM Allworks WHERE [All Works Calendar].[Fy Year - Fy Qtr].[2005].lastchild
23. List Hours Worked and Total Labor for each employee for 2005, along with the labor rate (Total labor / Hours worked). Sort the employees by labor rate descending, to see the employees with the highest labor rate at the top. WITH MEMBER [Measures].[Labor Rate] AS [Measures].[Total Labor]/[Measures].[Hoursworked], FORMAT_STRING = 'Currency' SELECT {[Measures].[Hoursworked],[Measures].[Total Labor], [Measures].[Labor Rate]} ON COLUMNS, non empty Order([Employees].[Full Name].members, [Measures].[Labor Rate], BDESC) on rows FROM Allworks WHERE[All Works Calendar].[Fy Year].&[2005]
25. This report uses the AllWorks cube and shows an employees labor cost by weekending date with a grand total at the bottom. Cascading parameters are used to select the employee and dates.
26. A report of region sales percentage by category for a selected year.
28. A breakdown of the labor cost of an employee by quarter. The line graph shows the percentage of labor of the employee for the quarter. The chart at the bottom shows the breakdown by job. These use the AllWorks cube created earlier.
29. The custom MDX code that allows the previous report to run
30. This scorecard has a drill down capability that will allow you to see regions, states and city Sales goals. In addition it is hot linked to a report that shows Dollar Sales.
32. This excel spreadsheet has taken data from a cube and created a chart. This chart and its parameter will then be published using Excel Services to make it available on a Sharepoint server.
33. Another chart built using Excel with data from a cube. This chart allows the tracking of sales data by Category on a selected year. In addition it tracks the selected category percentage of sales vs. the parent on a separate axis. This chart with its parameters was published to SharePoint Server using Excel Services.
35. SharePoint site created with document folders to hold Excel spreadsheets published with Excel Services, SSRS reports, and Performance Point Dashboards. Two web parts are on the front page showing a Excel chart and SSAS KPI published through Performance Point.
36. The previously shown Employee Labor Analysis Report deployed to SharePoiont as part of A Performance Point Dashboard.
37. An SSRS reported scheduled to be autogenerated using SharePoint scheduling and subsription services