SQL Server 2008 Data Mining

SSAS 2008 Data Mining

Lynn Langit MSDN Developer Evangelist – Southern California

http://blogs.msdn.com/SoCalDevGal

Session Prerequisites

Working SQL Server 2005 Developer

Understanding of OLAP concepts

Working SQL Server Analysis Server 2005 Developer

Interest in or basic knowledge of Data Mining concepts

Session Objectives and Agenda

Understand what, why, when & how of SQL Server 2008 Data Mining

Examine the core functionality of the Data Mining Extensions

Hear about the new and/or advanced functionality of Data Mining

What and Why Data Mining? Predictive Analytics Presentation Exploration Discovery Passive Interactive Proactive Role of Software Business Insight Canned reporting Ad-hoc reporting OLAP Data mining

OLAP vs. Data Mining

DM - From Scenarios to Tasks

From Tasks to Techniques

BI for Everyone Enterprise – Performance Point Individual – Excel Project – Share Point

Microsoft’s Predictive Analytics Data Mining SQL extensions (DMX) Application Developer Data Mining Specialist Microsoft Dynamics CRM Analytics Foundation SQL Server 2005 Business Intelligence Development Studio Microsoft SQL Server 2008 Analysis Services Information Worker Data Mining Add-ins for the 2007 Microsoft Office system Microsoft SQL Server 2008 Data Mining BI Analyst Custom Algorithms

Data Mining Add-ins for Office 2007 Table Analysis Tools for Excel 2007 Data Mining Template for Visio 2007 Data Mining Client for Excel 2007 Information Worker BI Analyst Data Mining Specialist

Microsoft Data Mining Lifecycle CRISP-DM SSAS (Data Mining) Excel SSAS (DSV) Query Excel SSIS SSAS SSRS Excel Your Apps SSIS SSAS Excel Data www.crisp-dm.org Business Understanding Data Understanding Data Preparation Modeling Evaluation Deployment

Understand & Prepare specifics

Demo 1 – Explore / Clean / Partition Data

Modeling Specifics

Demo 3 – Modeling

Evaluation Specifics

Demo 4 – Evaluation

Data Mining – Logical Model Mining Model Mining Model Training Data DB data Client data Application data Data Mining Engine Data To Predict Predicted Data Mining Model DB data Client data Application data “ Just one row ” Data Mining Engine

Data Mining - Physical Model Analysis Services Server Mining Model Data Mining Algorithm Data Source Your Application OLE DB/ ADOMD/ XMLA Deploy BI Dev Studio (Visual Studio) App Data

Data Mining Interfaces – APIs XMLA Over TCP/IP XMLA Over HTTP Analysis Server (msmdsrv.exe) OLAP Data Mining Server ADOMD.NET .Net Stored Procedures Microsoft Algorithms Third Party Algorithms OLEDB for OLAP/DM ADO/DSO Any Platform, Any Device C++ App VB App .Net App AMO Any App ADOMD.NET WAN DM Interfaces

Configuration

Model Creation/Management

Database Administrators

Session Mining Models

Model Application

Permissions on models

Permissions on data sources

Deployment

Browse

Copy to Excel

Drillthrough

Query

Default

Advanced

Excel Services

Manage models and structures

Export/Import

Rename

Connection

Database

Trace

Excel Functions*

DMPREDICTTABLEROW ( Connection, ModelName, PredictionResult, TableRowRange [, string CommaSeparatedColumnNames] )

DMPREDICT ( Connection, Model, PredictionResult, Value1, Name1, [...,Value32, Name32] )

DMCONTENTQUERY (Connection, Model, PredictionResult [, WhereClause])

Data Mining Extensions (DMX) CREATE MINING MODEL CreditRisk (CustID LONG KEY, Gender TEXT DISCRETE, Income LONG CONTINUOUS, Profession TEXT DISCRETE, Risk TEXT DISCRETE PREDICT) USING Microsoft_Decision_Trees INSERT INTO CreditRisk (CustId, Gender, Income, Profession, Risk) Select CustomerID, Gender, Income, Profession,Risk From Customers Select NewCustomers.CustomerID, CreditRisk.Risk, PredictProbability(CreditRisk.Risk) FROM CreditRisk PREDICTION JOIN NewCustomers ON CreditRisk.Gender=NewCustomer.Gender AND CreditRisk.Income=NewCustomer.Income AND CreditRisk.Profession=NewCustomer.Profession

DMX Column Expressions

Predictable Columns

Source Data Columns

Functions

Predict

“ Workhorse”

Discrete scalar values

Continuous scalar values

Associative nested tables

Sequence nested tables

Time Series

Overloaded to

PredictAssociation

PredictSequence

PredictTimeSeries

PredictProbability

PredictSupport

PredictHistogram

Cluster

ClusterProbability

GetNodeId

IsInNode

Arithmetic operators

Stored Procedure

Subselect

Select from nested tables

Data Mining Interfaces – XMLA ++ XMLA Over TCP/IP XMLA Over HTTP Analysis Server (msmdsrv.exe) OLAP Data Mining Server ADOMD.NET .Net Stored Procedures Microsoft Algorithms Third Party Algorithms OLEDB for OLAP/DM ADO/DSO Any Platform, Any Device C++ App VB App .Net App AMO Any App ADOMD.NET WAN DM Interfaces

New to SQL Server 2008

Microsoft_Time_Series algorithm improved

now includes ARIMA in addition to the existing ARTxp method, and a blending algorithm is now used to deliver more accurate and stable predictions, both short and long term, from a hybrid model. In addition, a new prediction mode allows you to add new data to time series models.

Holdout Support added

You can easily partition your data into training and test sets that are stored in the mining structure and are available to query after processing.

Ability to build mining models based on filtered subsets added

(e.g. just male customers), this means that you no longer have to create multiple mining structures and re-read the source data for such variations over a dataset

Drillthrough functionality extended

makes all mining structure columns available, not just columns included in the model

allows you to build more compact models without sacrificing the ability to producing actionable output reports like targeted mailing lists.

Cross-validation added

allows users to quickly validate their modeling approach by automatically building temporary models and evaluating accuracy measures across K folds. The feature is available through a new cross-validation tab under Accuracy Charts in Business Intelligence Development Studio, in addition to being accessible programmatically via a stored procedure call.

Summary

Data Mining in SQL Server 2008 is mature, powerful and accessible

Can use Excel 2007

Familiar client for BI – OLAP cubes AND Data Mining models

Model Creators / Users

Excel Data or Server Data

SSAS and Excel both support the full DM Cycle

Data Understanding

Data Preparation

Modeling

Validation

Deployment

DM Webcasts Fri, 02 Nov 2007 MSDN Webcast: Build Smart Web Applications with SQL Server Data Mining (Level 200) Thu, 08 Nov 2007 MSDN Webcast: Building Adaptive Applications with SQL Server Data Mining (Level 300) Mon, 19 Nov 2007 MSDN Webcast: Extending and Customizing SQL Server Data Mining (Level 300) Fri, 30 Nov 2007 MSDN Webcast: Creating Visualizations for SQL Server Data Mining (Level 300) Thu, 01 Nov 2007 TechNet Webcast: Deliver Actionable Insight Throughout Your Organization with Data Mining (Part 1 of 3): Your First Project with SQL Server Data Mining (Level 200) Thu, 15 Nov 2007 TechNet Webcast: Deliver Actionable Insight Throughout Your Organization with Data Mining (Part 2 of 3): Understand SQL Server Data Mining Add-ins for the 2007 Office System (Level 200) Thu, 29 Nov 2007 TechNet Webcast: Deliver Actionable Insight Throughout Your Organization with Data Mining (Part 3 of 3): Use Predictive Intelligence to Create Smarter KPIs (Level 200)

BI Resources from Lynn Langit 1. “Foundations of SQL Server 2005 Business Intelligence” (published by APress in April 2007) 2. http://blogs.msdn.com/SoCalDevGal 3. “Building Business Intelligence Solutions with SQL Server 2008” (MSPress Fall 2008)

DM Resources Technical Communities, Webcasts, Blogs, Chats & User Groups http://www.microsoft.com/communities/default.mspx Microsoft Developer Network (MSDN) & TechNet http://microsoft.com/msdn http://microsoft.com/technet Trial Software and Virtual Labs http://www.microsoft.com/technet/downloads/trials/default.mspx Microsoft Learning and Certification http://www.microsoft.com/learning/default.mspx SQL Server Data Mining http://www.sqlserverdatamining.com http://www.microsoft.com/bi/bicapabilities/data-mining.aspx