Data Mining Beyond Adventure Works (Redmond WA 10/3/2009)

Data Mining beyond Adventure Works Mark Tabladillo Ph D Ph.D. MTabladillo <(at)> solidq.com October 3 2009 3,

Approach of this Presentation • Emphasize – Conceptual value of data mining – Relationship of data mining to the real world ld • Reserve – Specific procedures and mechanics – Specific mathematics – Production implementation © 2009 Mark Tabladillo Ph.D. 2

Data Mining Definitions • Data mining is the automatic or semi- automatic process of exploring data for t ti f l i d t f meaningful or useful patterns. • Data mining algorithms typically use estimation or optimization to achieve results (as opposed to only calculations). © 2009 Mark Tabladillo Ph.D. 5

Microsoft Data Mining • Microsoft Data Mining refers to Microsoft’s specific implementation of Mi ft’ ifi i l t ti f certain common data mining algorithms for the th DMX (D t Mi i E t (Data Mining Extensions) i ) language. • Also called SQL Server Data Mining, the technology is integrated into SQL Server rather than presented as an independent application. © 2009 Mark Tabladillo Ph.D. 6

Data Mining Tasks • Supervised – Answer known, what is correlated? • Unsupervised – Answer unknown (unspecified), what are the groups? • Forecasting – Given a trend, what is next? , Value Slide © 2009 Mark Tabladillo Ph.D. 7

The Data Mining Algorithms • Microsoft Naive Bayes • Microsoft Linear R Mi ft Li Regression i • Microsoft Decision Trees • Microsoft Time Series • Microsoft Clustering • Microsoft Sequence Clustering • Microsoft Association Rules • Microsoft Neural Networks • Microsoft Logistic Regression • Text Mining © 2009 Mark Tabladillo Ph.D. 9

The Analyze Tab Menu Option Data Mining Algorithm Analyze Key Influencers Naïve Bayes Detect Categories Clustering Fill from Example Logistic Regression Forecast Time Series Highlight Exceptions Clustering Scenario Analysis (Goal Seek) Logistic Regression Scenario Analysis (What If) Logistic Regression Prediction Calculator Logistic Regression Shopping Basket Analysis Association Rules © 2009 Mark Tabladillo Ph.D. 10

Demo One: National League Baseball • Directions: You Y are on the management team for the th tt f th Atlanta Braves. To better serve the team, you hhave b been i t t d b th owner t instructed by the to group the players by considering both their position and th i salary. iti d their l © 2009 Mark Tabladillo Ph.D. 11

Demo One: National League Baseball • The following rules apply: – You must make more than one group – Each group must have at least two players – Players of different position may be in the same group © 2009 Mark Tabladillo Ph.D. 12

Demo One: National League Baseball • Individual attributes can be used to make groups • Historical statistics can be used to group new players • Both supervised and unsupervised p p algorithms can be applied to the same data © 2009 Mark Tabladillo Ph.D. 13

Demo Two: Government Forecasting • Directions: The P id t is ki Th President i asking your opinion on i i how the following numbers will increase over th next f the t few months. B th Because thithis project is sensitive, you do not know what these numbers measure. H th b However, bbasedd on the available history, make your best projection f th next six periods. j ti for the t i i d © 2009 Mark Tabladillo Ph.D. 14

Demo Two: Government Forecasting 8 7 6 5 4 3 2 1 0 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2008 2008 2008 2008 2008 2008 2008 2008 2008 2008 2008 2008 © 2009 Mark Tabladillo Ph.D. 15

Demo Two: Government Forecasting 12 10 8 6 4 2 0 Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug 2007 2007 2007 2007 2008 2008 2008 2008 2008 2008 2008 2008 2008 2008 2008 2008 2009 2009 2009 2009 2009 2009 2009 2009 © 2009 Mark Tabladillo Ph.D. 16

Demo Two: Government Forecasting • Rapid response is as useful as prediction • Seek intelligent correlations among related metrics • Projections depend on time frame – modeling is continual g © 2009 Mark Tabladillo Ph.D. 17

Supervised Algorithms • Microsoft Naive Bayes • Microsoft Linear R Mi ft Li Regression i • Microsoft Decision Trees • Microsoft Neural Networks • Microsoft Logistic Regression Value Slide © 2009 Mark Tabladillo Ph.D. 19

Resources • MarkTab.NET Links, video resources and information for data mining • Data Mining with Microsoft SQL Server 2008 by Jamie MacLennan (Author), ZhaoHui Tang (Author), Bogdan Crivat (Author) • Smart Business Intelligence Solutions with Microsoft® SQL Server® 2008 (PRO-Developer) by Lynn Langit (Author), Matthew Roche (Author) • Solid Quality Mentors © 2009 Mark Tabladillo Ph.D. 21

Bonus: Sequence Clustering Ideas • Trading players in professional sports • Assigning l A i i players t certain positions to t i iti • Moving from city to city • Store path at the mall • Cancer treatment path • Taking up a musical instrument • Taking up sports • Blogging • Viral news © 2009 Mark Tabladillo Ph.D. 24

Data Mining Beyond Adventure Works (Redmond WA 10/3/2009)

0 comments

Notes on slide 1

Favorites, Groups & Events