This document provides an overview of a SQL Server 2008 for Business Intelligence short course. It discusses the course instructor's background and specialties. The course will cover creating a data warehouse, OLAP cubes, and reports. It will also discuss data mining concepts like why it's used, common algorithms, and include a hands-on lab. Data mining algorithms that will be covered include classification, clustering, decision trees, and neural networks.

1. SQL Server 2008 for Business IntelligenceUTS Short Course

2. Peter GfaderSpecializes in C# and .NET (Java not anymore)TestingAutomated testsAgile, ScrumCertified Scrum TrainerTechnology aficionado SilverlightASP.NETWindows Forms

3. Admin StuffAttendanceYou initial sheetHands On LabYou get me to initial sheetHomeworkCertificate At end of 5 sessionsIf I say if you have completed successfully 

4. Course WebsiteCourse Timetable & Materialshttp://www.ssw.com.au/ssw/Events/2010UTSSQL/Resourceshttp://sharepoint.ssw.com.au/Training/UTSSQL/

5. Course Overview

6. Last week(s)Other cube browsersMicrosoft Data AnalyzerProclarityExcel 2003/2007/2010Excel servicesThinslicerPerformance PointPower Pivot

7. Create report on top of NorthwindTop 10 customers (Table)Top 10 products (Table)Top 10 employees (Table)1 chart that shows the top 10 customers1 usage of the gauge control (surprise me)Homework

8. The plan

9. Step by step to BICreate Data WarehouseCopy data to data warehouse Create OLAP CubesCreate ReportsBrowse the cubeDo some Data MiningDiscovering relationshipsPredict future events

10. AgendaWhat is Data Mining?Why?UsesAlgorithmsDemoHands on Lab

11. What is Data Mining?“Data mining is the use of powerful software tools to discover significant traits or relationships,from databases or data warehouses and often used to predict future events”

12. What is Data Mining?It exploits statistical algorithms Once the “knowledge” is extracted it:Can be used to discoverCan be used to predict values of other cases

13. Why Data Mining?MarketingWho picks the movie? The kids, the wife, meWho are our Customers and what sort of films do they hire?Is a 30 year old woman with 2 children going to hire Arnie’s latest filmValidationIs this data sensible? Terminator 2 and Toy StoryPredictionSales Next Year

14. Get new information from data, future trends, past trends, outlier, maximums, minimumsAnalyse data from different perspectives and summarizing it into useful informationNew information toincrease revenuecuts costsor both :-)Why? Its all about money

15. Who are our biggest customers?What are customers buying with cigars?What are the customer retention levels of our branches?Which customers have bought olives, feta cheese but no ciabatta bread?Which regions have the highest male/female ratio of single 20 somethings?Which region has lowest customer retention levels and list out lost customers?Which Questions are Data Mining?

16. Ad hoc queryDrill through to detailsBusiness Intelligence toolWhat’s not data mining

17. Huge amount of data

18. Good raw material  good data miningSamples should be representativeSamples "similar" to domainNot all-seeing crystal ballVerify and Validate!Data - Uncover patterns in samples

19. OLAPIs about fast ad hoc queryingAnalysis by dimensions and measuresGives precise answersData MiningMay use RDBMS or OLAP sourceIs about discovering and predictingGives imprecise answersOLAP is not a prerequisite for data mining, but it almost always comes firstOLAP versus Data Mining(learning to ride a bike before a car)

20. Classification algorithms predictone or more discrete variables, based on the other attributes in the datasetRegression algorithms predictone or more continuous variables, such as profit or loss, based on other attributes in the datasetSegmentation algorithms dividedata into groups, or clusters, of items that have similar propertiesAssociation algorithms find correlations between different attributes in a datasetSequence analysis algorithms summarize frequent sequences or episodes in data, such as a Web path flowTypes of Data Mining Algorithms

21. ClusteringTime SeriesDecision TreesNaïve BayesAssociationLinear RegressionComplete Set Of AlgorithmsWays to analyze your dataNeural NetworkSequence ClusteringLogistic Regression

22. Split dataEach of branch is like an attributeBrightness = amount of dataDecision trees

23. Decision Trees (1)Decision Trees assign (classify) each case to one of a few (discrete) broad categories of selected attribute (variable) and explains the classification with few selected input variablesThe process of building is recursive partitioning – splitting data into partitions and then splitting it up moreInitially all cases are in one big box

24. Decision Trees (2)The algorithm tries all possible breaks in classes using all possible values of each input attribute; it then selects the split that partitions data to the purest classes of the searched variableSeveral measures of purityThen it repeats splitting for each new classAgain testing all possible breaksUnuseful branches of the tree can be pre-pruned or post-pruned

25. Decision Trees (3)Decision trees are used for classification and predictionTypical questions:Predict which customers will leaveHelp in mailing and promotion campaignsExplain reasons for a decisionWhat are the movies young female customers like to buy?

26. Decision Trees – Who Decides

27. Naïve BayesBayes FormulaUses statistics to say falls into certain category or not with probabilitySpam filtering: score of spam (Bayes)Testing only a particular attribute

28. Naïve BayesQuickly builds mining models that can be used for classification and predictionIt calculates probabilities for each possible state of the input attribute, given each state of the predictable attributeThis can later be used to predict an outcome of the predicted attribute based on the known input attributes This makes the model a good option for exploring the data

29. Cluster Analysis (1)Grouping data into clustersObjects within a cluster have high similarity based on the attribute valuesThe class label of each object is not knownSeveral techniquesPartitioning methodsHierarchical methodsDensity based methodsModel based methodsAnd more…

30. Cluster Analysis (2)Segments a heterogeneous population into a number of more homogenous subgroups or clustersSome typical questions:Discover distinct groups of customersIdentification of groups of houses in a cityIn biology, derive animal and plant taxonomiesFind outliers

31. ClusteringAnnual IncomeAge

32. Time seriesTimebaseddata  prediction

33. Sequence clusteringNumbers orders stronger associationsDirection of association (not necessary the other direction)

34. If you own certain stocks ' you own maybe other ones as wellProbability = thickness of lineAssociation

35. Let system learn how to classify dataNeural Network adapts to the new dataFormulate statement/hypothesisOutcome is know(Data / Surveys)1. 70% data to train network (outcome is known)2. 30% of data to test network (outcome is known)3. New data (no survey needed, predict from network)Other example: OCR Neural Nets

36. Both have directionsSequence Clustering has probability number and colourThey are very similar. The difference is that Association analyses items that occur together whereas sequence clustering analyses items that follow one another.An example is that Sequence Clustering might be used by credit card companies to spot fraud, e.g. a petrol station refill followed by another petrol station refill followed by a big purchase = fraud (different transactions)Whereas Association will be more like: when someone buys popcorn at the cinemas, they also buy a drink (same transaction)Difference between algorithms: Association and Sequence

37. Conclusion: When To Use What

38. Visual Numerics3rd party algorithmshttp://www.vni.com/company/whitepapers/ MicrosoftBIwithNumericalLibraries.pdfThere is more...

39. Excel Data MiningMicrosoft SQL Server 2008 Data Mining Add-ins for Microsoft Office 2007http://www.microsoft.com/downloads/en/details.aspx?familyid=896A493A-2502-4795-94AE-E00632BA6DE7&displaylang=en

40. Train station / airport Who is the bad guyFarmers Find the best cropsSupermarket Find to figure out how to get you to buy more, where the expensive itemsOther usages of data miningFind patterns - Profiling

41. SSIS 2008 - Data profiling taskGet a profile of the data in a table potential candidate keyslength of data values in columnsNull percentage of rowsdistribution of values....Tip

42. Video: Simple data mining modelhttp://www.sqlservercentral.com/articles/Video/65055/Video: Data mining and Reporting Serviceshttp://www.sqlservercentral.com/articles/Video/64190/Data Mining Algorithmshttp://msdn.microsoft.com/en-us/library/ms175595.aspxResources 1

43. Jamie MacLennanhttp://blogs.msdn.com/b/jamiemac/Richard Lees on BIhttp://richardlees.blogspot.com/Book Data Mining with Microsoft SQL Server 2008http://www.amazon.com/gp/product/0470277742?ie=UTF8&tag=sqlserverda09-20&linkCode=as2&camp=1789&creative=9325&creativeASIN=0470277742Resources 2

44. SummaryWhy Data Mining?UsesAlgorithmsDemoHands on Lab

45. 3things…PeterGfader@ssw.com.auhttp://blog.gfader.com/twitter.com/peitor

46. Thank You!Gateway Court Suite 10 81 - 91 Military Road Neutral Bay, Sydney NSW 2089 AUSTRALIA ABN: 21 069 371 900 Phone: + 61 2 9953 3000 Fax: + 61 2 9953 3105 info@ssw.com.auwww.ssw.com.au