Your SlideShare is downloading. ×

Applied Semantic Search with Microsoft SQL Server


Published on

Text mining is projected to dominate data mining, and the reasons are evident: we have more text available than numeric data. Microsoft introduced a new technology to SQL Server 2012 called Semantic …

Text mining is projected to dominate data mining, and the reasons are evident: we have more text available than numeric data. Microsoft introduced a new technology to SQL Server 2012 called Semantic Search. This session's detailed description and demos give you important information for the enterprise implementation of Tag Index and Document Similarity Index. The demos include a web-based Silverlight application, and content documents from Wikipedia. We'll also look at strategy tips for how to best leverage the new semantic technology with existing Microsoft data mining.

Published in: Business
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Applied EnterpriseSemantic MiningMark Tabladillo, Ph.D. (MVP, MCAD .NET, MCITP, MCT)PASS SQL Saturday #198 Vancouver BCFebruary 16, 2013
  • 2. Photos © 2013 Mark Tabladillo, All Rights Reserved
  • 3. Photos © 2013 Mark Tabladillo, All Rights Reserved
  • 4. NetworkingInteractive
  • 5. About MarkTabTraining and Consulting with Ph.D. – Industrial Engineering, Georgia TechData Mining Resources and Blog at Training and consulting internationally across many industries – SAS and Microsoft Contributed to peer-reviewed research and legislation Mentoring doctoral dissertations at the accredited University of Phoenix Presenter
  • 6. Quick LookMy Semantic Search
  • 7. InteractiveName three things you want from enterprise textmining
  • 8. IntroductionSQL Server 2012 has new Programmability Enhancements Statistical Semantic Search File Tables Full-Text Search ImprovementsThese combined technologies make SQL Server 2012 a strong contender in textmining
  • 9. OutlineWhy Microsoft is competitive for data miningDefinitions: what is text mining?History: how Microsoft’s semantic search was bornWhat is inside semantic search Logical model Demos PerformanceMicrosoft Resources
  • 10. Why Microsoft isCompetitive for DataMiningBased on 2012 and 2013 Surveys
  • 11. Gartner 2013 Magic Quadrant for Business Intelligence and Analytics Platforms Retrieved from – February 5, 2013
  • 12. Gartner 2013 Magic Quadrant for Data Warehouse Database Management Systems Retrieved from – January 31, 2013
  • 13. KDNuggets 2012
  • 14. DefinitionsWhat is text mining?
  • 15. DefinitionData mining is the automated or semi-automated process ofdiscovering patterns in data Text mining is the automated or semi-automated process of discovering patterns from textual dataMachine learning is the development and optimization ofalgorithms for automated or semi-automated pattern discovery
  • 16. Purposes Phrase Goal “Data Mining” Inform actionable decisions “Text Mining” “Machine Determine best performing Learning” algorithm
  • 17. MarkTab Decision Cycle GO Synthesis Analysis (art) (science) Science needs science fiction -- MarkTab
  • 18. MarkTab Decision Cycle GO Synthesis Analysis (art) (science)
  • 19. HistoryHow Microsoft’s semantic search came to be
  • 20. HistoryJuly 2008 Microsoft purchases Powerset for US$100 Million Google Dismisses Semantic Search powerset-for-100m-plus/ cx_ag_0701powerset.html
  • 21. HistoryMarch 2009 Google announces “snippets” as relevant to search The media picks this story up as “semantic search” results.html#!/2009/03/two-new-improvements-to-google-results.html
  • 22. HistoryFebruary 2012 Google announces Knowledge Graph, an explicit application of semantic search
  • 23. HistoryApril 2012 Microsoft purchases 800+ patents from AOL for US$1 Billion Among the patents are semantic search and metadata querying – older than Google
  • 24. What is inside SemanticSearchText Mining introduced for SQL Server 2012
  • 25. Future: Most data is TextTwo Research Types• Quantitative research = data mining• Qualitative research = text miningThe future is combining both
  • 26. Statistical Semantic SearchComprises some aspects of text miningIdentifies statistically relevant key phrasesBased on these phrases, can identify (by score) similar documents
  • 27. FileTablesBuilt on existing SQL Server FILESTREAM technologyFiles and documents Stored in special tables in SQL Server Accessed if they were stored in the file system
  • 28. Full-Text Search EnhancementsProperty search: search on tagged properties (such as author or title)Customizable NEAR: find words or phrases close to one anotherNew Word Breakers and Stemmers (for many languages)
  • 29. Logical ModelHow semantic search works
  • 30. From Documents to Output Office Varchar PDF NVarchar Rowset Output with Scores
  • 31. (iFilter Required) iFilters Full-Text Documents Keyword Index “FTI” Semantic Key Phrase Semantic Index – Semantic Document Database Tag Index Similarity Index “DSI” “TI”
  • 32. Languages Currently SupportedTraditional Chinese Simplified ChineseGerman British EnglishEnglish PortugueseFrench Chinese (Hong Kong SAR, PRC)Italian SpanishBrazilian Chinese (Singapore)Russian Chinese (Macau SAR)Swedish
  • 33. Phases of Semantic Indexing Full Text Keyword Index “FTI” Semantic Document Similarity Index “DSI” Semantic Key Phrase Index – Tag Index “TI”
  • 34. Interactive DemoSQL Server Management Studio
  • 35. Semantic Search andSQL Server Data MiningSQL Server Data Tools: data mining plus text mining
  • 36. PerformanceThe Million-Dollar Edge
  • 37. Integrated Full Text Search (iFTS)Improved Performance and Scale: Scale-up to 350M documents for storage and search iFTS query performance 7-10 times faster than in SQL Server 2008 Worst-case iFTS query response times less than 3 sec for corpus Similar or better than main database search competitors(2012, Michael Rys, Microsoft)
  • 38. Linear Scale of FTI/TI/DSIFirst known linearly scaling end-to-end Search and Semantic product in the industry Time in Seconds vs. Number of Documents (2011 – K. Mukerjee, T. Porter, S. Gherman – Microsoft)
  • 39. Text Mining ReferencesVideo Search Search (Books Online) – explains the demo
  • 40. Microsoft ResourcesLinks
  • 41. SoftwareSQL Server 2012 Enterprise(includes database engine, Analysis Services, SSMS and SSDT) Office 2012 Professional
  • 42. Organizations Professional Association for SQL Server Atlanta MDF Atlanta Microsoft BI Users Group Business-Intelligence-Users/PASS Business Analytics Conference http://www.passbaconference.comMicrosoft TechEd North America
  • 43. InteractiveTakeaways
  • 44. ConclusionSQL Server Data Mining 2012 provides data mining and semantic searchThe core technology allows document similarity matchingThe results can be combined with SQL Server Data Mining (such asAssociation Analysis)
  • 45. ConnectData Mining Resources and blog http://marktab.netData Mining Training and Consulting (especially Microsoft and SAS)