Downloaded 15 times
Uploaded byMark Tabladillo
Applied Enterprise Semantic Search 201305
The document discusses the advancements in SQL Server 2012 regarding text and data mining, highlighting its competitive edge in the market due to new programmability enhancements and semantic search capabilities. It explains the evolution of Microsoft's semantic search technology, including key acquisitions and developments over the years, and outlines the technical aspects of semantic indexing and performance improvements. Additionally, it emphasizes the importance of integrating both quantitative and qualitative research in future data analysis.
More Related Content
Similar to Applied Enterprise Semantic Search 201305
Applied Enterprise Semantic Search 201305
- 1. Applied Enterprise Semantic Mining MarkTabladillo, Ph.D. (MVP, MCAD .NET, MCITP, MCT) PASS SQL Saturday #220 Atlanta GA May 18, 2013
- 2.
- 3. About MarkTab Training andConsulting with http://marktab.com Data Mining Resources and Blog at http://marktab.net Twitter @marktabnet
- 4. Quick Look My SemanticSearch
- 5. Interactive Name three thingsyou want from enterprise text mining
- 6. Introduction SQL Server 2012has new Programmability Enhancements Statistical Semantic Search File Tables Full-Text Search Improvements These combined technologies make SQL Server 2012 a strong contender in text mining
- 7. Outline Why Microsoft iscompetitive for data mining Definitions: what is text mining? History: how Microsoft’s semantic search was born What is inside semantic search Logical model Demos Performance Microsoft Resources
- 8. Why Microsoft is Competitivefor Data Mining Based on 2012 and 2013 Surveys
- 9. Gartner 2013 Magic Quadrantfor Business Intelligence and Analytics Platforms Retrieved from http://www.gartner.com/technology/reprints.do?id=1-1DZLPEH&ct=130207&st=sb – February 5, 2013
- 10. Gartner 2013 Magic Quadrantfor Data Warehouse Database Management Systems Retrieved from http://www.gartner.com/technology/reprints.do?id=1-1DU2VD4&ct=130131&st=sb – January 31, 2013
- 11.
- 12. Definitions What is textmining?
- 13. Definition Data mining isthe automated or semi-automated process of discovering patterns in data Text mining is the automated or semi-automated process of discovering patterns from textual data Machine learning is the development and optimization of algorithms for automated or semi-automated pattern discovery
- 14. Purposes Phrase Goal “Data Mining” “TextMining” Inform actionable decisions “Machine Learning” Determine best performing algorithm
- 15. MarkTab Decision Cycle Analysis (science) Synthesis (art) GO Scienceneeds science fiction -- MarkTab
- 16.
- 17. History How Microsoft’s semanticsearch came to be
- 18. History July 2008 Microsoft purchasesPowerset for US$100 Million Google Dismisses Semantic Search http://venturebeat.com/2008/06/26/microsoft-to-buy-semantic-search-engine- powerset-for-100m-plus/ http://www.forbes.com/2008/07/01/powerset-msft-search-tech-intel- cx_ag_0701powerset.html
- 19. History March 2009 Google announces“snippets” as relevant to search The media picks this story up as “semantic search” http://googleblog.blogspot.com/2009/03/two-new-improvements-to-google- results.html#!/2009/03/two-new-improvements-to-google-results.html
- 20. History February 2012 Google announcesKnowledge Graph, an explicit application of semantic search http://mashable.com/2012/02/13/google-knowledge-graph-change-search/
- 21. History April 2012 Microsoft purchases800+ patents from AOL for US$1 Billion Among the patents are semantic search and metadata querying – older than Google http://www.theregister.co.uk/2012/04/09/aol_microsoft_patent_deal/
- 22. What is insideSemantic Search Text Mining introduced for SQL Server 2012
- 23. Future: Most datais Text Two Research Types • Quantitative research = data mining • Qualitative research = text mining The future is combining both
- 24. Statistical Semantic Search Comprisessome aspects of text mining Identifies statistically relevant key phrases Based on these phrases, can identify (by score) similar documents
- 25. FileTables Built on existingSQL Server FILESTREAM technology Files and documents Stored in special tables in SQL Server Accessed if they were stored in the file system
- 26. Full-Text Search Enhancements Propertysearch: search on tagged properties (such as author or title) Customizable NEAR: find words or phrases close to one another New Word Breakers and Stemmers (for many languages)
- 27. Logical Model How semanticsearch works
- 28.
- 29. (iFilter Required) Documents Full-Text Keyword Index “FTI” iFilters Semantic Document SimilarityIndex “DSI” Semantic Database Semantic Key Phrase Index – Tag Index “TI”
- 30. Languages Currently Supported TraditionalChinese German English French Italian Brazilian Russian Swedish Simplified Chinese British English Portuguese Chinese (Hong Kong SAR, PRC) Spanish Chinese (Singapore) Chinese (Macau SAR)
- 31. Phases of SemanticIndexing Full Text Keyword Index “FTI” Semantic Key Phrase Index – Tag Index “TI” Semantic Document Similarity Index “DSI” http://msdn.microsoft.com/en-us/library/gg492085.aspx#SemanticIndexing
- 32. Interactive Demo SQL ServerManagement Studio
- 33. Semantic Search and SQLServer Data Mining SQL Server Data Tools: data mining plus text mining
- 34.
- 35. Integrated Full TextSearch (iFTS) Improved Performance and Scale: Scale-up to 350M documents for storage and search iFTS query performance 7-10 times faster than in SQL Server 2008 Worst-case iFTS query response times less than 3 sec for corpus Similar or better than main database search competitors (2012, Michael Rys, Microsoft)
- 36. Linear Scale ofFTI/TI/DSI First known linearly scaling end-to-end Search and Semantic product in the industry Time in Seconds vs. Number of Documents (2011 – K. Mukerjee, T. Porter, S. Gherman – Microsoft)
- 37. Text Mining References Video http://channel9.msdn.com/Shows/DataBound/DataBound-Episode-2-Semantic- Search http://www.microsoftpdc.com/2009/SVR32 SemanticSearch (Books Online) – explains the demo http://msdn.microsoft.com/en-us/library/gg492075.aspx Paper http://users.cis.fiu.edu/~lzhen001/activities/KDD2011Program/docs/p213.pdf
- 38.
- 39. Software SQL Server 2012Enterprise (includes database engine, Analysis Services, SSMS and SSDT) http://www.microsoft.com/sqlserver/en/us/get-sql-server/try-it.aspx Microsoft Office 2012 Professional http://office.microsoft.com/en-us/try
- 40. Organizations Professional Association forSQL Server http://www.sqlpass.org Atlanta MDF http://www.atlantamdf.com/ Atlanta Microsoft BI Users Group http://www.meetup.com/Atlanta-Microsoft- Business-Intelligence-Users/ PASS Business Analytics Conference http://www.passbaconference.com Microsoft TechEd North America http://northamerica.msteched.com/
- 41.
- 42. Connect Data Mining Resourcesand blog http://marktab.net Data Mining Training and Consulting (especially Microsoft and SAS) http://marktab.com
- 43. Conclusion SQL Server DataMining 2012 provides data mining and semantic search The core technology allows document similarity matching The results can be combined with SQL Server Data Mining (such as Association Analysis)