Knowledge Mining with Azure Search
•Download as PPTX, PDF•
1 like•246 views
The document discusses Azure Search and how it can be used for knowledge mining and decision making. It provides an overview of Cognitive Search Architecture which allows ingesting customer data from various sources and enriching it with built-in and custom skills before exploring the indexed data. Examples of skills that can be used include optical character recognition, text recognition, face detection and more. These skills can also be chained together for advanced capabilities.
Recommended
Recommended
More Related Content
What's hot
What's hot (13)
Similar to Knowledge Mining with Azure Search
Similar to Knowledge Mining with Azure Search (20)
More from AI Leadership Institute
More from AI Leadership Institute (15)
Recently uploaded
Recently uploaded (20)
Knowledge Mining with Azure Search
- 1. Knowledge Mining with Azure Search Minería del Conocimiento para la toma de Decisión Noelle LaCharite
- 2. Connect, Create, Innovate Noelle LaCharite Principal Program Manager, Developer Experience Applied AI and Cognitive Services @noellelacharite Connect with me on LinkedIn
- 3. “What is success? To laugh often and much; to win the respect of intelligent people and the affection of children; to earn the appreciation of honest critics and endure the betrayal of false friends; to appreciate the beauty; to find the best in others; to leave the world a bit better, whether by a healthy child, a garden patch Or a redeemed social condition; to know even one life has breathed easier because you have lived. This is to have succeeded!” ― Bessie Anderson Stanley Adapted by Ralph Waldo Emerson
- 4. Dr. Vannevar Bush in 1945
- 8. Seattle, WA Microsoft Campus 47.61164, -122.3331 Buenos Aires, Argentina Web Document Update 28.532270, -81.380906 Tokyo, Japan Twitter Feed Update 35.699441, 139.683795 Brisbane, AU Web Document Update -27.481616, 152.971464
- 9. 2.5 quintillion bytes per day
- 11. 80% of business relevant information is unstructured
- 14. Cognitive Search Architecture Annotated Documents Customer Data Search Index INGEST ENRICH EXPLORE
- 15. Document Cracking • Different types of data sources • Blob Storage • Azure SQL • Cosmos DB • Azure Tables • MySQL • Azure Files ( private preview) • File formats supported in blob storage
- 16. Built-in skills Cognitive skills Custom skills Azure Databricks Machine Learning VMs Azure Machine Learning Your custom skill goes here!
- 17. Skills can be chained OCR (text recognition) custom skill 2 custom skill 1 handwritten text recognition face detection face detection
- 18. Cognitive Services capabilities Infuse your apps, websites, and bots with human-like intelligence
Editor's Notes
- In 1945 Dr. Vannevar Bush wrote an essay called ‘As We May Think’ which was published in the Atlantic Monthly. In essay he ideated around something he called the ‘MEMEX’ Doug Engelbart came across the essay shortly after its publication, and keeping the memex in mind, he began work that would eventually result in the invention of the mouse, the word processor, the hyperlink and concepts of new media for which these groundbreaking inventions were merely enabling technologies
- In 1945 he wrote an essay called ‘As We May Think’ which was published in the Atlantic Monthly. In essay he ideated around something he called the ‘MEMEX’
- Bush envisioned the memex as a device in which individuals would compress and store all of their books, records, and communications, "mechanized so that it may be consulted with exceeding speed and flexibility". Coincidentally, Doug Engelbart came across the essay shortly after its publication, and keeping the memex in mind, he began work that would eventually result in the invention of the mouse, the word processor, the hyperlink and concepts of new media for which these groundbreaking inventions were merely enabling technologies
- This slide and the next slide is about telling the story that in just the few moments that the speaker was talking millions of updates are taking place to facts, documents on the web, twitter updates spanning the entire planet. And How Bing was able to index the content and keep it up to date for users…
- These days, there is a lot more data than could fit into one of these Memex’s. A huge amount of data is now generated each day – a total of 2.5 quintillion bytes of data per day. Autoencoders for compression We are generating all this information through IOT devices, logs of our usage online and on mobile applications, images, video, audio, and even documents that we generate. Data, next to compute and algorithms, is seen as one of the drivers of the AI revolution.
- But one of the biggest challenges that we have is that a lot of that data has latent content that we have not yet been understood or unleashed. This is an actual picture of a customer that shall not be named – as it turns out the best way to do a Denial of Service on a legal firm is to provide them data that is not structured…
- Much of this data ends up in the unstructured form. It’s estimated that 80-90% of business relevant information is unstructured and is primarily text. The goal of my team is to create platform components that can enable you to better extract the value of unstructured data. Think of some of the most important documents you have dealt with: Your employment contract, the deed of your house, your tax form. A few months ago, I learned that if I filled a portion of my taxes incorrectly I could be charged several thousands of dollars per year.
- We started this journey with Azure Search, a down-to-earth search as a service product with all the capabilities that you would expect from a search service. Compare Paas with Saas Highlight: Synonyms Boosting Scoring Lexical Analyzer Scenario of web store You connected your data source to our service, we would index the information in the data source. We were able to tell when the data sources changed to keep your search index up to date. It supports different data sources such as CosmosDB, Blob storage and Azure SQL. We received so We started this journey with Azure Search, a down-to-earth search as a service product with all the capabilities that you would expect from a search service. Compare Paas with Saas Highlight: Synonyms Boosting Scoring Lexical Analyzer Scenario of web store You connected your data source to our service, we would index the information in the data source. We were able to tell when the data sources changed to keep your search index up to date. It supports different data sources such as CosmosDB, Blob storage and Azure SQL. We received some cle management free keyword search faceting language analyzers geospatial support suggestions/auto-complete customizable scoring proximity search synonyms etc. ar feedback that files were not all structured (like JSON files). Our customers wanted us to be able to read a variety of documents. me clear feedback that files were not all structured (like JSON files). Our customers wanted us to be able to read a variety of documents.
- Talking Points: need to add
- MongoDB?
- Customers lack data science skills and roles Vision: Image-processing algorithms to smartly identify, caption and moderate your pictures Computer vision: Distill actionable information from images Content Moderator: Automatically moderate potentially offensive images, text and videos Customer Vision Service: Train a web service to recognize specific content in images Face: Identify human faces and emotions in images Video indexer: Easily extract insights from your videos to enrich your applications Speech: Convert spoken audio into text, use voice for verification, or add speaker recognition to your app Bing Speech: Convert speech to text and text to speech Speaker Recognition: Use speech to identify and authenticate individual speakers Custom Speech Service: Overcome speech recognition barriers like speaking style, background noise, and vocabulary Translator Speech: Easily conduct real-time speech translation on your app Language: Enable your apps to process natural language with pre-built scripts, evaluate sentiment and learn how to recognize what users want Bing Spell Check: Add spell checking functionality to your app Language Understanding (LUIS): Add language understanding intelligence to your apps with minimal effort Linguistic Analysis: Easily parse complex text with language analysis Text Analytics: Easily evaluate sentiment, language, and key phrases to understand what users want Translator Text: Easily conduct machine translation for 60+ languages Knowledge: Map complex information and data in order to solve tasks such as intelligent recommendations and semantic search Knowledge Exploration Service: Enable interactive search experiences over structured data via natural language inputs Entity Linking Service: Power your app's data links with named entity recognition and disambiguation Academic Knowledge: Tap into the wealth of academic content in the Microsoft Academic Graph using the Academic Knowledge API QnA Maker: Distill information into an easy-to-navigate FAQ for bot services Customer Decision Service: Create custom experiences with adaptive, contextual decision-making Search: Add Bing Search APIs to your apps and harness the ability to comb billions of webpages, images, videos, and news with a single API call Bing Autosuggest: Give your app intelligent autosuggest options for searches Bing News Search: Search for news and get comprehensive results Bing Web Search: Get enhanced search details from billions of web documents Bing Entity Search: Enrich your experiences by identifying and augmenting entity information from the web Bing Image Search: Search for images and get comprehensive results Bing Video Search: Search for videos and get comprehensive results Bing Custom Search: Create tailored site search or vertical search experiences for topics you care about Labs: Cognitive Services Labs are early preview limited availability leading innovation APIs and SDKs that allow developers to start experimenting with Microsoft’s latest and greatest Cognitive Services. Project Prague: SDK to incorporate gesture-based controls into your apps. Quickly define and implement customized hand gestures, creating a more natural user experience. Limited private preview availability at launch. Project Cuzco: API to help developers find events associated with Wikipedia entities. Begin with a Wikipedia entity, and receive a list of related events organized by time. Project Johannesburg: API to calculate route logistics for with deeper location intelligence to account for specific enterprise requirements. IE: weight, height length, hazardous materials, etc. Project Nanjing: API to calculate isochrones - time and distance-based recommendations for enterprise route optimization. Project Abu Dhabi: API to create distance matrices, enabling you to calculate a histogram of travel times, and serve as stepping stone for enterprise route optimization. Project Wollongong: API to help ‘score’ the attractiveness of a location, based on how many of a particular amenity are within a specific distance. Ex: restaurants, parks, transit stops. With Cognitive Services, developers can easily add intelligent features – such as emotion and sentiment detection, vision and speech recognition, knowledge, search and language understanding – into their applications. The collection will continuously improve, adding new APIs and updating existing ones. Cognitive Services includes: <click> Vision: From faces to feelings, allow apps to understand images and video <click> Speech: Hear and speak to users by filtering noise, identifying speakers, and understanding intent <click> Language: Process text and learn how to recognize what users want <click> Knowledge: Tap into rich knowledge amassed from the web, academia, or your own data <click> Search: Access billions of web pages, images, videos, and news with the power of Bing APIs Why choose these APIs? They work, and it’s easy. <click> Easy: The APIs are easy to implement because of the simple REST calls. There’s a common way to implement, and you can get started with all of them for free simply by going to one place, one website, www.microsoft.com/cognitive. Flexible: We’ve got a breadth of intelligence and knowledge APIs so developers will be able to find what intelligence feature they need. And, importantly, they all work on whatever language, framework, or platform developers choose. So, developers can integrate into their apps—iOS, Android, Windows—using their own tools they know and love. Tested: Tap into an ever-growing collection of powerful AI algorithms developed by experts. Developers can trust the quality and expertise build into each API by experts in their field from Microsoft’s Research organization, Bing, and Azure machine learning and these capabilities are used across many Microsoft first party products such as Cortana, Bing and Skype. Transition: When it comes to real-world applications for Cognitive Services, the sky is the limit! Let’s look at some examples. <click>