Understanding Pre-Built AI: AI for Every Developer
•Download as PPTX, PDF•
2 likes•435 views
Standard talk track I use for sharing our vision for making AI accessible and discoverable to all developers.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Understanding Pre-Built AI: AI for Every Developer
Similar to Understanding Pre-Built AI: AI for Every Developer (20)
More from AI Leadership Institute
More from AI Leadership Institute (8)
Recently uploaded
Recently uploaded (20)
Understanding Pre-Built AI: AI for Every Developer
- 1. What Developers Need to Know About Building Intelligent Apps Noelle LaCharite Principal Program Manager, Developer Experience Cognitive Services and Azure ML @noellelacharite Connect with me on LinkedIn
- 2. “What is success? To laugh often and much; to win the respect of intelligent people and the affection of children; to earn the appreciation of honest critics and endure the betrayal of false friends; to appreciate the beauty; to find the best in others; to leave the world a bit better, whether by a healthy child, a garden patch Or a redeemed social condition; to know even one life has breathed easier because you have lived. This is to have succeeded!” ― Ralph Waldo Emerson
- 6. Let Cognitive Services be your Super Power
- 8. Cognitive Services Infuse your apps, websites, and bots with human-like intelligence Speech Language Knowledge SearchVision Speech to text (Speech Transcription) Speaker ID and authentication Customized speech Transcription (complex word, noisy environment) Real-time speech translation Spell checking Contextual language understanding – customized intent analysis Sentiment analysis, key phrase detection text translation up to 60+ languages Customized decision making with reinforcement learning Q&A service and bot training Automatic search suggestions Comprehensive news, image, and video results Entity information augmentation Tailored and customized search experiences Image Tagging, Thumbnails OCR, Handwriting recognition Customized image recognition Face detection Video insights - people detection, visual search, keyframe detection, object recognition Image and video moderation Labs Entity Identification and Linking Academic Content Index and search Knowledge Content index and search Augmented entity information Tailored search experiences Text Moderation Emotion recognition Text to speech
- 9. IT Automation Customer Intelligence & Recommendations Research & Discovery Market Intelligence Predictive Maintenance & Resource Optimization Fraud Analysis & Investigation Customer-facing Chatbots Retail Intelligence Edge Surveillance & Security Smart Networking Business Process Automation & Internal bots Intelligent Search Supply Chain Management Diagnosis & Treatment Systems Adaptive Learning Regulatory Intelligence
- 10. Knowledge Cognitive Search Custom Speech Understanding Custom Voice Personality Azure Bot Service Conversational AI
- 15. Cognitive Services COMPUTER VISION | CUSTOM VISION | VIDEO INDEXER | SEARCH Microsoft Azure AZURE ML | IOT EDGE | AZURE STACK | AZURE STORAGE
- 20. Cognitive Services COMPUTER VISION | OCR | LUIS | TEXT ANALYTICS | TRANSLATOR | SPELL CHECK | SPEECH | SEARCH Microsoft Azure POWER BI | APP INSIGHTS | DYNAMICS 365 | AZURE SEARCH
- 25. • • • • Cognitive Services LUIS | TEXT ANALYTICS | TRANSLATOR | SPELL CHECK | SPEECH | WEB / NEWS SEARCH |IMAGE SEARCH | COMPUTER VISION | EMOTION API | FACE API Microsoft Azure AZURE ML | POWER BI | APP INSIGHTS
- 28. Cognitive Services LUIS | TEXT ANALYTICS | TRANSLATOR | SPELL CHECK | SPEECH | CUSTOM DECISION SERVICE | QnA MAKER Microsoft Azure AZURE SEARCH | POWER BI | APP INSIGHTS n
- 31. Cognitive Services COMPUTER VISION | VIDEO INDEXER | OCR | BING IMAGE SEARCH | WEB SEARCH | BING VISUAL SEARCH Microsoft Azure AZURE ML | POWER BI | APP INSIGHTS n
Editor's Notes
- We have been working over the last year to transform our Cognitive Services into enterprise-grade solutions that can solve our leading customers’ and partners’ business and IT challenges. And that’s exactly the evolution we would like to share with you today. Cognitive Services are intelligent, business-ready, and easy to deploy services that can be customized for your specific business scenario. The APIs can be combined, like building blocks, to infuse your business with AI, and rapidly accelerate time to market. We are going to demonstrate how you can get started today, using our advanced artificial intelligence technology to solve the complex, real-world business and IT challenges many of you face. With Microsoft Cognitive Services, you can build the way you want to build and customize AI for your business.
- We have been working over the last year to transform our Cognitive Services into enterprise-grade solutions that can solve our leading customers’ and partners’ business and IT challenges. And that’s exactly the evolution we would like to share with you today. Cognitive Services are intelligent, business-ready, and easy to deploy services that can be customized for your specific business scenario. The APIs can be combined, like building blocks, to infuse your business with AI, and rapidly accelerate time to market. We are going to demonstrate how you can get started today, using our advanced artificial intelligence technology to solve the complex, real-world business and IT challenges many of you face. With Microsoft Cognitive Services, you can build the way you want to build and customize AI for your business.
- Why choose these APIs? They work, and it’s easy. Easy: The APIs are easy to implement because of the simple REST calls. Being REST APIs, there’s a common way to implement and you can get started with all of them for free simply by going to one place, one website, www.microsoft.com/cognitive. (You don’t have to hunt around to different places.) Flexible: We’ve got a breadth of intelligence and knowledge APIs so developers will be able to find what intelligence feature they need; and importantly, they all work on whatever language, framework, or platform developers choose. So, devs can integrated into their apps—iOS, Android, Windows—using their own tools they know and love (such as python or node.js, etc.). Tested: Tap into an ever-growing collection of powerful AI algorithms developed by experts. Developers can trust the quality and expertise build into each API by experts in their field from Microsoft’s Research organization, Bing, and Azure machine learning and these capabilities are used across many Microsoft first party products such as Cortana, Bing and Skype.
- We have been working over the last year to transform our Cognitive Services into enterprise-grade solutions that can solve our leading customers’ and partners’ business and IT challenges. And that’s exactly the evolution we would like to share with you today. Cognitive Services are intelligent, business-ready, and easy to deploy services that can be customized for your specific business scenario. The APIs can be combined, like building blocks, to infuse your business with AI, and rapidly accelerate time to market. We are going to demonstrate how you can get started today, using our advanced artificial intelligence technology to solve the complex, real-world business and IT challenges many of you face. With Microsoft Cognitive Services, you can build the way you want to build and customize AI for your business.
- Vision: Image-processing algorithms to smartly identify, caption and moderate your pictures Computer vision: Distill actionable information from images Content Moderator: Automatically moderate potentially offensive images, text and videos Customer Vision Service: Train a web service to recognize specific content in images Face: Identify human faces and emotions in images Video indexer: Easily extract insights from your videos to enrich your applications Speech: Convert spoken audio into text, use voice for verification, or add speaker recognition to your app Bing Speech: Convert speech to text and text to speech Speaker Recognition: Use speech to identify and authenticate individual speakers Custom Speech Service: Overcome speech recognition barriers like speaking style, background noise, and vocabulary Translator Speech: Easily conduct real-time speech translation on your app Language: Enable your apps to process natural language with pre-built scripts, evaluate sentiment and learn how to recognize what users want Bing Spell Check: Add spell checking functionality to your app Language Understanding (LUIS): Add language understanding intelligence to your apps with minimal effort Linguistic Analysis: Easily parse complex text with language analysis Text Analytics: Easily evaluate sentiment, language, and key phrases to understand what users want Translator Text: Easily conduct machine translation for 60+ languages Knowledge: Map complex information and data in order to solve tasks such as intelligent recommendations and semantic search Knowledge Exploration Service: Enable interactive search experiences over structured data via natural language inputs Entity Linking Service: Power your app's data links with named entity recognition and disambiguation Academic Knowledge: Tap into the wealth of academic content in the Microsoft Academic Graph using the Academic Knowledge API QnA Maker: Distill information into an easy-to-navigate FAQ for bot services Customer Decision Service: Create custom experiences with adaptive, contextual decision-making Search: Add Bing Search APIs to your apps and harness the ability to comb billions of webpages, images, videos, and news with a single API call Bing Autosuggest: Give your app intelligent autosuggest options for searches Bing News Search: Search for news and get comprehensive results Bing Web Search: Get enhanced search details from billions of web documents Bing Entity Search: Enrich your experiences by identifying and augmenting entity information from the web Bing Image Search: Search for images and get comprehensive results Bing Video Search: Search for videos and get comprehensive results Bing Custom Search: Create tailored site search or vertical search experiences for topics you care about Labs: Cognitive Services Labs are early preview limited availability leading innovation APIs and SDKs that allow developers to start experimenting with Microsoft’s latest and greatest Cognitive Services. Project Prague: SDK to incorporate gesture-based controls into your apps. Quickly define and implement customized hand gestures, creating a more natural user experience. Limited private preview availability at launch. Project Cuzco: API to help developers find events associated with Wikipedia entities. Begin with a Wikipedia entity, and receive a list of related events organized by time. Project Johannesburg: API to calculate route logistics for with deeper location intelligence to account for specific enterprise requirements. IE: weight, height length, hazardous materials, etc. Project Nanjing: API to calculate isochrones - time and distance-based recommendations for enterprise route optimization. Project Abu Dhabi: API to create distance matrices, enabling you to calculate a histogram of travel times, and serve as stepping stone for enterprise route optimization. Project Wollongong: API to help ‘score’ the attractiveness of a location, based on how many of a particular amenity are within a specific distance. Ex: restaurants, parks, transit stops. With Cognitive Services, developers can easily add intelligent features – such as emotion and sentiment detection, vision and speech recognition, knowledge, search and language understanding – into their applications. The collection will continuously improve, adding new APIs and updating existing ones. Cognitive Services includes: <click> Vision: From faces to feelings, allow apps to understand images and video <click> Speech: Hear and speak to users by filtering noise, identifying speakers, and understanding intent <click> Language: Process text and learn how to recognize what users want <click> Knowledge: Tap into rich knowledge amassed from the web, academia, or your own data <click> Search: Access billions of web pages, images, videos, and news with the power of Bing APIs Why choose these APIs? They work, and it’s easy. <click> Easy: The APIs are easy to implement because of the simple REST calls. There’s a common way to implement, and you can get started with all of them for free simply by going to one place, one website, www.microsoft.com/cognitive. Flexible: We’ve got a breadth of intelligence and knowledge APIs so developers will be able to find what intelligence feature they need. And, importantly, they all work on whatever language, framework, or platform developers choose. So, developers can integrate into their apps—iOS, Android, Windows—using their own tools they know and love. Tested: Tap into an ever-growing collection of powerful AI algorithms developed by experts. Developers can trust the quality and expertise build into each API by experts in their field from Microsoft’s Research organization, Bing, and Azure machine learning and these capabilities are used across many Microsoft first party products such as Cortana, Bing and Skype. Transition: When it comes to real-world applications for Cognitive Services, the sky is the limit! Let’s look at some examples. <click>
- Today’s presentation will cover two main things: Message: We are going to show you two things: what scenarios you and other top customers are building, and how Microsoft Cognitive services can be used to enable these solutions One , this is what our many of you and our additional top customers are doing today. We have spoken with the leading organizations who have implemented Cognitive Services in the last year to understand the use cases they are enabling. We have further spoken with our top Azure customers to understand the AI initiatives they are focused on in the near term. And, we have a network of early access program partners looking to use our MSFT Cog Svs for a number of initiatives. The scenarios that we have prioritized represent the most frequent requests from our top customers in our ecosystem. Second, these solutions can be built using the tools and services we have in our portfolio today. We have a portfolio of intelligent, powerful, enterprise-ready, Cognitive Services. The scenarios we are focused on today will harness the existing capabilities of Cognitive Services, enabling us to easily build any of this solutions this year. In addition, the scenarios discussed today will combine Cognitive Services with other Azure services, further uniting our ecosystem and creating a unified experience for you – our customers. This session will focus on how you can customize Cognitive Services for your unique business scenario
- Let’s take an example. Think of the JFK demo that you just saw. All of that rich information extracted from the JFK Files data and indexed and searchable. What if we could now add custom speech recognition to it that can recognize the unique words in that data? What if you could have a conversational bot on JFK Files that you can chat with? And wouldn’t it be great if it could answer back in a custom voice of your own choosing? Let me now invite Noelle and Wilson from my team to show off such a bot.
- Do you need an API that gives you actionable information about images used in your app? The Computer Vision API gives you the tools to understand the contents of any image. Create tags identifying objects, beings, or actions present in the image, and then craft coherent sentences to describe it. Whether you want to execute better image search, or you want to create an assistive app for the visually impaired, the Computer Vision API helps get the job done.
- Detect human faces and compare similar ones, organize people into groups according to visual similarity, and identify previously tagged people in images.
- Image moderation Enhance your ability to detect potentially offensive or unwanted images through machine-learning based classifiers, custom blacklists, and optical character recognition (OCR). Text moderation Helps you detect potential profanity in more than 100 languages and match text against your custom lists automatically. Content Moderator also checks for possible personally identifiable information (PII). Video moderation Enable the scoring of possible adult content in videos. Video moderation is currently deployed in preview on Azure Media Services. Human review tool Humans can effectively augment machine learning models in situations where the prediction confidence warrants assistance or when decisions must be tempered with a real world context. Enjoy visibility, flexibility and control with a human review tool that supervises the results of your algorithms.
- Predictive maintenance involves a number of specific components, each enabled by a core set of Cognitive Services and Azure Services: Anomaly detection: in process manufacturing, identify when something is abnormal. In discrete manufacturing, identify a potential issue or defect – early in the production cycle. We can train our vision services to understand what “normal” processing looks like, and then immediately flag anything that is “abnormal” Root cause analysis: use AI to triangulate and find source of issue or defect, to keep it from spreading or getting worse. Cognitive Services, with human-like intelligence, can track and trace issue history and aggregate the string of logic and factors that may have influenced the issue along the way. Expert finder / similarity analysis: Cognitive services search can be customized to your internal business operations. It will catalog all of your previous maintenance notes, part malfunctions, etc., and provide you with the capability to quickly identify similar maintenance issues and streamline resolution. Cognitive Services also uses fuzzy matching – so in the case where a name was typed incorrectly, or there are spelling mistakes or errors, Cognitive Services will still be able to link together similar issues. Further, the internal search function can also carry information such as the former expert technicians responsible for the maintenance issue, or an internal IP in order to create efficiency when maintenance does occur Resolutions / Auto correction: based on aggregated data on maintenance issues, and previous or historical fixes, Cognitive Services can intelligently recommend best resolution. Further, if you want to integrate the AI solution with your control systems, Cognitive Services can be programmed to automatically fix the issue once it has determined the fix. For example, if the system identified that the issue was due to rising temperatures, it could integrate with the control system to change the temperature and fix the issue. All of this can be displayed in real time, in a dashboard, to provide employees with insight into the status of their assets. Predictive maintenance is a solution that requires processing of high volumes of data, with low latency. Sometimes, connectivity can be a challenge as well. Our predictive maintenance solution will leverage Azure Stack, IoT edge, Azure ML and storage. The solution requires real-time sensory inputs and historical pattern analysis to proactively address issues and defects and prevent part failures. These services combined will create an intelligent system that can better (more efficiently, and quickly) identify warning signs of failure, and streamline maintenance resolution process. Cognitive Services is then an intelligence layer on top of the predictive capabilities that will determine best course of action based on all of the available data
- Convert audio to text, understand intent, and convert text back to speech for natural responsiveness. Speech Intent Recognition can, in addition to returning recognized text from audio inputs, also return structured information about the speech to apps that parse the intent of the speaker and drive further actions by the app. Models trained by LUIS service are used to generate the intent.
- There are a number of specific components involved within BPA. I’m sure each of you can think of areas of your business where these solutions would be applicable. The first area of BPA is automated data capture. Using our intelligent language tools: LUIS, translator, text analytics, spell check, as well as our vision services like OCR and computer vision, we will enable developers to easily extract information for important documents. The value proposition for this is simple: Cognitive Services can offload manual labor and reduce human error in the collection of data via unstructured files, such as images, emails, resumes, social media posts, and even handwritten doctors notes. Not only is the data collection process tedious and non-systematic, but our customers have expressed concern about errors in information collection and processing. AI can play a significant role in parsing out and identify entities within large documents and across data sources, and speed the process of data labeling and cleaning for departments such as Human Resources, Legal, Finance and Customer Support. A second core element of BPA is automated form filing and document processing. Again using our natural language processing tools, such as LUIS and Text Analytics, and vision, Cognitive Services can analyze and detect a variety of forms for type, completeness, errors and to identify the important entities within documents or correspondence. Through this intelligent analysis, Cognitive Services can automatically process documents and reduce the manual steps in between. The solution is used by developers to automate filing and or large document processing with greater speed and precision. [For example, the solution could automate the filling and filing of a return merchandise form, summarize a large document, identify medical form and incorrect information, or process / file loan applications.] With intelligent data capture, and automated form filing, there is the opportunity to automate entire business processes, or workflows. When stringed together, Cognitive Services for BPA can automate entire workflows with efficiency and accuracy, freeing your employees to focus on the tasks that will deliver greater impact to your business. Here, you have the opportunity to customize our tools to extract information specific to your use case and determine the decision logic for the next process steps. All of this can be done with simple tuning of the parameters and our model will continue to learn and improve with your data. The automated workflow scenarios using Cognitive Services will bring intelligence without the need for a human touchpoint. [For example, tasks such as opening new accounts, getting insurance quotes, or finding new talent to hire, can become automated and completed in a fraction of the time.] And last, we’ll talk through BPA’s ability to analyze claims and automate processing. Cognitive services can bring new capabilities to claims processing using intelligent search, automated data capture and standards / compliance tracking. Data is often manually entered between systems due to system silos, contributing to high costs and potential of errors. BPA can remove manual touchpoints and automate the lower-level claims, enabling adjusters to focus on the areas where they will return the highest impact. Further, Cognitive Services can act as an augmentation tool, providing adjusters with information in real time on claim status and synthesizing all available data to assist recommendations and resolutions. In general, Business process automation is powered by a combination of Cognitive Services and Azure Services, though as developers you have the power to integrate additional services into the solution as you see fit for your business. Language tools, such as LUIS for language understanding, and text analytics, and translator Vision for information extraction, and character recognition Speech for recording, reporting and interpreting customer requests Search (both internal and web search), across mediums (images, free form text, etc.) Visualization and data insights will be displayed using a variety of Azure services such as Power BI, app insights and dynamics 365
- Understanding and analyzing unstructured text is an increasingly popular field and includes a wide spectrum of problems such as sentiment analysis, key phrase extraction, topic modeling/extraction, aspect extraction and more.
- . Try now the Speech Translator demo App on : https://github.com/MicrosoftTranslator/SpeechTranslator . Try now the Document Translator demo, translating (batches of) Word or pdf documents while preserving the formatting : https://www.microsoft.com/en-us/translator/doctranslator.aspx Language detection The Translator Text API automatically detects the language of the text that’s sent before translating it. If your application simply needs to know what language the text is in, you can also call the API to detect the language of any text string. Translation Add speech translation, for any of the 9 supported languages, and text translation, for any of the 60 supported languages, to your app. Grow your potential user base by localizing your app and its content with clear translations. Custom translation system Build a custom translation system, using as little as 1,000 parallel sentences or start out simply by providing a dictionary of company specific words. Collaborative Translation Framework (CTF) Improve translations by creating a specific user group that provides suggestions to improve the translations. Users suggest translations and designated approvers either approve or deny changes. These updated translations can then be used for the company’s specific Hub to further improve its custom system.
- These are a number of AI/ML services that enable Customer feedback analytics Cognitive services, combined with azure services, can transform a business’ ability to use customer data and effectively gauge customer satisfaction, product feedback, predict churn and understand customer emotions. Customer feedback analytics combines a variety of tools such as: Language tools such as LUIS, text analytics, translator and spell check can be combined to interpret a customer’s natural way of communication to understand sentiment or true intent behind feedback Further, speech tools can be used to transcribe support calls and process feedback in real time, whether it’s to intelligently route a caller, or identify whether or not customer had a good experience Vision tools too can be used to determine customer sentiment or product feedback – whether it’s through reviewing social media posts or even detecting customer’s facial emotion while shopping or interacting Search can be used to find content related to product, service or customer on web to identify new avenues of feedback data. Azure used to create a dashboard for organizations to display insights in real time – power BI, app insights, dynamics 365
- QnA Maker is primarily meant to provide a FAQ data source which you can query from your Bot/Application. Although developers will find this useful, content owners will especially benefit from this tool. QnA Maker is a completely no-code way of managing the content that powers your Bot/Application.
- The Bing Search API adds intelligent search to your app, combing hundreds of billions of webpages, images, videos, and news to provide relevant results with no ad requirements. The results can be automatically customized to your users' locations or markets, increasing relevancy by staying local.
- Leverage Bing Custom Search to build a search that fits your needs with powerful ranking, a global-scale search index, and document processing. Setup is easy and fast. The core technology works in three steps: it identifies on-topic sites, applies the Bing ranker, and delivers relevant search results while allowing you to adjust the parameters at any time.
- Access to strong documentation, sample code and community resources is critical for developers to be able to understand and become users of Cognitive Services. Customize these links based on your own resources or use the ones listed here.
- Need to update social logos.