Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Microsoft Cognitive Services at a Glance
Similar to Microsoft Cognitive Services at a Glance (20)
More from Marvin Heng
More from Marvin Heng (18)
Recently uploaded
Recently uploaded (20)
Microsoft Cognitive Services at a Glance
- 1. Microsoft Cognitive Services at a Glance Marvin Heng | @hmheng www.techconnect.io
- 4. Microsoft AI
- 5. Azure Cognitive Services From faces to feelings, allow your apps to understand images and video Hear and speak to your users by filtering noise, identifying speakers, and understanding intent Process text and learn how to recognize what users want Tap into rich knowledge amassed from the web, academia, or your own data Access billions of web pages, images, videos, and news with the power of Bing APIs
- 6. Why Azure Cognitive Services ?
- 7. Cognitive Services Emotion Computer Vision Face Video Indexer Form Recognizer Speech To Text Text To Speech Speech Translation Speaker Recognition Immersive Reader Language Understanding QnA Maker Text Analytics Translator Anomaly Detector Content Moderator Metrics Advisor Personalizer Bing Autosuggest Bing Custom Search Bing Entity Search Bing Image Search Bing News Search Bing Spell Check Bing Video Search Bing Visual Search Bing Web Search Custom Vision
- 8. Bing Search
- 9. Bing Search • Allow developers to integrate a search function to their apps that allows users to find webpages, images, news, locations, and more without advertisements • For knowledge mining
- 10. Bing Search Autosuggest Entity Search Custom Search Image Search News Search Video Search Visual Search Spell Check Local Business
- 11. Speech
- 12. Speech-to-Text • Speech-to-text service • Improves meeting efficiency by transcribing conversations in real-time • Help safeguard data with industry-leading security and compliance certifications. • Integrates with a variety of meeting conference solutions including Microsoft Teams and other third-party meeting software. • SDK is available.
- 13. Speech-to-Text
- 14. Speaker Recognition “who is speaking?”
- 15. Speaker Verification • Text-dependent verification means speakers need to choose the same passphrase to use during both enrollment and verification phases. • Text-independent verification means speakers can speak in everyday language in the enrollment and verification phrases.
- 16. Text-to-Speech • Convert text into human-like synthesized speech. • Offer 75+ standard in more than 45 languages and locales, and 5 neural voices • Tune voice output by easily adjusting rate, pitch, pronunciation, pauses, and more. • Speech synthesis • Asynchronous synthesis of long audio • Speech Synthesis Markup Language (SSML)
- 19. Speech Translation • Speech-to-text translation with recognition results. • Speech-to-speech translation. • Support for translation to multiple target languages. • Interim recognition and translation results.
- 21. Use case of Speech Service
- 22. Language
- 23. Language Understanding • Applies custom machine-learning intelligence to a user's conversational, natural language text to predict overall meaning, and pull out relevant, detailed information. • Often used in Chatbots or conversational bots.
- 25. Use Cases of Language Understanding • Automate capturing order for your application • Automate social media feedback & response • Conversation bot for your HR service, IT service, or other customer services. • Integrate with speech services for enabling your app to responding voice request from users.
- 27. QnA Maker • Natural Language Processing (NLP) service • Create a natural conversational layer over your data • Find the most appropriate answer for any input from your custom knowledge base (KB) of information
- 28. Development Lifecycle for QnA Maker
- 29. Immersive Reader • Embed text reading and comprehension capabilities into applications • Features: • Reading aloud, • translating languages, and • focusing attention through highlighting • No machine learning expertise is required.
- 30. Shows content in a minimal reading view
- 31. Displays pictures of commonly used words
- 32. Highlights nouns, verbs, adjectives, and adverbs
- 33. Reads your content out loud to you
- 34. Translates your content into another language
- 35. Breaks down words into syllables
- 36. What proves that Immersive Reader helps people with reading? • A 2017 study by RTI International showed that reading comprehension among groups of fourth-grade students improved an average of 10 percent. Read the research.
- 38. Translator • Support over 70+ languages & customizable. • Easy integration via REST API
- 39. Text Analytics • For text mining and text analysis • Understand the context in a conversation better • Sentiment analysis, opinion mining, key phrase extraction, language detection, and named entity recognition • More than 20 languages being supported (reference)
- 41. Text Analytics for Health • Extract information from unstructured English-language text in clinical documents such as: patient intake forms, doctor's notes, research papers and discharge summaries
- 42. Decision
- 43. Personalizer • Provide information about your users and content and receive the top action to show your users. • No need to clean and label data before using Personalizer. • Provide feedback to Personalizer when it is convenient to you. • View real-time analytics. • Use Personalizer as part of a larger data science effort to validate existing experiments.
- 44. Personalizer
- 45. Personalizer
- 46. Where can I use Personalizer? • Personalize what article is highlighted on a news website. • Display a personalized "recommended item" on a shopping website. • Suggest user interface elements such as filters to apply to a specific photo.
- 47. Where can I use Personalizer? • Send information (features) about your users and the content (actions) to personalize. Personalizer responds with the top action. • Send feedback to Personalizer about how well the ranking worked as a number typically between 0 and 1.
- 48. Vision
- 49. Computer Vision • Computer vision is an area of artificial intelligence (AI) in which software systems are designed to perceive the world visually, though cameras, images, and video. • Computer vision is one of the core areas of artificial intelligence (AI), and focuses on creating solutions that enable AI-enabled applications to "see" the world and make sense of it.
- 50. Use Cases of Computer Vision • Analyze an image and suggest an appropriate caption. • Suggest relevant tags that could be used to index an image. • Categorize an image. • Identify objects in an image. • Detect faces and people in an image. • Recognize celebrities and landmarks in an image. • Read text in an image.
- 51. What can CV tell us? • A black and white photo of a city • A black and white photo of a large city • A large white building in a city
- 52. Not only that! It tags too! • Tagging • Type of identified object • Bounding Box • Set of coordinates (Top, left, width and height)
- 54. Categorization in 86-category taxonomy abstract_ animal_horse building_street food_grilled others_ outdoor_road people_hand plant_tree text_menu abstract_net animal_panda dark_ food_pizza outdoor_ outdoor_sportsf ield people_many object_screen text_sign abstract_nonph oto building_ drink_ indoor_ outdoor_city outdoor_stoner ock people_portrait object_sculptur e trans_bicycle abstract_rect building_arch drink_can indoor_churchw indow outdoor_field outdoor_street people_show sky_cloud trans_bus abstract_shape building_brickw all dark_fire indoor_court outdoor_grass outdoor_water people_tattoo sky_sun trans_car abstract_texture building_church dark_fireworks indoor_doorwin dows outdoor_house outdoor_watersi de people_young people_swimmi ng trans_trainstatio n animal_ building_corner sky_object indoor_markets tore outdoor_mount ain people_ plant_ outdoor_pool animal_bird building_doorwi ndows food_ indoor_room outdoor_oceanb each people_baby plant_branch text_ animal_cat building_pillar food_bread indoor_venue outdoor_playgro und people_crowd plant_flower text_mag animal_dog building_stair food_fastfood dark_light outdoor_railway people_group plant_leaves text_map
- 56. Optical character recognition Faith CAN MOVE MOUNTAINS
- 57. Some Additional Capabilities • Detect image • Detect image color schemes • Generate thumbnails • Moderate content
- 59. Custom Vision • Azure Custom Vision is an image recognition service that lets you build, deploy, and improve your own image identifiers. • An image identifier applies labels (which represent classes or objects) to images, according to their visual characteristics. • The Custom Vision service uses a machine learning algorithm to analyze images.
- 60. What can Custom Vision do? • Classification • Object Detection • Export as standalone offline model for your app development.
- 62. Face Detection
- 63. Face Verification Verification result: The two faces belong to the same person. Confidence is 0.93468.
- 66. Video Indexer • Video Indexer provides ability to extract deep insights (with no need for data analysis or coding skills) using machine learning models based on multiple channels (voice, vocals, visual). • The service enables deep search, reduces operational costs, enables new monetization opportunities, and creates new user experiences on large archives of videos (with low entry barriers).
- 67. Video Indexer • Keywords extraction • Named entities extraction • Topic inference • Artifacts Sentiment analysis: Identifies positive, negative, and neutral sentiments from speech and visual text.
- 68. Video Indexer
- 69. Use Cases of Video Indexer • Deep search • Content creation • Accessibility. • Monetization • Content moderation • Recommendations
- 70. Video Indexer Face detection Celebrity identification Account-based face identification Visual text recognition Visual content moderation Labels identification Scene segmentation Shot detection Black frame detection Keyframe extraction Rolling credits Animated characters detection Editorial shot type detection Audio transcription Automatic language detection Multi-language speech identification and transcription Two channel processing Closed captioning Noise reduction Transcript customization (CRIS) Speaker enumeration Speaker statistics Textual content moderation Audio effects Emotion detection Translation
- 71. Form Recognizer • Extract text and data from business’s forms and documents. • Easily extract text and structure, with simple REST API • Pre-trained model: • Receipt • Business Card • Layouts • Custom Trained Model • Supports printed and handwritten forms, PDFs and images. • Container support
- 72. What can you do with Form Recognizer? • Automate written text > digital text conversion • Automate capturing receipt data • Automate converting business card into digital contacts
- 74. Sample Form Recognizer tool • Client library / REST API quickstart (all languages, multiple scenarios) • Web UI quickstarts • Train with labels - sample labeling tool • REST samples (GitHub) • Extract text, selection marks and table structure from documents • Extract layout data - Python
- 75. Sample Form Recognizer tool • Train custom models and extract form data • Train without labels - Python • Train with labels - Python • Extract data from invoices • Extract invoice data - Python • Extract data from sales receipts • Extract receipt data - Python • Extract data from business cards • Extract business card data - Python
- 77. Things you may be interested in. •AI Labs@hmheng (GitHub): https://github.com/hmheng/AILabs
- 78. Extras Add-on: • Cognitive Service • https://github.com/Azure-Samples/cognitive-services-python-sdk- samples/tree/deb4483ec1db33138e32ce472a775f3ed874fb80?WT.mc_id=AI-MVP-5003227 • Personalizer • https://azure.microsoft.com/en-us/services/cognitive-services/personalizer?WT.mc_id=AI-MVP-5003227 • https://personalizationdemo.azurewebsites.net • Speech-To-Text • https://azure.microsoft.com/en-us/services/cognitive-services/speech-to-text?WT.mc_id=AI-MVP-5003227 • Speech • https://azure.microsoft.com/en-us/services/cognitive-services/speech-to-text?WT.mc_id=AI-MVP-5003227 • https://docs.microsoft.com/en-us/azure/cognitive-services/speech-service/speech-synthesis- markup?WT.mc_id=AI-MVP-5003227
- 79. Extras Add-on: • Bing Search • https://docs.microsoft.com/en-us/azure/cognitive-services/bing-web-search?WT.mc_id=AI-MVP-5003227 • Personalizer • https://azure.microsoft.com/en-us/services/cognitive-services/personalizer?WT.mc_id=AI-MVP-5003227 • https://personalizationdemo.azurewebsites.net • Speech-To-Text • https://azure.microsoft.com/en-us/services/cognitive-services/speech-to-text?WT.mc_id=AI-MVP-5003227 • Speech • https://azure.microsoft.com/en-us/services/cognitive-services/speech-to-text?WT.mc_id=AI-MVP-5003227 • https://docs.microsoft.com/en-us/azure/cognitive-services/speech-service/speech-synthesis- markup?WT.mc_id=AI-MVP-5003227
- 80. Extras Add-on: • Form Recognizer: • https://docs.microsoft.com/en-us/azure/cognitive-services/form- recognizer/language-support?WT.mc_id=AI-MVP-5003227 • https://docs.microsoft.com/en-us/azure/cognitive-services/form- recognizer/quickstarts/label-tool?WT.mc_id=AI-MVP-5003227 • Form OCR Testing Tool • On Azure @ https://fott.azurewebsites.net/ • GitHub @ https://github.com/microsoft/OCR-Form- Tools/?ocid=AID2463683&WT.mc_id=AI-MVP-5003227
- 81. Thank you! Marvin Heng | @hmheng www.techconnect.io