Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Build Stuff 2021 Azure Video Indexer Advanced data extraction from video and audio content.pptx
Similar to Build Stuff 2021 Azure Video Indexer Advanced data extraction from video and audio content.pptx (20)
More from icebeam7
More from icebeam7 (17)
Recently uploaded
Recently uploaded (20)
Build Stuff 2021 Azure Video Indexer Advanced data extraction from video and audio content.pptx
- 1. Lithuania, 2021 Azure Video Indexer: Advanced data extraction from video and audio content Luis Beltrán
- 2. Speaker Bio Luis Beltrán • Researcher at Tomás Bata University in Zlín, Czech Republic • Lecturer at Tecnológico Nacional de México en Celaya • Xamarin, Azure, Artificial Intelligence @darkicebeam luis@luisbeltran.mx
- 3. Agenda • Introduction • Azure Video Analyzer for Media o Portal o Developer Portal • Demo
- 4. Introduction
- 5. Azure Cognitive Services Computer Vision Face API Custom Vision Text-to-Speech Speech-to-Text Speech Translator Speaker Recognition Language Understanding Text Translator Text Analytics QnA Maker Anomaly Detector Content Moderator Personalizer microsoft.com/cognitive Vision Speech Language Decision
- 6. Azure Applied AI Services • Go live with AI solutions in days—not months. • Generate tangible value for your organization quickly with AI services for common business processes. • Azure Applied AI Services bring together Azure Cognitive Services, task-specific AI, and business logic to offer you turnkey AI services for common business processes.
- 8. Imagine if you could… • Automatically create metadata (spoken text, written text, faces, places, objects etc) for any video file. • As a call center operator, scan for frustrated customers or count time spent on hold. • Collaboratively upload & edit videos in real-time, search to find key moments, and speed up time to publish new content • Enable your viewers to search by Name, Search by Keyword, Search by Shot Location, key objects • Distribute playable clips of live video content internally or externally without long download or encode times • Create automated summaries or highlight reels of their video content based on scene detection, specific people, and motion within the video • Translate your content into other languages for global
- 9. Video AI is the key to solve these challenges • Improve Content Discoverability • Increase Content Value • Personalize the Viewing Experience • Uncover Hidden Content Insights • Reduce Manual Labor • Increase Revenue / Drive Viewing o Targeted Advertising o Predictive modeling & recommendations
- 10. Azure Video Analyzer for Media
- 11. Azure Video Analyzer for Media • Azure Video Analyzer for Media is a service that extracts valuable insights. • It uses machine learning models that can be further customized and trained. • The video insights include face identification, text recognition, object labels, scene segmentations… • Additional insights are extracted from audio, such as transcription and emotion detection. • You can use these results to improve search, extract clips, create thumbnails, and more, thus enhancing user engagement. Video Analyzer Analyze the visual and audio channels in a video, and index its content.
- 14. Subscribe to the API • In order to use Video Indexer, you need to create a subscription. • You can sign up for a trial: o Up to 600 minutes of free indexing using the Video Indexer Portal o Up to 2400 free minutes using the API. • If you sign up for a trial, you will have a subscription created automatically. videoindexer.ai
- 15. Azure Video Analyzer for Media Portal • Upload videos o From URL o From File o Most common media formats are supported (MOV, WMV, MPG, AVI, MP4…) • Features are indexed
- 17. Identify people in a video
- 18. Get a full transcript, translate it to another language
- 19. Search content
- 20. Get your Account Id
- 21. Azure Video Analyzer for Media Developer Portal api-portal.videoindexer.ai
- 22. 3 simple steps
- 23. Get your Subscription Key
- 24. Exploring the Azure Video Analyzer Developer Portal (I)
- 25. Sending requests using the API console, get the code
- 26. Exploring the Azure Video Analyzer Developer Portal (II)
- 28. Demo
- 30. Takeaways
- 32. Call to Action • Video Analyzer for Media Portal https://www.videoindexer.ai/ • Video Analyzer for Media Developer Portal https://api-portal.videoindexer.ai/ • Microsoft Learn: Create computer vision solutions with Azure Cognitive Services – Analyze Video https://docs.microsoft.com/en-us/learn/modules/analyze-video/
- 33. Q & A
- 34. Thank you for your attention Luis Beltrán about.me/luis-beltran
Editor's Notes
- Your goal is to help travelers search and filter to find videos uploaded by others. Proper indexing of media and extracting insights is a challenge. The Video Indexer can extract insights from posted videos to help build a stunning platform with easy to implement features.
- Azure Cognitive Services is a suite of services and APIs backed by machine learning that enables developers to incorporate intelligent features such as facial recognition in photos and videos, sentiment analysis in text, and language understanding into their applications.
- Video Indexer Its purpose is to transform raw video content into content that is searchable, discoverable, and more engaging to the user. Want to generate a video transcript, index words spoken in the video or written on a whiteboard, or create a list of keywords from topics discussed in the video? Video Indexer can do all this and more. It can even find individuals in the video, and sometimes tell who they are. Video Indexer is both a service and an API. The service is accessed through a Web portal. It allows you to upload videos and examine the information generated from them. The Video Indexer API is a REST API that does everything the portal does, and also allows you to access the information that is generated when videos are indexed.
- Video Indexer provides regarding each video that it indexes, and this information is available not only in the portal, but through the Video Indexer API.
- Open the Video Indexer portal in your browser and select Get Started. Then select Sign in with your work account (AAD) to sign in with a work or school account, or Sign in with your personal Microsoft account to use a personal Microsoft account. Answer Yes if prompted to let this app access your info. Select the Upload button. Then select enter a file url to upload a video from a URL. Paste the following URL into the URL field, and enter "Overview of the Microsoft AI School" as the video name. Then select Upload to begin the upload. https://topcs.blob.core.windows.net/public/Machine-Learning-in-IoT-solutions_high.mp4 Machine Learning in IoT Solutions When indexing is complete, you will receive an e-mail notification for each video with a link to the video and a short description of what was found in it, such as people, topics, and keywords. Wait for all three videos to finish indexing, and then proceed to the next exercise.
- In Video Indexer, insights are aggregated views of the knowledge extracted from a video, such as faces, keywords, and sentiment. For example, you can see the faces of people appearing in the video, as well as time ranges and percentages for each face shown. Video Indexer cross-references the faces that it finds against a database of thousands of famous people and automatically identifies them. You can see for yourself by opening the "Microsoft in Education" video in the portal. Microsoft CEO Satya Nadella appears in that video, and Video Indexer recognizes him. Video Indexer automatically generates video transcripts based on its built-in speech and speaker recognition services. It even provides facilities for editing the information that it generated so you can correct errors in transcripts, put names to faces that weren't recognized, and more. The "Insights" tab shows people featured in the video, keywords generated from the video, topics identified in the video, brands featured in the video, and even emotions found in the video. You can select Play next for any of these items and cycle through the corresponding points in the video. In this example, Video indexer found two people in the video. It was unable to identify them because they don't appear in its database of famous people. However, you can lend a helping hand by identifying them yourself. Enable editing by selecting the Edit icon in the upper-right corner. Then select the pencil icon next to "Unknown #1" and enter "Sonya Koptyev" as the person's name. Finish up by pressing Enter to save the change. Repeat this step for the "Unknown #2" in the video. This person's name is "Seth Juarez". Want to see a full transcript of the video? Select Timeline at the top of the page. Video Indexer uses a deep neural network (DNN) to aid in converting speech to text, but such conversions are rarely perfect. Here, too, you can help out by editing words and phrases that weren't converted properly. To demonstrate, make sure you're still in editing mode and change "High Amsonia captive." to "Hi, I'm Sonya Koptyev.” Select Insights, and then search for the word "intelligence." This time, the results are conceptual topics that include the search term. Video Indexer has the ability to translate transcripts into a variety of languages, including German, Dutch, Spanish, French, Czech, Korean, and Japanese. To demonstrate, select Timeline again, select the world icon, and select a language other than English from the drop-down list.
- The "Insights" tab shows people featured in the video, keywords generated from the video, topics identified in the video, brands featured in the video, and even emotions found in the video. You can select Play next for any of these items and cycle through the corresponding points in the video. In this example, Video indexer found two people in the video. It was unable to identify them because they don't appear in its database of famous people. However, you can lend a helping hand by identifying them yourself. Enable editing by selecting the Edit icon in the upper-right corner. Then select the pencil icon next to "Unknown #1" and enter "Sonya Koptyev" as the person's name. Finish up by pressing Enter to save the change. Repeat this step for the "Unknown #2" in the video. This person's name is "Seth Juarez". Want to see a full transcript of the video? Select Timeline at the top of the page. Video Indexer uses a deep neural network (DNN) to aid in converting speech to text, but such conversions are rarely perfect. Here, too, you can help out by editing words and phrases that weren't converted properly. To demonstrate, make sure you're still in editing mode and change "High Amsonia captive." to "Hi, I'm Sonya Koptyev.” Select Insights, and then search for the word "intelligence." This time, the results are conceptual topics that include the search term. Video Indexer has the ability to translate transcripts into a variety of languages, including German, Dutch, Spanish, French, Czech, Korean, and Japanese. To demonstrate, select Timeline again, select the world icon, and select a language other than English from the drop-down list.
- Want to see a full transcript of the video? Select Timeline at the top of the page. Video Indexer uses a deep neural network (DNN) to aid in converting speech to text, but such conversions are rarely perfect. Here, too, you can help out by editing words and phrases that weren't converted properly. To demonstrate, make sure you're still in editing mode and change "High Amsonia captive." to "Hi, I'm Sonya Koptyev.” Select Insights, and then search for the word "intelligence." This time, the results are conceptual topics that include the search term. Video Indexer has the ability to translate transcripts into a variety of languages, including German, Dutch, Spanish, French, Czech, Korean, and Japanese. To demonstrate, select Timeline again, select the world icon, and select a language other than English from the drop-down list.
- Once a video is indexed, you can search its contents. Type "suggestion" into the search box at the top of the page and press Enter. Confirm that the search results include four instances in which the word "suggestion" was found in the video. Select Insights, and then search for the word "intelligence." This time, the results are conceptual topics that include the search term.
- The Video Indexer portal provides a window into the videos that you index and lets you see a wealth of information extracted from them. But the real power of Video Indexer lies in the Video Indexer API, which lets you submit videos for indexing programmatically and access the results using a REST API. In Exercise 4, you will build an app that uses this API to expose content in the videos you indexed in Exercise 1. But to call the API, you must first subscribe to it and obtain an API key that is transmitted in each request. In this exercise, you will create a Video Indexer API subscription and retrieve the API key created for it.
- Open the Video Indexer API portal in your browser and select SIGN IN in the top-right corner. Sign in with your Microsoft account — the same one you used to sign in to the Video Indexer portal. Answer Yes if prompted to let this app access your info. Select Products, and then select Authorization. Select the Subscribe button. The subscription will be created. Select Products again, and then select Authorization. You will now see a list of the subscriptions you have, so select the Product Authorization Subscription. Select the Subscribe button. The subscription will be created. Select Products again, and then select Authorization. You will now see a list of the subscriptions you have, so select the Product Authorization Subscription. Select the Show button next to the Primary Key. Copy the API key to the clipboard, and then select Hide to hide it again. Now that you have an API key, you can write apps that call the Video Indexer API. The API key travels in an HTTP header in each request. Without a valid API key, the Video Indexer API fails requests placed to it. It is the API's way of ensuring that the caller is authorized.
- Open the Video Indexer API portal in your browser and select SIGN IN in the top-right corner. Sign in with your Microsoft account — the same one you used to sign in to the Video Indexer portal. Answer Yes if prompted to let this app access your info. Select Products, and then select Authorization. Select the Subscribe button. The subscription will be created. Select Products again, and then select Authorization. You will now see a list of the subscriptions you have, so select the Product Authorization Subscription. Select the Subscribe button. The subscription will be created. Select Products again, and then select Authorization. You will now see a list of the subscriptions you have, so select the Product Authorization Subscription. Select the Show button next to the Primary Key. Copy the API key to the clipboard, and then select Hide to hide it again. Now that you have an API key, you can write apps that call the Video Indexer API. The API key travels in an HTTP header in each request. Without a valid API key, the Video Indexer API fails requests placed to it. It is the API's way of ensuring that the caller is authorized.