Download to read offline
Uploaded byLuis Beltran
Cloud Skills Challenge.pptx
This document outlines steps for a cloud skills challenge involving mobile apps, AI services, and computer vision. It describes creating a Xamarin mobile app to record and upload videos to the Azure Video Indexer service to generate insights. A Custom Vision model is then trained for object detection and integrated back into the mobile app to provide customized insights using both services. The full process and technologies used include Xamarin, Azure Video Indexer, and Custom Vision.
More Related Content
![Cognitive Services APIs [Mihai Negrea]](https://cdn.slidesharecdn.com/ss_thumbnails/cognitiveservicesapismihainegrea-161102082437-thumbnail.jpg?width=640&height=640&fit=bounds)
More from Luis Beltran
![Getting Started with Apache Spark: Big Data Made Simple [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/apachesparkgettingstarted-260203175547-8361bcc3-thumbnail.jpg?width=640&height=640&fit=bounds)
Cloud Skills Challenge.pptx
- 1. Cloud Skills Challenge LuisBeltrán
- 2. Mobile apps: XamarinAI: Custom Vision object detection AI: Video Indexer insights from videos
- 3. Step 1. Recorda video using the mobile app Technologies Involved: - Xamarin
- 4. Step 2. Uploadthe video from the mobile app to Video Indexer service Technologies Involved: - Xamarin - Azure Video Indexer
- 5. Step 3. Getinsights from Video Indexer service Technologies Involved: - Azure Video Indexer
- 6. Step 4. Traina Custom Vision model for object detection
- 7. Technologies Involved: - Xamarin -Azure Video Indexer - Custom Vision Step 5. Get custom insights using the mobile app and Custom Vision
- 8. Thank you foryour attention! Special thanks to: Carla Mamani Anthony Quiranza Open-source project available on GitHub: https://github.com/icebeam7/FaceMaskVideoDetector
Editor's Notes
- #2 Hello everyone, my name is Luis I am a student from Czech Republic. I hope you’re enjoying the event and getting inspiration from the projects developed by students who participated at the Discover AI event. This is my entry for the Cloud Skills Challenge.
- #3 During the Cloud Skills Challenge I focused my learning experience in two areas: Mobile applications and artificial intelligence. I learned how to develop mobile apps for Android and iOS devices using Xamarin, how to extract insights from videos using Video Indexer and how to detect objects in images using Azure Custom Vision. It was fun and easy, and the best part is that you can create something with this knowledge, so let’s take a look.
- #4 I created a mobile application with several options. The first one allows me to record a video within the application. As you can see, I wear a facemask at some point of the video.
- #5 Second option is to upload this video to my video indexer workspace. Here you can see that the video is sent from the mobile app.
- #6 Once the video is uploaded and indexed, I can check the insights using the Video Indexer. But what if I want to know if the person is wearing a mask at specific times or not?
- #7 So that’s where Custom Vision comes in hand. Without any single line of code, I can create a Custom Vision object detection project, then upload images and draw a rectangle where the object of interest is present. After submitting several images, I can train a machine learning model that will detect if there’s a facemask in the picture and where it is located.
- #8 I can go back to my mobile app and integrate it there. The video indexer services extracts frames or images, then I send these pictures to Custom Vision service and it will tell me if there’s a facemask present or not.
- #9 So that’s it. This project was created in collaboration with two friends, Carla from Bolivia and Anthony from Ecuador. Special thanks to them, without their hard work this fun project wouldn’t have been created. The solution is available on GitHub by the way. Thanks to the organizers of this challenge, it was great to learn new things and I definitely had fun integrating all the parts. And of course, thanks to all attendees for being here.