Azure Cognitive Services Bring AI to your applications in 3 steps.pptx
•Download as PPTX, PDF•
0 likes•219 views
Slides for my talk about Azure Cognitive Services at Back 2 School event.
Recommended
More Related Content
Similar to Azure Cognitive Services Bring AI to your applications in 3 steps.pptx
Similar to Azure Cognitive Services Bring AI to your applications in 3 steps.pptx (20)
More from Luis Beltran
More from Luis Beltran (20)
Recently uploaded
Recently uploaded (20)
Azure Cognitive Services Bring AI to your applications in 3 steps.pptx
- 1. Azure Cognitive Services: Bring AI to your applications in 3 steps! Carla Mamani MLSA Gold Luis Beltrán Microsoft MVP
- 2. Luis Beltrán • Researcher at Tomas Bata University in Zlín, Czech Republic • Lecturer at Tecnologico Nacional de Mexico en Celaya, Mexico • Microsoft MVP in AI and Developer Technologies
- 8. AI and ML in Azure Azure is a flexible platform for AI and ML • Quickly create, train, host, and deploy models from any Python environment with Azure services for data science and ML. • Or you can use pre-built AI solutions for vision, speech, language, and decision to embed cutting-edge experiences in your applications.
- 9. Carla Vanesa Mamani Chavez • Student Computer Science • U.M.S.A • La Paz - Bolivia • Microsoft Learn Student Ambassadors • Certified MIE (Microsoft Innovative Educator) • Leader Student Ambassador La Paz
- 12. How do they work?
- 13. Speech
- 14. Speech to Text example in Speech Studio speech.microsoft.co m
- 15. Language
- 16. Text Analytics API example
- 17. Vision
- 19. Decision
- 21. Step 1 – Create an Azure resource Option 1. Specific Cognitive Service resource from the list/Marketplace
- 22. Option 2. General Cognitive Services resource
- 24. Step 2 – Grab the keys, region, and endpoint
- 25. Step 3 – Incorporate them in your app
- 26. Demo #1: Knowledge Base QnA Maker Azure Bot Services Direct Line Speech To Text Text to speech
- 27. Step 1 : Create a new QnA Maker Service
- 28. Step 2:
- 29. Step 3:
- 30. Step 4:
- 31. Step 5:
- 32. Step 6
- 33. Step 7
- 34. Step 8
- 35. Step 9
- 36. Step 10
- 37. Step 11
- 38. Step 11
- 39. Demo #2: Text Analytics
- 40. Step 3 – Incorporate them in your app
- 41. AI: The building blocks for the future
- 42. Call for action Azure Cognitive Services microsoft.com/cognitive docs.microsoft.com/en-us/learn/
- 43. Azure Cognitive Services: Bring AI to your applications in 3 steps! Carla Mamani @carly.chavez1 Luis Beltrán @darkicebeam Thank you for your attention!
Editor's Notes
- Hello and welcome to today's Azure Back to School contribution. In this video you are going to learn about Azure Cognitive Services, a cloud technology that allows you to easily integrate Artificial Intelligence in your applications. We are Carla Vanesa Mamani Chavez and Luis Beltran and are delighted to participate in this event. Special thanks to Dwayne Natwick for organizing this great initiative. So, let's get started
- Before that, just a brief introduction about ourselves. I'm Luis, a Microsoft MVP in AI and Developer Technologies from Mexico. I work as a lecturer at Tecnologico Nacional de Mexico un Celaya although I'm currently pursuing my PhD at Tomas Bata University in Zlín, Czech Republic. I enjoy talking about cloud computing with Azure, mobile applications development with Xamarin, Artificial Intelligence with deep neural networks, and .NET as a platform in general. You can find me on twitter as @darkicebeam and here you also have my email in case you want to reach out and talk about any of these topics. My colleague Carla will introduce herself a bit later.
- Currently, we live in a world where everyone is connected through Internet. Millions of data are being generated every single minute and a relevant amount of that total is created by the human-machine interaction, such as when we interact with a conversational bot or when we talk to a digital assistant.
- In fact, artificial intelligence makes life easier for humans. We can give a command to a smart TV in order to change the channel or find our favorite program, so manual interaction with a remote control is no longer needed. Talking to a machine using natural language and making it to understand our intentions is an application of artificial intelligence. The machine learning model that is integrated in its software uses natural language processing in order to recognize words, translate them into text that will be analyzed to detect what we want and some context, such as the date, time or place. We don't need to use a special voice command to interact with this machine, it feels like talking to a friend
- We actually live surrounded by applications of artificial intelligence. For example, we can interact with digital screens that can track our sight and make decisions based on our eyes movement, or we can simplify things by using our fingers. As we mentioned earlier, there is also the conversational bot interaction, we can chat with this software element and get information, assistance in our requests or make things happen such as a flight reservation or a question answered from a knowledge base. The third example is an online store that displays a customized appearance or product recommendations that most probably we will be interested in, bringing an effective benefit to the company, since this products are not randomly selected but chosen from our previous shopping history, past searches and even what our friends have bought in the past. Yes, AI can extract relevant information from our relationships with other people.
- AI is not only the future but the present. We live in an Internet era where there is an explosion of data every minute. This data is generated mostly by mobile phones and applications such as WhatsApp, Twitter, Facebook and others. Internet of Things, Nanotechnology, Drones, and Blockchain, among others are current trends and technologies that draw a lot of attention by companies. They integrate the data into Mixed reality applications so they can add virtual elements to physical locations; Genetic Engineering is the process of using recombinant DNA (rDNA) technology to alter the genetic makeup of an organism, and this was possible thanks to lots of information stored from the past. And the future is bright, in a few months or years we will see emerging and promising technologies around us, maybe flying cars or smart cities will become a reality too! The key here is to learn about Artificial intelligence and use it to help us make decisions.
- So what do we need to analyze data. Here you can see an example of a deep neural network called Convolutional Neural Network that recognizes handwritten digits. In order to actually identify them, the network must be trained with data. So we can provide thousands of pictures of every digit and add a label that says what digit it is. This is supervised learning because we provide data with tags. Then this image is processed or converted into a matrix representation because, you know, a picture is an array of bytes that represent the color in hexadecimal format. Then we perform matrix multiplication, which in terms of deep learning, means that we apply a filter; the purpose is to detect low-level features, such as borders, edges, lines. As we continue to apply filters, we will be able to detect higher-level features such as shapes, circles, and other patterns that distinguish every digit from another. You can see actually that there are several matrices multiplications, the result of each set of operations is a convolutional layer and in-between these layers we also apply a small filter, known as pooling, which reduces the number of dimensions so the matrices will be smaller every time; at the end we will connect it to a fully-connected neural network that we will use to detect the right output, so after this process it will learn that this input is actually a two. Nobody said that artificial intelligence would be easy. But, if we need to create smart solutions for a better future, will there be a way to accelerate the AI learning or implementation?
- The answer is yes. Cloud leaders such as Microsoft provide workspaces to either create our own machine learning models with enough computation resources or use AI services already trained by Microsoft that will help us to integrate smart capabilities in our applications, even if we don't have any previous AI background or experience. And now my colleague Carla will introduce herself and will talk about Azure Cognitive Services.
- I belong to the global program MLSA.
- These are the APIs, SDKs and services available that aim to help developers to create intelligent applications without the use of "direct artificial intelligence" and without the need for data science skills and knowledge. Azure Cognitive Services enable developers to easily add cognitive features into their applications. Azure Cognitive Services enable applications to see, hear, speak, understand and even begin to reason. The Azure Cognitive Services portfolio can be divided into four main pillars: Vision, Speech, Language, and Decision.
- Easy: Because, we can simply send a HTTP Request to pre-trained models and services that have been created thanks to innovative research in AI. In some cases, we can even create customizable AI that adjust to our needs. All we need to use Cognitive Services is to get a key and use it in our app requests. Flexible: We can make ….. Also We can integrate ….. This platform has been tested by Microsoft itself and their partners. You can find great tutorials at Microsoft Docs and Microsoft Learn. And the team is very active at Stackoverflow and GitHub in case there are questions and issues about Cognitive Services.
- You prepare your data, for example in this case you see some text in Spanish. You include it in your HTTP request along with an key to authorize the request. Then, you send it to an endpoint and you'll get a response as a JSON string that you can process in your application to extract the results and present it to your users. Funciona con keys y endpoint xD FIN JAJJAJA
- Let's see the four categories of Cognitive Services. First, we have Speech. It's a set of services that allow you to Convert speech to text and text to natural sounding speech. Moreover, the Speech Translation capability allows you to perform real-time speech translation, easily integrated via service. Finally, we have Speaker Recognition. It's an API whose use is to determine the identity of an unknown speaker. The input audio of the unknown speaker is matched against a group of selected speakers and, if a match is found, the identity of the speaker is returned.
- Here is a screenshot of Speech Studio, a portal where you can test the Speech service in different ways. It is available at speech.microsoft.com You can use your microphone to talk and whatever you said will be detected. No lo pruebes, solo mencionalo
- The second group of Cognitive Services is Language. It's a set of APIs that allow applications to process natural language with precompiled scripts, evaluate sentiment and learn to recognize what users want. It is the base of conversational bots, particularly QnA Maker and LUIS service are useful for that.
- As we can see in the example. This service identifies the language, the key phrases of the text, the sentiment within the text. Such as whether it is positive or negative. Maybe neutral. Some key phrases are also detected.
- The third category is Vision. This enables our applications to see and describe what is happening in a picture or video. The Computer Vision API is able to analyze the content in multimedia files, so we can get a description that represents an scene, what objects are recognized in a photo or even identify celebrities such as actors or famous people. Face API detects and identifies people and facial attributes such as hair, emotion –if a person is happy, sad or angry-, even an estimated age is possible to get with this service; finally, Custom Vision allows us to easily create a custom image recognition or object detection model from our own images to fit our business needs.
- This is a Custom Vision example, you can see on the left that we provide pictures of different tools, then a model is trained and after that it we can test it with a new image and it will be able to recognize what object is present in the photo. Then we can integrate this custom model in the mobile, web, desktop applications that we develop.
- And the final category is Decision, which supports actions based on data analysis. Sometimes we want to detect outliers that might be present in our data, such as a variation in temperature or a fraud bank operation. With Anomaly Detector API we can identify this kind of issues or event predict them before they happen. Content Moderator detects potentially offensive or unwanted content, such as racist pictures or adult content that should not be allowed in an application. Finally Personalizer allows us to create rich experiences for each users, so we can use it for example to present relevant products to our customers, saving them some time and increasing the revenue probability of our company, since customers will have the products they are interested in very easily.
- You can see a Personalizer example in this picture. Depending on the day, time and weather, a relevant news is presented to the user; then this model is smart because it can learn if the user actually interacted with the information or not, so the next news that will be shown to the customer will continue to be relevant.
- We said in the title of this talk that all you needed to add AI in your apps is 3 steps and here they are. The first one is to go to the Azure Portal and create a Cognitive Service. There are two ways, you can either select one from the AI + Machine Learning category
- Or you can actually go to the Marketplace and select again, a specific service such as Personalizer, Translator, Custom Vision or the general Cognitive Services resource.
- The difference is that the keys from this general Cognitive Service resource can be used for several services including Vision, Language, and Speech. So just fill in the blanks with the required information, such as the Subscription, resource group, region, name and pricing tier. The individual services offer a free tier so you won't spend a single penny.
- Then you accept the terms and hit on create. After a few minutes the resource will be created. Access it and you'll enter a dashboard with several options. Pick Keys and Endpoint and copy the primary key, location, and endpoint, paste them into a notepad or somewhere else.
- Finally, use this information in your code. Of course, this depends on the technology, programming language and cognitive service. In this example you see a node js application; we add the azure/ai-text-analytics library; then in our code we can create a TextAnalyticsClient that uses the endpoint and keys for authentication of each operation. After that, we can call the specific methods such as analyze sentiment or detect language, which take an array of strings, then send a request to the Azure endpoint and we get the result so we can use it in the application. For this artificial intelligence part, only few lines of code are needed! Now Carlita will demonstrate a powerful scenario of Cognitive Services that involve several technologies.
- --- Let's get started --- We will be use two cognitive services. These are : QnA Maker and Speech. Also, We will be use azure bot services.
- First, Let's choose a knowledge base. This can be some FAQ questions and answers. Well, click on the create a knowledge base menu. In Step 1: we are going to create a new service. Click on the button. Create a QnA Service. And Enter the necessary data requested by the new service in Azure.
- Step 2: We return to the QnA maker portal. In the blue box. Let's select the tenant on which we are working. In the green box. Let's select the azure subscription we are working on. In the yellow box. Let's select the new service we created. Finally, let's select the language. ----------------------------------------------------------------------------- Volvemos al portal de QnA maker. En el cuadro rojo. Vamos a seleccionar el tenant en el cual estamos trabajando. En el cuadro verde. Vamos a seleccionar la suscripción de azure en el cual estamos trabajando. En el cuadro amarillo. Vamos a seleccionar el nuevo servicio que creamos. Finalmente, vamos a seleccionar el idioma.
- Step 3: Let's name it. For example: MLSA PROGRAM KB
- Step 4: Here- we will add our links or documents containing our question and answer pairs.
- Finally, Click on the button. Create your KB
- Step 6: Here we will be able to edit and train our knowledge base which contains the questions and answers recognized from our document or website. Once this procedure is done. We click on the button marked with yellow. Finally, click on publish. Paso 6: Aqui nosotros vamos a poder editar y entranar nuestra base de conocimientos. Una vez realizado este procedimeinto. Damos clic en la color marcado con amarillo. Finalmente, Damos click en publicar.
- Step 7: Congratulations. We have successfully created the knowledge base! Now, let's proceed to create the bot. We will do this with azure bot services. Click on the button Create Bot. Paso 7: Felicidades. Creamos el base de conocimiento con exito! Ahora, Vamos a proceder a a crear el bot. Esto lo haremos con azure bot services. Click en el boton.
- Once inside the azure portal. Here, we complete the missing data. Finally, we click on the create button. Once the resource is created. We will go to channels >> And we will get the secret keys to put it in our JS code. Una vez dentro del portal del azure. Aqui, nosotros completamos los datos faltantes. Por ultimo, damos clic en el boton de crear. Una vez que el recurso ya este creado. Vamos a dirigirnos a canales >> Y obtendremos las claves secretas para ponerla en nuestro codigo JS.
- Step 9 Finally. We will create the last cognitive service. This is speech. Also, we complete all the missing data. Paso 9 Por ultimo. Crearemos el ultimo servicio cognitivo. Este es el de speech. Tambien, completamos todos los datos faltantes.
- Finally. We get our keys and connection points. From the speech service Finalmente. Obtenemos nuestras claves y puntos de conexion. Del servicio de speech.
- First, we add the library. As we see in images marked in yellow color. Second, we add the key of cognitive services.
- Finally, we add the key of channel. This is key secret of direct line. Well, That's all. Now! Let's see the demonstration.
- That was impressive, thank you Carlita. Here we present a second demo, based on the steps that you saw earlier; in this case let's demonstrate it with a web application, the user will enter a message and we can get the language and sentiment.
- To conclude, AI is really important nowadays because it will help us to build the future. Mixed reality, quantum computing, blockchain. All are AI and ML applications, so let's be part of that future.
- If you want to learn more, visit microsoft.com/cognitive and check the documentation, tutorials, and more; you can also go to Microsoft Learn and find interesting learning paths which combine theory and practice; you don't even need an Azure subscription to test the Cognitive Services in applications, so we highly recommend you to visit the Learn portal
- That will be all from us, thank you Carlita. Here you have your contact details in case you want to learn more about Cognitive Services. Thank you again Dwayne Natwick for organizing the Azure Back to School initiative, enjoy the rest of the publications. See you next time