Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Mobile & Cognitive Services | Harnessing the Power of IoT – Xamarin Experience London 2017


Published on

By Jorge Borralho – Microsoft Tech Lead @Xpand IT

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Mobile & Cognitive Services | Harnessing the Power of IoT – Xamarin Experience London 2017

  1. 1. Jorge Borralho Mobile & Cognitive Services
  2. 2. We need smarter apps but the processing power of our smartphones is not enough for everything
  3. 3. We need smarter services that can be easily integrated with our apps
  4. 4. What are Cognitive Services? The Cognitive Services are a collection of complimentary ‘services’ (29 and counting) to assist developers with the processing of natural input sources such as speech, images and emotion. The Cognitive Services are organized into five pillars: Speech, Language, Vision, Knowledge and Search. By combining different Cognitive Services developers can build intelligent apps with powerful algorithms using just a few lines of code.
  5. 5. Cognitive Services Vision SpeechLanguage KnowledgeSearch Computer Vision Content Moderator Emotion Face Video Indexer LUIS Spell Check Linguistic Analysis Text Analytics Translator Web Language Model Speech Speaker Recognition Translator Custom Academic Entity Linking Exploration QnA Maker Recommendations Autosuggest Image News Video Web Entity Custom Vision Service Custom Language Understanding Custom Speech Custom Search Custom Decision Service
  6. 6. Language Language Understanding Intelligent Service PREVIEW Teach your apps to understand commands from your users Bing Spell Check API Detecting and correcting spelling mistakes in your app Web Language Model API PREVIEW Use the power of predictive language models trained on web-scale data Translator Speech API Easily conduct real-time speech translation with a simple REST API call Text Analytics API PREVIEW Easily evaluate sentiment and topics to understand what users want Translator Text API Easily conduct machine translation with a simple REST API call Linguistic Analysis API PREVIEW Simplify complex language concepts and parse text with the Linguistic Analysis API.
  7. 7. Computer Vision APIs Computer Vision API Distil actionable information from images Content moderator Automated image, text and video moderation Video Indexer PREVIEW Unlock video insights Face API Detect, analyse, organise and tag faces in photos Emotion API PREVIEW Personalise user experiences with emotion recognition Custom Vision Service PREVIEW Easily customise your own state-of-the-art computer vision models for your unique use case.
  8. 8. Speech APIs Custom Speech Service PREVIEW Overcome speech recognition barriers like speaking style, background noise and vocabulary Bing Speech API Convert speech to text and back again to understand user intent Speaker Recognition API PREVIEW Use speech to identify and authenticate individual speakers
  9. 9. Search APIs Bing Autosuggest API Give your app intelligent autosuggest options for searches Bing News Search API Search for news and get comprehensive results Bing Web Search API Get enhanced search details from billions of web documents Bing Entity Search API Enrich your app by infusing knowledge search into existing content Bing Image Search API Search for images and get comprehensive results Bing Video Search API Search for videos and get comprehensive results Bing Custom Search PREVIEW An easy-to-use, advert-free, commercial-grade search tool that lets you deliver the results you want.
  10. 10. Knowledge APIs Recommendations API PREVIEW Predict and recommend items that your customers want Knowledge Exploration Service PREVIEW Enable interactive search experiences over structured data via natural language inputs Entity Linking Intelligence Service API PREVIEW Power your app’s data links with named entity recognition and disambiguation. Academic Knowledge API PREVIEW Tap into the wealth of academic content in the Microsoft Academic Graph QnA Maker API PREVIEW Distil information into conversational, easy-to- navigate answers. Custom Decision Service PREVIEW A cloud-based, contextual decision-making API that sharpens with experience
  11. 11. Computer Vision Description: { "Tags": [ "water", "mountain", "outdoor", "surfing", "blue", "wave", "riding", "man", "ocean", "top", …], "Captions": [ { "Text": "a man riding a wave on a surfboard in the water", "Confidence": 0.437939942 } ] } Tags: [ { "Name": "sky", "Confidence": 0.9965861 }, { "Name": "water", "Confidence": 0.9905544 }, { "Name": "mountain", "Confidence": 0.945281565 }, { "Name": "outdoor", "Confidence": 0.9212041 }, { "Name": "water sport", "Confidence": 0.783058345 }, { "Name": "sport", "Confidence": 0.6974723 }, { "Name": "blue", "Confidence": 0.684881747 }, { "Name": "wave", "Confidence": 0.6782906 } ]
  12. 12. Linguistic Analysis Programmatically analyze text and get: • Language • Key phrase extraction • Sentiment analysis • Topic detection examples: I had a wonderful experience! The rooms were wonderful and the staff was helpful. I had a terrible time at the hotel. The staff was rude and the food was awful.
  13. 13. Video Indexing
  14. 14. Language Understanding Intelligent Service - LUIS Build custom language models: LUIS provides simple tools that enable you to build your own language models (intents/entities) which allow any application/bot to understand your commands and act accordingly Define Concepts (intents and entities) Provide Examples (uterances) Deploy Active Learning
  15. 15. Language Understanding Intelligent Service - LUIS Example: “News about flight delays” { "entities":[{"entity":flight_delays", "type":"Topic"}], "intents":[ {"intent":"FindNews", “score":0.99853384}, {"intent":"None", "score":0.07289317}, {"intent":"ReadNews", "score":0.0167122427}, {"intent":"ShareNews", "score":1.0919299E-06} ] }
  16. 16. Language Understanding Intelligent Service - LUIS Powering bots: • Microsoft bot framework • Bots built with Node.js or C# • Dialogs to model a conversation • Many types of dialogs
  17. 17. Roll your own with REST APIs Simple to add: just a few lines of code required Integrate into the language and platform of your choice Breadth of offerings helps you find the right API for your app Built by experts in their field from Microsoft Research, Bing, and Azure Machine Learning Quality documentation, sample code, and community support Easy Flexible Tested GET A KEY Cognitive Services
  18. 18. Q&A Cognitive Services
  19. 19. Jorge Borralho Harnessing the power of IoT
  20. 20. The Internet of Things isn’t a technology revolution…
  21. 21. …IoT is a business revolution, enabled by technology
  22. 22. Capitalizing on IoT Connect your things Turn data into insights Transform your business
  23. 23. The IoT Customer Journey Objective Increase efficiency by reducing costs and eliminating bottlenecks Organizational readiness This is a great place to start if you have: • An identified business process you want to gain visibility into • An understanding of the kind of data you need in order to support your business objective • Devices capable of providing the type of data required
  24. 24. Objective Drive innovation and increase revenue using your existing assets Organizational readiness This is a great place to start if you have : • The ability to interpret incoming data from devices • An understanding of the impacted business groups and processes • A plan for using data to drive business process improvements The IoT Customer Journey
  25. 25. The IoT Customer Journey Objective Create new lines of business and new business models Organizational readiness This is a great place to start if you have : • A strong baseline of historical device data (e.g. usage trends, repair records) • Visibility into real-time operational status • Predictive capability from advanced analytics • Stakeholder commitment to support change efforts
  26. 26. IoT projects are complex Time-consuming to stand up Hard to scaleIncompatible with current infrastructure Difficult to maintain cohesive security
  27. 27. Gain defense in depth with Azure IoT Suite security
  28. 28. Obtain flexibility with an open approach
  29. 29. Devices Azure IoT Services Back end systems and processes IoT Hub Stream Analytics Azure MLAzure Functions Stream Analytics Azure MLAzure Functions *Preview
  30. 30. Q&A Harnessing the power of IoT