Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Jennifer Marsman, Principal Software Development Engineer, Microsoft at MLconf ATL 2017

Game of Drones: Using IoT, Machine Learning, Drones, and Networking to Solve World Hunger
Drones are increasingly used in various commercial and consumer scenarios – from agriculture drones (providing farmers with crop and irrigation patterns) to consumer drones (that follow you around as you engage in action sports), to drone racing. Drones are outfitted with a large number of sensors (cameras, accelerometers, gyros, etc.), and can continuously stream these signals in real time for analysis.

This talk introduces the landscape of the various drone technologies that are currently available, and shows you how to acquire and analyze the real-time signals from the drones to design intelligent applications in an IoT pipeline. We will demonstrate how to leverage machine learning models that perform real-time facial detection along with predictions of age, gender, emotion, and object recognition using the signals acquired from the drones. You will walk away understanding the basics of how to develop applications that utilize and visualize these real-time insights.

This talk includes fun with drones, how to tackle the problem of world hunger, and some Game of Thrones silliness. It is targeted at data scientists, students, researchers, and IT professionals who have an interest in building intelligent applications using drones and machine learning. It will be a fun and exciting exploration as we demonstrate a drone with the power of recognizing faces, ages, genders, emotions, and objects. You will learn how to leverage these same machine learning models to imbue intelligence into drones or other applications.

  • Login to see the comments

Jennifer Marsman, Principal Software Development Engineer, Microsoft at MLconf ATL 2017

  1. 1. Azure ML Analytics Cameras, Drones, sensors Weather Data (Rain, Wind, Pollen) Recommendations (daily best practices) FarmBeats Gateway FarmBeats Gateway
  2. 2. 2000 acres in upstate NY Horticulture, animal farming, dairy, etc. 100 acres of farm in Carnation, WA Rented out to small farmers - Primarily horticulture
  3. 3. Camera
  4. 4. Fusing it all together Spatio-temporal view of the farm Sensors & UAVs Yield estimation Precision Irrigation Pest Infection Fertilizer application …Ag Services
  5. 5. probabilistic graphical models that embed Gaussian processes is used to extrapolate from the sensor data points to the full territory. This model seeks to balance spatial and visual smoothness: are measuring physical properties of the soil and the environment, the sensor readings for locations that are nearby should be similar (spatial smoothness). • Areas that look similar should have similar sensor values. For example, a recently irrigated area has more moisture and hence looks darker (visual smoothness).
  6. 6. Dragon Drones
  7. 7. Dragon Drones
  8. 8. Drone Auto-pilot App Point selection mode Features  Simple user interface  Supports two flight modes o point-selection mode o area-selection mode  Estimates flight time  Stores path history and telemetry  Transfers video from drone to IoT edge  Supports DJI Phantom2 and Inspire 1 Area selection mode
  9. 9. Drone Auto-pilot App
  10. 10. Dragon Drones
  11. 11. Area Coverage Algorithm Drone stops and goes or reduce its speed at each waypoint. 8 waypoints 16 waypoints 1 2 4 3 5 6 8 7 1 2 4 3 5 6 8 7 10 11 9 12 13 14 15 16
  12. 12.  Omni-directional  Directional ◦ Front – low drag ◦ Side – high drag Front side Front side Omni-directional drones can have directionality by attaching a paper here
  13. 13. Yaw Control Algorithm  Given a path and wind information, the drone changes its yaw in order to save energy. Exploiting wind for acceleration Exploiting wind for deceleration Avoiding wind for acceleration Exploiting drag for deceleration Exploiting wind for acceleration Exploiting drag for deceleration Avoid drag Avoid drag
  14. 14. Dragon Drones
  15. 15. Snapshot of available wireless technologies Range Throughput *Not to scale, Very rough estimates HaLow 3G, LTE NB-IoT WiMax, TVWS Cat0
  16. 16. dbm Frequency -60 -100 “White spaces” 470 MHz 700 MHz What are TV White Spaces? 25 0 MHz 7000 MHz TV ISM (Wi- Fi) 698470 2400 51802500 5845 are Unoccupied TV ChannelsWhite Spaces 54-88 170-216 Wireless Mic
  17. 17. Mawingu Project Collaboration between Kenya’s Ministry of Information and Communications, Microsoft, and Mawingu Networks. Pilot delivering low-cost wireless broadband access to previously unserved locations near Nanyuki. To maximize coverage and bandwidth, while keeping costs to a minimum, the Mawingu network relies on a combination of “license- exempt” wireless technologies, including Wi-Fi and TVWS. First deployment of solar-powered based stations together with TVWS to deliver high-speed Internet access to areas currently lacking even basic electricity. Base stations allow end-users to charge devices.
  18. 18. Roll your own with REST APIs Simple to add: just a few lines of code required Integrate into the language and platform of your choice Breadth of offerings helps you find the right API for your app Built by experts in their field from Microsoft Research, Bing, and Azure Machine Learning Quality documentation, sample code, and community support Easy Flexible Tested GET A KEY
  19. 19. Microsoft Cognitive Services Give your apps a human side
  20. 20. Computer Vision API Distill actionable information from images Video API Analyze, edit, and process videos within your app Face API Detect, identify, analyze, organize, and tag faces in photos Emotion API Personalize experiences with emotion recognition Vision
  21. 21. How do I use them? POST,Tags &subscription-key=<Your subscription key> { "tags": [ { "name": "outdoor", "score": 0.976 }, { "name": "bird", "score": 0.95 } ], "description": { "tags": [ "outdoor", "bird" ], "captions": [ { "text": "partridge in a pear tree", "confidence": 0.96 } ] } }
  22. 22.
  23. 23.
  24. 24. Demo Microsoft Cognitive Services: Facial verification: Can the “dragon” drone find Khaleesi?
  25. 25. us/research/project/farmbeats-iot-agriculture/ technology/21707242-unused-tv-spectrum-and- drones-could-help-make-smart-farms-reality-tv- dinners
  26. 26.