Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

What's Cooking in the Cloud?

In this talk we explore ways to leverage new and improved features of the Google Cloud Platform for your applications - based on an Android demo using Cloud Vision API to detect sleeping students in class. We also look at a number of patterns for mobile backends with Firebase and AppEngine and discuss machine learning and Cloud Speech API.

Talk given at a Google Developer Group meetup in Aarhus by Kasper Løvborg Jensen on April 28, 2016.

  • Login to see the comments

  • Be the first to like this

What's Cooking in the Cloud?

  1. 1. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University What’s Cooking in the Cloud? Expanding the Menu for App Developers Kasper Løvborg Jensen
  2. 2. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Chief Wizard, Founder Leafcastle Labs Associate Professor Aarhus University School of Engineering Organizer Google Developer Group Aarhus @kasperljkasper@leafcastlelabs.com
  3. 3. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University GCP NEXT ‘16 Conference •  Machine Learning for all! •  Cloud Vision for all! •  Cloud Speech API for all (soon)! •  StackDriver (works with AWS)! •  …lots more!
  4. 4. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Outline •  Why cloud? •  Mobile backends – Experiences – Patterns •  New powerful Tools/APIs – Machine Learning – Speech API – Cloud Vision •  Demo: TA-800
  5. 5. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University
  6. 6. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Why I am Excited about Cloud / GCP •  Backends for mobile/wearable/web apps –  Low/No Maintenance –  Scalability –  Reliability –  Security •  APIs / Services for applications –  Integrate/synch with existing Services –  Add powerful features/functionality through APIs •  Great for startups –  Business models and focus on creating value –  Cost scales with growth –  Speed of iteration and experiments
  7. 7. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University 7
  8. 8. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University In My Steps
  9. 9. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University
  10. 10. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University
  11. 11. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University GCP Scalability
  12. 12. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University App Backend Needs •  Storage •  Computing •  Network / serving •  Cloud Messaging •  Analytics •  Tools •  …
  13. 13. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Google Cloud Platform Source: GCP NEXT 2016 keynote
  14. 14. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Mobile Backends? Source: Google, Android, Apple, Chrome
  15. 15. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Mobile Backends “Building a backend service for a mobile app is similar to building a web-based service, with some additional requirements: •  Limit on-device data storage •  Synchronize data across multiple devices •  Handle the offline case gracefully •  Send notifications and messages •  Minimize battery drain” hAps://cloud.google.com/soluEons/mobile/mobile-app-backend-services
  16. 16. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Backend Patterns M.C. Escher
  17. 17. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University App Engine w. EndPoints Modified from: hAps://cloud.google.com/soluEons/mobile/mobile-app-backend-services
  18. 18. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Pure Firebase hAps://cloud.google.com/soluEons/mobile/mobile-app-backend-services
  19. 19. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Firebase + App Engine hAps://cloud.google.com/soluEons/mobile/mobile-app-backend-services
  20. 20. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Firebase + MVMs | Containers hAps://cloud.google.com/soluEons/mobile/mobile-app-backend-services
  21. 21. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Machine Learning •  Machine Learning •  Vision API •  Speech API •  Translate API Source: Google
  22. 22. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University FROM THIS
  23. 23. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University TO THIS
  24. 24. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Source: www.tensorflow.org
  25. 25. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University But wait, isn’t Machine Learning EVIL?
  26. 26. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University If by evil, you mean AWESOME!
  27. 27. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Cloud Speech APi
  28. 28. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Speech API •  Solves really hard problems! – Natural language recognition – Open vocabulary – Noisy environments – 80+ Different languages hAps://cloud.google.com/speech/
  29. 29. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Speech Horror Story ca. 2005 •  Distributed speech recognition in app as part of my PhD –  On PDA running Windows Mobile with feature extraction through custom DLL (c/c++) –  Live stream features to Sphinx IV DSR on Tomcat server that could only handle one connection/request at a time due to memory requirements –  Months of building/training speech, grammar and acoustic models
  30. 30. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University
  31. 31. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Live or Batch? •  Batch – Upload audio sample(s) and get results when recognition complete •  Live – Stream audio to server and receive partial results as they become available hAps://cloud.google.com/speech/
  32. 32. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Cloud Vision API
  33. 33. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Cloud Vision API •  Solves really hard problems! –  Extract and label objects in arbitrary image without context –  Can be anything –  Find details like facial features, tilt and “emotions“ –  Lighting conditions –  Blurry pictures –  Vast amount of cameras and settings –  …
  34. 34. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Case Study •  Problem: Students falling asleep during lectures •  Solution: Need person some sort of contraption to detect students falling asleep and then wake them up “Never send a man to do a machine’s job” -Gandalf
  35. 35. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University TA-800 •  IF we wanted to build a Terminator as a teaching assistant – Running Android – Powered by Google Cloud / Skynet •  Detect and “terminate” inactive students during lectures – High precision Nerf gatling gun
  36. 36. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University TA-800 Hardware •  Nexus 6 or 6p •  ATLAS prototype •  Nerf minigun
  37. 37. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University TA-800 Behavior Protocol While(true){ – Detect situation •  Identify features and classify place/situation – Acquire potential targets •  Detect if sleeping – TERMINATE Wake up targets – Report to master }
  38. 38. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Architecture Master TA-800
  39. 39. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Demo
  40. 40. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University TA-800 Hack/Demo •  Image processing based on Cloud Vision API example app https://github.com/GoogleCloudPlatform/ cloud-vision/tree/master/android/CloudVision •  Firebase integration based on Firebase chat example https://github.com/firebase/AndroidChat •  Total dev time:~6 hours (50%+ time on UI) •  Will be on GitHub:
  41. 41. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Cloud Vision API hAps://cloud.google.com/vision/
  42. 42. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Using Cloud Vision API •  Specify request (JSON) – Image •  Content/source – Features •  Type •  Max results •  Handle result (JSON) •  Android: use CloudVision API –  com.google.apis:google-api-services- vision:v1-rev2-1.21.0!
  43. 43. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Cloud Vision Detection Types DETECTION_TYPES = [! 'TYPE_UNSPECIFIED',! 'FACE_DETECTION',! 'LANDMARK_DETECTION',! 'LOGO_DETECTION',! 'LABEL_DETECTION',! 'TEXT_DETECTION',! 'SAFE_SEARCH_DETECTION',! ]!
  44. 44. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Rich Response Data as JSON •  Labels –  Things/situations in the image •  Faces –  Faces, location in picture –  Happy, angry, sad, surprised –  Face “landmarks”: eyes, ears, … –  Tilt, … •  Logos –  Known brands and logos •  Landmarks –  “Eiffel Tower”, “AROS museum”, … –  Also including e.g. location metadata •  Texts –  OCR, finding text in the image
  45. 45. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University Your turn! GCP: https://cloud.google.com/ List of (most) products: https://cloud.google.com/products/ Playground: https://cloud-playground.appspot.com/playground/ Launcher: https://cloud.google.com/launcher/ Firebase: https://www.firebase.com/
  46. 46. Kasper Løvborg Jensen Leafcastle Labs | Aarhus University THANK YOU! Questions? @kasperlj kasper@leafcastlelabs.com

×