In this talk we explore ways to leverage new and improved features of the Google Cloud Platform for your applications - based on an Android demo using Cloud Vision API to detect sleeping students in class. We also look at a number of patterns for mobile backends with Firebase and AppEngine and discuss machine learning and Cloud Speech API.
Talk given at a Google Developer Group meetup in Aarhus by Kasper Løvborg Jensen on April 28, 2016.
08448380779 Call Girls In Civil Lines Women Seeking Men
What's Cooking in the Cloud?
1. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
What’s Cooking in the Cloud?
Expanding the Menu for App Developers
Kasper Løvborg Jensen
2. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
Chief Wizard, Founder
Leafcastle Labs
Associate Professor
Aarhus University School of Engineering
Organizer
Google Developer Group Aarhus
@kasperljkasper@leafcastlelabs.com
3. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
GCP NEXT ‘16 Conference
• Machine Learning for all!
• Cloud Vision for all!
• Cloud Speech API for all (soon)!
• StackDriver (works with AWS)!
• …lots more!
4. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
Outline
• Why cloud?
• Mobile backends
– Experiences
– Patterns
• New powerful Tools/APIs
– Machine Learning
– Speech API
– Cloud Vision
• Demo: TA-800
6. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
Why I am Excited about Cloud / GCP
• Backends for mobile/wearable/web apps
– Low/No Maintenance
– Scalability
– Reliability
– Security
• APIs / Services for applications
– Integrate/synch with existing Services
– Add powerful features/functionality through APIs
• Great for startups
– Business models and focus on creating value
– Cost scales with growth
– Speed of iteration and experiments
15. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
Mobile Backends
“Building a backend service for a mobile app is
similar to building a web-based service, with
some additional requirements:
• Limit on-device data storage
• Synchronize data across multiple devices
• Handle the offline case gracefully
• Send notifications and messages
• Minimize battery drain”
hAps://cloud.google.com/soluEons/mobile/mobile-app-backend-services
21. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
Machine Learning
• Machine Learning
• Vision API
• Speech API
• Translate API
Source: Google
28. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
Speech API
• Solves really hard problems!
– Natural language recognition
– Open vocabulary
– Noisy environments
– 80+ Different languages
hAps://cloud.google.com/speech/
29. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
Speech Horror Story ca. 2005
• Distributed speech recognition in
app as part of my PhD
– On PDA running Windows Mobile
with feature extraction through
custom DLL (c/c++)
– Live stream features to Sphinx IV
DSR on Tomcat server that could
only handle one connection/request
at a time due to memory
requirements
– Months of building/training speech,
grammar and acoustic models
31. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
Live or Batch?
• Batch
– Upload audio sample(s) and get
results when recognition
complete
• Live
– Stream audio to server and
receive partial results as they
become available
hAps://cloud.google.com/speech/
33. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
Cloud Vision API
• Solves really hard problems!
– Extract and label objects in arbitrary image
without context
– Can be anything
– Find details like facial features, tilt and
“emotions“
– Lighting conditions
– Blurry pictures
– Vast amount of cameras and settings
– …
34. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
Case Study
• Problem: Students falling asleep during
lectures
• Solution: Need person some sort of
contraption to detect students falling
asleep and then wake them up
“Never send a man to
do a machine’s job”
-Gandalf
35. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
TA-800
• IF we wanted to build a Terminator as a
teaching assistant
– Running Android
– Powered by Google Cloud / Skynet
• Detect and “terminate” inactive students
during lectures
– High precision Nerf gatling gun
40. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
TA-800 Hack/Demo
• Image processing based on Cloud Vision API
example app
https://github.com/GoogleCloudPlatform/
cloud-vision/tree/master/android/CloudVision
• Firebase integration based on Firebase chat
example
https://github.com/firebase/AndroidChat
• Total dev time:~6 hours (50%+ time on UI)
• Will be on GitHub:
42. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
Using Cloud Vision API
• Specify request (JSON)
– Image
• Content/source
– Features
• Type
• Max results
• Handle result (JSON)
• Android: use CloudVision API
– com.google.apis:google-api-services-
vision:v1-rev2-1.21.0!
44. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
Rich Response Data as JSON
• Labels
– Things/situations in the image
• Faces
– Faces, location in picture
– Happy, angry, sad, surprised
– Face “landmarks”: eyes, ears, …
– Tilt, …
• Logos
– Known brands and logos
• Landmarks
– “Eiffel Tower”, “AROS museum”, …
– Also including e.g. location metadata
• Texts
– OCR, finding text in the image
45. Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
Your turn!
GCP:
https://cloud.google.com/
List of (most) products:
https://cloud.google.com/products/
Playground:
https://cloud-playground.appspot.com/playground/
Launcher:
https://cloud.google.com/launcher/
Firebase:
https://www.firebase.com/