PixelsCamp 2017 - Cloud Functions for Firebase and Machine Learning APIs

Cloud Functions
& Machine Learning APIs
@carlosazaustre
carlosazaustre.es

@carlosazaustre
Carlos Azaustre
Telecommunications Engineer
Front-End Developer
CTO & CoFounder Chefly
@carlosazaustre
https://carlosazaustre.es
youtube.com/carlosazaustre

@carlosazaustre
What is Firebase?
Google Platform to build apps fast without managing infrastructure

@carlosazaustre
Firebase Services
● Realtime NoSQL Database
● User Authentication
● Storage
● Hosting
● Push Notifications
● And Cloud Functions!
(and much more)

@carlosazaustre
Cloud Functions
Develop a Backend without
managing servers.
Create functions that are triggered
by Firebase products such as
changes to data in Database, new
user sign ups via Auth, file uploads
to Storage, etc...

@carlosazaustre
How works Cloud Functions?
Firebase Product Trigger
● Database
● Storage
● Auth
● HTTPS
● Hosting
● PubSub
Function Handler
● Code write in Node.js (v6)
● You can use GCP easily
● You can use Google APIs
● You can use NPM libraries
● You can use 3rd party APIs

@carlosazaustre
What are Machine Learning APIs?
Google Cloud Platform provides
several API’s to use in opur
projects such as
● Vision API
● Speech API
● Translation API
● Natural Language API
● Video Intelligence API
Or you can train your models
with TensorFlow and Machine
Learning Engine

@carlosazaustre
Practical Example

@carlosazaustre
What is Hot Dog or Not App?

@carlosazaustre
Frontend
Cloud Storage
Firebase
Cloud Functions
Firebase
Cloud Vision API
LabelDetection
Database
Firebase
App Architecture

@carlosazaustre
1. Take a Picture
const file = event.target.files[0]
const task = firebase.storage().ref(`/uploads/${file.name}`)
task.on('state_changed', snapshot => {
// Update progress indicator
}, error => {
// Show error details
}, () => {
// Upload completed!
this.imageUrl = task.snapshot.downloadURL
})
1
2

@carlosazaustre
2. Trigger a Cloud Function
exports.isHotdog = functions.storage.object()
.onChange(event => {
...
})
/uploads/hotdog.jpg
functions.storage.object().onChange isHotdog <Function>

@carlosazaustre
3. Take a File reference on Google Cloud Storage
exports.isHotdog = functions.storage.object()
.onChange(event => {
const object = event.data
const file = `gs://${object.bucket}/${object.name}`
})
/uploads/hotdog.jpg gs://bucketName/hotdog.jpg

@carlosazaustre
Cloud Vision API
● Label Detection
● Explicit Content Detection
● Logo Detection
● Landmark Detection
● Optical Character Recognition
● Face Detection
● Image Attributes
● Web Detection

@carlosazaustre
Cloud Vision API
"labelAnnotations": [
{ "mid": "/m/0hhrcvd", "description": "frankfurter würstchen", "score": 0.96459
{ "mid": "/m/01b9xk", "description": "hot dog", "score": 0.9400299 },
{ "mid": "/m/0kdzn", "description": "sausage", "score": 0.93496704 },
{ "mid": "/m/09370d", "description": "bockwurst", "score": 0.9349343 },
{ "mid": "/m/01sn4c", "description": "kielbasa", "score": 0.9139112 },
{ "mid": "/m/079yjd", "description": "knackwurst", "score": 0.8636358 },
{ "mid": "/m/03f7_0", "description": "bologna sausage", "score": 0.8028193 },
{ "mid": "/m/038rz", "description": "german food", "score": 0.76827973 },
{ "mid": "/m/04pfxj", "description": "cervelat", "score": 0.7612521 },
{ "mid": "/m/06l9l9", "description": "vienna sausage", "score": 0.7339414 },

@carlosazaustre
4. Use LabelDetection from GC Vision API
return vision.labelDetection({ source: { imageUri: file } })
.then(response => {
const labels = response[0].labelAnnotations
let isHotdog = false
labels.map(label => {
if(label.description === `hot dog`) {
isHotdog = true
}
})
return admin.database().ref(`/uploads/photo`)
.update({ isHotdog })
})
const file = `gs://${object.bucket}/${object.name}`

@carlosazaustre
5. Frontend App reacts to changes
//AppImage.vue
created() {
firebase.database()
.ref(`/upload`)
.on(value, snapshot => {
let isHotdog = snapshot.val().photo.isHotdog
this.$emit(`upload`, { isHotdog })
})
App.vue
AppHeader.vue AppImage.vue
emit (‘upload’, isHotdog)props: isHotdog

@carlosazaustre
https://github.com/carlosazaustre/pixelscamp-firebase-ml
https://hotdog-7e6aa.firebaseapp.com/

PixelsCamp 2017 - Cloud Functions for Firebase and Machine Learning APIs

Recommended

Recommended

More Related Content

Similar to PixelsCamp 2017 - Cloud Functions for Firebase and Machine Learning APIs

Similar to PixelsCamp 2017 - Cloud Functions for Firebase and Machine Learning APIs (20)

More from Carlos Azaustre

More from Carlos Azaustre (19)

Recently uploaded

Recently uploaded (20)

PixelsCamp 2017 - Cloud Functions for Firebase and Machine Learning APIs