SlideShare a Scribd company logo
1 of 61
Machine Intelligence
at Google Scale
Vision, Video, NLP, Speech, Dialogflow
TensorFlow, Cloud ML Engine, AutoML
Guillaume Laforge
Developer Advocate
Google Cloud
@glaforge
InfoQ.com: News & Community Site
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
google-ml-services
• Over 1,000,000 software developers, architects and CTOs read the site world-
wide every month
• 250,000 senior developers subscribe to our weekly newsletter
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• 2 dedicated podcast channels: The InfoQ Podcast, with a focus on
Architecture and The Engineering Culture Podcast, with a focus on building
• 96 deep dives on innovative topics packed as downloadable emags and
minibooks
• Over 40 new content items per week
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Presented at QCon London
www.qconlondon.com
Confidential & Proprietary
@glaforge
How did we escape the AI winter?
Ongoing research
on neural networks
More labeled
datasets to
learn from
More scalable
compute power to
train bigger models
Confidential & Proprietary
[dog]
Google Photos
@glaforge
Machine Learning is everywhere at Google
@glaforge
Machine Learning is everywhere at Google
@glaforge
The Machine Learning Spectrum
TensorFlow
Machine
Learning APIs
Academia,
R&D
Easy-to-use
APIs
ML as a Service,
for developers
Custom ML models,
For data scientists
@glaforge
The Machine Learning Spectrum
Academia,
R&D
Easy-to-use
APIs
ML as a Service,
for developers
Custom ML models,
for data scientists
Machine learning is learning from
examples and experience
Let’s try some
human-powered
image detection
@glaforge
How would we do this without ML?
CC-BY-SA 2.0 Wikimedia Commons
https://commons.wikimedia.org/wiki/File:Apple_in_lightbox.png
@glaforge
How would we do this without ML?
CC-BY-SA 2.0 Wikimedia Commons
https://commons.wikimedia.org/wiki/File:Apple_in_lightbox.png
@glaforge
CC-BY-SA 2.0 Wikimedia Commons
https://commons.wikimedia.org/wiki/File:Apple_in_lightbox.png
How would we do this without ML?
@glaforge
CC-BY 4.0 Wikimedia Commons https://commons.wikimedia.org/wiki/File:Mop_and_bucket.jpg
What about a dog and a mop? Easy, right?
@glaforge
Not so fast...
CC-BY-SA-2.5 Wikimedia Commons https://commons.wikimedia.org/wiki/File:Komondor_Westminster_Dog_Show_crop.jpg
CC-BY-2.0 Wikimedia Commons https://commons.wikimedia.org/wiki/File:2014_Westminster_Kennel_Club_Dog_Show_(12487315865).jpg
CC-BY-2.0 Petful https://www.flickr.com/photos/petsadviser-pix/16395099127
CC-BY-SA-2.0 Jeffrey Beall https://www.flickr.com/photos/denverjeffrey/6903790333
@glaforge
Machine Learning tools by Google at your disposal
Use your own data to train models Machine Learning as an API
Cloud
Vision API
Cloud
Translation APICloud
Natural Language
API
Cloud
Speech API
Cloud Machine
Learning Engine
TensorFlow
Cloud Video
Intelligence
Conversational
Interfaces
Cloud AutoML
Vision API
Complex image detection with
a simple REST request
Logo Detection
@glaforge
Face detection
"detectionConfidence" : 0.93568963,
"joyLikelihood" : "VERY_LIKELY",
"panAngle" : 4.150538,
"sorrowLikelihood" : "VERY_UNLIKELY",
"tiltAngle" : -19.377356,
"underExposedLikelihood" : "VERY_UNLIKELY",
"blurredLikelihood" : "VERY_UNLIKELY"
"faceAnnotations" : [
{
"headwearLikelihood" : "VERY_UNLIKELY",
"surpriseLikelihood" : "VERY_UNLIKELY",
rollAngle" : -4.6490049,
"angerLikelihood" : "VERY_UNLIKELY",
"landmarks" : [
{
"type" : "LEFT_EYE",
"position" : {
"x" : 691.97974,
"y" : 373.11096,
"z" : 0.000037421443
}
},
...
],
"boundingPoly" : {
"vertices" : [
{
"x" : 743,
"y" : 449
},
...
@glaforge
"landmarkAnnotations": [
{
"mid": "/m/0348s6",
"description": "Paris Hotel and Casino",
"score": 80,
"boundingPoly": {
"vertices": [
{
"x": 117,
"y": 479
},
...
]
},
"locations": [
{
"latLng": {
"latitude": 36.11221,
"longitude": -115.172596
}
}
]
}
]
Landmark detection
CC-BY-SA-3.0 Wikimedia Commons https://commons.wikimedia.org/wiki/File:Las-Vegas-Paris-Hotel-Eiffel-Tower-8307.jpg
@glaforge
Web annotations
{
"entityId": "/m/016ms7",
"score": 1.44038,
"description": "Ford Anglia"
}
{
"entityId": "/m/0gff2yr",
"score": 5.92256,
"description": "ArtScience Museum"
}
{
"entityId": "/m/0h898pd",
"score": 7.4162,
"description": "Harry Potter (Literary Series)"
}
CC-BY 2.0 Rev Stan: https://www.flickr.com/photos/revstan/6865880240
@glaforge
Web annotations
"fullMatchingImages": [{
"url":
"https://upload.wikimedia.org/wikipedia/commons/6/6d/Flying_Ford_Angl
ia_from_Harry_Potter_and_the_Chamber_of_Secrets_at_the_ArtScience_Mus
eum,_Singapore_-_20120608.jpg",
"score": 0.34952533
},
...
]
"partialMatchingImages": [{
"url":
"https://muckysock.files.wordpress.com/2012/06/img_2730.jpg",
"score": 0.887808
},
...
]
"pagesWithMatchingImages": [{
"url":
"https://www.haikudeck.com/harry-potter-and-chamber-of-secrets--educa
tion-presentation-SKZRnAO2UH",
"score": 53.212971
},
...
]
CC-BY 2.0 Rev Stan: https://www.flickr.com/photos/revstan/6865880240
@glaforge
CC-BY-SA-2.5 Wikimedia Commons https://commons.wikimedia.org/wiki/File:Komondor_Westminster_Dog_Show_crop.jpg
CC-BY-2.0 Wikimedia Commons https://commons.wikimedia.org/wiki/File:2014_Westminster_Kennel_Club_Dog_Show_(12487315865).jpg
CC-BY-2.0 Petful https://www.flickr.com/photos/petsadviser-pix/16395099127
CC-BY-SA-2.0 Jeffrey Beall https://www.flickr.com/photos/denverjeffrey/6903790333
In case you were wondering…
@glaforge
CC-BY-SA-2.5 Wikimedia Commons https://commons.wikimedia.org/wiki/File:Komondor_Westminster_Dog_Show_crop.jpg
CC-BY-2.0 Wikimedia Commons https://commons.wikimedia.org/wiki/File:2014_Westminster_Kennel_Club_Dog_Show_(12487315865).jpg
CC-BY-2.0 Petful https://www.flickr.com/photos/petsadviser-pix/16395099127
CC-BY-SA-2.0 Jeffrey Beall https://www.flickr.com/photos/denverjeffrey/6903790333
In case you were wondering…
@glaforge
In case you were wondering…
?
textile
fur
CC-BY-SA-2.5 Wikimedia Commons https://commons.wikimedia.org/wiki/File:Komondor_Westminster_Dog_Show_crop.jpg
CC-BY-2.0 Wikimedia Commons https://commons.wikimedia.org/wiki/File:2014_Westminster_Kennel_Club_Dog_Show_(12487315865).jpg
CC-BY-2.0 Petful https://www.flickr.com/photos/petsadviser-pix/16395099127
CC-BY-SA-2.0 Jeffrey Beall https://www.flickr.com/photos/denverjeffrey/6903790333
Natural Language API
Extract entities, sentiment,
and syntax from text
@glaforge
Extract entities
Joanne "Jo" Rowling, pen names J. K. Rowling and Robert Galbraith,
is a British novelist, screenwriter and film producer best known as the
author of the Harry Potter fantasy series
@glaforge
Extract entities
Joanne "Jo" Rowling, pen names J. K. Rowling and Robert Galbraith,
is a British novelist, screenwriter and film producer best known as the
author of the Harry Potter fantasy series
@glaforge
Extract entities
Joanne "Jo" Rowling, pen names J. K. Rowling and Robert Galbraith,
is a British novelist, screenwriter and film producer best known as the
author of the Harry Potter fantasy series
{
"name": "Joanne 'Jo' Rowling",
"type": "PERSON",
"metadata": {
"mid": "/m/042xh",
"wikipedia_url": "http://en.wikipedia.org/wiki/J._K._Rowling"
}
{
"name": "British",
"type": "LOCATION",
"metadata": {
"mid": "/m/07ssc",
"wikipedia_url": "http://en.wikipedia.org/wiki/United_Kingdom"
}
{
"name": "Harry Potter",
"type": "PERSON",
"metadata": {
"mid": "/m/078ffw",
"wikipedia_url":
"http://en.wikipedia.org/wiki/Harry_Potter"
}
@glaforge
Analyze sentiment
“The food was excellent, I would definitely go back!”
{
"documentSentiment": {
"score": 0.8,
"magnitude": 0.8
}
}
@glaforge
Analyze syntax
Speech API
Speech to text transcription in
over 80 languages
@glaforge
Speech API features
Speech Recognition
Recognizes over
110 languages &
variants.
Powered by deep
learning neural
networking to power
your applications.
Real-time results
Can stream text
results, returning
partial recognition
results as they
become available.
Can also be run
on buffered or
archived audio
files.
Noise Robustness
No need for signal
processing or
noise cancellation
before calling API.
Can handle noisy
audio from a
variety of
environments.
Context-Aware
Can provide
context hints for
improved
accuracy.
Especially useful
for device and app
use cases.
@glaforge
Speech timestamps
"transcript": "Hello World...",
"confidence": 0.96596134,
"words": [
{
"startTime": "1.400s",
"endTime": "1.800s",
"word": "Hello"
},
{
"startTime": "1.800s",
"endTime": "2.300s",
"word": "World"
},
...
]
Search for text
in audio files
Dialogflow
Build natural and rich
conversational experiences
@glaforge
Build chatbots and conversational interfaces
@glaforge
Build chatbots and conversational interfaces
Translation API
Translate text in 100+ languages
@glaforge
— connecting guests through translation
● 60% of Airbnb bookings
connect people who use
the app in different
languages
● Using the Translation API
to translate listings,
reviews, and conversations
significantly improves a
guest’s likelihood to book
@glaforge
Calling the
translation API
import com.google.cloud.translate.*;
import com.google.cloud.translate.Translate.*;
Translate translate =
TranslateOptions.getDefaultInstance()
.getService();
String text = "Hello, world!";
Translation translation =
translate.translate(
text,
TranslateOption.sourceLanguage("en"),
TranslateOption.targetLanguage("de"));
System.out.printf("Translation: %s%n",
translation.getTranslatedText());
@glaforge
Neural machine translation
Learn more: bit.ly/nyt-ai-awakening
@glaforge
Neural machine translation improvements ⚡
Original Spanish Text
El señor Dursley era el director de
una empresa llamada Grunnings,
que fabricaba taladros. Era un
hombre corpulento y rollizo, casi sin
cuello, aunque con un bigote
inmenso. La señora Dursley era
delgada, rubia y tenía un cuello
casi el doble de largo de lo habitual,
lo que le resultaba muy útil, ya que
pasaba la mayor parte del tiempo
estirándolo por encima de la valla
de los jardines para espiar a sus
vecinos
First generation translation
Mr. Dursley was the director of a
company called Grunnings, which
made drills. He was a big beefy
man, almost neckless, albeit with
a huge mustache. Mrs. Dursley
was thin and blonde and had a
neck almost twice longer than
usual, so it was very useful, since
he spent most of the time
stretching it over the fence of the
gardens to spy on their neighbors
Neural Machine Translation
Mr. Dursley was the director of a
company called Grunnings, which
manufactured drills. He was a big,
plump man, almost without a
neck, but with a huge mustache.
Mrs. Dursley was thin, blond, and
had a neck almost twice as long
as usual, which was very useful,
since she spent most of the time
stretching it over the garden fence
to spy on her neighbors
Video Intelligence API
Understand your video’s entities
at shot, frame, or video level
@glaforge
{
"description": "Bird's-eye view",
"language_code": "en-us",
"locations": {
"segment": {
"start_time_offset": 71905212,
"end_time_offset": 73740392
},
"confidence": 0.96653205
}
}
Video API Response:
Label detection
@glaforge
{
"description": "Portrait",
"language_code": "en-us",
"locations": {
"segment": {
"start_time_offset": 116991989
"end_time_offset": 118243219
},
"confidence": 0.8332939
}
}
Video API Response:
Label detection
TensorFlow
Google’s Open Source framework
for deep neural networks
@glaforge
TensorFlow — Google’s 2nd gen. OSS deep learning library
● Provides APIs in Python and C++ (Java & Go experimental)
○ To describe Machine Learning models
○ To implement Machine Learning algorithms
● Supported:
○ Regression models
○ Neural networks & Deep learning
■ Convolutional Neural Networks
■ Recurrent Neural Networks
■ LSTM Neural Networks
Cloud Machine
Learning Engine
Train your models,
run predictions,
directly in the cloud
@glaforge
Build, train and serve your own models
@glaforge
Cloud Machine Learning Engine
Train models and run predictions
for your TensorFlow models
in the cloud, as a fully managed service,
on CPUs, GPUs or TPUs
gcloud ml jobs submit training job22 --package-path=trainer
--module-name=trainer.task2 --staging-bucket=gs://ml-demo/jobs
--config=config.yaml -- --train_dir=gs://ml-demo/jobs/train22
Bridging the Machine
Learning gap
API easy to
call for a
developer
Data scientists
with ML
background
Use your own
data to
customize
pre-trained
models
AutoML [ALPHA]
Fine-tune pre-trained models
with your own datasets
Summary
Guillaume Laforge
Developer Advocate
Google Cloud
@glaforge
@glaforge
Video — cloud.google.com/video-intelligence
Vision — cloud.google.com/vision
Speech — cloud.google.com/speech
Natural Language — cloud.google.com/natural-language
Dialogflow — dialogflow.com
Translation — cloud.google.com/translation
AutoML — cloud.google.com/automl
TensorFlow — tensorflow.org
ML Engine — cloud.google.com/ml-engine
Try them all in your browser!
@glaforge
Machine Learning
Crash Course
https://developers.google.com/
machine-learning/crash-course/
@glaforge
How Google does
Machine Learning
https://www.coursera.org/
learn/google-machine-learning
Thanks
for your
attention
Watch the video with slide synchronization on
InfoQ.com!
https://www.infoq.com/presentations/google-
ml-services

More Related Content

More from C4Media

More from C4Media (20)

Shifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CDShifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CD
 
CI/CD for Machine Learning
CI/CD for Machine LearningCI/CD for Machine Learning
CI/CD for Machine Learning
 
Fault Tolerance at Speed
Fault Tolerance at SpeedFault Tolerance at Speed
Fault Tolerance at Speed
 
Architectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep SystemsArchitectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep Systems
 
ML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.jsML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.js
 
Build Your Own WebAssembly Compiler
Build Your Own WebAssembly CompilerBuild Your Own WebAssembly Compiler
Build Your Own WebAssembly Compiler
 
User & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix ScaleUser & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix Scale
 
Scaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's EdgeScaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's Edge
 
Make Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home EverywhereMake Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home Everywhere
 
The Talk You've Been Await-ing For
The Talk You've Been Await-ing ForThe Talk You've Been Await-ing For
The Talk You've Been Await-ing For
 
Future of Data Engineering
Future of Data EngineeringFuture of Data Engineering
Future of Data Engineering
 
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreAutomated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
 
Navigating Complexity: High-performance Delivery and Discovery Teams
Navigating Complexity: High-performance Delivery and Discovery TeamsNavigating Complexity: High-performance Delivery and Discovery Teams
Navigating Complexity: High-performance Delivery and Discovery Teams
 
High Performance Cooperative Distributed Systems in Adtech
High Performance Cooperative Distributed Systems in AdtechHigh Performance Cooperative Distributed Systems in Adtech
High Performance Cooperative Distributed Systems in Adtech
 
Rust's Journey to Async/await
Rust's Journey to Async/awaitRust's Journey to Async/await
Rust's Journey to Async/await
 
Opportunities and Pitfalls of Event-Driven Utopia
Opportunities and Pitfalls of Event-Driven UtopiaOpportunities and Pitfalls of Event-Driven Utopia
Opportunities and Pitfalls of Event-Driven Utopia
 
Datadog: a Real-Time Metrics Database for One Quadrillion Points/Day
Datadog: a Real-Time Metrics Database for One Quadrillion Points/DayDatadog: a Real-Time Metrics Database for One Quadrillion Points/Day
Datadog: a Real-Time Metrics Database for One Quadrillion Points/Day
 
Are We Really Cloud-Native?
Are We Really Cloud-Native?Are We Really Cloud-Native?
Are We Really Cloud-Native?
 
CockroachDB: Architecture of a Geo-Distributed SQL Database
CockroachDB: Architecture of a Geo-Distributed SQL DatabaseCockroachDB: Architecture of a Geo-Distributed SQL Database
CockroachDB: Architecture of a Geo-Distributed SQL Database
 
A Dive into Streams @LinkedIn with Brooklin
A Dive into Streams @LinkedIn with BrooklinA Dive into Streams @LinkedIn with Brooklin
A Dive into Streams @LinkedIn with Brooklin
 

Recently uploaded

Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 

Recently uploaded (20)

Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024
 
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
Long journey of Ruby Standard library at RubyKaigi 2024
Long journey of Ruby Standard library at RubyKaigi 2024Long journey of Ruby Standard library at RubyKaigi 2024
Long journey of Ruby Standard library at RubyKaigi 2024
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfLinux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdf
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfHow Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 

Machine Intelligence at Google Scale

  • 1. Machine Intelligence at Google Scale Vision, Video, NLP, Speech, Dialogflow TensorFlow, Cloud ML Engine, AutoML Guillaume Laforge Developer Advocate Google Cloud @glaforge
  • 2. InfoQ.com: News & Community Site Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ google-ml-services • Over 1,000,000 software developers, architects and CTOs read the site world- wide every month • 250,000 senior developers subscribe to our weekly newsletter • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • 2 dedicated podcast channels: The InfoQ Podcast, with a focus on Architecture and The Engineering Culture Podcast, with a focus on building • 96 deep dives on innovative topics packed as downloadable emags and minibooks • Over 40 new content items per week
  • 3. Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon London www.qconlondon.com
  • 5. @glaforge How did we escape the AI winter? Ongoing research on neural networks More labeled datasets to learn from More scalable compute power to train bigger models
  • 7. @glaforge Machine Learning is everywhere at Google
  • 8. @glaforge Machine Learning is everywhere at Google
  • 9. @glaforge The Machine Learning Spectrum TensorFlow Machine Learning APIs Academia, R&D Easy-to-use APIs ML as a Service, for developers Custom ML models, For data scientists
  • 10. @glaforge The Machine Learning Spectrum Academia, R&D Easy-to-use APIs ML as a Service, for developers Custom ML models, for data scientists
  • 11. Machine learning is learning from examples and experience
  • 12.
  • 14. @glaforge How would we do this without ML? CC-BY-SA 2.0 Wikimedia Commons https://commons.wikimedia.org/wiki/File:Apple_in_lightbox.png
  • 15. @glaforge How would we do this without ML? CC-BY-SA 2.0 Wikimedia Commons https://commons.wikimedia.org/wiki/File:Apple_in_lightbox.png
  • 16. @glaforge CC-BY-SA 2.0 Wikimedia Commons https://commons.wikimedia.org/wiki/File:Apple_in_lightbox.png How would we do this without ML?
  • 17. @glaforge CC-BY 4.0 Wikimedia Commons https://commons.wikimedia.org/wiki/File:Mop_and_bucket.jpg What about a dog and a mop? Easy, right?
  • 18. @glaforge Not so fast... CC-BY-SA-2.5 Wikimedia Commons https://commons.wikimedia.org/wiki/File:Komondor_Westminster_Dog_Show_crop.jpg CC-BY-2.0 Wikimedia Commons https://commons.wikimedia.org/wiki/File:2014_Westminster_Kennel_Club_Dog_Show_(12487315865).jpg CC-BY-2.0 Petful https://www.flickr.com/photos/petsadviser-pix/16395099127 CC-BY-SA-2.0 Jeffrey Beall https://www.flickr.com/photos/denverjeffrey/6903790333
  • 19. @glaforge Machine Learning tools by Google at your disposal Use your own data to train models Machine Learning as an API Cloud Vision API Cloud Translation APICloud Natural Language API Cloud Speech API Cloud Machine Learning Engine TensorFlow Cloud Video Intelligence Conversational Interfaces Cloud AutoML
  • 20. Vision API Complex image detection with a simple REST request
  • 22. @glaforge Face detection "detectionConfidence" : 0.93568963, "joyLikelihood" : "VERY_LIKELY", "panAngle" : 4.150538, "sorrowLikelihood" : "VERY_UNLIKELY", "tiltAngle" : -19.377356, "underExposedLikelihood" : "VERY_UNLIKELY", "blurredLikelihood" : "VERY_UNLIKELY" "faceAnnotations" : [ { "headwearLikelihood" : "VERY_UNLIKELY", "surpriseLikelihood" : "VERY_UNLIKELY", rollAngle" : -4.6490049, "angerLikelihood" : "VERY_UNLIKELY", "landmarks" : [ { "type" : "LEFT_EYE", "position" : { "x" : 691.97974, "y" : 373.11096, "z" : 0.000037421443 } }, ... ], "boundingPoly" : { "vertices" : [ { "x" : 743, "y" : 449 }, ...
  • 23. @glaforge "landmarkAnnotations": [ { "mid": "/m/0348s6", "description": "Paris Hotel and Casino", "score": 80, "boundingPoly": { "vertices": [ { "x": 117, "y": 479 }, ... ] }, "locations": [ { "latLng": { "latitude": 36.11221, "longitude": -115.172596 } } ] } ] Landmark detection CC-BY-SA-3.0 Wikimedia Commons https://commons.wikimedia.org/wiki/File:Las-Vegas-Paris-Hotel-Eiffel-Tower-8307.jpg
  • 24. @glaforge Web annotations { "entityId": "/m/016ms7", "score": 1.44038, "description": "Ford Anglia" } { "entityId": "/m/0gff2yr", "score": 5.92256, "description": "ArtScience Museum" } { "entityId": "/m/0h898pd", "score": 7.4162, "description": "Harry Potter (Literary Series)" } CC-BY 2.0 Rev Stan: https://www.flickr.com/photos/revstan/6865880240
  • 25. @glaforge Web annotations "fullMatchingImages": [{ "url": "https://upload.wikimedia.org/wikipedia/commons/6/6d/Flying_Ford_Angl ia_from_Harry_Potter_and_the_Chamber_of_Secrets_at_the_ArtScience_Mus eum,_Singapore_-_20120608.jpg", "score": 0.34952533 }, ... ] "partialMatchingImages": [{ "url": "https://muckysock.files.wordpress.com/2012/06/img_2730.jpg", "score": 0.887808 }, ... ] "pagesWithMatchingImages": [{ "url": "https://www.haikudeck.com/harry-potter-and-chamber-of-secrets--educa tion-presentation-SKZRnAO2UH", "score": 53.212971 }, ... ] CC-BY 2.0 Rev Stan: https://www.flickr.com/photos/revstan/6865880240
  • 26. @glaforge CC-BY-SA-2.5 Wikimedia Commons https://commons.wikimedia.org/wiki/File:Komondor_Westminster_Dog_Show_crop.jpg CC-BY-2.0 Wikimedia Commons https://commons.wikimedia.org/wiki/File:2014_Westminster_Kennel_Club_Dog_Show_(12487315865).jpg CC-BY-2.0 Petful https://www.flickr.com/photos/petsadviser-pix/16395099127 CC-BY-SA-2.0 Jeffrey Beall https://www.flickr.com/photos/denverjeffrey/6903790333 In case you were wondering…
  • 27. @glaforge CC-BY-SA-2.5 Wikimedia Commons https://commons.wikimedia.org/wiki/File:Komondor_Westminster_Dog_Show_crop.jpg CC-BY-2.0 Wikimedia Commons https://commons.wikimedia.org/wiki/File:2014_Westminster_Kennel_Club_Dog_Show_(12487315865).jpg CC-BY-2.0 Petful https://www.flickr.com/photos/petsadviser-pix/16395099127 CC-BY-SA-2.0 Jeffrey Beall https://www.flickr.com/photos/denverjeffrey/6903790333 In case you were wondering…
  • 28. @glaforge In case you were wondering… ? textile fur CC-BY-SA-2.5 Wikimedia Commons https://commons.wikimedia.org/wiki/File:Komondor_Westminster_Dog_Show_crop.jpg CC-BY-2.0 Wikimedia Commons https://commons.wikimedia.org/wiki/File:2014_Westminster_Kennel_Club_Dog_Show_(12487315865).jpg CC-BY-2.0 Petful https://www.flickr.com/photos/petsadviser-pix/16395099127 CC-BY-SA-2.0 Jeffrey Beall https://www.flickr.com/photos/denverjeffrey/6903790333
  • 29. Natural Language API Extract entities, sentiment, and syntax from text
  • 30. @glaforge Extract entities Joanne "Jo" Rowling, pen names J. K. Rowling and Robert Galbraith, is a British novelist, screenwriter and film producer best known as the author of the Harry Potter fantasy series
  • 31. @glaforge Extract entities Joanne "Jo" Rowling, pen names J. K. Rowling and Robert Galbraith, is a British novelist, screenwriter and film producer best known as the author of the Harry Potter fantasy series
  • 32. @glaforge Extract entities Joanne "Jo" Rowling, pen names J. K. Rowling and Robert Galbraith, is a British novelist, screenwriter and film producer best known as the author of the Harry Potter fantasy series { "name": "Joanne 'Jo' Rowling", "type": "PERSON", "metadata": { "mid": "/m/042xh", "wikipedia_url": "http://en.wikipedia.org/wiki/J._K._Rowling" } { "name": "British", "type": "LOCATION", "metadata": { "mid": "/m/07ssc", "wikipedia_url": "http://en.wikipedia.org/wiki/United_Kingdom" } { "name": "Harry Potter", "type": "PERSON", "metadata": { "mid": "/m/078ffw", "wikipedia_url": "http://en.wikipedia.org/wiki/Harry_Potter" }
  • 33. @glaforge Analyze sentiment “The food was excellent, I would definitely go back!” { "documentSentiment": { "score": 0.8, "magnitude": 0.8 } }
  • 35. Speech API Speech to text transcription in over 80 languages
  • 36. @glaforge Speech API features Speech Recognition Recognizes over 110 languages & variants. Powered by deep learning neural networking to power your applications. Real-time results Can stream text results, returning partial recognition results as they become available. Can also be run on buffered or archived audio files. Noise Robustness No need for signal processing or noise cancellation before calling API. Can handle noisy audio from a variety of environments. Context-Aware Can provide context hints for improved accuracy. Especially useful for device and app use cases.
  • 37. @glaforge Speech timestamps "transcript": "Hello World...", "confidence": 0.96596134, "words": [ { "startTime": "1.400s", "endTime": "1.800s", "word": "Hello" }, { "startTime": "1.800s", "endTime": "2.300s", "word": "World" }, ... ] Search for text in audio files
  • 38. Dialogflow Build natural and rich conversational experiences
  • 39. @glaforge Build chatbots and conversational interfaces
  • 40. @glaforge Build chatbots and conversational interfaces
  • 41. Translation API Translate text in 100+ languages
  • 42. @glaforge — connecting guests through translation ● 60% of Airbnb bookings connect people who use the app in different languages ● Using the Translation API to translate listings, reviews, and conversations significantly improves a guest’s likelihood to book
  • 43. @glaforge Calling the translation API import com.google.cloud.translate.*; import com.google.cloud.translate.Translate.*; Translate translate = TranslateOptions.getDefaultInstance() .getService(); String text = "Hello, world!"; Translation translation = translate.translate( text, TranslateOption.sourceLanguage("en"), TranslateOption.targetLanguage("de")); System.out.printf("Translation: %s%n", translation.getTranslatedText());
  • 44. @glaforge Neural machine translation Learn more: bit.ly/nyt-ai-awakening
  • 45. @glaforge Neural machine translation improvements ⚡ Original Spanish Text El señor Dursley era el director de una empresa llamada Grunnings, que fabricaba taladros. Era un hombre corpulento y rollizo, casi sin cuello, aunque con un bigote inmenso. La señora Dursley era delgada, rubia y tenía un cuello casi el doble de largo de lo habitual, lo que le resultaba muy útil, ya que pasaba la mayor parte del tiempo estirándolo por encima de la valla de los jardines para espiar a sus vecinos First generation translation Mr. Dursley was the director of a company called Grunnings, which made drills. He was a big beefy man, almost neckless, albeit with a huge mustache. Mrs. Dursley was thin and blonde and had a neck almost twice longer than usual, so it was very useful, since he spent most of the time stretching it over the fence of the gardens to spy on their neighbors Neural Machine Translation Mr. Dursley was the director of a company called Grunnings, which manufactured drills. He was a big, plump man, almost without a neck, but with a huge mustache. Mrs. Dursley was thin, blond, and had a neck almost twice as long as usual, which was very useful, since she spent most of the time stretching it over the garden fence to spy on her neighbors
  • 46. Video Intelligence API Understand your video’s entities at shot, frame, or video level
  • 47. @glaforge { "description": "Bird's-eye view", "language_code": "en-us", "locations": { "segment": { "start_time_offset": 71905212, "end_time_offset": 73740392 }, "confidence": 0.96653205 } } Video API Response: Label detection
  • 48. @glaforge { "description": "Portrait", "language_code": "en-us", "locations": { "segment": { "start_time_offset": 116991989 "end_time_offset": 118243219 }, "confidence": 0.8332939 } } Video API Response: Label detection
  • 49. TensorFlow Google’s Open Source framework for deep neural networks
  • 50. @glaforge TensorFlow — Google’s 2nd gen. OSS deep learning library ● Provides APIs in Python and C++ (Java & Go experimental) ○ To describe Machine Learning models ○ To implement Machine Learning algorithms ● Supported: ○ Regression models ○ Neural networks & Deep learning ■ Convolutional Neural Networks ■ Recurrent Neural Networks ■ LSTM Neural Networks
  • 51. Cloud Machine Learning Engine Train your models, run predictions, directly in the cloud
  • 52. @glaforge Build, train and serve your own models
  • 53. @glaforge Cloud Machine Learning Engine Train models and run predictions for your TensorFlow models in the cloud, as a fully managed service, on CPUs, GPUs or TPUs gcloud ml jobs submit training job22 --package-path=trainer --module-name=trainer.task2 --staging-bucket=gs://ml-demo/jobs --config=config.yaml -- --train_dir=gs://ml-demo/jobs/train22
  • 54. Bridging the Machine Learning gap API easy to call for a developer Data scientists with ML background Use your own data to customize pre-trained models
  • 55. AutoML [ALPHA] Fine-tune pre-trained models with your own datasets
  • 57. @glaforge Video — cloud.google.com/video-intelligence Vision — cloud.google.com/vision Speech — cloud.google.com/speech Natural Language — cloud.google.com/natural-language Dialogflow — dialogflow.com Translation — cloud.google.com/translation AutoML — cloud.google.com/automl TensorFlow — tensorflow.org ML Engine — cloud.google.com/ml-engine Try them all in your browser!
  • 59. @glaforge How Google does Machine Learning https://www.coursera.org/ learn/google-machine-learning
  • 61. Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/google- ml-services