Submit Search
Upload
TensorFlow on GCP
•
8 likes
•
2,079 views
Hayato Yoshikawa
Follow
GCP, as a full managed TensorFlow training, prediction and data processing environment.
Read less
Read more
Technology
Report
Share
Report
Share
1 of 30
Download now
Download to read offline
Recommended
How to make GAE adapt the Great Firewall
How to make GAE adapt the Great Firewall
Hayato Yoshikawa
Google Cloud Dataflow meets TensorFlow
Google Cloud Dataflow meets TensorFlow
Hayato Yoshikawa
GCPUG.TW - GCP學習資源分享
GCPUG.TW - GCP學習資源分享
Simon Su
Google for モバイル アプリ 16:00: モバイル kpi 分析の新標準 fluentd + google big query
Google for モバイル アプリ 16:00: モバイル kpi 分析の新標準 fluentd + google big query
Google Cloud Platform - Japan
JCConf 2016 - Google Dataflow 小試
JCConf 2016 - Google Dataflow 小試
Simon Su
Hands on App Engine
Hands on App Engine
Simon Su
Data Science on Google Cloud Platform
Data Science on Google Cloud Platform
Virot "Ta" Chiraphadhanakul
Google Cloud Platform for Data Science teams
Google Cloud Platform for Data Science teams
Barton Rhodes
Recommended
How to make GAE adapt the Great Firewall
How to make GAE adapt the Great Firewall
Hayato Yoshikawa
Google Cloud Dataflow meets TensorFlow
Google Cloud Dataflow meets TensorFlow
Hayato Yoshikawa
GCPUG.TW - GCP學習資源分享
GCPUG.TW - GCP學習資源分享
Simon Su
Google for モバイル アプリ 16:00: モバイル kpi 分析の新標準 fluentd + google big query
Google for モバイル アプリ 16:00: モバイル kpi 分析の新標準 fluentd + google big query
Google Cloud Platform - Japan
JCConf 2016 - Google Dataflow 小試
JCConf 2016 - Google Dataflow 小試
Simon Su
Hands on App Engine
Hands on App Engine
Simon Su
Data Science on Google Cloud Platform
Data Science on Google Cloud Platform
Virot "Ta" Chiraphadhanakul
Google Cloud Platform for Data Science teams
Google Cloud Platform for Data Science teams
Barton Rhodes
Google Cloud Platform for the Enterprise
Google Cloud Platform for the Enterprise
VMware Tanzu
Gcp
Gcp
HimanshuPise1
[Giovanni Galloro] How to use machine learning on Google Cloud Platform
[Giovanni Galloro] How to use machine learning on Google Cloud Platform
MeetupDataScienceRoma
Recap NEXT 2017
Recap NEXT 2017
Mario Fahlandt
Google Cloud Computing compares GCE, GAE and GKE
Google Cloud Computing compares GCE, GAE and GKE
Simon Su
Getting started with BigQuery
Getting started with BigQuery
Pradeep Bhadani
Google compute engine - overview
Google compute engine - overview
Charles Fan
A Tour of Google Cloud Platform
A Tour of Google Cloud Platform
Colin Su
[English] Create Mobile LBS Application Using Maps API
[English] Create Mobile LBS Application Using Maps API
Google Cloud Platform - Japan
GeoPython - Mapping Data in Jupyter Notebooks with PixieDust
GeoPython - Mapping Data in Jupyter Notebooks with PixieDust
Margriet Groenendijk
使用 Raspberry pi + fluentd + gcp cloud logging, big query 做iot 資料搜集與分析
使用 Raspberry pi + fluentd + gcp cloud logging, big query 做iot 資料搜集與分析
Simon Su
Using Cloud Natural Language API as a preprocessor
Using Cloud Natural Language API as a preprocessor
Hayato Yoshikawa
Introduction to Terraform and Google Cloud Platform
Introduction to Terraform and Google Cloud Platform
Pradeep Bhadani
Google Cloud Platform: Prototype ->Production-> Planet scale
Google Cloud Platform: Prototype ->Production-> Planet scale
Idan Tohami
Provision GCP resources using Terraform @ GDG Craiova
Provision GCP resources using Terraform @ GDG Craiova
Pradeep Bhadani
GCP Best Practices for SRE Team
GCP Best Practices for SRE Team
Huynh Thai Bao
Google Developers Summit Tokyo - Google Cloud Platform で知る Google クラウドの「Googl...
Google Developers Summit Tokyo - Google Cloud Platform で知る Google クラウドの「Googl...
Google Cloud Platform - Japan
GCP Gaming 2016 Keynote Seoul, Korea
GCP Gaming 2016 Keynote Seoul, Korea
Chris Jang
GCP Gaming 2016 Seoul, Korea Gaming Analytics
GCP Gaming 2016 Seoul, Korea Gaming Analytics
Chris Jang
Top Advantages of Using Google Cloud Platform
Top Advantages of Using Google Cloud Platform
Kinsta WordPress Hosting
Exploring Google (Cloud) APIs with Python & JavaScript
Exploring Google (Cloud) APIs with Python & JavaScript
wesley chun
Using Google (Cloud) APIs
Using Google (Cloud) APIs
wesley chun
More Related Content
What's hot
Google Cloud Platform for the Enterprise
Google Cloud Platform for the Enterprise
VMware Tanzu
Gcp
Gcp
HimanshuPise1
[Giovanni Galloro] How to use machine learning on Google Cloud Platform
[Giovanni Galloro] How to use machine learning on Google Cloud Platform
MeetupDataScienceRoma
Recap NEXT 2017
Recap NEXT 2017
Mario Fahlandt
Google Cloud Computing compares GCE, GAE and GKE
Google Cloud Computing compares GCE, GAE and GKE
Simon Su
Getting started with BigQuery
Getting started with BigQuery
Pradeep Bhadani
Google compute engine - overview
Google compute engine - overview
Charles Fan
A Tour of Google Cloud Platform
A Tour of Google Cloud Platform
Colin Su
[English] Create Mobile LBS Application Using Maps API
[English] Create Mobile LBS Application Using Maps API
Google Cloud Platform - Japan
GeoPython - Mapping Data in Jupyter Notebooks with PixieDust
GeoPython - Mapping Data in Jupyter Notebooks with PixieDust
Margriet Groenendijk
使用 Raspberry pi + fluentd + gcp cloud logging, big query 做iot 資料搜集與分析
使用 Raspberry pi + fluentd + gcp cloud logging, big query 做iot 資料搜集與分析
Simon Su
Using Cloud Natural Language API as a preprocessor
Using Cloud Natural Language API as a preprocessor
Hayato Yoshikawa
Introduction to Terraform and Google Cloud Platform
Introduction to Terraform and Google Cloud Platform
Pradeep Bhadani
Google Cloud Platform: Prototype ->Production-> Planet scale
Google Cloud Platform: Prototype ->Production-> Planet scale
Idan Tohami
Provision GCP resources using Terraform @ GDG Craiova
Provision GCP resources using Terraform @ GDG Craiova
Pradeep Bhadani
GCP Best Practices for SRE Team
GCP Best Practices for SRE Team
Huynh Thai Bao
Google Developers Summit Tokyo - Google Cloud Platform で知る Google クラウドの「Googl...
Google Developers Summit Tokyo - Google Cloud Platform で知る Google クラウドの「Googl...
Google Cloud Platform - Japan
GCP Gaming 2016 Keynote Seoul, Korea
GCP Gaming 2016 Keynote Seoul, Korea
Chris Jang
GCP Gaming 2016 Seoul, Korea Gaming Analytics
GCP Gaming 2016 Seoul, Korea Gaming Analytics
Chris Jang
Top Advantages of Using Google Cloud Platform
Top Advantages of Using Google Cloud Platform
Kinsta WordPress Hosting
What's hot
(20)
Google Cloud Platform for the Enterprise
Google Cloud Platform for the Enterprise
Gcp
Gcp
[Giovanni Galloro] How to use machine learning on Google Cloud Platform
[Giovanni Galloro] How to use machine learning on Google Cloud Platform
Recap NEXT 2017
Recap NEXT 2017
Google Cloud Computing compares GCE, GAE and GKE
Google Cloud Computing compares GCE, GAE and GKE
Getting started with BigQuery
Getting started with BigQuery
Google compute engine - overview
Google compute engine - overview
A Tour of Google Cloud Platform
A Tour of Google Cloud Platform
[English] Create Mobile LBS Application Using Maps API
[English] Create Mobile LBS Application Using Maps API
GeoPython - Mapping Data in Jupyter Notebooks with PixieDust
GeoPython - Mapping Data in Jupyter Notebooks with PixieDust
使用 Raspberry pi + fluentd + gcp cloud logging, big query 做iot 資料搜集與分析
使用 Raspberry pi + fluentd + gcp cloud logging, big query 做iot 資料搜集與分析
Using Cloud Natural Language API as a preprocessor
Using Cloud Natural Language API as a preprocessor
Introduction to Terraform and Google Cloud Platform
Introduction to Terraform and Google Cloud Platform
Google Cloud Platform: Prototype ->Production-> Planet scale
Google Cloud Platform: Prototype ->Production-> Planet scale
Provision GCP resources using Terraform @ GDG Craiova
Provision GCP resources using Terraform @ GDG Craiova
GCP Best Practices for SRE Team
GCP Best Practices for SRE Team
Google Developers Summit Tokyo - Google Cloud Platform で知る Google クラウドの「Googl...
Google Developers Summit Tokyo - Google Cloud Platform で知る Google クラウドの「Googl...
GCP Gaming 2016 Keynote Seoul, Korea
GCP Gaming 2016 Keynote Seoul, Korea
GCP Gaming 2016 Seoul, Korea Gaming Analytics
GCP Gaming 2016 Seoul, Korea Gaming Analytics
Top Advantages of Using Google Cloud Platform
Top Advantages of Using Google Cloud Platform
Similar to TensorFlow on GCP
Exploring Google (Cloud) APIs with Python & JavaScript
Exploring Google (Cloud) APIs with Python & JavaScript
wesley chun
Using Google (Cloud) APIs
Using Google (Cloud) APIs
wesley chun
Introduction to Cloud Computing with Google Cloud
Introduction to Cloud Computing with Google Cloud
wesley chun
Introduction to Google Cloud Platform
Introduction to Google Cloud Platform
Pradeep Bhadani
Serverless Data Architecture at scale on Google Cloud Platform
Serverless Data Architecture at scale on Google Cloud Platform
MeetupDataScienceRoma
Introduction to Google Cloud
Introduction to Google Cloud
DSC IEM
Cloud Workflows What's new in serverless orchestration and automation
Cloud Workflows What's new in serverless orchestration and automation
Márton Kodok
[Kotlin Serverless 工作坊] 單元 1 - 開發環境建置
[Kotlin Serverless 工作坊] 單元 1 - 開發環境建置
Shengyou Fan
Accessing Google Cloud APIs
Accessing Google Cloud APIs
wesley chun
Google Home and Google Assistant Workshop: Build your own serverless Action o...
Google Home and Google Assistant Workshop: Build your own serverless Action o...
Bret McGowen - NYC Google Developer Advocate
How Google Cloud Platform can help in the classroom/lab
How Google Cloud Platform can help in the classroom/lab
wesley chun
Week #1 Training.pptx
Week #1 Training.pptx
prajeethpoojary
60分鐘完送百萬edm,背後雲端ci/cd實戰大公開
60分鐘完送百萬edm,背後雲端ci/cd實戰大公開
KAI CHU CHUNG
Week #1 Training ACEM.pptx
Week #1 Training ACEM.pptx
ShivamJha793068
Power your apps with Gmail, Google Drive, Calendar, Sheets, Slides & more
Power your apps with Gmail, Google Drive, Calendar, Sheets, Slides & more
wesley chun
Gaming analytics on gcp
Gaming analytics on gcp
Myunggeun Choi
GDSC BVCOENM - Google Cloud Study Jam October 2021 | Day 1 + Day 2
GDSC BVCOENM - Google Cloud Study Jam October 2021 | Day 1 + Day 2
GDSCBVCOENM
Exploring Google (Cloud) APIs & Cloud Computing overview
Exploring Google (Cloud) APIs & Cloud Computing overview
wesley chun
Build an AI/ML-driven image archive processing workflow: Image archive, analy...
Build an AI/ML-driven image archive processing workflow: Image archive, analy...
wesley chun
TIAD : Automate everything with Google Cloud
TIAD : Automate everything with Google Cloud
The Incredible Automation Day
Similar to TensorFlow on GCP
(20)
Exploring Google (Cloud) APIs with Python & JavaScript
Exploring Google (Cloud) APIs with Python & JavaScript
Using Google (Cloud) APIs
Using Google (Cloud) APIs
Introduction to Cloud Computing with Google Cloud
Introduction to Cloud Computing with Google Cloud
Introduction to Google Cloud Platform
Introduction to Google Cloud Platform
Serverless Data Architecture at scale on Google Cloud Platform
Serverless Data Architecture at scale on Google Cloud Platform
Introduction to Google Cloud
Introduction to Google Cloud
Cloud Workflows What's new in serverless orchestration and automation
Cloud Workflows What's new in serverless orchestration and automation
[Kotlin Serverless 工作坊] 單元 1 - 開發環境建置
[Kotlin Serverless 工作坊] 單元 1 - 開發環境建置
Accessing Google Cloud APIs
Accessing Google Cloud APIs
Google Home and Google Assistant Workshop: Build your own serverless Action o...
Google Home and Google Assistant Workshop: Build your own serverless Action o...
How Google Cloud Platform can help in the classroom/lab
How Google Cloud Platform can help in the classroom/lab
Week #1 Training.pptx
Week #1 Training.pptx
60分鐘完送百萬edm,背後雲端ci/cd實戰大公開
60分鐘完送百萬edm,背後雲端ci/cd實戰大公開
Week #1 Training ACEM.pptx
Week #1 Training ACEM.pptx
Power your apps with Gmail, Google Drive, Calendar, Sheets, Slides & more
Power your apps with Gmail, Google Drive, Calendar, Sheets, Slides & more
Gaming analytics on gcp
Gaming analytics on gcp
GDSC BVCOENM - Google Cloud Study Jam October 2021 | Day 1 + Day 2
GDSC BVCOENM - Google Cloud Study Jam October 2021 | Day 1 + Day 2
Exploring Google (Cloud) APIs & Cloud Computing overview
Exploring Google (Cloud) APIs & Cloud Computing overview
Build an AI/ML-driven image archive processing workflow: Image archive, analy...
Build an AI/ML-driven image archive processing workflow: Image archive, analy...
TIAD : Automate everything with Google Cloud
TIAD : Automate everything with Google Cloud
Recently uploaded
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
Neo4j
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Katpro Technologies
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
shyamraj55
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?
XfilesPro
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
ThousandEyes
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
Delhi Call girls
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
Scott Keck-Warren
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
hans926745
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
Delhi Call girls
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Safe Software
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
Allon Mureinik
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
Michael W. Hawkins
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
HostedbyConfluent
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
2toLead Limited
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
Malak Abu Hammad
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptx
OnBoard
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
Mark Billinghurst
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions
Recently uploaded
(20)
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping Elbows
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptx
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
TensorFlow on GCP
1.
GCP TensorFlow Google Cloud
ML Engine & Dataflow H.Yoshikawa (@hayatoy) TFUG #5 24 May 2017
2.
@hayatoy APAC TFUG GCPUG GDG Presenter
3.
4.
5.
Google Cloud ML
Engine Jupyter Notebook [DEMO] Online Prediction [DEMO] Google Cloud Dataflow
6.
Cloud ML Engine
7.
Cloud ML Engine TensorFlow Online
Prediction ←ktkr!
8.
Cloud ML Engine
9.
Cloud ML Engine
10.
11.
Package gcloud REST API Package Jupyter Notebook
←New!
12.
gcloud GCP gcloud ml-engine jobs
submit COMMAND [GLOBAL-FLAG …]
13.
REST API Python Client
Library from oauth2client.client import GoogleCredentials from googleapiclient import discovery credentials = GoogleCredentials.get_application_default() ml = discovery.build('ml', 'v1', credentials=credentials) job_req = ml.projects().jobs().create( parent=project_id, body=param) job_req.execute()
14.
Jupyter Notebook Notebook Cloud
ML Engine https://github.com/hayatoy/cloudml‑magic
15.
16.
Online Prediction ‑
Demo
17.
pix2pix GCPUG bit.ly/realgcpug
18.
pix2pix
19.
GCPUG Original Edge 20
epochs 200 epochs
20.
Online Prediction SavedModel v1 serving_input_fn Beta
21.
Cloud Dataflow
22.
Cloud Dataflow ETL tf.Transform Dynamic Work
Rebalancing GCE MapReduce
23.
Dataflow
24.
Dataflow
25.
BigQuery, GCS query =
"SELECT * FROM [project:dataset.table]" p = beam.Pipeline(options=options) (p | 'query image list' >> beam.Read(beam.io.BigQuerySource(query=query)) | 'download image' >> beam.FlatMap(download_image) | 'crop and resize image' >> beam.Map(crop_image) | 'make edge image' >> beam.Map(make_edge_image)
26.
1000 DWR Cap opt.max_num_workers = 10
27.
Dataflow
28.
29.
GCP
30.
Thank you! Cloud ML
Engine https://cloud.google.com/ml‑engine/ Cloud Dataflow https://cloud.google.com/dataflow/ Cloud ML Engine ‑ Jupyter Notebook Extension https://github.com/hayatoy/cloudml‑magic
Download now