This is one of two 45-60-min presentations to students or working professionals. You may know Google for search, YouTube, Android, Chrome, and Gmail, but did you know Google has many other cloud services? In this comprehensive yet still high-level overview of Google Cloud tools & APIs with the purpose of inspiring you as to what's possible. The session introduces Google's machine learning & other APIs, tools that have an immediate impact on projects, alleviating the need to think about computing infrastructure as well as dispensing with the need to have machine learning expertise. We'll wrap up w/online resources like videos & hands-on tutorials to get you started! The main takeaways are where to run your code, store your data, and analyze your data, all in the cloud!
The other version of this talk ("B") focuses more on serverless platforms.
Powerful Google developer tools for immediate impact! (2023-24 A)
1. Powerful Google developer
tools for immediate impact!
Santa Clara/San Jose :: Fall 2023
Wesley Chun
Principal, CyberWeb
@wescpy@
Principal Consultant, CyberWeb
● Mission: produce accelerated Python
developers, enable developers to be
successful using Google Cloud and
other Google developer tools & APIs
● Focus: Python, Google Cloud (GCP) &
Google Workspace (GWS) APIs; GAE
migrations; Google X-product sol'ns
● Services: technical consulting,
training, engineering, speaking, code
samples, hands-on tutorials, public
technical content (blogs, social, etc.)
About the speaker
Previous experience / background
● Software Engineer & Developer Advocate
○ Google, Sun, HP, Cisco, EMC, Xilinx &
○ Original Yahoo!Mail engineer/SWE
● Technical trainer, teacher, instructor
○ Teaching Math, Linux, Python since '83
○ Adjunct CS Faculty at local SV colleges
● Python community member
○ Popular Core Python series author
○ Python Software Foundation Fellow
● AB (Math/CS) & CMP (Music/Piano), UC
Berkeley and MSCS, UC Santa Barbara
● Adjunct Computer Science Faculty, Foothill
College (Silicon Valley)
GWS Dev Show
goo.gl/JpBQ40
GAE migration
bit.ly/3xk2Swi
2. Why and Agenda
● Cloud has taken industry by storm (all?)
● Not enough cloud computing in industry or higher-ed curriculum
● Cloud skills are in-demand in today's workforce
● How can Google developer tools help with student/work projects?
● Show how to start using these impactful Google tools today!
Cloud computing
& Google Cloud
1
Hosting your
code
2
Google APIs
primer
3
Machine
Learning
4 5
Other APIs to
consider
6
Wrap-up
01
Cloud computing
& Google Cloud
Intro and overview
3. Getting things done using someone else’s computers, especially
where someone else worries about maintenance, provisioning, system
administration, security, networking, failure recovery, upgrades, etc.
Google Compute Engine, Cloud Storage
AWS EC2 & S3; Rackspace; Joyent
Cloud service levels/"pillars"
SaaS
Software as a Service
PaaS
Platform as a Service
IaaS
Infrastructure as a Service
Google BigQuery, Cloud SQL, Vertex AI,
Cloud Datastore, NL, Vision, Pub/Sub
AWS Kinesis, RDS; Windows Azure SQL, Docker
Google Apps Script
Salesforce1/force.com
Google Workspace (was G Suite/Google Apps)
Yahoo!Mail, Hotmail, Salesforce, Netsuite, Office 365
Google App Engine, Cloud Functions
Heroku, Cloud Foundry, Engine Yard, AWS Lambda
4. Summary of responsibility
SaaS
Software as a Service
Applications
Data
Runtime
Middleware
OS
Virtualization
Servers
Storage
Networking
Applications
Data
Runtime
Middleware
OS
Virtualization
Servers
Storage
Networking
IaaS
Infrastructure as a Service
Applications
Data
Runtime
Middleware
OS
Virtualization
Servers
Storage
Networking
PaaS
Platform as a Service
Managed by YOU Managed by cloud vendor
Applications
Data
Runtime
Middleware
OS
Virtualization
Servers
Storage
Networking
on-prem
all you, no cloud
Theme - Spec/Reqs
Logistics - Design app
Space - Provision HW
Food - Build & serve app
Manage - Manage app
on-prem
(DIY)
IaaS
(Compute Engine)
PaaS
(App Engine)
SaaS
(Google Workspace
(formerly G Suite))
Pick theme
Plan party
Find space
Cook
On-call
Pick theme
Plan party
Rent hall
Cook
On-call
Pick theme
Plan party
Rent hall
Hire Caterer
Hire manager
Pick theme
Hire planner
Rent hall
Hire caterer
Hire manager
Imagine you’re hosting a party...
6. Running Code: App Engine
Got a great app idea? Now what?
VMs? Operating systems? Big disk?
Web servers? Load balancing?
Database servers? Autoscaling?
With Google App Engine, you don't
think about those. Just upload
your code; we do everything else.
>
cloud.google.com/appengine
Running Code: Cloud Functions
Don't have an entire app? Just want
to deploy small microservices or
"RPCs" online globally? That's what
Google Cloud Functions are for!
(+Firebase version for mobile apps)
cloud.google.com/functions
firebase.google.com/products/functions
7. Running Code: Cloud Run
Got a containerized app? Want its
flexibility along with the convenience
of serverless that's fully-managed
plus auto-scales? Google Cloud Run is
exactly what you're looking for!
Need custom HW? Cloud Run on GKE
cloud.google.com/run
Running code: Apps Script
Apps Script: A customized serverless JS
runtime for automation, and extension
and integration with Google Workspace
(formerly G Suite), Google, or other
external services. Access 40+ different
Google services w/o using APIs or
writing OAuth code.
developers.google.com/apps-script
8. Accessing maps from
spreadsheets?!?
goo.gl/oAzBN9
This… with help from Google Maps & Gmail
function sendMap() {
var sheet = SpreadsheetApp.getActiveSheet();
var address = sheet.getRange("A2").getValue();
var map = Maps.newStaticMap().addMarker(address);
GmailApp.sendEmail('friend@example.com', 'Map',
'See below.', {attachments:[map]});
}
JS
g.co/codelabs/apps-script-intro
03
Google APIs
primer
How to access Cloud APIs
9.
10. General steps
1. Go to Cloud Console
2. Login to Google/Gmail account
(Workspace domain may require admin approval)
3. Create project (per application)
4. Enable APIs to use
5. Enable billing (CC, Free Trial, etc.)
6. Download client library(ies)
7. Create & download credentials
8. Write code*
9. Run code (may need to authorize)
Google APIs: how to use
Costs and pricing
● GCP: pay-per-use
● Google Workspace: subscription
● GCP Free Trial ($300/1Q, CC req'd)
● GCP "Always Free" tier
○ Some products have free tier
○ Daily or monthly quota
○ Must exceed to incur billing
● More on both programs at
cloud.google.com/free
*In your code
1. Import API client library
2. Create API client object
3. Use client to make API Calls
11. Cloud/GCP console
console.cloud.google.com
● Hub of all developer activity
● Applications == projects
○ New project for new apps
○ Projects have a billing acct
● Manage billing accounts
○ Financial instrument required
○ Personal or corporate credit cards,
Free Trial, and education grants
● Access GCP product settings
● Manage users & security
● Manage APIs in devconsole
● View application statistics
● En-/disable Google APIs
● Obtain application credentials
Using Google APIs
goo.gl/RbyTFD
API manager aka Developers Console (devconsole)
console.developers.google.com
12. Client libraries and credentials types
● Two different client library types
○ Platform-level client libraries (lower-level)
■ Multiple product groups as a "lowest-common denominator"
■ Install: developers.google.com/api-client-library
○ Product-level client libraries (higher-level)
■ Custom client libraries made specifically for 1 product or product group
■ Found on product or product group page(s)
● Three different credentials types
○ Simple: API keys (access public data)
■ Simplest form of authorization: an API key; tied to a project
○ Authorized: OAuth client IDs (access data owned by [human] user)
■ Provides additional layer of security via OAuth2 (RFC 6749)
○ Authorized: service accounts (access data owned by an app/robot user)
■ Provides additional layer of security via OAuth2 or JWT (RFC 7519)
Blog series:
dev.to/wescpy
&
Google APIs client
libraries for many
languages; demos in
developers.google.com/api-
client-library
cloud.google.com/apis/docs
/cloud-client-libraries
13. OAuth2 or
API key
HTTP-based REST APIs 1
HTTP
2
Google APIs request-response workflow
● Application makes request
● Request received by service
● Process data, return response
● Results sent to application
(typical client-server model)
04
How to analyze
your data
Machine Learning
14. AI & machine learning
Puppy or muffin?
Source: twistedsifter.com/2016/03/
puppy-or-bagel-meme-gallery
15. Machine learning is learning
from rules plus experience.
Inference Phase
Training Phase
answers
data
model ("rules")
Machine
Learning
Model
predictions
(new) data
16. ML "building block" APIs
● Gain insights from data using GCP's
pre-trained machine learning models
● Leverage the same technology as Google
Translate, Photos, and Assistant
● Requires ZERO prior knowledge of ML
● If you can call an API, you can use AI/ML!
● cloud.google.com/products/ai/building-blocks
Vision Video
Intelligence
Speech
(S2T & T2S)
Natural
Language
Translation
Machine Learning: Cloud Vision
Google Cloud Vision API lets developers
extract metadata and understand the
content of an image, identify & detect
objects/labels, text/OCR, landmarks,
logos, facial features, products, XC, etc.
cloud.google.com/vision
17. from google.cloud import vision
image_uri = 'gs://cloud-samples-data/vision/using_curl/shanghai.jpeg'
client = vision.ImageAnnotatorClient()
image = vision.types.Image()
image.source.image_uri = image_uri
response = client.label_detection(image=image)
print('Labels (and confidence score):')
print('=' * 30)
for label in response.label_annotations:
print(label.description, '(%.2f%%)' % (label.score*100.))
Vision: label annotation/object detection
$ python3 label-detect.py
Labels (and confidence score):
==============================
People (95.05%)
Street (89.12%)
Mode of transport (89.09%)
Transport (85.13%)
Vehicle (84.69%)
Snapshot (84.11%)
Urban area (80.29%)
Infrastructure (73.14%)
Road (72.74%)
Pedestrian (68.90%)
Vision: label annotation/object detection
g.co/codelabs/vision-python
18. Google Photos
The app has a
search bar?!?
blog.google/products/photos/smarter-photo-albums-without-work
Global view
Problem
● 1B ppl depend on seafood
● 85% at/over-fishing or recovering
● 20% caught illegal, undoc'd, unreg'd
● Analysts monitoring unscalable
One solution
● globalfishingwatch.org/map
● Machine-learning classifiers:
○ Ship type: cargo, tug, sail, fishing
○ Ship size
○ Gear: longline, purse seine, trawl
○ Movement tracking: when and
where vessels are fishing
19. quickdraw.withgoogle.com
Cloud Vision demo: Quick Draw game
experiments.withgoogle.com/quick-draw
Machine Learning: Cloud Natural Language
Google Cloud Natural Language API
reveals the structure and meaning
of text, performing sentiment
analysis, content classification,
entity extraction, and syntactical
structure analysis; multi-lingual
cloud.google.com/language
20. Machine Learning: Cloud Video Intelligence
Google Cloud Video Intelligence
API makes videos searchable, and
discoverable, by extracting
metadata. Other features: object
tracking, shot change detection,
and text detection
cloud.google.com/video-intelligence
Machine Learning: Cloud Speech
Google Cloud Speech APIs enable
developers to convert
speech-to-text and vice versa
cloud.google.com/speech
cloud.google.com/text-to-speech
21. Machine Learning: Cloud Translation
Access Google Translate
programmatically through this
API; translate an arbitrary
string into any supported
language using state-of-the-art
Neural Machine Translation
cloud.google.com/translate
Machine Learning: AutoML
AutoML: a suite of cloud APIs for
developers with limited machine
learning expertise; auto-selects best
models & allows for further training
of those models for your data
(Translation, Vision, Natural Language,
Video Intelligence, Tables)
cloud.google.com/automl
cloud.google.com/automl-tables
22. Full Spectrum of AI & ML Offerings
App developer Data scientist,
developer
Data scientist, Researcher
(w/infrastructure access &
DevOps/SysAdmin skills)
AI Platform
(now Vertex AI)
Auto ML
Build custom models,
use OSS SDK on fully-
managed infrastructure
ML APIs
App developer,
data scientist
Use/customize pre-built
models
Use pre-built/pre-
trained models
Build custom models, use/
extend OSS SDK, self-manage
training infrastructure
Google and Jupyter Notebooks
Users have many ways to access an indispensable data science tool
● Google Cloud Vertex AI Workbench
○ cloud.google.com/notebooks
● Kaggle
○ kaggle.com
● Google Research CoLaboratory
○ colab.research.google.com
● Google Cloud Dataproc Hub
○ cloud.google.com/dataproc/docs/tutorials/dataproc-hub-overview
● Google Cloud Datalab
○ cloud.google.com/datalab/docs/how-to/working-with-notebooks
23. 05
Where to store
your data
And more ways to analyzing it
Storing Data: Cloud Firestore
The best of both worlds: the next
generation of Cloud Datastore
(w/product rebrand) plus features
from the Firebase realtime database
(For choosing between Firebase & Cloud Firestore: see
firebase.google.com/docs/firestore/rtdb-vs-firestore;
for choosing between Firestore Datastore & Firestore Native modes:
see cloud.google.com/datastore/docs/firestore-or-datastore)
cloud.google.com/firestore
24. ● Ordinary database - explicitly query database for
new updates (but when?)
● Realtime database - automatically receive deltas
when database updated (setup client listener object)
● Highly scalable database
● Multi-regional data replication
● Strong consistency (vs. eventual consistency): read
your writes!
Cloud Firestore key features
Cloud Firestore data model
collections subcollections
25. Cloud Firestore straightforward querying
github.com/GoogleCloudPlatform/python-docs-samples/blob/master/firestore/cloud-client/snippets.py
Storing and Analyzing Data: BigQuery
Google BigQuery: fully-managed data
warehouse for large-scale data
analytics with built-in machine
learning (BQML); issue SQL queries
across multi-terabytes of data. BQ
Sandbox lets anyone query up to
1TB/mo for free with no obligation
cloud.google.com/bigquery
26. BigQuery: querying Shakespeare words
TITLE = "The most common words in all of Shakespeare's works"
QUERY = '''
SELECT LOWER(word) AS word, sum(word_count) AS count
FROM [bigquery-public-data:samples.shakespeare]
GROUP BY word ORDER BY count DESC LIMIT 10
'''
rsp = BQ.jobs().query(body={'query': QUERY}, projectId=PROJ_ID).execute()
print('n*** Results for %r:n' % TITLE)
print('t'.join(col['name'].upper() # HEADERS
for col in rsp['schema']['fields']))
print('n'.join('t'.join(str(col['v']) # DATA
for col in row['f']) for row in rsp['rows']))
Top 10 most common Shakespeare words
$ python bq_shake.py
*** Results for "The most common words in all of Shakespeare's works":
WORD COUNT
the 29801
and 27529
i 21029
to 20957
of 18514
a 15370
you 14010
my 12936
in 11722
that 11519
27. ● BigQuery public data sets: cloud.google.com/bigquery/public-data
● BQ sandbox (1TB/mo free): cloud.google.com/bigquery/docs/sandbox (see blog post)
● Other public data sets: cloud.google.com/public-datasets (Google Cloud),
research.google/tools/datasets (Google Research), and Kaggle (kaggle.com)
● COVID-19 BigQuery data sets
○ How to use our data sets (see blog post)
○ JHU Coronavirus COVID-19 Global Cases data set
○ List of all COVID-19 data sets
● Cloud Life Sciences API: cloud.google.com/life-sciences (see blog post)
● Cloud Healthcare API: cloud.google.com/healthcare (see blog post)
BigQuery and public data sets
Google Workspace: Google Sheets
Sheets API gives you programmatic
access to spreadsheets; perform
(w/code) almost any action you can
do from the web interface as a user
developers.google.com/sheets
28. 06
Inspiration
Use multiple Google APIs
to create unique solutions
Hangouts Chat Productivity Tracker
Chat bot that's GCP & Google Workspace (formerly G Suite) aware
29. Productivity tracker Hangouts Chat bot
Google Workspace
(formerly G Suite)
GCP
Sheets Natural Language
START
or LOG
END
Hangouts
Chat
App
Engine
Cloud
SQL
30. App summary
● Motivation
● Do coding contract jobs during school year
● Needed to track time spent on work (and non-work)
● Who doesn't want to be more productive?
● Hangouts Chat bot framework & API... build bots to:
● Automate workflows
● Query for information
● Other heavy-lifting
● Google Workspace (formerly G Suite) app that leverages app-hosting, NL processing, and storage
tools from GCP
● Application source code
● github.com/gsuitedevs/hangouts-chat-samples/tree/master/python/productivity_tracker
● Presented at GDG Silicon Valley (Aug 2018)
● meetup.com/gdg-silicon-valley/events/252858660
Cloud image processing workflow
Archive and analyze Google Workspace
(formerly G Suite) images with GCP
34. ● Project goal: Imagining an actual enterprise use case and solve it!
● Specific goals: free-up highly-utilized resource, archive data to
colder/cheaper storage, analyze images, generate report for mgmt
● Download image binary from Google Drive
● Upload object to Cloud Storage bucket
● Send payload for analysis by Cloud Vision
● Write back-up location & analysis results into Google Sheets
● Blog post: goo.gle/3nPxmlc (original post); Cloud X-post
● Codelab: free, online, self-paced, hands-on tutorial
● g.co/codelabs/drive-gcs-vision-sheets
● Application source code
● github.com/wescpy/analyze_gsimg
App summary
07
Wrap-up
Summary & resources
35. Summary: Google developer tools
● Google provides more than just apps
○ More than search, YouTube, Android, Chrome, Gmail, and Docs
○ Google technology & integration available to developers via APIs
● Tour of Google (Cloud) APIs & developer tools
○ GWS: not just a set of productivity apps… you can code them too!
○ GCP: compute, storage, networking, security, data & machine learning tools
■ Serverless: frees developers from infrastructure
■ So you can focus on building solutions
● Interesting possibilities when using >1 Google product families/APIs
● With Google: run your apps, analyze & store your data
Other Google APIs & platforms
● Google Workspace (G Suite) (code Gmail, Drive, Docs, Sheets, Slides!)
○ developers.google.com/gsuite
● Firebase (mobile development platform and RT DB plus ML-Kit)
○ firebase.google.com and firebase.google.com/docs/ml-kit
● Google Data Studio (data visualization, dashboards, etc.)
○ datastudio.google.com/overview
○ goo.gle/datastudio-course
● Actions on Google/Assistant/DialogFlow (voice apps)
○ developers.google.com/actions
● YouTube (Data, Analytics, and Livestreaming APIs)
○ developers.google.com/youtube
● Google Maps (Maps, Routes, and Places APIs)
○ developers.google.com/maps
● Flutter (native apps [Android, iOS, web] w/1 code base[!])
○ flutter.dev
36. ● Documentation (most APIs have "Quickstarts")
○ GCP: cloud.google.com/{docs,appengine,functions,run,vision,automl,translate,language,
speech,texttospeech,video-intelligence,firestore,bigquery,compute,storage,gpu,tpu}
○ GWS: developers.google.com/{gsuite,drive,calendar,gmail,docs,sheets,slides,apps-script}
● Introductory "codelabs" ([free] self-paced, hands-on tutorials)
○ GWS APIs: g.co/codelabs/gsuite-apis-intro (featuring Drive API)
○ Apps Script: g.co/codelabs/apps-script-intro
○ App Engine: codelabs.developers.google.com/codelabs/cloud-app-engine-python
○ Cloud Functions: codelabs.developers.google.com/codelabs/cloud-starting-cloudfunctions
○ Cloud Run: codelabs.developers.google.com/codelabs/cloud-run-hello-python3
○ Others: g.co/codelabs (all Google codelabs) and g.co/codelabs/cloud (GCP-only)
● Videos: youtube.com/GoogleCloudPlatform (GCP) and goo.gl/JpBQ40 (GWS)
● Code samples: github.com/GoogleCloudPlatform (GCP) and github.com/googleworkspace (GWS)
● Cloud Free Trial (new users) and Always Free (daily/monthly tier) programs: cloud.google.com/free
● Know AWS/Azure? Compare with GCP products at cloud.google.com/docs/compare/aws
● Language support: cloud.google.com/{python,java,nodejs,go,php,ruby,dotnet}
Resources (industry)
● Google Cloud (GCP)
○ Code samples for students: goo.gle/hackathon-toolkit
○ Videos: youtube.com/GoogleCloudPlatform
○ Codelabs (free, self-paced, hands-on tutorials): g.co/codelabs/cloud
○ Cloud Skills Boost/QwikLabs (free only for students): go.qwiklabs.com/hey-student
● GCP docs/quickstarts - cloud.google.com/{docs,appengine,functions,run,vision,automl,translate,
language, speech,texttospeech,video-intelligence,firestore,bigquery,compute,storage,gpu,tpu}
● Like GCP? Wanna use it in class or your research lab? Send your profs to cloud.google.com/edu to
apply for GCP teaching/research or Cloud Skills Boost/Qwiklabs/training credits!
● Know AWS/Azure? Compare w/GCP at cloud.google.com/docs/compare/aws
● GCP Free Trial (ignore) and Always Free (tier) - cloud.google.com/free
● Mobile apps: Firebase - firebase.google.com and Flutter - flutter.dev
● Workspace - developers.google.com/{workspace,drive,docs,sheets,slides} and goo.gl/JpBQ40
● Other Google (non-GCP) codelabs: g.co/codelabs
● Developer Student Clubs - g.co/dev/dsc - Global Google student developer groups
● Internships & other career stuff - google.com/students
Resources (students)
37. Prepare for careers in a cloud-first
world. Ask your faculty to apply now.
g.co/edu/cloudready
Career Readiness
track: certification
On-demand
Get free access to Architecting
with Google Compute Engine
specialization on-demand on
Coursera.
(100% discount)
Self-paced labs
Get free access to relevant
Google Cloud quests & other
Cloud Skills Boost self-paced
labs powered by Qwiklabs.
(100% discount: 10+ labs)
Google Cloud Certified
Prepare to take the exam and
earn your Cloud certification.
(100% discount for faculty, 50% discount for
students per adherence to timelines and
other criteria)
Careers at Google
Software Engineer (SWE): not the only technical role @Google
● Application Engineer
● Business Analyst
● Developer Advocate (DA)
● Developer Programs Engineer (DPE)
● Hardware Engineer
● Mechanical Engineer
● Network Engineer
● Product Manager (PM)
● Program Manager (PgM or TPM)
● Quantitative Analyst (Quant)
● Release Engineer
● Research Scientist
● Security Engineer
● Site Reliability Engineer (SRE)
● Software Engineer in Test (SET)
● System Administrator
● Technical Writer (TW)
● UX Content Strategist
● UX Designer
● UX Engineer
● Student careers: google.com/students
● Technical development guide: g.co/techdevguide
● Sample technical interview
38. Bring me to your organization
... it is my job to help you!
● "Transfer of Info" tech talks
● Half- or full-day seminars
● Hands-on "codelab" workshops
● Multi-day training courses
● Engineering consulting
● Migration strategy & planning
● cyberwebconsulting.com
● appenginemigration.com
Slides
you're looking
at them now
Work
cyberwebconsulting.com
Books
corepython.com
Blog
dev.to/wescpy
Img proc post
goo.gle/3nPxmlc
GCP+GWS 101 (2019)
youtu.be/ri8Bfptgo9Q
...& 102 talks (2023)
youtu.be/3IQ4Yv80lJg
Progress bars
goo.gl/69EJVw
Thank you! Questions?
Wesley Chun
Principal Consultant, CyberWeb
Python, GCP & GWS specialist
@wescpy (Tw/X, SO, GH, IG, LI)