SlideShare a Scribd company logo
1 of 49
Download to read offline
A Quick Introduction to Google's Cloud
Technologies
Chris Schalk
Developer Advocate



@cschalk
Agenda

● Introduction

● Introduction to Google's Cloud Technologies

● App Engine Review

● Google's new Cloud Technologies
   ○ Google Storage
   ○ Prediction API
   ○ BigQuery

● Summary Q&A
Google's Cloud Technologies




                 Google App Engine



            Google                 Google
           BigQuery             Prediction API



                      Google
                      Storage
Google App Engine




A quick look at App Engine...
Cloud Development in a Box

● Downloadable SDK
● Application runtimes
    ○ Java, Python
● Local development tools
    ○ Eclipse plugin,
      AppEngine Launcher
● Specialized application
  services
● Cloud based dashboard
● Ready to scale
● Built in fault tolerance, load
  balancing
Quick App Engine Demo!
Specialized Services


Memcache         Datastore    URL Fetch




Mail             XMPP         Task Queue




Images           Blobstore    User Service

                             But, is that it?
No!
Now App Engine has access to all the
new Google Cloud Services...
Google's new Cloud Technologies
New Google Cloud Technologies


 ● Google Storage
    ○ Store your data in Google's cloud

 ● Prediction API
    ○ Google's machine learning tech in an API

 ● BigQuery
    ○ Hi-speed data analysis on massive scale

 ● ...
Google Storage for Developers
       Store your data in Google's cloud
What Is Google Storage?



 ● Store your data in Google's cloud
    ○ any format, any amount, any time

 ● You control access to your data
    ○ private, shared, or public

 ● Access via Google APIs or 3rd party tools/libraries
Sample Use Cases

 Static content hosting
 e.g. static html, images, music, video

 Backup and recovery
 e.g. personal data, business records

 Sharing
 e.g. share data with your customers

 Data storage for applications
 e.g. used as storage backend for Android, AppEngine, Cloud
 based apps

 Storage for Computation
 e.g. BigQuery, Prediction API
Google Storage Benefits


             High Performance and Scalability
             Backed by Google infrastructure




               Strong Security and Privacy
               Control access to your data



           Easy to Use
           Get started fast with Google & 3rd party tools
Google Storage Technical Details

 ● RESTful API
    ○ Verbs: GET, PUT, POST, HEAD, DELETE
    ○ Resources: identified by URI
    ○ Compatible with S3

 ● Buckets
    ○ Flat containers

 ● Objects
    ○ Any type
    ○ Size: 100 GB / object

 ● Access Control for Google Accounts
    ○ For individuals and groups
 ● Two Ways to Authenticate Requests
    ○ Sign request using access keys
    ○ Web browser login
Demo


● Tools:
   ○ GSUtil
   ○ GS Manager

● Upload / Download
Google Storage usage within Google



            Google                        Google
           BigQuery                    Prediction API




                                Haiti Relief Imagery      USPTO data




                Partner Reporting     Partner Reporting
Some Early Google Storage Adopters
Google Storage - Pricing
    ○ Free trial quota until Dec 31, 2011
        ■ For first project
        ■ 5 GB of storage
        ■ 25 GB download/upload data
            ■ 20 GB to Americas/EMEA, 5GB APAC
        ■ 25K GET, HEAD requests
        ■ 2,5K PUT, POST, LIST* requests

    ○ Production Storage
        ■ $0.17/GB/Month (Location US, EU)
        ■ Upload - $0.10/GB
        ■ Download
            ■ $0.15/GB Americas / EMEA
            ■ $0.30/GB APAC
        ■ Requests
            ■ PUT, POST, LIST - $0.01 / 1000 Requests
            ■ GET, HEAD - $0.01 / 10,000 Requests
        ■ Up to 99.9% SLA
Google Storage Summary


 ● Store any kind of data using Google's cloud infrastructure

 ● Easy to Use APIs

 ● Many available tools and libraries
    ○ gsutil, GS Manager
    ○ 3rd party:
        ■ Boto, CloudBerry, CyberDuck, JetS3t, and more
Google Prediction API
Google's prediction engine in the cloud
Google Prediction API as a simple example



      Predicts outcomes based on 'learned' patterns
How does it work?

                     "english" The quick brown fox jumped over the
The Prediction API             lazy dog.
finds relevant
                     "english" To err is human, but to really foul things
features in the                up you need a computer.
sample data during   "spanish" No hay mal que por bien no venga.
training.
                     "spanish" La tercera es la vencida.


The Prediction API
later searches for   ?          To be or not to be, that is the
                                question.
those features
                     ?          La fe mueve montañas.
during prediction.
A virtually endless number of applications...


 Customer    Transaction         Species           Message     Diagnostics
 Sentiment      Risk           Identification      Routing




  Churn      Legal Docket      Suspicious       Work Roster    Inappropriate
Prediction   Classification     Activity        Assignment        Content




Recommend      Political         Uplift             Email        Career
 Products       Bias            Marketing          Filtering   Counselling

                           ... and many more ...
Using the Prediction API

A simple three step process...


                                 Upload your training data to
              1. Upload          Google Storage




                                 Build a model from your data
              2. Train




              3. Predict         Make new predictions
Step 1: Upload
 Upload your training data to Google Storage
● Training data: outputs and input features
● Data format: comma separated value format (CSV)

   "english","To err is human, but to really ..."
   "spanish","No hay mal que por bien no venga."
   ...

   Upload to Google Storage
   gsutil cp ${data} gs://yourbucket/${data}
Step 2: Train
Create a new model by training on data

To train a model:

POST prediction/v1.3/training
{"id":"mybucket/mydata"}
Training runs asynchronously. To see if it has finished:

GET prediction/v1.3/training/mybucket%2Fmydata

{"kind": "prediction#training",...
,"training status": "DONE"}
Step 3: Predict
 Apply the trained model to make predictions on new data
POST
prediction/v1.3/training/mybucket%2Fmydata/predict
{ "data":{
   "input": { "text" : [
    "J'aime X! C'est le meilleur" ]}}}
Step 3: Predict
   Apply the trained model to make predictions on new data
POST prediction/v1.3/training/bucket%2Fdata/predict

{ "data":{
   "input": { "text" : [
    "J'aime X! C'est le meilleur" ]}}}

{ data : {
 "kind" : "prediction#output",
 "outputLabel":"French",
 "outputMulti" :[
   {"label":"French", "score": x.xx}
   {"label":"English", "score": x.xx}
   {"label":"Spanish", "score": x.xx}]}}
Step 3: Predict
   Apply the trained model to make predictions on new data

import httplib

header = {"Content-Type" : "application/json"}#...put new data in JSON
format in params variable
conn = httplib.HTTPConnection("www.googleapis.com")conn.request
("POST",
 "/prediction/v1.3/query/bucket%2Fdata/predict", params, header)print
conn.getresponse()
Google Prediction API Demo!


 ● Command line Demos
    ○ Training a model
    ○ Checking training status
    ○ Making predictions


 ● A complete Web application using the JavaScript
   API for Prediction
Prediction API Capabilities
Data
 ● Input Features: numeric or unstructured text
 ● Output: up to hundreds of discrete categories

Training
 ● Many machine learning techniques
 ● Automatically selected
 ● Performed asynchronously

Access from many platforms:
 ● Web app from Google App Engine
 ● Apps Script (e.g. from Google Spreadsheet)
 ● Desktop app
Prediction API - key features



 ● Multi-category prediction
    ○ Tag entry with multiple labels

 ● Continuous Output
    ○ Finer grained prediction rankings based on multiple labels

 ● Mixed Inputs
    ○ Both numeric and text inputs are now supported


Can combine continuous output with mixed inputs
Google BigQuery
Interactive analysis of large datasets in Google's cloud
Introducing Google BigQuery


 ● Google's large data adhoc analysis technology
    ○ Analyze massive amounts of data in seconds

 ● Simple SQL-like query language

 ● Flexible access
     ○ REST APIs, JSON-RPC, Google Apps Script
Why BigQuery?
Working with large data is a challenge
Many Use Cases ...




    Interactive                                 Trends
                               Spam
       Tools                                   Detection




                     Web               Network
                  Dashboards          Optimization
Key Capabilities of BigQuery

 ● Scalable: Billions of rows

 ● Fast: Response in seconds

 ● Simple: Queries in SQL

 ● Web Service
    ○ REST
    ○ JSON-RPC
    ○ Google App Scripts
Using BigQuery

Another simple three step process...


                                   Upload your raw data to
              1. Upload            Google Storage




                                   Import raw data into
              2. Import
                                   BigQuery table



              3. Query             Perform SQL queries on
                                   table
Writing Queries

Compact subset of SQL
   ○ SELECT ... FROM ...
     WHERE ...
     GROUP BY ... ORDER BY ...
     LIMIT ...;

Common functions
   ○ Math, String, Time, ...

Statistical approximations
     ○ TOP
     ○ COUNT DISTINCT
BigQuery via REST
GET /bigquery/v1/tables/{table name}

GET /bigquery/v1/query?q={query}
Sample JSON Reply:
{
    "results": {
      "fields": { [
       {"id":"COUNT(*)","type":"uint64"}, ... ]
      },
      "rows": [
       {"f":[{"v":"2949"}, ...]},
       {"f":[{"v":"5387"}, ...]}, ... ]
    }
}
Also supports JSON-RPC
Security and Privacy

Standard Google Authentication
 ● Client Login
 ● OAuth
 ● AuthSub

HTTPS support
 ● protects your credentials
 ● protects your data

Relies on Google Storage to manage access
Large Data Analysis Example
Wikimedia Revision History




Wikimedia Revision history data from: http://download.wikimedia.
org/enwiki/latest/enwiki-latest-pages-meta-history.xml.7z
Using BigQuery Shell
Python DB API 2.0 + B. Clapper's sqlcmd
http://www.clapper.org/software/python/sqlcmd/
BigQuery from a Spreadsheet
BigQuery from a Spreadsheet
Recap

  ● Google App Engine
     ○ Application development platform for the
       cloud

  ● Google Storage
     ○ High speed cloud data storage on Google's
       infrastructure

  ● Prediction API
     ○ Google's machine learning technology able to
       predict outcomes based on sample data

  ● BigQuery
     ○ Interactive analysis of very large data sets
     ○ Simple SQL query language access
Further info available at:

● Google App Engine
   ○ http://code.google.com/appengine

● Google Storage for Developers
   ○ http://code.google.com/apis/storage

● Prediction API
   ○ http://code.google.com/apis/predict

● BigQuery
   ○ http://code.google.com/apis/bigquery
Thank you!



Questions?
 ● @cschalk

More Related Content

What's hot

Connecta Event: Big Query och dataanalys med Google Cloud Platform
Connecta Event: Big Query och dataanalys med Google Cloud PlatformConnecta Event: Big Query och dataanalys med Google Cloud Platform
Connecta Event: Big Query och dataanalys med Google Cloud Platform
ConnectaDigital
 

What's hot (20)

Big Query Basics
Big Query BasicsBig Query Basics
Big Query Basics
 
Google BigQuery - Features & Benefits
Google BigQuery - Features & BenefitsGoogle BigQuery - Features & Benefits
Google BigQuery - Features & Benefits
 
Google Cloud Next 2021 Recap
 Google Cloud Next 2021 Recap Google Cloud Next 2021 Recap
Google Cloud Next 2021 Recap
 
Connecta Event: Big Query och dataanalys med Google Cloud Platform
Connecta Event: Big Query och dataanalys med Google Cloud PlatformConnecta Event: Big Query och dataanalys med Google Cloud Platform
Connecta Event: Big Query och dataanalys med Google Cloud Platform
 
StackEngine Demo - Docker Austin
StackEngine Demo - Docker AustinStackEngine Demo - Docker Austin
StackEngine Demo - Docker Austin
 
Big Query - Utilizing Google Data Warehouse for Media Analytics
Big Query - Utilizing Google Data Warehouse for Media AnalyticsBig Query - Utilizing Google Data Warehouse for Media Analytics
Big Query - Utilizing Google Data Warehouse for Media Analytics
 
Using Google Cloud for Marketing Analytics: How the7stars, the UK’s largest i...
Using Google Cloud for Marketing Analytics: How the7stars, the UK’s largest i...Using Google Cloud for Marketing Analytics: How the7stars, the UK’s largest i...
Using Google Cloud for Marketing Analytics: How the7stars, the UK’s largest i...
 
Google App Engine 7 9-14
Google App Engine 7 9-14Google App Engine 7 9-14
Google App Engine 7 9-14
 
Un orquestador en la nube: Azure Data Factory (por Carlos Sacristán)
Un orquestador en la nube: Azure Data Factory (por Carlos Sacristán)Un orquestador en la nube: Azure Data Factory (por Carlos Sacristán)
Un orquestador en la nube: Azure Data Factory (por Carlos Sacristán)
 
Google Cloud Data Platform - Why Google for Data Analysis?
Google Cloud Data Platform - Why Google for Data Analysis?Google Cloud Data Platform - Why Google for Data Analysis?
Google Cloud Data Platform - Why Google for Data Analysis?
 
Google BigQuery
Google BigQueryGoogle BigQuery
Google BigQuery
 
Google Cloud Platform at Vente-Exclusive.com
Google Cloud Platform at Vente-Exclusive.comGoogle Cloud Platform at Vente-Exclusive.com
Google Cloud Platform at Vente-Exclusive.com
 
How to Choose a Data Warehouse
How to Choose a Data WarehouseHow to Choose a Data Warehouse
How to Choose a Data Warehouse
 
DevTest Labs en Azure (por Iván Cañizares)
DevTest Labs en Azure (por Iván Cañizares)DevTest Labs en Azure (por Iván Cañizares)
DevTest Labs en Azure (por Iván Cañizares)
 
Reach New Heights with Amazon Redshift
Reach New Heights with Amazon RedshiftReach New Heights with Amazon Redshift
Reach New Heights with Amazon Redshift
 
Google Cloud Platform & rockPlace Big Data Event-Mar.31.2016
Google Cloud Platform & rockPlace Big Data Event-Mar.31.2016Google Cloud Platform & rockPlace Big Data Event-Mar.31.2016
Google Cloud Platform & rockPlace Big Data Event-Mar.31.2016
 
Google BigQuery is the future of Analytics! (Google Developer Conference)
Google BigQuery is the future of Analytics! (Google Developer Conference)Google BigQuery is the future of Analytics! (Google Developer Conference)
Google BigQuery is the future of Analytics! (Google Developer Conference)
 
Pick a Winner: How to Choose a Data Warehouse
Pick a Winner: How to Choose a Data WarehousePick a Winner: How to Choose a Data Warehouse
Pick a Winner: How to Choose a Data Warehouse
 
Introduction to Big Data using AWS Services
Introduction to Big Data using AWS ServicesIntroduction to Big Data using AWS Services
Introduction to Big Data using AWS Services
 
An indepth look at Google BigQuery Architecture by Felipe Hoffa of Google
An indepth look at Google BigQuery Architecture by Felipe Hoffa of GoogleAn indepth look at Google BigQuery Architecture by Felipe Hoffa of Google
An indepth look at Google BigQuery Architecture by Felipe Hoffa of Google
 

Viewers also liked

App engine devfest_mexico_10
App engine devfest_mexico_10App engine devfest_mexico_10
App engine devfest_mexico_10
Chris Schalk
 
What is Google App Engine
What is Google App EngineWhat is Google App Engine
What is Google App Engine
Chris Schalk
 
Clase 6 alumnos con necesidades educativas especiales
Clase 6 alumnos con necesidades educativas especialesClase 6 alumnos con necesidades educativas especiales
Clase 6 alumnos con necesidades educativas especiales
YaribelDominguez1016
 
la falta de mkt en los lugare capitals emblemáticos de san luis potosi
la falta de mkt en los lugare capitals emblemáticos de san luis potosila falta de mkt en los lugare capitals emblemáticos de san luis potosi
la falta de mkt en los lugare capitals emblemáticos de san luis potosi
Alexis Ruiz de Montalvo
 

Viewers also liked (20)

Intro to Google's Cloud Technologies
Intro to Google's Cloud TechnologiesIntro to Google's Cloud Technologies
Intro to Google's Cloud Technologies
 
Google App Engine's Latest Features
Google App Engine's Latest FeaturesGoogle App Engine's Latest Features
Google App Engine's Latest Features
 
App engine devfest_mexico_10
App engine devfest_mexico_10App engine devfest_mexico_10
App engine devfest_mexico_10
 
Building AJAX Mashups with Google APIs
Building AJAX Mashups with Google APIsBuilding AJAX Mashups with Google APIs
Building AJAX Mashups with Google APIs
 
Open Social Intro Facebookmeetup
Open Social Intro FacebookmeetupOpen Social Intro Facebookmeetup
Open Social Intro Facebookmeetup
 
Devfest09 App Engine Java
Devfest09  App Engine  JavaDevfest09  App Engine  Java
Devfest09 App Engine Java
 
A technical overview of OpenSocial
A technical overview of OpenSocialA technical overview of OpenSocial
A technical overview of OpenSocial
 
Introduction to Google Cloud platform technologies
Introduction to Google Cloud platform technologiesIntroduction to Google Cloud platform technologies
Introduction to Google Cloud platform technologies
 
Devfest09 OpenSocial Enterprise
Devfest09 OpenSocial EnterpriseDevfest09 OpenSocial Enterprise
Devfest09 OpenSocial Enterprise
 
App Engine Overview @ Google Hackathon SXSW 2010
App Engine Overview @ Google Hackathon SXSW 2010App Engine Overview @ Google Hackathon SXSW 2010
App Engine Overview @ Google Hackathon SXSW 2010
 
Intro to Google's AJAX apis @ Rich Web Experience East 08
Intro to Google's AJAX apis @ Rich Web Experience East 08Intro to Google's AJAX apis @ Rich Web Experience East 08
Intro to Google's AJAX apis @ Rich Web Experience East 08
 
What is Google App Engine
What is Google App EngineWhat is Google App Engine
What is Google App Engine
 
Building Integrated Applications on Google's Cloud Technologies
Building Integrated Applications on Google's Cloud TechnologiesBuilding Integrated Applications on Google's Cloud Technologies
Building Integrated Applications on Google's Cloud Technologies
 
Building Kick Ass Video Games for the Cloud
Building Kick Ass Video Games for the CloudBuilding Kick Ass Video Games for the Cloud
Building Kick Ass Video Games for the Cloud
 
Building Integrated Applications on Google's Cloud Technologies
Building Integrated Applications on Google's Cloud TechnologiesBuilding Integrated Applications on Google's Cloud Technologies
Building Integrated Applications on Google's Cloud Technologies
 
Google App Engine Overview and Update
Google App Engine Overview and UpdateGoogle App Engine Overview and Update
Google App Engine Overview and Update
 
Spanish iii unit 3 prj.
Spanish iii unit 3 prj.Spanish iii unit 3 prj.
Spanish iii unit 3 prj.
 
Clase 6 alumnos con necesidades educativas especiales
Clase 6 alumnos con necesidades educativas especialesClase 6 alumnos con necesidades educativas especiales
Clase 6 alumnos con necesidades educativas especiales
 
la falta de mkt en los lugare capitals emblemáticos de san luis potosi
la falta de mkt en los lugare capitals emblemáticos de san luis potosila falta de mkt en los lugare capitals emblemáticos de san luis potosi
la falta de mkt en los lugare capitals emblemáticos de san luis potosi
 
AIAMN_LL_Award_20161 (1)
AIAMN_LL_Award_20161 (1)AIAMN_LL_Award_20161 (1)
AIAMN_LL_Award_20161 (1)
 

Similar to Quick Intro to Google Cloud Technologies

Google cloud platform
Google cloud platformGoogle cloud platform
Google cloud platform
rajdeep
 

Similar to Quick Intro to Google Cloud Technologies (20)

Introduction to Google's Cloud Technologies
Introduction to Google's Cloud TechnologiesIntroduction to Google's Cloud Technologies
Introduction to Google's Cloud Technologies
 
Building Apps on Google Cloud Technologies
Building Apps on Google Cloud TechnologiesBuilding Apps on Google Cloud Technologies
Building Apps on Google Cloud Technologies
 
Intro to new Google cloud technologies: Google Storage, Prediction API, BigQuery
Intro to new Google cloud technologies: Google Storage, Prediction API, BigQueryIntro to new Google cloud technologies: Google Storage, Prediction API, BigQuery
Intro to new Google cloud technologies: Google Storage, Prediction API, BigQuery
 
Google cloud platform
Google cloud platformGoogle cloud platform
Google cloud platform
 
Google Cloud Platform Update
Google Cloud Platform UpdateGoogle Cloud Platform Update
Google Cloud Platform Update
 
Introduction to Google Cloud Platform Technologies
Introduction to Google Cloud Platform TechnologiesIntroduction to Google Cloud Platform Technologies
Introduction to Google Cloud Platform Technologies
 
Exploring Google APIs with Python
Exploring Google APIs with PythonExploring Google APIs with Python
Exploring Google APIs with Python
 
Google Cloud for Data Crunchers - Strata Conf 2011
Google Cloud for Data Crunchers - Strata Conf 2011Google Cloud for Data Crunchers - Strata Conf 2011
Google Cloud for Data Crunchers - Strata Conf 2011
 
Image archive, analysis & report generation with Google Cloud
Image archive, analysis & report generation with Google CloudImage archive, analysis & report generation with Google Cloud
Image archive, analysis & report generation with Google Cloud
 
Entrepreneurship Tips With HTML5 & App Engine Startup Weekend (June 2012)
Entrepreneurship Tips With HTML5 & App Engine Startup Weekend (June 2012)Entrepreneurship Tips With HTML5 & App Engine Startup Weekend (June 2012)
Entrepreneurship Tips With HTML5 & App Engine Startup Weekend (June 2012)
 
Exploring Google APIs with Python
Exploring Google APIs with PythonExploring Google APIs with Python
Exploring Google APIs with Python
 
Powerful Google developer tools for immediate impact! (2023-24 A)
Powerful Google developer tools for immediate impact! (2023-24 A)Powerful Google developer tools for immediate impact! (2023-24 A)
Powerful Google developer tools for immediate impact! (2023-24 A)
 
Computing at scale
Computing at scaleComputing at scale
Computing at scale
 
GDD Brazil 2010 - Google Storage, Bigquery and Prediction APIs
GDD Brazil 2010 - Google Storage, Bigquery and Prediction APIsGDD Brazil 2010 - Google Storage, Bigquery and Prediction APIs
GDD Brazil 2010 - Google Storage, Bigquery and Prediction APIs
 
Easy path to machine learning (Spring 2020)
Easy path to machine learning (Spring 2020)Easy path to machine learning (Spring 2020)
Easy path to machine learning (Spring 2020)
 
Modern Thinking área digital MSKM 21/09/2017
Modern Thinking área digital MSKM 21/09/2017Modern Thinking área digital MSKM 21/09/2017
Modern Thinking área digital MSKM 21/09/2017
 
Easy path to machine learning (2022)
Easy path to machine learning (2022)Easy path to machine learning (2022)
Easy path to machine learning (2022)
 
Accessing Google Cloud APIs
Accessing Google Cloud APIsAccessing Google Cloud APIs
Accessing Google Cloud APIs
 
Scale with a smile with Google Cloud Platform At DevConTLV (June 2014)
Scale with a smile with Google Cloud Platform At DevConTLV (June 2014)Scale with a smile with Google Cloud Platform At DevConTLV (June 2014)
Scale with a smile with Google Cloud Platform At DevConTLV (June 2014)
 
Google App Engine – niekonwencjonalna platforma aplikacji SaaS do Twojego nas...
Google App Engine – niekonwencjonalna platforma aplikacji SaaS do Twojego nas...Google App Engine – niekonwencjonalna platforma aplikacji SaaS do Twojego nas...
Google App Engine – niekonwencjonalna platforma aplikacji SaaS do Twojego nas...
 

More from Chris Schalk

Javaedge 2010-cschalk
Javaedge 2010-cschalkJavaedge 2010-cschalk
Javaedge 2010-cschalk
Chris Schalk
 
App Engine Presentation @ SFJUG Sep 2010
App Engine Presentation @ SFJUG Sep 2010App Engine Presentation @ SFJUG Sep 2010
App Engine Presentation @ SFJUG Sep 2010
Chris Schalk
 

More from Chris Schalk (15)

How to build Kick Ass Games in the Cloud
How to build Kick Ass Games in the CloudHow to build Kick Ass Games in the Cloud
How to build Kick Ass Games in the Cloud
 
GDD 2011 - How to build kick ass video games for the cloud
GDD 2011 - How to build kick ass video games for the cloudGDD 2011 - How to build kick ass video games for the cloud
GDD 2011 - How to build kick ass video games for the cloud
 
Google App Engine's Latest Features
Google App Engine's Latest FeaturesGoogle App Engine's Latest Features
Google App Engine's Latest Features
 
Building Multi-platform Video Games for the Cloud
Building Multi-platform Video Games for the CloudBuilding Multi-platform Video Games for the Cloud
Building Multi-platform Video Games for the Cloud
 
Building Enterprise Applications on Google Cloud Platform Cloud Computing Exp...
Building Enterprise Applications on Google Cloud Platform Cloud Computing Exp...Building Enterprise Applications on Google Cloud Platform Cloud Computing Exp...
Building Enterprise Applications on Google Cloud Platform Cloud Computing Exp...
 
Introduction to Google's Cloud Technologies
Introduction to Google's Cloud TechnologiesIntroduction to Google's Cloud Technologies
Introduction to Google's Cloud Technologies
 
Javaedge 2010-cschalk
Javaedge 2010-cschalkJavaedge 2010-cschalk
Javaedge 2010-cschalk
 
Google Cloud Technologies Overview
Google Cloud Technologies OverviewGoogle Cloud Technologies Overview
Google Cloud Technologies Overview
 
Introducing App Engine for Business
Introducing App Engine for BusinessIntroducing App Engine for Business
Introducing App Engine for Business
 
Google App Engine for Business 101
Google App Engine for Business 101Google App Engine for Business 101
Google App Engine for Business 101
 
What's new in App Engine and intro to App Engine for Business
What's new in App Engine and intro to App Engine for BusinessWhat's new in App Engine and intro to App Engine for Business
What's new in App Engine and intro to App Engine for Business
 
App Engine Presentation @ SFJUG Sep 2010
App Engine Presentation @ SFJUG Sep 2010App Engine Presentation @ SFJUG Sep 2010
App Engine Presentation @ SFJUG Sep 2010
 
App engine cloud_comp_expo_nyc
App engine cloud_comp_expo_nycApp engine cloud_comp_expo_nyc
App engine cloud_comp_expo_nyc
 
App Engine Overview Cloud Futures Publish
App Engine Overview Cloud Futures PublishApp Engine Overview Cloud Futures Publish
App Engine Overview Cloud Futures Publish
 
Google App Engine and Social Apps
Google App Engine and Social AppsGoogle App Engine and Social Apps
Google App Engine and Social Apps
 

Recently uploaded

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc
 

Recently uploaded (20)

UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
الأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهالأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهله
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDM
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Quantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation ComputingQuantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation Computing
 
API Governance and Monetization - The evolution of API governance
API Governance and Monetization -  The evolution of API governanceAPI Governance and Monetization -  The evolution of API governance
API Governance and Monetization - The evolution of API governance
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
Modernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using BallerinaModernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using Ballerina
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
 

Quick Intro to Google Cloud Technologies

  • 1. A Quick Introduction to Google's Cloud Technologies Chris Schalk Developer Advocate @cschalk
  • 2. Agenda ● Introduction ● Introduction to Google's Cloud Technologies ● App Engine Review ● Google's new Cloud Technologies ○ Google Storage ○ Prediction API ○ BigQuery ● Summary Q&A
  • 3. Google's Cloud Technologies Google App Engine Google Google BigQuery Prediction API Google Storage
  • 4. Google App Engine A quick look at App Engine...
  • 5. Cloud Development in a Box ● Downloadable SDK ● Application runtimes ○ Java, Python ● Local development tools ○ Eclipse plugin, AppEngine Launcher ● Specialized application services ● Cloud based dashboard ● Ready to scale ● Built in fault tolerance, load balancing
  • 7. Specialized Services Memcache Datastore URL Fetch Mail XMPP Task Queue Images Blobstore User Service But, is that it?
  • 8. No! Now App Engine has access to all the new Google Cloud Services...
  • 9. Google's new Cloud Technologies
  • 10. New Google Cloud Technologies ● Google Storage ○ Store your data in Google's cloud ● Prediction API ○ Google's machine learning tech in an API ● BigQuery ○ Hi-speed data analysis on massive scale ● ...
  • 11. Google Storage for Developers Store your data in Google's cloud
  • 12. What Is Google Storage? ● Store your data in Google's cloud ○ any format, any amount, any time ● You control access to your data ○ private, shared, or public ● Access via Google APIs or 3rd party tools/libraries
  • 13. Sample Use Cases Static content hosting e.g. static html, images, music, video Backup and recovery e.g. personal data, business records Sharing e.g. share data with your customers Data storage for applications e.g. used as storage backend for Android, AppEngine, Cloud based apps Storage for Computation e.g. BigQuery, Prediction API
  • 14. Google Storage Benefits High Performance and Scalability Backed by Google infrastructure Strong Security and Privacy Control access to your data Easy to Use Get started fast with Google & 3rd party tools
  • 15. Google Storage Technical Details ● RESTful API ○ Verbs: GET, PUT, POST, HEAD, DELETE ○ Resources: identified by URI ○ Compatible with S3 ● Buckets ○ Flat containers ● Objects ○ Any type ○ Size: 100 GB / object ● Access Control for Google Accounts ○ For individuals and groups ● Two Ways to Authenticate Requests ○ Sign request using access keys ○ Web browser login
  • 16. Demo ● Tools: ○ GSUtil ○ GS Manager ● Upload / Download
  • 17. Google Storage usage within Google Google Google BigQuery Prediction API Haiti Relief Imagery USPTO data Partner Reporting Partner Reporting
  • 18. Some Early Google Storage Adopters
  • 19. Google Storage - Pricing ○ Free trial quota until Dec 31, 2011 ■ For first project ■ 5 GB of storage ■ 25 GB download/upload data ■ 20 GB to Americas/EMEA, 5GB APAC ■ 25K GET, HEAD requests ■ 2,5K PUT, POST, LIST* requests ○ Production Storage ■ $0.17/GB/Month (Location US, EU) ■ Upload - $0.10/GB ■ Download ■ $0.15/GB Americas / EMEA ■ $0.30/GB APAC ■ Requests ■ PUT, POST, LIST - $0.01 / 1000 Requests ■ GET, HEAD - $0.01 / 10,000 Requests ■ Up to 99.9% SLA
  • 20. Google Storage Summary ● Store any kind of data using Google's cloud infrastructure ● Easy to Use APIs ● Many available tools and libraries ○ gsutil, GS Manager ○ 3rd party: ■ Boto, CloudBerry, CyberDuck, JetS3t, and more
  • 21. Google Prediction API Google's prediction engine in the cloud
  • 22. Google Prediction API as a simple example Predicts outcomes based on 'learned' patterns
  • 23. How does it work? "english" The quick brown fox jumped over the The Prediction API lazy dog. finds relevant "english" To err is human, but to really foul things features in the up you need a computer. sample data during "spanish" No hay mal que por bien no venga. training. "spanish" La tercera es la vencida. The Prediction API later searches for ? To be or not to be, that is the question. those features ? La fe mueve montañas. during prediction.
  • 24. A virtually endless number of applications... Customer Transaction Species Message Diagnostics Sentiment Risk Identification Routing Churn Legal Docket Suspicious Work Roster Inappropriate Prediction Classification Activity Assignment Content Recommend Political Uplift Email Career Products Bias Marketing Filtering Counselling ... and many more ...
  • 25. Using the Prediction API A simple three step process... Upload your training data to 1. Upload Google Storage Build a model from your data 2. Train 3. Predict Make new predictions
  • 26. Step 1: Upload Upload your training data to Google Storage ● Training data: outputs and input features ● Data format: comma separated value format (CSV) "english","To err is human, but to really ..." "spanish","No hay mal que por bien no venga." ... Upload to Google Storage gsutil cp ${data} gs://yourbucket/${data}
  • 27. Step 2: Train Create a new model by training on data To train a model: POST prediction/v1.3/training {"id":"mybucket/mydata"} Training runs asynchronously. To see if it has finished: GET prediction/v1.3/training/mybucket%2Fmydata {"kind": "prediction#training",... ,"training status": "DONE"}
  • 28. Step 3: Predict Apply the trained model to make predictions on new data POST prediction/v1.3/training/mybucket%2Fmydata/predict { "data":{ "input": { "text" : [ "J'aime X! C'est le meilleur" ]}}}
  • 29. Step 3: Predict Apply the trained model to make predictions on new data POST prediction/v1.3/training/bucket%2Fdata/predict { "data":{ "input": { "text" : [ "J'aime X! C'est le meilleur" ]}}} { data : { "kind" : "prediction#output", "outputLabel":"French", "outputMulti" :[ {"label":"French", "score": x.xx} {"label":"English", "score": x.xx} {"label":"Spanish", "score": x.xx}]}}
  • 30. Step 3: Predict Apply the trained model to make predictions on new data import httplib header = {"Content-Type" : "application/json"}#...put new data in JSON format in params variable conn = httplib.HTTPConnection("www.googleapis.com")conn.request ("POST", "/prediction/v1.3/query/bucket%2Fdata/predict", params, header)print conn.getresponse()
  • 31. Google Prediction API Demo! ● Command line Demos ○ Training a model ○ Checking training status ○ Making predictions ● A complete Web application using the JavaScript API for Prediction
  • 32. Prediction API Capabilities Data ● Input Features: numeric or unstructured text ● Output: up to hundreds of discrete categories Training ● Many machine learning techniques ● Automatically selected ● Performed asynchronously Access from many platforms: ● Web app from Google App Engine ● Apps Script (e.g. from Google Spreadsheet) ● Desktop app
  • 33. Prediction API - key features ● Multi-category prediction ○ Tag entry with multiple labels ● Continuous Output ○ Finer grained prediction rankings based on multiple labels ● Mixed Inputs ○ Both numeric and text inputs are now supported Can combine continuous output with mixed inputs
  • 34. Google BigQuery Interactive analysis of large datasets in Google's cloud
  • 35. Introducing Google BigQuery ● Google's large data adhoc analysis technology ○ Analyze massive amounts of data in seconds ● Simple SQL-like query language ● Flexible access ○ REST APIs, JSON-RPC, Google Apps Script
  • 36. Why BigQuery? Working with large data is a challenge
  • 37. Many Use Cases ... Interactive Trends Spam Tools Detection Web Network Dashboards Optimization
  • 38. Key Capabilities of BigQuery ● Scalable: Billions of rows ● Fast: Response in seconds ● Simple: Queries in SQL ● Web Service ○ REST ○ JSON-RPC ○ Google App Scripts
  • 39. Using BigQuery Another simple three step process... Upload your raw data to 1. Upload Google Storage Import raw data into 2. Import BigQuery table 3. Query Perform SQL queries on table
  • 40. Writing Queries Compact subset of SQL ○ SELECT ... FROM ... WHERE ... GROUP BY ... ORDER BY ... LIMIT ...; Common functions ○ Math, String, Time, ... Statistical approximations ○ TOP ○ COUNT DISTINCT
  • 41. BigQuery via REST GET /bigquery/v1/tables/{table name} GET /bigquery/v1/query?q={query} Sample JSON Reply: { "results": { "fields": { [ {"id":"COUNT(*)","type":"uint64"}, ... ] }, "rows": [ {"f":[{"v":"2949"}, ...]}, {"f":[{"v":"5387"}, ...]}, ... ] } } Also supports JSON-RPC
  • 42. Security and Privacy Standard Google Authentication ● Client Login ● OAuth ● AuthSub HTTPS support ● protects your credentials ● protects your data Relies on Google Storage to manage access
  • 43. Large Data Analysis Example Wikimedia Revision History Wikimedia Revision history data from: http://download.wikimedia. org/enwiki/latest/enwiki-latest-pages-meta-history.xml.7z
  • 44. Using BigQuery Shell Python DB API 2.0 + B. Clapper's sqlcmd http://www.clapper.org/software/python/sqlcmd/
  • 45. BigQuery from a Spreadsheet
  • 46. BigQuery from a Spreadsheet
  • 47. Recap ● Google App Engine ○ Application development platform for the cloud ● Google Storage ○ High speed cloud data storage on Google's infrastructure ● Prediction API ○ Google's machine learning technology able to predict outcomes based on sample data ● BigQuery ○ Interactive analysis of very large data sets ○ Simple SQL query language access
  • 48. Further info available at: ● Google App Engine ○ http://code.google.com/appengine ● Google Storage for Developers ○ http://code.google.com/apis/storage ● Prediction API ○ http://code.google.com/apis/predict ● BigQuery ○ http://code.google.com/apis/bigquery