Gcp intro-20160721

Introduction
CDNetworks inc.
Product Manager / Haeseung Lee

before we start…
• Pros & Cons of Cloud Computing?

Pros of Cloud Computing
• say ‘goodbye’ to costly systems
• access from innumerable options
• software expense
• cooked food
• lower traditional servers’ cost
• data centralization
• data recovery
• sharing capabilities
• cloud security
• free cloud storage
• instantly test
Cost Saving
Reliability
Manageability
Strategic Edge = Edge Computing

Cons of Cloud Computing
• net connections
• low bandwidth
• affected quality
• security issues
• non-negotiable agreements
• cost comparison
• no hard drive
• lack of full support
• incompatiblity
• lack of insight into your network
• minimal flexibility
Downtime
Security and Privacy
Vulnerability to attack
Limited control and flexibility
Platform Dependency(aka ‘Lock in’)
Cloud Computing costs

Cloud Vendors? = Our Competitors!

Public Cloud Adoption
either or both Azure offerings. 7 percent are using Google App Engine and 6 percent are using
Google IaaS, while 10 percent are using either or both of Google IaaS/PaaS.

Private Cloud Adoption 2016 vs. 2015
©2016 RightScale, Inc. All rights reserved.
This work by RightScale is licensed under a Creative Commons Attribution 4.0 International License.
In comparison to last year, there were noticeable increases in adoption rates of every private cloud
technology. VMware vSphere/vCenter went up eleven percent (33 to 44 percent). OpenStack and
VMware vCloud Suite both increased six percent (13 to 19 percent).

Why choose Google Cloud Platform?
• enables developers to build, test and deploy
applications on Google’s highly-scalable, secure, and
reliable infrastructure.
• computing, storage, big data/machine learning, and
application services for web, mobile, analytics, and
backend solutions.

Google’s Infrastructure
. Google has been building for the past 15 years:
Massive, Powerful infrastructures such as datacenter and
high-speed fiber optic networks.
. Data Centers / Backbone / 70+ edge POPs in 33 Countries /
edge caching platform

Cloud Regions and Zones
• Regions
• Regions are specific geographical locations where you can run your
resources
• are collections of zones
• Regional resources are available to resources in any zones in the region
• Zones
• Isolated locations within a region
• Zonal resources are only available in that zone
• Multi-regional resources - below are one or more multi-regional deployment in
addition to any regional deployment
• Google App Engine and its feature
• Google Cloud Database
• Google Cloud Storage
• Google BigQuery

Innovative, Customer-Friendly Pricing
• Sub-Hour billing
• Sustained-use discounts
• automatically reward users who run virtual machines for over 25% of any
calendar month
• Compute Engine custom machine types
• pay only for the resources you need for your application
• online pricing calculator

©Google Inc. or its affiliates. All rights reserved. Do not distribute. 12
Image by Connie
Zhou
Next
Fully automated
Now
User managed, user
configured, user maintained
1st
wave:
Colocation
2nd
wave:
Virtualized
datacenters
3rd
wave:
A global,
elastic cloud
The Future of Cloud Computing
Notes:

©Google Inc. or its affiliates. All rights reserved. Do not distribute.
IaaS and PaaS
14
PaaS
Preset run-times
Java, Go, PHP, Python...
Focus is application logic
Pay for what you use
Less management overhead
IaaS
Raw compute, storage and
network
More granular control
Pay for what you allocate
More management overhead
Towards
managed
infrastructure
(DevOps)
Towards
managed
services
(NoOps)
Compute Engine App Engine

Google Cloud Platform
15
BigQuery
Big Data
Pub/Sub Dataflow Dataproc Datalab
Compute
Compute EngineApp Engine Container Engine
Storage
Cloud
Storage
Cloud SQL Cloud
Datastore
Bigtable
Machine Learning
Speech APIMachine
Learning
Translate APIVision API

Google Cloud Platform Console
• https://console.gloud.google.com

Projects
• All GCP services are associated
with a project that is used to:
• Track resources and quota ranges
• Enable billing
• Manage permissions and credentials
• Enable services and APIs
• Projects use three identifying
attributes:
• Project Name
• Project Number
• Project ID - aka Application ID
• Interact with projects using the
Cloud Console or the Cloud
Resource Manager APIAlpha

Project Permissions - primitive roles
Owner
Billing
Administrator
ViewerEditor
Invite members
Remove members
Can delete project
includes Editor
rights
Deploy applications
Modify code
Configure services
includes Viewer
rights
Read-only access Manage billing
Add administrators
Remove
administrators
A project can have multiple owners, editors, viewers and billing administrator.
- Primitive roles -above
- curated roles - new IAM roles that give finer-grained access control than the primitive
roles

Identity and Access Management
8
Who can do what on which resource

IAM roles - Curated Roles
IAM Roles - Curated Roles
IAM
✔ compute.instances.delete
✔ compute.instances.get
✔ compute.instances.list
✔ compute.instances.setMachineType
✔ compute.instances.start
✔ compute.instances.stop
. . .
List of Permissions
InstanceAdmin
Role
project_a
Google
Group

Service Accounts and IAM
• Service Accounts
authentication with keys
• google manage keys, key rotation for
Compute engine and App engine
• Can assign an IAM role to
the service account
• Can also assign
serviceAccountActor role
to users/groups
Service Accounts and IAM
14
● Service accounts
authenticate with keys
○ Google manages keys, key
rotation for Compute Engine and
App Engine
● Can assign an IAM role to the
service account
● Can also assign
ServiceAccountActor role to
users/groups
Service Account InstanceAdmin Role Compute Instances
Service Account
Identity IAM Role Resource
User/Group ServiceAccountActor Role
Notes:
Users require a username and password to authenticate. Apps use a key. One

Google App Engine and
Google Cloud Datastore

NetworkingCompute Big Data
Machine
Learning
Storage
Operations and
Tools
Compute
Engine
Container
Engine
App
Engine

What is Google App Engine
• A platform(platform as a service)
for building scalable web
applications and mobile backends
• App Engine makes deployment,
maintenance, and scalability easy
so you can focus on innovation

700million
“App Engine enabled us to focus on developing the
application. We wouldn’t have gotten here without the
ease of development that App Engine gave us.”
Bobby Murphy, CTO
Snapchat sends
photos and videos each day Google App Engine
scaled seamlessly
during growth to
millions of users
Small team is able
to innovate quickly
and expand
globally
Snapchat
App Engine - Best Practice

App Engine Standard Environment
• Managed runtimes for specific versions of Java,
Python, PHP & Go
• Autoscale workloads to meet demand
• Free daily quota, usage based pricing
• SDKs for development, testing and deployment
• Need to conform to sandbox constraints:
• no writing to local file system
• request timeouts at 60 seconds
• Limit on 3rd-party software installations

Example App Engine Standard Workflow
- Web Applications
10
Project
App Engine
App Servers
Application
instances
Application
instances
Application
instances
App Engine can access
a variety of services
using dedicated APIs
1 Develop & test the web
application locally
2
Use the SDK to deploy
to App Engine
3
App Engine automatically
scales & reliably serves
your web application
Task
queues
Scheduled
tasks
Search
Memcache
Logs
Notes:

App Engine Flexible EnvironmentBeta
• Build, deploy containerized apps with a click
• Standard runtimes - Python, Java, Go, Node.js 
- with no sandbox constraints
• Custom runtime support for any languages
that support HTTP requests
• During beta pricing based on Compute Engine
usage
• Local development relies on Docker
• Standard runtimes can access App Engine
services: Datastore, Memcache, task queues,
logging, users, and so on.

App Engine Standard vs Flexible Environment
Standard Environment Flexible Environment
Instance startup Milliseconds Minutes
SSH access No Yes (not default)
Scaling Manual, basic, automatic Manual, automatic
Write to local disk No Yes (ephemeral)
Support for 3rd party
binaries
No Yes
Network access Via App Engine services Yes
Customizable stack No Yes
18

Google Cloud Endpoints
• Build your own API running on App
Engine Standard
• Expose your API using a RESTful interface
• Includes support for OAuth 2.0
authorization
• Generate client libraries
• Support Java and Python server-side
code
• Includes App Engine features
• scaling
• denial of service protection
• high availability
• Support iOS, Android, and JavaScript
Clients

Compute
Machine
Learning
Storage
Operations and
Tools
Cloud
Storage
Cloud
SQL
Cloud
Datastore
Cloud
Bigtable
Networking Big Data

Google Cloud Datastore
• Database designed for application
backends
• NoSQL store for billions of rows
• Schemaless access, no need to think
about underlying data structure
• Local development tools
• Automatic scaling and fully managed
• Built-in redundancy
• Supports ACID transactions
• Includes a free daily quotas
• Access from anywhere through a
RESTful interface

Compute
Machine
Learning
Storage
Operations and
Tools
Cloud
Storage
Cloud
SQL
Cloud
Datastore
Cloud
Bigtable
Networking Big Data

Google Cloud Storage
• High performance, internet-scale,
immutable BLOB(binary large
object) storage
• Not a file system(but can be
accessed as one via 3rd-party
tools such as Cloud Storage
Fuse)
• Simple administration and does
not require capacity mgmt
• Data encryption in-flight and at
rest
• All storage classes accessed
through the same APIs

Cloud Storage Classes
Cloud Storage Classes
Standard
Storage
DRA
Storage
Nearline
Storage
Standard Storage
provides the highest
durability, availability
and performance with
low latency and is ideal
for use with website
content distribution and
video streaming
Durable Reduced
Availability Storage
offers the same
durability as Standard
Storage but with a lower
availability SLA at a
reduced cost
Nearline Storage
offers low-cost, highly
durable storage service
for data archiving, online
backup, and disaster
recovery, without having
to wait hours or days to
retrieve or access your
data
Each option comes with detailed pricing

Cloud Storage Integration
Cloud Storage Integration
1
Import
and
export
tables
Object
storage,
logs,
Datastore
backups
Startup
scripts,
images
and
general
object
storage
Import
and
export
tables
App
Engine
Compute
Engine
Cloud Storage
Cloud SQL
BigQuery

DEMO
1. Getting Started with GCP
2. Deploying Applications using Google App Engine and Cloud
Datastore

Google Cloud SQL
• Google-managed MySQL
relational database in the cloud
• Pay-per-Use model
• RESTful API for management
• Affordability and performance
• Good Security
• Vertical scaling(read and write)
• Horizontal scaling(read)
• Seamless integration with App
Engine and Compute Engine

Managed
backups
Google
security
Automatic
replication
Familiar
Flexible
pricing
Cloud SQL
Features

Cloud SQL Integration
Cloud SQL can be used with
App Engine using standard
drivers like Connector/J for
Java or MySQLdb for Python.
App Engine applications are
authorized to access Cloud
SQL, and the instance can be
configured to follow one
application.
Compute Engine instances
can be authorized to
access Cloud SQL
instances using an external
IP address.
Cloud SQL instances can
be configured with a
preferred zone - to stay
close to the Compute
Engine infrastructure.
Cloud SQL can be used with
external applications and clients
by authorizing IP addresses or
networks using CIDR notation.
Standard tools like MySQL
Workbench can be used to
administer databases. External
read replicas can be configured.
External
service

Google Cloud SQL second generationBeta
• Same features as first generation
with higher performance, storage
capacity at lower cost
• up to 7x throughput and 20x storage
capacity of first generation instances
• less expensive than first generation for
most use cases

Comparing Storage Options (1 of 2)
NoSQLBLOB SQL
Cloud
Storage
Cloud
Bigtable
Cloud
Datastore
Cloud SQL
Good for:
Structured and unstructured
binary or object data
Good for:
Getting started, App Engine
applications
Good for:
“Flat” data, Heavy read/write,
events, analytical data
Good for:
Web frameworks,
existing applications
Use cases:
Images, large media files,
backups
Use cases:
User profiles,
product catalog
Use cases:
AdTech, Financial and IoT
data
Use cases:
User credentials, customer
orders

Comparing Storage Options (2 of 2)
33
Cloud Datastore Cloud Storage Cloud SQL
(1st
Generation)
Bigtable
Storage type NoSQL,
document
Object (BLOB)
store
Relational SQL NoSQL, wide-column
Overall capacity Terabytes + Petabytes + up to 500 GB Petabytes +
Unit size 1 megabyte /
entity
5 TB / object Standard MySQL
limits
Recommended -
Individual values: ~10 MB per
cell
All values per row: ~100 MB
Transactions Yes No Yes No
Complex queries No No Yes No

Google Container Engine
Google Compute Engine

What is a Container?
• Virtualization at the
operating system layer
• Separates operating
system from application
code and dependencies
• Isolated individual
processes
• Popular implementations
include Docker and rkt

Why use Containers?
• Support consistency across
development, testing, and
production environments
• Loose coupling between
application and operating system
layers
• Much simpler to migrate
workloads between on-premises
and cloud environments
• Support agile development and
operations ©Google Inc. or its affiliates. All rights reserved. Do not distribute.
● Virtualization at the operating
system layer
● Separates operating system
from application code and
dependencies
● Isolates individual processes
● Popular implementations
include Docker and rkt
3
What is a Container?
Operating system
Shared libraries
Container
Code and libraries
Container
Code and libraries
Container
Code and libraries
Virtual Machine

Kubernetes(aka ‘k8s’)
• Open Source container cluster
orchestration system
• automates deployment,
scaling, and operations for
container clusters
• Based on Google’s experience
over 10+ years
• Built for a multi-cloud world:
• public, private, hybrid

Features of Kubernetes
• Workload portability
• run in many environments, across cloud providers
• implementation is open and modular
• Rolling updates
• upgrade application with zero downtime
• Autoscaling
• automatically adapt to changes in workload
• Persistent storage
• abstracts details of how storage is provided from how it is consumed
• Multi-Zone clusters
• run a single cluster in multiple zones
• Alpha
on Google Cloud Platform
• Load balancing
• external IP address routed traffic to correct port

NetworkingCompute Big Data
Machine
Learning
Storage
Operations and
Tools
Compute
Engine
Container
Engine
App
Engine

Google Container Engine(1/2)
• Fully managed cluster
management and orchestration
system for running containers
• based on Kubernetes
• uses Compute Engine instances and
resources
• Complimentary services:
• Google Cloud Container Builder Beta -
Create Docker container images from app
code in Google Cloud Storage
• Google Container Registry - Secure,
Private Docker image storage

Google Container Engine(2/2)
• Uses a declarative syntax to
manage applications
• declare desired application configuration,
Container Engine implements, manages
• Decouples operational,
development concerns
• Manage and maintains
• logging, health management, monitoring
• Easily update Kubernetes
versions as they are released

Container Engine=k8s :: concepts
- Nodes: node is working machine in a k8s cluster, and in Google Container
engine, the machine is always a Computer Engine Instance.
- Pods: Pod is a group of one or more containers, shared storage, and
configuration data relating to those containers. It is common for production
applications running in k8s to include multiple, relatively tightly-coupled
containers in a single pod.
- Replication Controllers: Replication Controller works to ensure that the
requested number of pod replicas are always available and running at a
given time. automatically adds or remove pods as required to maintain a
desired state.
- Services: defines a logical set of pods and a way to access them using an IP
address and port number pair.

Comparing Compute Options
9
Compute
Engine
Container
Engine
App Engine
Standard
App Engine
Flexible
Language
support
Any Any Java, Python,
Go & PHP
Any
Service model IaaS Hybrid PaaS PaaS
Primary use
case
General
computing
workloads
Container-based
workloads
Web and
mobile
applications
Web and mobile
applications,
container-based
workloads

Google Compute Engine(1/2)
• Run large-scale workloads on
virtual machines hosted on
Google’s infrastructure
• Robust networking features
• default, custom networks
• firewall rules
• regional HTTP(S) load balancing
• network load balancing
• subnetworks
• High CPU, high memory,
standard and shared-core
machine types

Google Compute Engine(2/2)
• Persistent disks
• standard, SSD, local SSD
• snapshots
• Resize disks, migrate instances with
no downtime
• Instance metadata and startup
scripts
• Advanced APIs for auto-scaling and
instance group management
• Innovative pricing
• per-minute billing, sustained use discounts
• preemptive instances
• high throughput to storage at no extra cost
• custom mashine types - only pay for the
hardware you need

DEMO
1. Deploying Applications using Google Container Engine
2. Deploying Applications using Google Compute Engine

Google Cloud Networking
Google Cloud Operations and Tools
Google Big Data
Google Machine Learning(“TensorFlow”)
Amazon Web Services case
Messaging system (such as ‘Kafka’, etc.)
Object Storage, File system introduction
…
goes to next part.

References
. Google Cloud Platform Fundamentals: https://sites.google.com/
a/google.com/cloud-platform-training/cloud-platform-training/
cp100-v2
. Google Developers - codelab: https://
codelabs.developers.google.com/?cat=Cloud

Compute
Machine
Learning
Storage
Operations and
Tools
Cloud
Storage
Cloud
SQL
Cloud
Datastore
Cloud
Bigtable
Networking Big Data

Google Cloud Bigtable
• Fully managed, NoSQL, wide-
column database service for large
workload application - TB to PB
• Integrated
• accessed using HBase API
• native compatibility with big data, Hadoop
ecosystem
• Protected
• replicated storage
• data encryption in-flight and at rest
• Role-based ACLs
• Proven
• drives major applications such as Google
Analytics and Gmail

Bigtable Access Patterns
Cloud Bigtable
Application API
Data can be read from and written to Cloud Bigtable through a data
service layer like: Managed VMs, the HBase REST Server, a Java
Server using the HBase client. Typically this will be to serve data to
applications, dashboards and data services.
Streaming
Data can be streamed in (written event by event) through a variety
of popular stream processing frameworks like:
Cloud Dataflow Streaming, Spark Streaming, Storm.
Batch Processing
Data can be read from and written to Cloud Bigtable through batch
processes like: Hadoop MapReduce, Dataflow, Spark. Often,
summarized or newly calculated data is written back to Cloud
Bigtable or to a downstream database.

Cloud Bigtable Integration
Cloud Bigtable
Google Cloud Dataflow
Use Cloud Dataflow connector for Bigtable for batch and
streaming operations in pipelines
Google Cloud Dataproc
Use Bigtable HBase client to integrate Hadoop jobs with
Cloud Dataproc
On-premises, cloud-based Hadoop
Use Bigtable HBase client to integrate with Hadoop
clusters
External
service

Gcp intro-20160721

In this document

More Related Content

What's hot

Viewers also liked

Similar to Gcp intro-20160721

Recently uploaded

Gcp intro-20160721