Next Generation Cloud Computing With Google - RightScale Compute 2013


Published on

Speaker: Martin Gannholm - Lead Engineer, Google

Google Cloud Platform provides everything you need to build, run, and scale social, mobile, and online applications. Already, tens of thousands of popular applications like Khan Academy, Angry Birds, SnapChat, and Pulse are benefiting from the power of running on top of Google infrastructure. Come join Google as we go deep on how to best leverage our technology with RightScale to build your next masterpiece.

Published in: Technology, News & Politics
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Next Generation Cloud Computing With Google - RightScale Compute 2013

  1. 1. Google Compute EngineGoogle Cloud StorageEvan Anderson, GCE Tech Lead (VM Environment) : Compute25 April 2013
  2. 2. The Plan•  Overview of Google Cloud Platform•  Google Compute Engine•  Google Cloud Storage•  Questions / ad hoc demos
  3. 3. Overview ofGoogle Cloud Platform
  4. 4. Google InfrastructurePerformanceRedundancyDisaster RecoveryAudits & CertificationsSecurityEnergy EfficientGlobal Data Centers99.95% Uptime SLACompute Storage ServicesCompute Engine (IaaS)App Engine (PaaS)Cloud Storage (Object)Cloud SQL (Relational)BigQueryCloud EndPointsCachingQueuesand more...Cloud PlatformGoogle Services
  5. 5. Cloud Based Analytics PlatformStoreBackends +MapReduceExtract &TransformHadoopBigQueryAPISQLAnalyzeinteractively ServeLogstoreCloud StorageDatastoreLog dataUnstructureddataStructured data InteractiveDashboards + appsApplicationlevel codeCustom logic & 3rdparty librariesBI toolsGoogleSpreadsheets
  6. 6. Overview ofGoogle Compute Engine
  7. 7. Google Compute Engine•  Run almost any application on Googles infrastructure•  Offers virtual machines, storage, networking•  Allows services to scale efficiently with little work
  8. 8. Google Compute Engine in pictures•  tools, UI & API•  project•  instances•  network &firewalls•  storage
  9. 9. Google Compute Engine in picturesgcutilAPIs ConsoleGoogle Clientlibraries
  10. 10. Google Compute Engine in is built on AppEngine.•  Uses public GCE APIs!•  No special sauce!
  11. 11. Google Compute Engine in picturesLinux VMs•  Modern CPU•  1, 2, 4, 8 CPUs•  3 memory sizes(RAM per CPU):o  standard: 3.75GBo  highcpu: 0.9Go  highmem: 6.5G•  Root access
  12. 12. Google Compute Engine in picturesExternal Network•  External IPs•  FirewallsPrivate Network•  Isolated at the project level•  Internal DNSInstance name ⇄ DNS name
  13. 13. Google Compute Engine in picturesStorage options:•  Persistent•  Scratch•  Cloud StorageOr:•  App Engine•  Google Cloud SQL•  ... anything you canconnect to!(Cassandra, MongoDB...)
  14. 14. •  Consistent CPU, network, and persistent disk performance.•  Dedicated scratch spindles for 4-cpu and 8-cpu VMs.•  Private networks for each project that span zones and regions.•  All data encrypted at rest.•  Building a consistent foundation on Googles infrastructure:o  So you dont have to worry about VM provisioning or location!Strong Isolation of Virtual Machines
  15. 15. Frictionless Oauth2 with Service Accounts•  Frictionless Access to Google APIs•  Synthetic, managed identity for VMs and code – no passwords required!•  Google Compute calling Google APIso  Examples: Cloud Storage, App Engine task queue API•  App Engine calling Compute Engine APIo  Use App Engine as orchestrator
  16. 16. (since Google I/O)Whats new inGoogle Compute Engine
  17. 17. First, one thing thats not new...•  Google Compute Engine is currently in Beta.•  Its used by dozens of Google projects internally.•  Recommended to build a multi-zone, multi-server architecture.•  GCE scheduled maintenance:o  Up to 2 weeks every 20 weeks, one zone at a time.•  RightScales management layer can migrate servers across zones.
  18. 18. •  For more memory relative to virtual cores, use high memory machine types.•  For more virtual cores relative to memory, use high CPU machine types.•  Machine types without scratch disk are discounted relative to scratch disks.New: High memory and high CPU machine types
  19. 19. •  Add ability to hot-add and hot-remove external IPs from a running VM.•  Even across zones!•  Most useful with static external IPs.New: Dynamically move external IPs
  20. 20. •  Add and remove persistent disks to a running VM.•  Create a persistent disk from a stored disk image.•  Launch a VM from a persistent root disk.•  The same persistent disk can be attached read-only to multiple VMs.New: More Flexibility with Persistent Disks
  21. 21. •  Create snapshots of existing persistent disks and apply them to new disks.•  Works across zones and regions!•  5GiB of data on 100GiB persistent disk == about 5GiB snapshot••  So, snapshots can be used to snapshot a read-write disk, create a new disk,and mount the new disk on N instances in read-only mode!New: Persistent Disk Snapshots
  22. 22. •  Added a --format flag for the list subcommands.•  The flag accepts the following values: table, sparse, json, csv, and names.•  --format=names allows gcutil to be used with Unix tool pipelines:o  gcutil  listinstances  -­‐-­‐format=names  |        xargs  gcutil  deleteinstance  -­‐-­‐force  New: Better command line scripting: gcutil --format
  23. 23. New: more new stuff!•  Added new European zones.o  europe-west1-ao  europe-west1-b•  Instance Cloning and Move Between Zones•  Serial console output from a VM instance•  Updated images & kernels a few times
  24. 24. Demos!
  25. 25.
  26. 26. Command line$  gcutil  addinstance  INSTANCE_NAME  -­‐-­‐machine_type=n1-­‐standard-­‐1  -­‐-­‐zone=us-­‐central1-­‐a    $  gcutil  ssh  INSTANCE_NAME    <do  work  on  the  instance>  <...>  <later>    $  gcutil  deleteinstance  INSTANCE_NAME  
  27. 27. By the way... These demos are App Engine apps!
  28. 28. Stories from the field(We love our partners!)
  29. 29. MapR Terasort RecordMapR World Record Previous RecordSort Time (s) 54 62Number of Servers 1003 1460Number of Cores 4012 11680Number of Disks 1003 5840Time to Build Cluster Minutes Months
  30. 30. •  Sorting depends on the speed of the slowest node.•  4-CPU and larger nodes have a dedicated spindle.•  High network bandwidth between nodes.o  No need to play games for switch or rack placement.•  CPU and memory isolation based on linux cgroups.o  (Developed at Google to solve performance consistency!)Consistency is the key to speed in distributed systems
  31. 31. RightScale and Google to-date:•  Complete integration with Google Compute Engine•  Customers already running in betaRightScale and Google are better together!•  Combine the speed of GCE with the flexibility of RightScale to provision entire environments in minutes•  Reproduce existing deployments for development and testing with the click of a button•  Generate rich media and push to outlets like YouTube, G+, and other Google Properties without beingcharged for bandwidth!RightScale Seamless GCE Integration
  32. 32. High-endnetworkingGeo-DistributedVery highReliabilityStrongperformanceGovernanceMulti-CloudOne-clickDeploymentTemplatizedWorkloadLow CostAutomationBetter Together
  33. 33. Google Cloud Storage
  34. 34. Its an object store.AKA: blobstore, key-value store
  35. 35. •  Strong (read-after-write) data consistency•  Streaming uploads and resumable transfers, range read support•  Objects can be terabytes in size, with resumable uploads and downloads,and range read support.•  Serve static data via HTTP directly from Cloud Storage.•  OAuth 2.0 or interoperable authentication•  Signed URLs allow delegating access to non-authenticated users.Overview
  36. 36. •  We replicate to data centers spanning multiple geographically diverse citieso  reducing the risk of natural disasters•  We can tolerate hardware failures to any part of the systemo  disk drives, servers, network routers, data centers, network links, etc•  Data is delivered via Googles worldwide networko  same network as YouTube, Gmail, etc•  We exercise a deep set of system tests before new releases•  Regular monitoring•  99.9% Service Level Agreemento and available by design
  37. 37. •  Scalable...o  requests, objects, GiBo  scales down well - affordable, robust infrastructure  Geographically redundant hosting for a single web page? Sure!•  Cloud Storage is the glue between all Google Cloud Serviceso  BigQuery, Compute Engine, App Engine, etc...•  Reliable, available, durable...Why should you care?
  38. 38. •  Durable Reduced Availability storage class (lower cost storage for backups)•  Static website hosting•  Cross-origin resource sharing for web and mobile clients•  Integration with App Engine (new GCS Client Library)•  Detailed permissions settings•  Access logs, enabling audit of storage activity•  Short-lived signed URLs for delegated download or uploadContinuous Innovation: recently added features...
  39. 39. •  Cloud Storage is an HTTP serviceo  serve data directly over HTTPo  APIs for all Cloud Storage features•  Our general purpose tools are built on these APIs•  The exact same APIs are used by all of our developers and partners•  XML API is compatible with APIs from similar services: AWS S3, EucalyptusHTTP APIs (interoperable XML and Google standard JSON)
  40. 40. RightScale Google Offer
  41. 41. •  GCE access is available through RightScale today•  RightScale usage charges are waived for the first year for new customers•  Get access to the Google Compute for a low flat rate through RightScaleRightScale Offer