Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Pivotal Greenplum Cloud Marketplaces - Greenplum Summit 2019


Published on

Greenplum Summit 2019
Jon Roberts, Principal Engineer

Published in: Software
  • Be the first to comment

Pivotal Greenplum Cloud Marketplaces - Greenplum Summit 2019

  1. 1. © Copyright 2019 Pivotal Software, Inc. All rights Reserved. Jon Roberts Principal Engineer Twitterless Pivotal Greenplum Cloud Marketplaces
  2. 2. Cover w/ Image ■ FAST ■ Leverage the Cloud ■ Same Experience Across Clouds ■ Secure Goals for Cloud Deployments
  3. 3. Goal 1 - Fast ● Companies use Greenplum for SPEED ● Cloud Deployments Must be Fast too
  4. 4. Performance Tuning What is Tuned? ● Virtual Machine ● Operating System ● Disk ● Memory ● Network ● Marketplace Template How is it Measured? ● "gpcheckperf" (Greenplum Utility) for Network and Disk ● TPC-DS Benchmark ● Cloud Vendor Specs
  5. 5. TPC-DS Performance Test Score ● Transaction Processing Performance Council (TPC) ● Members include: ○ Pivotal, Cloudera, HP, IBM, Microsoft, MapR, Oracle, RedHat, Teradata, Intel, VMWare, Dell, and many others ● Decision Support (DS): Standard for Big Data / Data Warehousing ● Star Schema with 24 Tables and 99 Queries ● 3TB of data ● 1 and 5 Users
  6. 6. Score is a Function of Duration and Hardware Larger Score = Faster TPC-DS Performance Test
  7. 7. TPC-DS Performance Test
  8. 8. Goal 2 - Leverage The Cloud ● Take Advantage of Cloud-Only Features ○ On-Demand Provisioning ○ Node Replacement ○ Disk Snapshots ○ Upgrades ○ Optional Installations ○ Web Based
  9. 9. On-Demand Provisioning ● Deployments Take less than 1 Hour to Complete ● Removes Barriers to Evaluate and Buy ● Empowers Business Units Azure Resource Group Deployment AWS CloudFormation GCP Deployment Manager
  10. 10. Node Replacement Pivotal Greenplum Self-Healing ● ANY Node Failure gets Automatically Replaced and Recovered ● Full Recovery in as little as 5 Minutes ○ On-Premises Recovery can last for Days! ● Online Recovery for Standby and Segment Hosts ● pgBouncer pause before Rebalance VM VM VMVM VM X Demos in Pivotal Booth!
  11. 11. Node Replacement Pivotal Greenplum Self-Healing Single Master ● Maintains High Availability ● No Performance Loss ● Fast Recovery with Self-Healing ● Save $$ on Infrastructure and Licensing Costs Interconnect sdw1 Standby Seg1 Seg2 Seg3 Seg4 sdw2 Seg5 Seg6 Seg7 Seg8 sdw3 Seg9 Seg10 Seg11 Seg12 ... mdw Master
  12. 12. Disk Snapshots gpsnap ● Schedule, Create, List, Delete, and Restore Snapshots with "gpsnap" and "gpcronsnap" ● IaaS Snapshots Provide Fast Backup of a Volume ● Full Cluster Backup Measured in Minutes ● Automatically Configured to take a Weekly Snapshot Backup ● Snapshots are executed in Parallel so they are very FAST! Data Volume Snapshot Restore Demos in Pivotal Booth!
  13. 13. Upgrades gprelease ● Notification of New Version Availability with gpcronrelease (Executes Weekly) ● Installation of New Version with gprelease ● Existing Optional Packages (MADlib, PostGIS, Command Center, etc) Re-Installed and Upgraded if Needed Demos in Pivotal Booth!
  14. 14. Optional Installations gpoptional ● Deployment Parameters to Install Components ● Or Post Deployment Tool gpoptional ● Included Packages ○ Command Center ○ Data Science R and Python ○ MADlib ○ PostGIS ○ PL/R Demos in Pivotal Booth!
  15. 15. Web Based Greenplum Browser ● Schema Browser ● SQL Queries ● Self-Signed SSL Certificate ● Open Source
  16. 16. Goal 3 - Same Experience Across Clouds ● Similar Deployment ● Same Tools ● Same Software Versions
  17. 17. Parameters - Basics Parameter AWS Azure GCP Name? Stack Name Deployment Name Deployment Name Where Deployed? Availability Zone Resource Group + Location Zone SSH Key? Key Name SSH Public Key N/A Who Can Access? SSH Location SSH Location SSH Location Subnet CIDR ClusterSubnet Subnet Subnet Instance Type? Instance Type+Storage Instance Type+Storage Instance Type Instance Storage? N/A N/A Node Storage How Many? Instance Count Instance Count Node Count ● GCP SSH Key is Managed Automatically ● Azure Deployments are in a Resource Group as well as in a Location ● AWS and Azure Storage is set by Instance Type for Optimal Performance ● GCP Disk Size does not impact performance Demos in Pivotal Booth!
  18. 18. Parameters - AWS Parameter AWS Name? Stack Name SSH Key? Key Name Who Can Access? SSH Location Where Deployed? Availability Zone Subnet CIDR Subnet Instance Type? Instance Type+Storage How Many? Instance Count
  19. 19. Parameters - Azure Parameter Azure Name? Deployment Name SSH Key? SSH Public Key Who Can Access? SSH Location Subnet Subnet Where Deployed? Resource Group + Location
  20. 20. Parameters - Azure Parameter Azure Instance Type? Instance Type+Storage How Many? Instance Count
  21. 21. Parameters - GCP Parameter GCP Name? Deployment Name Where Deployed? Zone Subnet Subnet Instance Type? Instance Type How Many? Node Count Instance Storage? Node Storage Who Can Access? SSH Location Dynamic SSH Keys
  22. 22. Parameters - Optional Installs Parameter AWS Azure GCP Install? Command Center Command Center Command Center Install? MADlib MADlib MADlib Install? Data Science Python Data Science Python Data Science Python Install? Data Science R Data Science R Data Science R Install? PL/R PL/R PL/R Install? PostGIS PostGIS PostGIS ● Optional Installs performed by "gpoptional"
  23. 23. Optional Installs AWS Azure GCP Same Experience Across Clouds
  24. 24. Goal 4 - Secure ● Must Be Secure ● Customers Must Have Confidence In Solution
  25. 25. Secure ● Templates and Machine Images Reviewed by Cloud Vendor ● Best Practices Implemented ○ Password Authentication Disabled ○ Greenplum MD5 Encrypted Password Authentication ○ Root Login Disabled ● Data At Rest Encryption Available ● Network Isolation
  26. 26. © Copyright 2019 Pivotal Software, Inc. All rights Reserved.© Copyright 2019 Pivotal Software, Inc. All rights Reserved. Frequently Asked Questions Greenplum on Cloud Marketplaces
  27. 27. FAQs ● Encryption ● Documentation ● Feature Comparison ● Licensing
  28. 28. Encryption ● AWS ○ EBS Encryption Option ● Azure ○ Partner: Thales Vormetric ● GCP ○ Automatic
  29. 29. Documentation ● Release Notes ○ Detailed Information ○ Located On Each Marketplace Listing ● Overview ○ One Pager ○ Located on Each Marketplace Listing
  30. 30. Documentation AWS Azure GCP
  31. 31. Cloud Marketplace Feature Comparison Feature AWS Azure GCP Marketplace BYOL License Yes Yes Yes Hourly License Yes Yes Yes Tools GP Browser Yes Yes Yes pgBouncer Yes Yes Yes Optional Installer Yes Yes Yes Upgrade Utility Yes Yes Yes Recovery Self Healing Yes Yes Yes Snapshots Backup / Restore Yes Yes Yes Disaster Recovery Yes No Yes
  32. 32. Licensing Free, BYOL, or Metered ● Free: Pivotal Web Services ○ Greenplum Tile: Fully Managed, Automated Maintenance, Multi-Tenancy ● Bring Your Own License (BYOL) ○ AWS and GCP: 1 core = 2 vCPUs (hyperthreading) ○ Azure: 1 core = 1 vCPU ○ Same Support as On-Premise ○ 90 Day Evaluation in EULA (but without Support) ● Metered ○ Pivotal Greenplum additional $0.50 per core per hour
  33. 33. #ScaleMatters © Copyright 2019 Pivotal Software, Inc. All rights Reserved.