What It Means to be a Massively Parallel PostgreSQL DBA Greenplum Summit at PostgresConf US 2018 Pravin Rao and Randy Willard

2. ■
■
■
■ Power of Parallel
■ Extensions
■

4. ●
●
●
●
●
●
●
●

5. ●
●
●
●
●
●
●
●
●
●
●●

7. Messaging
Interface

8. ●
●
●
●
●
●
●
●

10. ➔➔➔➔

11. ●
●
●
●
●
●
●
●
●

13. ●
●
●
●
●
●
●
●
●
●
●
●
●
●

14. ●
●
●
●
●
●
●
SELECT username
FROM users_tbl
WHERE
username = 'artoo’
AND
cryptpwd = crypt('test', cryptpwd);
SELECT
madlib.logregr_train( 'patients', -- source table
'patients_logregr', -- output table
'second_attack', -- labels
'ARRAY[1, treatment, trait_anxiety]', -- features
NULL, -- grouping columns
20, -- max iterations
'irls' ); -- optimizer

15. ●
●
create or replace function distance_between(text, text)
returns float as $$
SELECT ST_Distance(gg1, gg2, true)/1000.00 AS sphere_dist
FROM (SELECT
get_loc($1) as gg1,
get_loc($2) as gg2
) AS foobar ;
$$
language SQL;

17. ●
●
●
●
●
●
●

18. ●
●
●
●
●
●
●
●
●
●

19. ●
●
●
●
●
●

24. Master Host
Standby
Master Host
Interconnect
Client
Segment Instance
Segment Host
Segment Instance
Segment Instance
Segment Instance
Segment Instance
Segment Host
Segment Instance
Segment Instance
Segment Instance
Segment Instance
Segment Host
Segment Instance
Segment Instance
Segment Instance
…
●
●
●
●
●
●
●
●
●

25. Client Redundant
Interconnect
Sync &
Failover
Processes
Standby
Master
(Passive)
Primary
RAID 5
Protection
A
1
B
1
C
1
A
2
B
2
C
2
A
1
B
1
C
1
A
2
B
2
C
2
Mirror
Data Segment Mirroring
Master Segments

26. • Columnar storage compresses better
• Optimized for retrieving a subset of the
columns when querying
• Compression can be set differently per
column: gzip (1-9), quicklz, delta, RLE
• Row oriented faster when returning
all columns
• HEAP for many updates and deletes
• Use indexes for drill through queries
TABLE ‘SALES’
Column-orientedRow-oriented
Jun Oct Year
-1
Year
-2
External Partitions
• Less accessed partitions
on external storage
• Seamlessly query all data
• All major Hadoop
distributions
• Amazon S3 storage
• Files + Others in
development
Nov DecJul Aug Sep

27. ●
●
●
●
●
●
●
Segment Host
100
Order
Order#
Order
Date
Customer
ID
10
6
Oct 20 2005 34
10
7
Oct 20 2005
21
310
8
Oct 20 2005 15
10
9
Oct 20 2005
10
211
0
Oct 20 2005 82
11
1
Oct 20 2005 55
10
0 Oct 20 2005 34
10
1
Oct 20 2005
21
310
2
Oct 20 2005 15
10
3
Oct 20 2005
10
210
4
Oct 20 2005 82
10
5
Oct 20 2005 55
101
102
103
Segment Host
104
105
106
107
Segment Host
108
109
110
111
Hash distribute
on column
Order #

28. ●
●
●
●
●
●
Segment Host 1
Segment 1A
Segment 1B
Segment 1C
Segment 1D
Segment Host 2
Segment 2A
Segment 2B
Segment 2C
Segment 2D
Segment Host 3
Segment 3A
Segment 3B
Segment 3C
Segment 3D
December 2018
November 2018
October 2018
September 2018
August 2018
July 2018
June 2018
May 2018
April 2018
March 2018
February 2018
January 2018
Example of
Range
Partition

29. Segment 1A Segment 1B Segment 1C Segment 1D
Segment 2A Segment 2B Segment 2C Segment 2D
Segment 3A Segment 3B Segment 3C Segment 3D
Segment 1A Segment 1B Segment 1C Segment 1D
Segment 2A Segment 2B Segment 2C Segment 2D
Segment 3A Segment 3B Segment 3C Segment 3D
Segment 1A Segment 1B Segment 1C Segment 1D
Segment 2A Segment 2B Segment 2C Segment 2D
Segment 3A Segment 3B Segment 3C Segment 3D
Partitioning
SELECT COUNT(*)
FROM order_lineitems
WHERE order_datetime BETWEEN date '2010-11-01'
AND date '2010-11-02'
VS
Hash Partition Multi-Level Partition
Segment 1A Segment 1B Segment 1C Segment 1D
Segment 2A Segment 2B Segment 2C Segment 2D
Segment 3A Segment 3B Segment 3C Segment 3D

30. Segment 1A Segment 1B Segment 1C Segment 1D
Segment 2A Segment 2B Segment 2C Segment 2D
Segment 3A Segment 3B Segment 3C Segment 3D
Segment 1A Segment 1B Segment 1C Segment 1D
Segment 2A Segment 2B Segment 2C Segment 2D
Segment 3A Segment 3B Segment 3C Segment 3D
Segment 1A Segment 1B Segment 1C Segment 1D
Segment 2A Segment 2B Segment 2C Segment 2D
Segment 3A Segment 3B Segment 3C Segment 3D
SELECT COUNT(*)
FROM orders
WHERE order_date >= ‘Oct 20 2007’
AND order_date < ‘Oct 27 2007’
&
Evenly distribute orders data across all segments Only scans the relevant order partitions
Segment 1A Segment 1B Segment 1C Segment 1D
Segment 2A Segment 2B Segment 2C Segment 2D
Segment 3A Segment 3B Segment 3C Segment 3D

31. ●
●
●
●
●
●
●

34. ●
●
●
●
●

35. •
•
•
•
•
•
–
•
•
•
•
•

37. • Greenplum Command Center – see next slide
• gp_text full text indexing (SOLR) directly linked into database with full SQL support for
Lucene search predicates
• Parallel Utilities
• gpstart/gpstop – all start/stop operations
• gpssh/gpscp – allow for parallel admin across the cluster
• gpcheckperf/gpconfig/gpseginstall
• Others to handle all manner of things within the cluster
• Coming Soon
• PKS for Greenplum
Pivotal Kubernetes Services for Greenplum

41. • Marketplace offerings on all major cloud platforms
• Data Warehousing with Greenplum – written by our own Marshall Presser
https://content.pivotal.io/ebooks/data-warehousing-with-greenplum
• Greenplum Documentation - http://gpdb.docs.pivotal.io/
• MadLib - http://madlib.apache.org/
Docs and Download available
https://azuremarketplace.microso
ft.com/en-us/marketplace/apps/pi
votal.pivotal-greenplum1
https://aws.amazon.com/marketp
lace/pp/B06XKQ8Z3H