1. Let’s turn your PostgreSQL
into columnar store with
cstore_fdw
Jan Holčapek
PgConf EU 2015 Vienna

2. What’s cstore_fdw?
Foreign data wrapper allowing for storing and reading data in
a columnar fashion.

3. Trivia: foreign data wrapper
Extensibility feature of PostgreSQL introduced in 9.1.
Means for accessing data outside the database.
Plenty of them
https://wiki.postgresql.org/wiki/Foreign_data_wrappers
Two of them officially supported
file_fdw
postgres_fdw

4. Trivia: columnar storage
Row-oriented store: data for a particular row are stored
consecutively on a disk.
Column-oriented store: data for a particular column are
stored consecutively on a disk.
col 1 col 2 ... ... ... ... col N-1 col N
col 1 col 2 ... ... ... ... col N-1 col N

5. Naïve example
Table
100 columns
10 million rows
1 billion values
Sum of a single column
Row store
Read 1 billion values to use only 10 million of them
Column store
Read 10 million values actually needed

6. Benefits of columnar storage
Less I/O
Projections
Better compression and/or encoding
Column-level aggregates

7. What’s cstore_fdw again?
Foreign data wrapper allowing for storing data to and reading
data from the CStore file format.

8. CStore file format
Derived from ORC format
Optimized Row Columnar employed in Apache Hive
Row stripes, row blocks, skip lists
col 1 col 2 ... col N-1 col N
val 1 val 5 val 9
val 2 val 6 val 10
val 3 val 7 ...
val 4 val 8

9. How to get it
Download and compile manually
git clone git@github.com:citusdata/cstore_fdw.git
cd cstore_fdw
make
sudo make install
edit postgresql.conf
add cstore_fdw to shared_preload_libraries
restart postgresql
Wait until there’s a Fedora package
https://bugzilla.redhat.com/show_bug.cgi?id=1265101
dnf install postgresql-cstore_fdw

10. How to get it working
As a superuser
CREATE EXTENSION cstore_fdw;
GRANT USAGE ON FOREIGN DATA WRAPPER TO public;
As a regular user
CREATE SERVER cstore_server FOREIGN DATA WRAPPER cstore_fdw;
CREATE FOREIGN TABLE foo (...)
SERVER cstore_server
OPTIONS (
filename ’/tmp/foo’,
stripe_row_count 150000,
block_row_count 10000,
compression ’pglz’ -- or ’none’
);

11. What we can do with cstore
Version 1.3 released in July 2015
Create foreign table
Alter foreign table
Add or drop columns
Load data using COPY
Load data using INSERT INTO … SELECT
Unload data using COPY TO

12. What we cannot do with cstore
Plain insert
Update
Upsert
Delete data
Truncate

13. Conclusion
Not a good match for OLTP workload
Great match for OLAP workload

14. Demo!
create native and cstore tables, load it with data
compare loading times and disk utilization
simple sum with date limit
sum with group by with date limit
slightly more complex query

15. That’s all folks!
Give cstore_fdw a try. Thank you!
Let Citus Data guys know how you like it or what you are missing. Thank you!
Submit feedback on my talk. Thank you!