PostgreSQL is the new
NoSQL
QUENTIN ADAM
CEO @ CLEVER CLOUD
@WAXZCE ON TWITTER
AT DEVOXX 2018
Who am I ?
Quentin ADAM from the Clever Cloud
@waxzce on twitter – github- soundcloud – instagram ….
https://www.youtube.com/c/MaiaMater “Interviews en toge”
https://www.youtube.com/c/clevercloudplatform
or search “Quentin ADAM”
💡☁️
Clever Cloud
💡☁️
Git push
It’s free to try it out
Deploy
&
Application sustainability
European Cloud & on premises
Data @ Clever Cloud
Data have gotten big
noSQL and big data world
CAP theorem
Make your choices
• Transactions
• Consistency
• Schema
• Constraint
• Event flow
• Request capabilities
• Scale (volume, numbers…)
• Indexes
• ACL
• …
Rush to the noSQL landscape
Store
And we discover some comfort
on the old RDBMS
Request/Analyze
SQL
SQL-ish
Luckily someone is improving our home
Postgres story
• SINCE 1995
• First name Ingres
Popularity
Playlist management app, multiple
backend, playlist history
To do
example
entities
• user management
• Playlists
• Tracks history (deezer, spotify, soundcloud)
Track data is heterogeneous
We need schemaless
Arbitrary data depth
External data is
already json
BTW, why do not use
noSQL?
“I need unique user on email”
constraints
“which Tracks are played at
8pm to 9pm week days?”
Complex data
exploratory is
hard
“… in paris area”
Complex data
exploratory is
hard
“… by male users”
Need some
relationships in your
data store?
“…order by..”
“…UI need pagination…”
SQL have been done to show
data
Show data VS Analitycs
So we are on PG for good reasons
Let’s create our tracks table
Uuid as primary key
& no serial
“we have 2000000 users on our
product”
http://www.my-platform.com/users-creation-
confirm/42?valid_key=jgqsdkfhqksdjfhkjsdhf2342345
Using it to count things >_<
Same id for various entities on
your database <_>
What is my ID?
Generate it on the code level
Put a name on things
Computer science problem
Blog article by the awesome @clementd (former Clever Cloud CTO)
https://www.clever-cloud.com/blog/engineering/2015/05/20/why-auto-increment-is-a-terrible-idea/
Let’s create our tracks table
USE enum
USE timestamptz
Let’s create our tracks table
Postgres have two json type
• Json: text input, parsed each time we need it
• Jsonb: decomposed binary format (ie: almost already parsed)
TLDR: use jsonb
Json/jsonb data on your DB
Let’s create our tracks table
Request json
Or test existence
Use sql function on it
Plenty of capacity for json • Array length, data
manipulation, filters,
jsonpath, prettyfier
Temporal function with pgsql
Count(*) on nosql is hard?
=> Use SQL
Index
choose db (and version)
ORM is not your best friend here, sadly
Jdbc drivers
Workaround
Workaround
Manage it like regular json in
your code
Data is going big and
query slow…
Use parallel queries: split queries to multiple CPU
• max_parallel_workers & max_parallel_workers_per_gather +
dynamic_shared_memory_type in configuration
Hash & Scan improved by GPU
• PG-Storm http://heterodb.github.io/pg-strom/
• PG extension with CUDA (nvidia chip)
• Useful for Hash and Scan
(I never used it)
Am I locked on only one node? Clustering?
Master/slave
Using live replication
Split request on your
code
Or use pgpool
Able to make requests navigation
Use replica for specific
purpose
BI for example, keep production DB stable
Logic Replication (partial)
Since 10
Pub/Sub
• Notify
• Listen (have to be
supported by the
driver)
Quorum
Since 10
(I never used it yet)
Foreign Data Wrapper
Request Postgres using Postgres
Table sharding
Since 10/11
(I never used it yet)
https://pgdash.io/blog/postgres-11-sharding.html
Table sharding
Since 10/11
(I never used it yet)
https://pgdash.io/blog/postgres-11-sharding.html
Table sharding
Since 10/11
(I never used it yet)
https://pgdash.io/blog/postgres-11-sharding.html
Since 10/11
(I never used it yet)
Foreign
Data
Wrapper
(posgres)
Table
Sharding
Split a
table
accros
servers
DB stability is driven
by predictability
Use proxy for your DB, as PG bouncer
Get metrics, learn, improve
• http://dalibo.github.io/
powa/
Free dev plan on Clever Cloud
One click launch
PG 11.1 with auto update
I’m @waxzce on twitter
Give CLEVER-CLOUD.COM a try
Free credits coupon (both work):
💡☁️🦋
devoxx_be_18
Thx for listening

PostgreSQL is the new NoSQL - at Devoxx 2018

Editor's Notes

  • #12 Log à rechercher
  • #14 Le slide est laid
  • #46 exemples
  • #47 Schema
  • #51 Mettre une exemple
  • #55 Changer les exemple
  • #56 À virer
  • #63 Rajouter un point clair sur les index et expliquer que c’est cool en json
  • #64 Réexpliquer le mécanisme de choix de DB
  • #65 Démontre sur les ORM
  • #66 Screen shot du code de jdbc en mode BTW
  • #77 Faire un schéma
  • #80 EXPLAIN EXPLAIN
  • #82 Il faut aller plus lmoin cas d’usage de migration ou de BI