A 5min lightning talk on the whys and benefits of running PostgreSQL on ZFS. TL;DR: Use ZFS, even in the cloud.

1. PostgreSQL on ZFS
Bits Hopefully Structured the way you Left Them:
The scary parts followed by a Happy Ending™.

2. “All good lightning talks have a little FUD.”

3. … except it’s not FUD.

4. … and few care or are aware
(thank you cloud providers!)

5. Cloud providers have duped a generation of
operators into thinking bitrot does not exist.
It does. The reality has been hidden from plain sight,
but it’s there… lurking… silently, waiting…

6. –Many (myself included)
"If you are not using ZFS, you are losing data."

7. This has been studied recently for SSDs…

9. ECC on Flash works until it doesn’t.
Probability will rip your face if you do not acknowledge
its existence and its control over your life.

12. TL;DR: 4.2% -> 34% of SSDs have one UBER per year

13. TL;DR: 4.2% -> 34% of SSDs have one UBER per year
How many SSDs in that shiny box 'ya got there?
How many boxes are running?

14. TL;DR: 4.2% -> 34% of SSDs have one UBER per year
How many SSDs in that shiny box 'ya got there?
How many boxes are running?
(1-(1-uberRate)^(numDisks)) = Probability of UBER/server/year
(1-(1-0.042)^(20)) = 58%
(1-(1-0.34)^(20)) = 99.975%

15. External Factors for UBER on SSDs:
• Temperature
• Bus Power Consumption
• Data Written by the System Software
• Workload changes due to SSD failure

16. In a Datacenter…
…no-one can hear your bits scream.

18. Well, maybe they can…

19. Take care of your bits.
Your bits are taking care of you.

20. Take care of your bits.
Your bits are taking care of you.
Answer their cry for help.

21. Similar studies and research exist for:
• Fibre Channel
• SAS
• SATA
• Tape
• SANs
• Cloud Object Stores

22. "Groupon’s Deal on FreeBSD"
FreeBSD Journal, July/August 2015
https://www.freebsdfoundation.org/past-issues/freebsd-in-the-enterprise/
https://www.freebsdfoundation.org/wp-content/uploads/2015/12/vol2_no4_groupon.pdf

23. PostgreSQL and ZFS were meant for each other

24. PostgreSQL and ZFS were meant for each other
• Because bitrot happens

25. PostgreSQL and ZFS were meant for each other
• Because bitrot happens
• Because its fast

26. PostgreSQL and ZFS were meant for each other
• Because bitrot happens
• Because its fast
2-4µs/pwrite(2)!!

27. PostgreSQL and ZFS were meant for each other
• Because bitrot happens
• Because its fast

28. PostgreSQL and ZFS were meant for each other
• Because bitrot happens
• Because its fast

29. PostgreSQL and ZFS were meant for each other
• Because bitrot happens
• Because its fast
P.S. This was observed on 10K RPM spinning rust.

30. PostgreSQL and ZFS were meant for each other
• Because bitrot happens
• Because its fast
• Because restoring from backups is AWFUL!
# zfsnap snapshot -rv -a 25h tank/pgdata
# zfs list -r -t snapshot
# zfs rollback -r tank/pgdata@hourly-2016-09-14_14.52.00—25h

31. PostgreSQL and ZFS were meant for each other
• Because bitrot happens
• Because its fast
• Because restoring from backups is AWFUL!
# zfsnap snapshot -rv -a 25h tank/pgdata
# zfs list -r -t snapshot
# zfs rollback -r tank/pgdata@hourly-2016-09-14_14.52.00—25h
This happens in seconds!
It’s YUGE people, absolutely YUGE!

32. PostgreSQL and ZFS were meant for each other
• Because bitrot happens
• Because its fast
• Because restoring from backups is AWFUL!
• Because compression is a performance win,
even on SSDs
# zfs set compression=lz4 tank/pgdata
(wtb publishing of benchmarks any year now…
you know who you are…)

33. PostgreSQL and ZFS were meant for each other
• Because bitrot happens
• Because its fast
• Because restoring from backups is AWFUL!
• Because compression is a performance win,
even on SSDs
• Because compression is a space win (2.2:1
compression for most PG data)

34. PostgreSQL and ZFS were meant for each other
• Because bitrot happens
• Because its fast
• Because restoring from backups is AWFUL!
• Because compression is a performance win,
even on SSDs
• Because compression is a space win (2.2:1
compression for most PG data)
• Because zfs snap; zfs send; ssh… ; zfs recv

35. PostgreSQL and ZFS were meant for each other
• Because bitrot happens
• Because its fast
• Because restoring from backups is AWFUL!
• Because compression is a performance win,
even on SSDs
• Because compression is a space win (2.2:1
compression for most PG data)
• Because zfs snap; zfs send; ssh… ; zfs recv
• Because caching compressed records is win
https://www.illumos.org/issues/6950

36. It even runs on Linux!

37. It even runs on Linux!
Don’t become cranky and
jaded and relearn the
perils of data sadness.

39. Winning Strategy

40. Winning Strategy
Be aware.
But take the Blue Pill.

41. +

43. Not running PostgreSQL on ZFS is tantamount
to professional negligence. Discuss.