Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Functional Database Strategies

231 views

Published on

A talk on Functional Database Strategies at Scala By The Bay 2016

Published in: Engineering
  • Be the first to comment

Functional Database Strategies

  1. 1. Functional Database Strategies Scalæ by the Bay 2016
  2. 2. YesQL
  3. 3. by Jason Swartz @swartzrock
  4. 4. Functional Database Strategies
  5. 5. Step One Buy A Functional Database
  6. 6. Silly! Databases Are mutable, not functional!
  7. 7. Well? Aren’tthey?
  8. 8. 1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
  9. 9. 1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
  10. 10. Let’s Talk About Immutable Tables
  11. 11. Create-Read- Update-Delete
  12. 12. GET /issues GET /issues/{id} POST /issues PUT /issues/{id} Issue Endpoints
  13. 13. How Do These Events Affect The Database?
  14. 14. 1. POST /issues title=’Config ELB’ +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+
  15. 15. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+
  16. 16. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
  17. 17. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Not Bad.
  18. 18. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Do You Know How We Got Here?
  19. 19. +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Do You Know How We Got Here?
  20. 20. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Do You Know How We Got Here?
  21. 21. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Why is ‘assignee’ NULL?
  22. 22. Mutable Table Rows Lose History
  23. 23. Immutable Table Rows KeepTheir History
  24. 24. Let’s Try To Lock Down Our State
  25. 25. 1. POST /issues title=’Config ELB’ +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+
  26. 26. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+
  27. 27. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:19 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
  28. 28. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:19 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
  29. 29. 1. GET /issues/1 +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:19 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
  30. 30. Tables Are Mutable, But Table Rows Should Be Immutable
  31. 31. In Other Words, Tables Should Be Append-Only
  32. 32. How Do You Make An Append-Only Table?
  33. 33. One: Don’t Let Your DB User Make Changes
  34. 34. Grant select, insert on issues to my-db-user; -- tested on Postgresql
  35. 35. Thank You! Goodbye!
  36. 36. Two: Pick The Right Columns
  37. 37. 1. GET /issues/1 +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:19 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
  38. 38. create table issues ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), title text, assignee int, done boolean default false )
  39. 39. 1. GET /issues/1 +------+-------------+------------+------------+----------+-------+ | id | created | issue_id | title | assignee | done | +------+-------------+------------+------------+----------+-------+ | 1 | 09-18 18:13 | 1 | Config ELB | NULL | false | +------+-------------+------------+------------+----------+-------+ | 2 | 09-18 18:16 | 1 | Config ELB | 10 | false | +------+-------------+------------+------------+----------+-------+ | 3 | 09-18 18:19 | 1 | Config ELB | NULL | false | +------+-------------+------------+------------+----------+-------+ +------+-------------+------------+------------+----------+-------+ | 4 | 09-18 18:24 | 1 | Config ELB | NULL | true | +------+-------------+------------+------------+----------+-------+
  40. 40. select * from issues where issue_id = :issue_id order by id desc limit 1;
  41. 41. That’s The Basics Of Immutability In Table Rows
  42. 42. 1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
  43. 43. 1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
  44. 44. SQL:2003 expands Groups into Windows
  45. 45. Works Great In Postgresql
  46. 46. ● Aggregation functions, eg sum(), rank(), avg() ● Window definitions with over() ● Grouping with partition by, order Window Functions
  47. 47. +------+------------+------------+----------+-------+ | id | issue_id | title | assignee | done | +------+------------+------------+----------+-------+ | 1 | 1 | Config ELB | NULL | false | +------+------------+------------+----------+-------+ | 2 | 1 | Config ELB | 10 | false | +------+------------+------------+----------+-------+ | 3 | 2 | Bug to fix | 11 | false | +------+------------+------------+----------+-------+ | 4 | 2 | Bug to fix | 11 | true | +------+------------+------------+----------+-------+
  48. 48. +------+------------+------------+----------+-------+ | id | issue_id | title | assignee | done | +------+------------+------------+----------+-------+ | 1 | 1 | Config ELB | NULL | false | +------+------------+------------+----------+-------+ | 2 | 1 | Config ELB | 10 | false | +------+------------+------------+----------+-------+ | 3 | 2 | Bug to fix | 11 | false | +------+------------+------------+----------+-------+ | 4 | 2 | Bug to fix | 11 | true | +------+------------+------------+----------+-------+
  49. 49. with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day' ) select * from latest_issues where row_number = 1
  50. 50. with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day' ) select * from latest_issues where row_number = 1
  51. 51. with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day' ) select * from latest_issues where row_number = 1
  52. 52. with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day' ) select * from latest_issues where row_number = 1
  53. 53. +------+------------+------------+----------+-------+------------+ | id | issue_id | title | assignee | done | row_number | +------+------------+------------+----------+-------+------------+ | 1 | 1 | Config ELB | NULL | false | 2 | +------+------------+------------+----------+-------+------------+ | 2 | 1 | Config ELB | 10 | false | 1 | +------+------------+------------+----------+-------+------------+ | 3 | 2 | Bug to fix | 11 | false | 2 | +------+------------+------------+----------+-------+------------+ | 4 | 2 | Bug to fix | 11 | true | 1 | +------+------------+------------+----------+-------+------------+
  54. 54. +------+------------+------------+----------+-------+------------+ | id | issue_id | title | assignee | done | row_number | +------+------------+------------+----------+-------+------------+ | 1 | 1 | Config ELB | NULL | false | 2 | +------+------------+------------+----------+-------+------------+ | 2 | 1 | Config ELB | 10 | false | 1 | +------+------------+------------+----------+-------+------------+ | 3 | 2 | Bug to fix | 11 | false | 2 | +------+------------+------------+----------+-------+------------+ | 4 | 2 | Bug to fix | 11 | true | 1 | +------+------------+------------+----------+-------+------------+
  55. 55. +------+------------+------------+----------+-------+------------+ | id | issue_id | title | assignee | done | row_number | +------+------------+------------+----------+-------+------------+ | 1 | 1 | Config ELB | NULL | false | 2 | +------+------------+------------+----------+-------+------------+ | 2 | 1 | Config ELB | 10 | false | 1 | +------+------------+------------+----------+-------+------------+ | 3 | 2 | Bug to fix | 11 | false | 2 | +------+------------+------------+----------+-------+------------+ | 4 | 2 | Bug to fix | 11 | true | 1 | +------+------------+------------+----------+-------+------------+
  56. 56. That Was Window Functions
  57. 57. 1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
  58. 58. 1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
  59. 59. You Know How To Maintain State
  60. 60. Do We Still Need State?
  61. 61. Let’s Talk About Event-Sourcing
  62. 62. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | 10 | true | +------+-------------+------------+----------+-------+
  63. 63. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | 10 | true | +------+-------------+------------+----------+-------+ Events States
  64. 64. Now We’re Storing Events, Not States
  65. 65. create table issue_events ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), originator text, payload text )
  66. 66. 1. POST /issue/1/event ‘Originator: 4a48239-8a..’ payload=’<Update val=”done=true”>’ +----+-------------+----------+------------+---------+ | id | created | issue_id | originator | payload | +----+-------------+----------+------------+---------+ | 14 | 09-18 18:50 | 1 | 4a482... | <...> | +----+-------------+----------+------------+---------+
  67. 67. Create Events And Simulate The State
  68. 68. 1. Create-Issue Issue(“Config ELB”, null, false); Real Events Virtual States
  69. 69. 1. Create-Issue 2. Assign-Issue Issue(“Config ELB”, 10, false); Real Events Virtual States
  70. 70. 1. Create-Issue 2. Assign-Issue 3. Complete-Issue Issue(“Config ELB”, 10, true); Real Events Virtual States
  71. 71. So Why Use Event-Sourcing?
  72. 72. 1. High Write Performance 2. Potential for Command/Query Separation 3. Auditable 4. Replayable 5. Undo-able 6. Monitorable Reasons For Event-Sourcing
  73. 73. It’s Like Having Control Over The Versions Of Your State Changes
  74. 74. It’s Like Having Control Over The Versions Of Your Data
  75. 75. It’s Like Git For Your Data
  76. 76. 1. Frankly, It’s Weird 2. Requires Events. No Events, No Event-Sourcing. 3. As Of November 2016, It’s Still Non-Standard Reasons Against Event-Sourcing
  77. 77. Wait! We’re Scala developers!
  78. 78. Who Cares About Being Non-Standard?
  79. 79. That About Sums Up Event Sourcing
  80. 80. 1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
  81. 81. 1. Immutable Rows 2. Window Functions 3. Events, Not State 4. DB Interactions Agenda
  82. 82. Remember That Your Database is mutable.
  83. 83. Avoid Sharing Your State
  84. 84. ● Avoid shared mutable SESSIONS ● Avoid shared mutable CURSORS ● Mutating state? Cool! But MODEL IT FIRST ● Execute state changes at THE EDGE Safe Database Interactions
  85. 85. Doobie A Typelevel Project
  86. 86. Are these steps? Or a Monad?
  87. 87. That About Sums Up Database Interactions
  88. 88. Okay, Actually That’s The Entire Talk Unless There’s More Time
  89. 89. Functional Database Strategies Scalæ by the Bay 2016
  90. 90. by Jason Swartz @swartzrock
  91. 91. Thank You For Attending
  92. 92. Fin
  93. 93. THIS SPACE LEFT BLANK

×