Your SlideShare is downloading. ×
FleetDB A Schema-Free Database in Clojure
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

FleetDB A Schema-Free Database in Clojure

2,805
views

Published on

Published in: Technology, Business

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,805
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
20
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. FleetDB A Schema-Free Database in Clojure Mark McGranaghan January 8, 2010
  • 2. Motivation Walkthrough Implementation QA
  • 3. Motivation
  • 4. Motivation optimize for agile development
  • 5. Walkthrough Client Quickstart Basic Queries Concurrency Tools
  • 6. Client Quickstart (use ’fleetdb.client) (def client (connect)) (client ["ping"]) => "pong"
  • 7. Insert (client ["insert" "accounts" {"id" 1 "owner" "Eve" "credits" 100}])
  • 8. Insert (client ["insert" "accounts" {"id" 1 "owner" "Eve" "credits" 100}]) => 1
  • 9. Select (client ["select" "accounts" {"where" ["=" "id" 1]}])
  • 10. Select (client ["select" "accounts" {"where" ["=" "id" 1]}]) => [{"id" 1 "owner" "Eve" "credits" 100}]
  • 11. Select with Conditions (client ["select" "accounts" {"where" [">=" "credits" 150]}])
  • 12. Select with Order and Limit (client ["select" "accounts" {"order" ["credits" "asc"] "limit" 2}])
  • 13. Update (client ["update" "accounts" {"credits" 105} {"where" ["=" "owner" "Eve"]}])
  • 14. Explain (I) (client ["explain" ["select" "accounts" {"where" ["=" "owner" "Eve"]}]])
  • 15. Explain (I) (client ["explain" ["select" "accounts" {"where" ["=" "owner" "Eve"]}]]) => ["filter" ["=" "owner" "Eve"] ["collection-scan" "accounts"]]
  • 16. Create Index (client ["create-index" "accounts" "owner"])
  • 17. Explain (II) (client ["explain" ["select" "accounts" {"where" ["=" "owner" "Eve"]}]]) => ["index-lookup" ["accounts" "owner" "Eve"]]]
  • 18. Multi-Write
  • 19. Multi-Write (client ["multi-write" [write-query-1 write-query-2 ...]])
  • 20. Multi-Write (client ["multi-write" [write-query-1 write-query-2 ...]]) => [result-1 result-2 ...]
  • 21. Checked-Write
  • 22. Checked-Write (client ["checked-write" read-query expected-read-result write-query])
  • 23. Checked-Write (client ["checked-write" read-query expected-read-result write-query]) => [true write-result]
  • 24. Checked-Write (client ["checked-write" read-query expected-read-result write-query]) => [true write-result] => [false actual-read-result]
  • 25. Implementation Background Organization Key ideas
  • 26. Background http://clojure.org/data_structures http://clojure.org/sequences http://clojure.org/state
  • 27. Organization
  • 28. Organization fleetdb.core: pure functions
  • 29. Organization fleetdb.core: pure functions fleetdb.embedded: identity and durability
  • 30. Organization fleetdb.core: pure functions fleetdb.embedded: identity and durability fleetdb.server: network interface
  • 31. fleetdb.core Pure functions
  • 32. fleetdb.core Pure functions Databases as Clojure data structures
  • 33. fleetdb.core Pure functions Databases as Clojure data structures Read: db + query → result
  • 34. fleetdb.core Pure functions Databases as Clojure data structures Read: db + query → result ‘Write’: db + query → result + new db
  • 35. fleetdb.core Query Planner
  • 36. fleetdb.core Query Planner Implements declarative queries
  • 37. fleetdb.core Query Planner Implements declarative queries Database + query → plan
  • 38. fleetdb.core Query Planner Implements declarative queries Database + query → plan ["select" "accounts" {"where" ["=" "owner" "Eve"]}]
  • 39. fleetdb.core Query Planner Implements declarative queries Database + query → plan ["select" "accounts" {"where" ["=" "owner" "Eve"]}] ["filter" ["=" "owner" "Eve"] ["record-scan" "accounts"]]
  • 40. fleetdb.core Query Planner Implements declarative queries Database + query → plan ["select" "accounts" {"where" ["=" "owner" "Eve"]}] ["filter" ["=" "owner" "Eve"] ["record-scan" "accounts"]] ["index-lookup" ["accounts" "owner" "Eve"]]
  • 41. fleetdb.core Query Executor
  • 42. fleetdb.core Query Executor Database + plan → result
  • 43. fleetdb.core Query Executor Database + plan → result ["filter" ["=" "owner" "Eve"] ["record-scan" "accounts"]]
  • 44. fleetdb.core Query Executor Database + plan → result ["filter" ["=" "owner" "Eve"] ["record-scan" "accounts"]] (filter (fn [r] (= (r "owner") "Eve")) (vals (:rmap (db "accounts"))))
  • 45. fleetdb.embedded Wraps fleetdb.core
  • 46. fleetdb.embedded Wraps fleetdb.core Adds identity and durability
  • 47. fleetdb.embedded Wraps fleetdb.core Adds identity and durability Databases in atoms
  • 48. fleetdb.embedded Wraps fleetdb.core Adds identity and durability Databases in atoms Append-only log
  • 49. fleetdb.embedded Read Path
  • 50. fleetdb.embedded Read Path Dereference database atom
  • 51. fleetdb.embedded Read Path Dereference database atom Pass to fleetdb.core
  • 52. fleetdb.embedded Read Path Dereference database atom Pass to fleetdb.core Return result
  • 53. fleetdb.embedded Write Path
  • 54. fleetdb.embedded Write Path Enter fair lock
  • 55. fleetdb.embedded Write Path Enter fair lock Dereference database atom
  • 56. fleetdb.embedded Write Path Enter fair lock Dereference database atom Pass to fleetdb.core
  • 57. fleetdb.embedded Write Path Enter fair lock Dereference database atom Pass to fleetdb.core Append query to log
  • 58. fleetdb.embedded Write Path Enter fair lock Dereference database atom Pass to fleetdb.core Append query to log Swap in new database value
  • 59. fleetdb.embedded Write Path Enter fair lock Dereference database atom Pass to fleetdb.core Append query to log Swap in new database value Return result
  • 60. fleetdb.server Wraps fleetdb.embedded
  • 61. fleetdb.server Wraps fleetdb.embedded Adds JSON client API
  • 62. Aside: Source Size
  • 63. Aside: Source Size Lines of Code core 630 embedded 130 server 120 lint 280 utilities 140 total 1300
  • 64. Takeaways
  • 65. Takeaways Clojure’s data structures are awesome
  • 66. Takeaways Clojure’s data structures are awesome Clojure is viable for infrastructure software
  • 67. Takeaways Clojure’s data structures are awesome Clojure is viable for infrastructure software Try FleetDB!
  • 68. Thanks for listening! Questions?
  • 69. http://FleetDB.org http://github.com/mmcgrana