Learn about Hitchhiker Trees from David Greenberg, a new functional, immutable, persistent variation of a fractal tree. In these slides, we'll learn how to understand immutable data strucutres and a variety of trees, introducing new concepts as we build up to the hitchhiker tree.

1. Exotic Functional Data
Structures:
Hitchhiker Trees
David Greenberg
9/17/16
Strange Loop

2. Who am I?

3. Functional
Data Structures
What are they, anyway?

4. Functional Data Structures
Immutable
7 + 1 = 8
But 7 is still 7

5. Functional Data Structures
x = [1, 2, 3]
y = x
y += [4]
if x == y:
print("I'm a sad panda”)

6. How to fix this?
x = [1, 2, 3]
y = x[:]
y += [4]
if x != y:
print("I'm a happy panda")

7. A List of Fruit

8. Mutation in an
Immutable World

9. What are
pointers?
(besides hard)

10. Pointers!

11. Pointers and Sharing

12. Doing Better with Pointers

13. Linked List

14. Editing the Linked List

15. Worse Case Performance

16. Philosophy of Identity
Q: When isn’t an apple an apple?
A: When an apple points to an orange points to a banana
isn’t an apple points to an orange points to a mango.

17. Trees

18. Binary Search Trees

19. Lookups are log2(n)
1 = 20
2 = 21
4 = 22
Elements
per
Level

20. Big O Analysis
We Care About the Dominating Factor

21. Performance
Analysis/Algebra
We have L levels
Lookups cost L
Only the last level matters
There are 2L-1 elements
Thus: n = 2L-1
log2(n) = L

22. Functional updates
Path Copying

23. Path Copying
Updates still log2(n)

24. Properties of Trees
Balanced
How do we maintain this?
How to order the values
Sort them
Trie

25. Changing Our
Cost Model
Where did the 2 come from in log2(n)?

26. More children
Fat nodes with ~B children

27. Going Wide

28. B Trees are Optimal for
Reads
Lower Bound of logB(n) for sorted lookups
Controlling the base of the logarithm is awesome
log2(1000) = 9.96
log5(1000) = 4.29
log100(1000) = 1.5
Going wide gives big constant speedups for free
Under our I/O cost model

29. B Tree Bookkeeping
Not as simple as a Binary Search Tree

30. Separate Node Types
Index & Data Nodes

31. B+ Tree
Reduce B to fit more levels on screen

32. Introducing
Fractal Trees

33. Fractal Trees

34. We can insert
faster
logb(n) is only for sorted lookups

35. Appending to a Log
Constant time to append
Already know the next index where we need to
insert
A B C D E

36. Fractal Trees

37. Fractal Insertion
Inserting 0

38. Walking Through Insertions
Inserting -1

39. Walking Through Insertions
Inserting 28

40. Walking Through Insertions
Inserting 29

41. Walking Through Insertions
Inserting -2

42. Walking Through Insertions
Inserting 11.5

43. Walking Through Insertions
Inserting 100

44. What about
Reads?

45. Looking up 20

46. Find the Path

47. Project Pending
Operations

48. Broken for Scans

49. Only Project Values Within
Range

50. Hitchhiker vs Fractal

51. Path Copying or Not!
Fractal Trees update in-place

52. Path Copying or Not!
Hitchhiker Trees use path-copying

53. Flush Control
Total I/O I/O per Flush Avg I/O per
Insert
B+ Tree 21 3 3
Fractal Tree 12 1 to 4 1.7
Hitchhiker Tree 5 5 0.7

54. Real Branching Factors
B+ Trees have fan out of 1000-2000
Hitchhiker Trees have fan out of 100-200
But Hitchhiker Tree buffers hold 900-1000
elements!

55. I want to try it!
On Github

58. Datacrypt is Pluggable
Backend Storage
I/O Management
Serialization
Sorting Algorithm

59. Works with Redis
Called the Outboard API

60. Outboard
Looks like a hash map
Data stored off-heap in Redis
Functional data structures mean free snapshots
After a VM restart, just reconnect to Redis
Lifetime of in-memory data doesn’t need to be
tied to lifetime of runtime memory

61. What’ll we build next?
Q&A
Thanks to:
Andy Chambers for JDBC Backend &
GC Improvements
Casey Marshall for S3 Backend

62. (Prefix) Tries

63. (Hash) Array Mapped Tries
We add the fat node trick from B trees
We hash keys first for even distribution
No need to store full hash: prefix is enough