Slides from my "Purely functional data structures" talk at Boiling Frogs 2016 conference. It covered some concepts from Chris Okasaki's thesis as well as Git internals as an example of legendary implementation.

1. Purely functional
data structures
@tkaczmarzyk
blog.kaczmarzyk.net
Tomek “Kosior” Kaczmarzyk

2. Purely functional
data structures
@tkaczmarzyk
blog.kaczmarzyk.net
Tomek “Kosior” Kaczmarzyk
meh...
?
WAT!?
Achievement unlocked!
WTF Level 3

3. whoami
Just wandering around the world,
developing software
@tkaczmarzyk
blog.kaczmarzyk.net
Tomek “Kosior” Kaczmarzyk
Recently hired by Siili Solutions

4. Agenda
1. Examples of PF data structures
2. GIT internals
- lists
- queues

8. “Bad programmers worry about the code.
Good programmers worry about
data structures and their relationships.”
Linus Torvalds

9. Part 1:
PF data structures
- how do they work?

12. Java

15. public class DefensiveList<T> {
private List<T> values;
public DefensiveList(List<T> values) {
values = new ArrayList<>(values);
}
public DefensiveList<T> add(T elem) {
List<T> copy = new ArrayList<>(values);
copy.add(elem);
return new DefensiveList<T>(copy);
}
}

17. functional lists

18. [1, 2, 3].head() = 1
[1, 2, 3].tail() = [2, 3]
[1, 2, 3].cons(7) = [7, 1, 2, 3]

19. 1

20. 2
1

21. 3
2
1

22. 3
2
1
Nil

23. 3 Nil
2
1
mylist =

24. 3 Nil
2
1
mylist =
= mylist.tail()

25. 3 Nil
2
1
7
mylist =
= mylist.tail()
= mylist.tail()
.cons(7)

26. functional queues

27. [1, 2, 3].head() = 1
[1, 2, 3].tail() = [2, 3]
[1, 2, 3].cons(7) = [7, 1, 2, 3]
[1, 2, 3].snoc(7) = [1, 2, 3, 7]

29. q = { [1, 2, 3], [6, 5, 4] }
front list
rear list
(reversed)

30. { [], [] }

31. snoc(2)
{ [2], [] }
{ [], [] }

32. snoc(2)
{ [2], [] }
{ [], [] }
snoc(3)
{ [2], [3] }

33. snoc(2)
{ [2], [] }
{ [], [] }
snoc(3)
{ [2], [3] }
snoc(4)
{ [2], [4, 3] }

34. snoc(2)
{ [2], [] }
{ [], [] }
snoc(3)
{ [2], [3] }
snoc(4)
{ [2], [4, 3] }
snoc(5)
{ [2], [5, 4, 3] }

35. tail
{ [2], [5, 4, 3] }
{ [], [5, 4, 3] } ???

36. tail
{ [2], [5, 4, 3] }
{ [], [5, 4, 3] } ???
{ [3, 4, 5], [] }

37. Computional
cost
$$$

38. { [1, 2], [] }

39. snoc
(3)
{ [1, 2], [3] }

40. snoc
snoc
(3)
(4)
{ [1, 2], [4, 3] }

41. snoc
snoc
(3)
(4)
tail
{ [2], [4, 3] }

42. snoc
snocsnoc
(3)
(4) (5)
tail
{ [2], [5, 4, 3] }

43. snoc
snoc tailsnoc
(3)
(4) (5)
tail
{ [3, 4, 5], [] }

44. snoc
snoc tailsnoc
(3)
tail
(4) (5)
tail
{ [4, 5], [] }

45. snoc
snoc tailsnoc
(3)
tail
(4) (5)
tail
O(1)
O(1) O(1) O(1) O(m)
{ [4, 5], [] }
O(1)

46. snoc
snoc tailsnoc
(3)
tail
(4) (5)
tail
O(1)
O(1) O(1) O(1) O(m)
{ [4, 5], [] }
O(1)

47. newList = mylist.snoc(3)
.snoc(4)
.tail()
.snoc(5)
.tail()
.tail();

48. newList = mylist.snoc(3)
.snoc(4)
.tail()
.snoc(5);
newList.tail();
newList.tail();
newList.tail();
newList.tail();

49. The problem
of multiple futures
Achievement unlocked!
Tribute

50. snoc
snoc tailsnoc
(3)
tail
(4) (5)
tail

51. snoc
snoc tailsnoc
(3)
tail
(4) (5)
tail
tail
tail
tail
tail tail
tail
snoc
(6)

52. lazy evaluation
& memoization
to the rescue!

53. class Suspension<T> {
private Supplier<T> suspendedFun;
private T calculatedValue;
public Suspension(Supplier<T> suspendedFun) {
this.suspendedFun = suspendedFun;
}
public T force() {
if (calculatedValue == null) {
calculatedValue = suspendedFun.get();
}
return calculatedValue;
}
}

54. Suspension<Integer> memoizedRandom
= new Suspension<>(() -> new Random().nextInt());
println(memoizedRandom.force()); // genereted here
println(memoizedRandom.force()); // cached value
println(memoizedRandom.force()); // on subsequent calls

55. interface Stream<T> {
Stream<T> tail();
Stream<T> append(Suspension<Stream<T>> other);
Stream<T> reverse();
boolean isEmpty();
}

56. interface Stream<T> {
Stream<T> tail();
Stream<T> append(Suspension<Stream<T>> other);
Stream<T> reverse();
boolean isEmpty();
}
class Empty<T> implements Stream<T> {
public Stream<T> tail() {
throw new UnsupportedOptionException(); }
public Stream<T> append(Suspension<Stream<T>> other){
return other.force();
}
public Stream<T> reverse() { return this; }
public boolean isEmpty() { return true; }
}

57. class NonEmpty<T> implements Stream<T> {
private T head;
private Suspension<Stream<T>> tail;
public NonEmpty(T head, Suspension<Stream<T>> tail) {
this.head = head;
this.tail = tail;
}
public Stream<T> tail() { return tail.force(); }
public Stream<T> append(Suspension<Stream<T>> other) {
return new NonEmpty<>(
this.head,
new Suspension<>(() -> this.tail().append(other)));
}
public Stream<T> reverse() {
// executes the full reverse...
}
public boolean isEmpty() { return false; }
}

58. class Queue<T> {
private Stream<T> front; private int lenF;
private Stream<T> rear; private int lenR;
private Queue(Stream front, int lenF, Stream rear, int lenR){
// ordinary assignments...
}
public static <T> Queue<T> queue(
Stream front, int lenF, Stream rear, int lenR){
if (lenR <= lenF) {
return new Queue<T>(front, lenF, rear, lenR);
} else
return new Queue<T>(
front.append(new Suspension<>(() -> rear.reverse())),
lenF + lenR,
new Empty<T>(),
0);
}
public Queue<T> tail() {
return queue(front.tail(), lenF - 1, rear, lenR);
}
}

59. Q { [1, 2, 3, 4], [7, 6, 5] }

60. Q { [2, 3, 4], [7, 6, 5] }
tail

61. Q {[3, 4].append(susp([7, 6, 5].rev())), []}
tail
tail
suspended reverse
is created here

62. Q {[4].append(susp([7, 6, 5].rev())), []}
tail
tail
suspended reverse
is created here
tail

63. Q {[5, 6, 7], []}
tail
tail
suspended reverse
is created here
tail tail
and forced here!
O(m)

64. Q {[5, 6, 7], []}
tail
tail
suspended reverse
is created here
tail tail
and forced here!
O(m)
at least
m x tail()
with O(1)

65. tail
tail tail tail
at least
m x tail()
with O(1)

66. tail
tail tail tail
at least
m x tail()
with O(1)
tail tail
tail tail

67. tail
tail tail tail
at least
m x tail()
with O(1)
tail tail
tail tail
it's the same (==)
Suspension!

68. tail
tail tail tail
at least
m x tail()
with O(1)
tail tail
tail tail
it's the same (==)
Suspension!
tail
tail

69. tail
tail tail tail
at least
m x tail()
with O(1)
tail tail
tail tail
it's the same (==)
Suspension!
tail
tail
different Suspensions
but each pays for m x O(1)

70. amortization
vs
worst-case

71. 1 op
number
0
1
20
system
response time
[s]
2 3 ... 100 200... 199
worst-case DS
amortized DS

73. there is a solution
for that as well
but let's skip the details :)

74. Part 2:
Git internals

75. becoming legendary

76. working
directory
repository

77. working
directory
repository
file A
file B

78. working
directory
repository
file A
file B
A1 B1

79. working
directory
repository
file A
file B
A1 B1snapshot
1

80. working
directory
repository
file A
file B
A1 B1snapshot
1
(v2)

81. working
directory
repository
file A
file B
A1 B1snapshot
1
B2snapshot
2(v2)

82. working
directory
repository
file A
file B
A1 B1snapshot
1
B2snapshot
2
(v2)
(v2)

83. working
directory
repository
file A
file B
A1 B1snapshot
1
B2snapshot
2
dir1
file C
(v2)
(v2)

84. working
directory
repository
file A
file B
A1 B1snapshot
1
snapshot
2
B2
dir1
file C
snapshot
3
A2
C1
(v2)
(v2)

86. ec11312 386ad51 6740724
1796e2f
cca7cf6

87. ec11312 386ad51 6740724
1796e2f
cca7cf6
fileA.txt fileB.txt dir1
fileC.txt

88. 100644 blob fa49b07797 fileA.txt
100644 blob 1f7a7a472a fileB.txt
040000 tree d8329fc1cc7 dir1
just a file!

89. repository
.git/objects
00
1c
ff
......
1674f7...
96e261...
ec1131...

91. commit
1
commit
2
commit
3

92. commit tree da3b911d93889f6f35ddf4
parent e78dca6f8f5e7dc825ded27094e0
author Kosior <t@kaczmarzyk.net>
committer Kosior <t@kaczmarzyk.net>
example commit :)
once again:
just a text file!

93. 88bd
2a3ac
9a789
8ffda

94. 88bd
2a3ac
9a789
8ffda master
hotfix

95. 88bd
2a3ac
9a789
8ffda
hotfix
HEAD
master

97. HEAD
master

98. HEAD
master

99. HEAD
master

100. HEAD
master

102. Is Git a purely functional
data structure!?

104. 88bd
2a3ac
9a789
8ffda
hotfix
master
a36c
HEAD

105. 88bd
2a3ac
9a789
8ffda
hotfix
master
a36c HEAD

106. 88bd
2a3ac
9a789
8ffda
hotfix
master
a36c HEAD7e89

107. 88bd
2a3ac
9a789
8ffda
hotfix
master
a36c
HEAD
7e89
6f3a2

108. 88bd
2a3ac
9a789
8ffda
hotfix
master
a36c
HEAD
7e89
6f3a2

109. 88bd
2a3ac
9a789
8ffda
hotfix
master
a36c
HEAD
7e89
6f3a2still available
(until gc)

110. the power of simplicity

111. recap
* immutability is powerful
* PF data structures
can be damn effective!
* they can become a way
to create legendary software :-)

112. Purely functional
data structures
@tkaczmarzyk
blog.kaczmarzyk.net
thanks!
Tomek “Kosior” Kaczmarzyk

113. Acknowledgements:
“Purely functional data structures” by Chris Okasaki
http://git-scm.com/book
Photo credits:
https://www.flickr.com/photos/jdhancock/9544541664/
https://www.flickr.com/photos/janimut/14609601810/
https://www.flickr.com/photos/macwagen/2819684/
https://www.flickr.com/photos/matt_gibson/2580896289/
https://www.flickr.com/photos/hktang/4243300265/
https://www.flickr.com/photos/funfilledgeorgie/10922459733/
https://www.flickr.com/photos/celinesphotographer/
4044569502/
https://www.flickr.com/photos/edsonhong1/5685375541/
https://www.flickr.com/photos/ch-weidinger/17341905336/
https://www.flickr.com/photos/toomuchdew/9284836919/