Incubaid is a technology incubator that was involved with Amplidata, a company developing dispersed storage systems. Amplidata's 1.6 version was primarily C++ code and was becoming difficult to develop and maintain due to issues like resource management, sockets, threads, and data corruption. Incubaid rewrote a component of Amplidata's system in OCaml in 1.5 days, demonstrating the viability of a functional programming approach. This led Amplidata to pursue a functional programming strategy using OCaml for its 2.0 version, though it ended up taking longer than estimated due to factors like adding new features. While functional programming provided benefits initially, there were also challenges from issues like OCaml language,

1. Who are ’we’?
How and why we bootstrapped Functional Programming
Evaluation
Summary
Functional Programming @ Ghent IT Valley
R. Slootmaekers N. Trangez
Incubaid Research Lab
Commercial Users of Functional Programming, 2012
R. Slootmaekers, N. Trangez FP@GhentITV

2. Who are ’we’?
How and why we bootstrapped Functional Programming
Evaluation
Summary
Outline
1 Who are ’we’?
2 How and why we bootstrapped Functional Programming
Amplidata’s unbreakable storage
our pain
2.0 plan and execution
3 Evaluation
Were we Happy?(2010)
Are we still happy?(2012)
R. Slootmaekers, N. Trangez FP@GhentITV

3. Who are ’we’?
How and why we bootstrapped Functional Programming
Evaluation
Summary
Incubaid is a technology incubator, active in datacenter
and cloud computing
Prior exits through Terremark, Telenet (Belgian telco),
Veritas/Symantec, Sun Microsystems,. . .
some of our current involvements: Amplidata , Awingu ,
CloudFounders , Dacentec , Racktivity
R. Slootmaekers, N. Trangez FP@GhentITV

4. Who are ’we’?
Amplidata’s unbreakable storage
How and why we bootstrapped Functional Programming
our pain
Evaluation
2.0 plan and execution
Summary
Outline
1 Who are ’we’?
2 How and why we bootstrapped Functional Programming
Amplidata’s unbreakable storage
our pain
2.0 plan and execution
3 Evaluation
Were we Happy?(2010)
Are we still happy?(2012)
R. Slootmaekers, N. Trangez FP@GhentITV

5. Who are ’we’?
Amplidata’s unbreakable storage
How and why we bootstrapped Functional Programming
our pain
Evaluation
2.0 plan and execution
Summary
Amplidata’s Dispersed Storage System
20 000ft view
diskA0
StorageNodeA diskA1
encoder
decoder
Client ........ diskA...
provisioning
policies
StorageNodeZ
metadata
client streams object data
encoder generates equations : Ax = b or x = A−1 b
policies determine how many disks | equations
provisioning determines which disks
send equations to StorageNodes
record metadata
R. Slootmaekers, N. Trangez FP@GhentITV

6. Who are ’we’?
Amplidata’s unbreakable storage
How and why we bootstrapped Functional Programming
our pain
Evaluation
2.0 plan and execution
Summary
Outline
1 Who are ’we’?
2 How and why we bootstrapped Functional Programming
Amplidata’s unbreakable storage
our pain
2.0 plan and execution
3 Evaluation
Were we Happy?(2010)
Are we still happy?(2012)
R. Slootmaekers, N. Trangez FP@GhentITV

7. Who are ’we’?
Amplidata’s unbreakable storage
How and why we bootstrapped Functional Programming
our pain
Evaluation
2.0 plan and execution
Summary
Pain
Pain: resource management
Pain: sockets
Pain: threads
Pain: data corruption
Pain: objects
development was grinding to a halt
R. Slootmaekers, N. Trangez FP@GhentITV

8. Who are ’we’?
Amplidata’s unbreakable storage
How and why we bootstrapped Functional Programming
our pain
Evaluation
2.0 plan and execution
Summary
Pain
Pain: resource management
Pain: sockets
Pain: threads
Pain: data corruption
Pain: objects
development was grinding to a halt
R. Slootmaekers, N. Trangez FP@GhentITV

9. Who are ’we’?
Amplidata’s unbreakable storage
How and why we bootstrapped Functional Programming
our pain
Evaluation
2.0 plan and execution
Summary
The 1.6 Codebase (Core)
Time (Blue):
Pareto applies
1 60% is IO
30% is encoding/decoding
size (Red):
0 > 50 kloc
StIO MdIO Codec Rest Codec ∼ 1 kloc
IO < 1 kloc
R. Slootmaekers, N. Trangez FP@GhentITV

10. Who are ’we’?
Amplidata’s unbreakable storage
How and why we bootstrapped Functional Programming
our pain
Evaluation
2.0 plan and execution
Summary
The Bet
I placed a bet to rewrite the StorageNode component in OCaml
Functionally identical
Same Performance
in 2 days. . .
1.5 day later the drop in replacement was demoed, and we
were allowed to Investigate alternatives to C++ for version 2.0
R. Slootmaekers, N. Trangez FP@GhentITV

11. Who are ’we’?
Amplidata’s unbreakable storage
How and why we bootstrapped Functional Programming
our pain
Evaluation
2.0 plan and execution
Summary
The Bet
I placed a bet to rewrite the StorageNode component in OCaml
Functionally identical
Same Performance
in 2 days. . .
1.5 day later the drop in replacement was demoed, and we
were allowed to Investigate alternatives to C++ for version 2.0
R. Slootmaekers, N. Trangez FP@GhentITV

12. Who are ’we’?
Amplidata’s unbreakable storage
How and why we bootstrapped Functional Programming
our pain
Evaluation
2.0 plan and execution
Summary
How is this possible?
component is very simple
perfect specs
existing component test suite
performance is mostly determined by network & disk speed
OCaml has object orientation.
R. Slootmaekers, N. Trangez FP@GhentITV

13. Who are ’we’?
Amplidata’s unbreakable storage
How and why we bootstrapped Functional Programming
our pain
Evaluation
2.0 plan and execution
Summary
Outline
1 Who are ’we’?
2 How and why we bootstrapped Functional Programming
Amplidata’s unbreakable storage
our pain
2.0 plan and execution
3 Evaluation
Were we Happy?(2010)
Are we still happy?(2012)
R. Slootmaekers, N. Trangez FP@GhentITV

14. Who are ’we’?
Amplidata’s unbreakable storage
How and why we bootstrapped Functional Programming
our pain
Evaluation
2.0 plan and execution
Summary
2.0 Estimates & plan
// in IO ⇒ 30 → 6
recycle codec
do the rest in comfort ⇒ 10 → 20
maybe improve metadata IO (?)
⇒ (time) 100% → 86%
⇒ (size) 100% → 40%
R. Slootmaekers, N. Trangez FP@GhentITV

15. Who are ’we’?
Amplidata’s unbreakable storage
How and why we bootstrapped Functional Programming
our pain
Evaluation
2.0 plan and execution
Summary
What happened?
We decided on Functional programming & picked OCaml.
use Light Weight Threads library.
Blinded by success.
rewrote codec (in C)
metadata → Arakoon
feature creep
⇒ (time) 100% → 40%
⇒ (size) 100% → 100% (90% OCaml, 9% C, 1% Asm)
R. Slootmaekers, N. Trangez FP@GhentITV

16. Who are ’we’?
How and why we bootstrapped Functional Programming Were we Happy?(2010)
Evaluation Are we still happy?(2012)
Summary
Outline
1 Who are ’we’?
2 How and why we bootstrapped Functional Programming
Amplidata’s unbreakable storage
our pain
2.0 plan and execution
3 Evaluation
Were we Happy?(2010)
Are we still happy?(2012)
R. Slootmaekers, N. Trangez FP@GhentITV

17. Who are ’we’?
How and why we bootstrapped Functional Programming Were we Happy?(2010)
Evaluation Are we still happy?(2012)
Summary
Were we Happy? (2010)
No:
Yes: Tools? what tools?
Type Inference rocks 32/64 bit issues
Superfast compiler Multicore issues
Fast Enough Code OCaml(build) learning curve
Nice C integration Emacs only
Refactoring OO does not fit well
Architecture?
R. Slootmaekers, N. Trangez FP@GhentITV

18. Who are ’we’?
How and why we bootstrapped Functional Programming Were we Happy?(2010)
Evaluation Are we still happy?(2012)
Summary
Outline
1 Who are ’we’?
2 How and why we bootstrapped Functional Programming
Amplidata’s unbreakable storage
our pain
2.0 plan and execution
3 Evaluation
Were we Happy?(2010)
Are we still happy?(2012)
R. Slootmaekers, N. Trangez FP@GhentITV

19. Who are ’we’?
How and why we bootstrapped Functional Programming Were we Happy?(2010)
Evaluation Are we still happy?(2012)
Summary
Are we still happy?(2012)
No:
OCaml language evolution.
Yes:
OCaml runtime evolution.
FP strategy works.
OCaml library evolution.
people & process mistakes.
R. Slootmaekers, N. Trangez FP@GhentITV

20. Who are ’we’?
How and why we bootstrapped Functional Programming
Evaluation
Summary
Summary
Doing everything in 1 programming language is plain silly.
(But you need pain and some guerilla tactics to convince
people)
Functional programming and distributed systems match
R. Slootmaekers, N. Trangez FP@GhentITV