A story of cross-company and cross-continent open-source collaboration. Last June at DockerCon in San Francisco, Solomon announced experimental support for Docker volume plugins, making it possible to build add-ons to Docker that manage persistent storage. The run-up to that announcement was a frenetic 9 months of cross-company and cross-timezone collaboration. At ClusterHQ, we were deeply involved in building the interface, and we also made our own Docker volume plugin Flocker available for migrating data volumes between nodes in a cluster. In this talk, I'll share stories on what it was like working with the Docker team and others in the ecosystem to build this API. We'll cover how you also can take advantage of Docker volume plugins to leverage stateful containers. I'll guide you through the Docker plugin model and show off some of the existing plugins so you can see how to enable stateful containers for your own use cases.

1. Behind the scenes with
Docker volume plugins
Luke Marsden
CTO, ClusterHQ
@lmarsden @clusterhq

2. Summer 2014
… trying to solve data problems for
Docker…

3. Docker Docker Docker Docker
Orchestration +
scheduling
Docker API

4. Docker Docker Docker Docker
Orchestration +
scheduling
Docker API
Networking extensions
Storage extensions
???

5. Wrapping sucks :(

6. December 2014:
CoreOS launched a rocket
Pressure on Docker to demonstrate
composability

7. December 2014: DockerCon EU

8. Docker Docker Docker Docker
Orchestration +
scheduling
Docker API
N S N S N S N S
Plugin mechanism

9. Late-bound
Composable
Optional

10. Healthy ecosystem
Lots of innovation

11. Powerstrip is was a prototyping tool for Docker extensions
Feb 2015: Launch Powerstrip

13. @crosbymichael@icecrime @progrium
@solomonstre@squaremobius@ chanezon@calavera
@binocarlos

14. The day before DockerCon June 2015
Docker 1.7 Experimental

16. Volume plugins in 1.8
Network plugins in 1.9

17. Here are some cool
things you can do with
docker volume plugins

18. dvol
github.com/clusterhq/dvol

19. Use case: git for data
storageb1
compute
dev’s
laptop c1
- commit_1

20. Use case: git for data
storageb1
compute
dev’s
laptop c1
b2
- commit_1- commit_1

21. Use case: git for data
storageb1
compute
dev’s
laptop c1
b2
- commit_1
- commit_2
- commit_1

22. Use case: git for data
storageb1
compute
dev’s
laptop c1
b2
- commit_1
- commit_2
- commit_1

23. Use case: git for data
storageb1
compute
dev’s
laptop c1
b2
- commit_1
- commit_2
- commit_1
b3
- commit_1

24. Use case: git for data
storageb1
compute
dev’s
laptop c1
b2
- commit_1
- commit_2
- commit_1
b3
- commit_1
- commit_2’

25. Demo 1
“git for data”

26. Flocker (+ Volume Hub)

29. Network attached block storages1 s2
+

30. ClusterHQ
ClusterHQ
Use case: relocate app
storagev1
compute
container
host
(VM or
bare metal)
c1 c2
container composition
v2

31. ClusterHQ
Use case: relocate app
storagev1
compute
container
host
(VM or
bare metal)
c1 c2
ClusterHQ
v2
container composition

32. Demo 2
relocate app

33. docs.clusterhq.com

34. Flocker + Swarm

35. ClusterHQ
Use case: HA when node fails
storagev1
compute
container
host
(VM or
bare metal)
c1
ClusterHQ
container composer + scheduler + SD

36. ClusterHQ
Use case: HA when node fails
storagev1
compute
container
host
(VM or
bare metal)
ClusterHQ
container composer + scheduler + SD

37. ClusterHQ
Use case: HA when node fails
storagev1
compute
container
host
(VM or
bare metal)
ClusterHQ
c1
container composer + scheduler + SD

38. Demo 3
Swarm will soon have HA support…
(Flocker also integrates with Mesos &
Kubernetes)

39. Summary

40. Docker plugins allow you to do some cool
stuff with volumes and networks
Write your own…
https://docs.docker.com/engine/extend/plugins/
… or use ours

41. Container data
management in dev
+
engine,
compose
+

42. + +
Container data
management in prod

43. Email me now!
luke@clusterhq.com

44. Try dvol + Flocker today for stateful
microservices
@lmarsden @clusterhq
clusterhq.com
We are hiring in Bristol + SF Bay Area
clusterhq.com/careers