This talk provides a history of containers from early chroot environments in 1979 to modern container technologies like Docker. It discusses how containers were used in the past to manage infrastructure, including the speaker's experience with IBM containers in 2007. It also summarizes why containers gained traction in recent years due to their portability and how they changed software development practices. Finally, it discusses the speaker's company SysEleven's experience transitioning from a proprietary container platform to OpenStack, and their plans to offer managed container services using orchestration tools like Kubernetes.

1. Container through the ages
Christoph Glaubitz
Cloud Architect at SysEleven GmbH
ContainerDays Hamburg, 28.06.2016

2. WHAT COULD YOU EXPECT FROM THIS TALK?
A little bit of history on containers
How we used containers in the past
What we did to manage our infrastructure

3. What is a container?
https://www.flickr.com/photos/sewm/

4. Simplified: Each container gets its own runtime
environment, but all share the same kernel.

5. With the advantage of just running one kernel.

6. But the disadvantage of just running one kernel.

7. Same kernel version for all the containers.

8. It is not possible to load different modules for containers.

9. OS type in containers must be the same like the host.
(Which is not true any more!)

10. The most simple way of containering is a traditional
chroot environment. Introduced in 1979.

11. Which isolates runtime environments, but does not isolate
process space and devices, and does nothing with
resource control.

12. In 2000, FreeBSD jails were introduced. chroot was
extended by further isolation mechanisms, like process
namespaces. Processes in one jail can only see processes
in the same jail.

13. In this time, Virtuozzo started with containers on Linux.

14. Without a large popularity, similar concepts landed in
mainline over the years

15. Namespaces to limit things, processes can see

16. cgroups
IPC (2.6.30)
Network (2.6.24)
Mount (2.4.19)
PID
User
UTS

17. cgroups to limit resources a process can use

18. cpu
memory
devices
blkio
[…]

19. My first contact with containers was around 2007
https://en.wikipedia.org/wiki/Star_Trek:_First_Contact#/media/File:Star_Trek_08-poster.png

20. When I worked in the Tivoli Storage Manager Team at
IBM, where we had to support AIX WPARs and Solaris
Zones.

21. The ideas are still the same, also in the times of
Application Containers.
Using the same kernel.
Isolate environments as strong as possible by using the
mentioned namespaces and cgroups.

22. WHY DID CONTAINERS GET THIS TRACTION IN
THE PAST FEW YEARS?

23. I think, it the easy way to create and deploy images.
https://www.flickr.com/photos/smemon/

24. We can run the image (almost) everywhere!

25. … in a chroot without isolation and resource control using
rkt fly by CoreOS

26. … in a small KVM (with an injected kernel), again using rkt,
thanks to CoreOS and Intel

27. … on FreeBSD using JetPack

28. … even on Windows!

29. And all this without any modification to the image!

30. It changed the way we think about software.

31. One application in one container.

32. Only one process where possible.

33. Applications being stateless…

34. … and more fault tolerant.

35. Smaller chunks of source code doing one thing…

36. … which could be deployed independent and often.

37. BUT THERE WILL BE DRAGONS!

38. Unfortunately it is also very easy to build crappy images!!
https://www.flickr.com/photos/57402879@N00/

39. In my experience, deployments fail more likely because of
misconfigured next-hops (like DB), instead of crappy
software.

40. Plugging services together is even more difficult in a
floating environment.

41. More management software is needed!

42. I strongly encourage you to read on this.
https://charity.wtf/2016/05/31/wtf-is-operations-serverless/
https://charity.wtf/2016/05/31/operational-best-practices-serverless/

43. "Microservices: because solving business problems is hard
but building loosely coupled fault-tolerant distributed
systems is easy."
https://twitter.com/neil_conway/status/743086761493008384

44. SO… WHAT ARE YOU GUYS AT SYSELEVEN
DOING?

45. RUNNING TWO TYPES OF INFRASTRUCTURE!

46. A NEW ONE, BASED ON OPENSTACK…

47. … AN OLD ONE, BASED ON PARALLELS
VIRTUOZZO…

48. … but most important: Provide, maintain and monitor the
infrastructure into which our customers can deploy their
code into.

49. ON TWO LEVELS!

50. 1 . The hardware and hypervisor "layer".

51. 2 . The running Virtual Environments/containers, with all
the services in it.

52. LET'S STEP INTO THE VIRTUOZZO PLATFORM

53. In the past, Virtuozzo was basically
image format
local storage
network isolation
Linux based container runtime
(but maintained outside of mainline)

54. … treating containers like usual virtual machines
with a complete Linux running inside
starting with the init system
sure, except of the kernel
target distribution has to be supported by virtuozzo
to set up the network, hostname, etc
this is extensible by shell scripts

55. But with a huge performance benefit over VMs
https://www.flickr.com/photos/17612257@N00/

56. SOME MORE HISTORY

57. From the early days, we have decent trending and alerting
for all the services running in the containers.

58. We even built a web frontend, which gathered data from
multiple nagios instances to give us a single overview.

59. … and an AdminTV with relevant graphs

60. We run a pool of hardware nodes…
… on which we schedule the containers of all customers.
https://www.flickr.com/photos/prinsotel/

61. In the beginning we track the containers and hardware
nodes in some kind of sheet.

62. Creating a new container was like…

63. Having a look into the sheet, and select a host that does
not run the same kind of instance of this customer

64. Having a look at the metrics to verify the host still has
resources

65. ssh into the host and create the container, using the
virtuozzo tools

66. Assigning an ip and a hostname like customer.project.app1

67. Enter the container, and install stuff by hand

68. Register the new container with its services to the
monitoring by hand

69. … we developed some tools to automate most of this stuff

70. … with a recommend system to return the best Hardware
for the requested container.
https://www.flickr.com/photos/jm3/

71. Hostnames are partly calculated. customer.project is pre-
filled, but the name app2 has to be selected by hand.

72. We came up with a tagging system to be a bit more
flexible.

73. But we still totally lack a customer facing API.

74. Everything works via phone and ticketing system.
https://www.flickr.com/photos/nekudo/

75. Over time, we built billing, a local DNS-API, SSL-Cert
"shop", some own metering-endpoints, Backups…

76. At some point, Configuration Management became cool

77. … and we added a Puppet Master to our infrastructure,
and registered all the containers to it. Based on this, we
automated much stuff, like provisioning of basic stuff to
the container and registering services to the monitoring.

78. We wrote much glue-code
https://www.flickr.com/photos/samcatchesides/

79. But we still treat the containers like computers.

81. THESE DAYS WE ARE RUNNING

82. HOW DOES A TRADITIONAL SETUP LOOK

83. MySQL master slave setup
Admin server
a bunch of App server
maybe some kind of search engine

84. We start the containers, using our internal toolchain.

85. This also triggers puppet to install required software in
the container, setting up configuration and register them
to our monitoring.

86. We hand over exactly this container to the customer.
https://www.flickr.com/photos/lindzgraham/

87. Customers deploy their software to the admin server.
From there it will be rsynced to the app server.
https://www.flickr.com/photos/auxesis/

88. Same thing for dev or test setups.
https://www.flickr.com/photos/christianjann/

89. All this worked very well for a long period of time!

90. But is very inflexible!

91. Installation of containers is strongly coupled to our
infrastructure.

92. So changes to the infra hurt.

93. We have startup dependencies rather than build
dependencies.

94. Don't get me wrong!

95. You can build such problems in the shiny new container
world as well!

96. Often a new container is just a clone of an old one!
https://www.flickr.com/photos/arenamontanus/

97. With things like hostname and monitoring stuff changed

98. There is
in it.
https://www.flickr.com/photos/trevorandmarjee/

99. So we keep the containers alive, no matter what happens.
In the worst case, we clone one back from the nightly
backup.
https://www.flickr.com/photos/vagawi/

100. Updates have to go to all containers…
https://www.flickr.com/photos/bovinity/

101. … rather than building one new image and replace old
containers.
https://www.flickr.com/photos/nicowa/

102. Because customers software is not packaged in any way…

103. … there is no overview of the versions of the deployed
software.

104. Rollbacks to a defined set of installed software and
deployed software are nearly impossible.
https://www.flickr.com/photos/thejesse/

105. Anyway.
This again worked fine for a very long time…

106. But these days we need a public facing API

107. We didn't have the man power to develop our own…

108. … and would have to replace too much of the inner core.

109. And we did not want to build another proprietary
again!
https://www.flickr.com/photos/simuh/

110. So we came up with OpenStack, and created…

112. We run managed setups on SysEleven Stack.

113. Some customers choose to run production fully managed,
and dev on self managed instances.

114. The core concept of running the managed setups is pretty
much the same as before.
https://www.flickr.com/photos/yoroy/

115. But with lessons learned from the old platform!
https://www.flickr.com/photos/pictoquotes/

116. The setups are flexible, and we can also run them on AWS.

117. It based on ansible, which creates VMs using the specific
API, and runs the deployment in it.

118. We enable customers to be able to do relevant tasks on
their own, and get the full power out of the Cloud.

119. Despite of this…

120. …in the past half year one question raised…

121. What about
?

122. Basically you can run every OS you like to. Yet Windows.

123. SO. YES!

124. Just install docker into the VM.

125. Sure. The containers run in VMs. But there is IMHO no
container vs. VM, while VMs are one possible
infrastructure for containers.

126. Are there future plans to provide managed container
services?

127. OF COURSE

128. In a first step, we will help you to get the orchestration
layer up and running.

129. We help you with setting up monitoring.

130. We yet know what cantainers are. We know the tools and
how to debug in the container world.

131. We will help you to get your deployment chains to work.

132. But that is not "managed conatainer services" at all!

133. We think about providing managed orchestration layers.

134. Like

135. … or Docker Swarm

136. We would manage and monitor all the services of the
conductor.

137. We may manage additional services like a gitlab, jenkins
or a private image registry.

138. The customer still is responsible all for the container
workloads.

139. Maybe in a far
https://www.flickr.com/photos/79909830@N04/

140. We could provided managed services running in
containers.

141. But there are many things to think about!
https://www.flickr.com/photos/dharmabum1964/

142. SOME RESOURCES
https://charity.wtf/2016/05/31/wtf-is-operations-serverless/
https://charity.wtf/2016/05/31/operational-best-practices-serverless/
https://twitter.com/neil_conway/status/743086761493008384
http://kubernetes.io/
https://docs.docker.com/swarm/
https://github.com/chrigl/heat-examples
https://chrigl.de/slides/sysconf15-docker/#/ecosystem
https://chrigl.de/slides/sysconf15-docker/#/resources

143. HAPPY

144. THANKS! QUESTIONS?
Contact me:
 c.glaubitz@syseleven.de
Get Awesome Hosting:
SysEleven.de