GlusterFS Container
Gluster Meetup June 2016,
Bangalore, India
Mohamed Ashiq Liyazudeen
Associate Software Engineer @redhat
@ IRC:ashiq on freenode,
github: github.com/MohamedAshiqrh,
mailto: ashiq333@gmail.com
Humble Devassy Chirammal
Senior Software Engineer @redhat
@ IRC : hchiramm on freenode,
github : github.com/humblec,
mailto : humble.devassy@gmail.com
Gluster Meetup June 2016,
Bangalore, India
Agenda
● GlusterFS
● Docker
● Why containers?
● GlusterFS Container
● Q&A
Gluster Meetup June 2016,
Bangalore, India
GlusterFS
● Open-source scale-out distributed file system.
● Aggregates various storage servers over
network interconnects into one large parallel
network file system.
● User space filesystem with no dedicated
metadata server that helps us achieve scalability.
Gluster Meetup June 2016,
Bangalore, India
Docker
● Docker offers a simple way to package an application
and its runtime dependencies into a single container.
● Docker Container uses shared kernel concept and are
much more efficient than hypervisors in system
resource terms.
● Docker is both a development tool and a runtime
environment.
Gluster Meetup June 2016,
Bangalore, India
Container Architecture
Source: Docker Inc.
Gluster Meetup June 2016,
Bangalore, India
Docker Concepts
● Docker Image is a static specification of what the
container should be in runtime, including the application
code inside the container and runtime configuration
settings. They are Read-only.
Gluster Meetup June 2016,
Bangalore, India
Docker Concepts
● Docker container always starts with an image and is
considered an instantiation of that image.
Gluster Meetup June 2016,
Bangalore, India
Why Container?
● Shared Kernel
● Environment Regardless
● Isolation
● Minimizes System Drift
● Shipping
● Cleaner updates and Manageable state change
● Storage Hyper-converged Environment Support
Gluster Meetup June 2016,
Bangalore, India
GlusterFS Container
● Systemd is used in GlusterFS container because
gluster needs to run more than one process in the
container.
● GlusterFS configuration is made persistent as
containers writable layer is not persistent in case of re-
spawn.
● Even though GlusterFS can operate on any networking
model, we prefer host networking (--net=host) of Docker
for better performance on storage network.
Gluster Meetup June 2016,
Bangalore, India
Steps Breakdown
● Pull the image
● Prepare the node for GlusterFS configuration
persistence(create directories /var/lib/glusterd,
/var/log/glusterfs and /etc/glusterfs).
● /var/lib/glusterd – volume management files
● /var/log/glusterfs – gluster log files
● /etc/glusterfs – glusterd management files
● GlusterFS prerequisites(like opening the required port).
● Start GlusterFS container.
Gluster Meetup June 2016,
Bangalore, India
GlusterFS Container
● # docker pull gluster/gluster-centos:gluster3u7_centos7
● # mkdir -p /etc/glusterfs /var/lib/glusterd
/var/log/glusterfs
● # modprobe dm_snapshot (in case of snapshot)
● # docker run --name gluster -d -v
/etc/glusterfs:/etc/glusterfs:z -v
/var/lib/glusterd:/var/lib/glusterd:z -v
/var/log/glusterfs:/var/log/glusterfs:z -v
/sys/fs/cgroup:/sys/fs/cgroup:ro --net=host
--privileged=true -v /dev:/dev gluster/gluster-
centos:gluster3u7_centos7
Gluster Meetup June 2016,
Bangalore, India
GlusterFS docker run
As seen before its a long command, lets check why all
the options are required.
● As Gluster needs to run more than one service in a
container, It needs systemd. For systemd to run in a
container “--privileged=true”.
● Host Networking (--net=host) of Docker for better
performance on storage network.
Gluster Meetup June 2016,
Bangalore, India
Network options
● Docker Network “none”, adds a container to a
container-specific network stack.
● Docker Network ”bridge”, represents the bridge network
(default docker0).
● Docker Network ”host”, adds a container on the hosts
network stack.
Gluster Meetup June 2016,
Bangalore, India
Persistence of Gluster configuration
● -v /etc/glusterfs:/etc/glusterfs:z -v
/var/lib/glusterd:/var/lib/glusterd:z -v
/var/log/glusterfs:/var/log/glusterfs:z
● The above ensures that the gluster configuration are
persistent, In the command.
● As we mention “:z” - SELinux labels are automatically
reset to svirt_sandbox_file_t so that the container can
interact with the Host directory.
Gluster Meetup June 2016,
Bangalore, India
Gluster Peer Probe
Gluster Meetup June 2016,
Bangalore, India
Trusted Storage Pool
Gluster Meetup June 2016,
Bangalore, India
References
● http://humblec.com/building-glusterfs-in-a-docker-container/
● http://humblec.com/gluster_containers_in_kubernetes_clust
● http://humblec.com/glusterfs-containers-docker-kubernetes-
● https://hub.docker.com/r/gluster/
● https://github.com/gluster/docker
● https://github.com/gluster/glusterfs-kubernetes-openshift
Gluster Meetup June 2016,
Bangalore, India
Q & A

GlusterFS Containers

  • 1.
  • 2.
    Gluster Meetup June2016, Bangalore, India Mohamed Ashiq Liyazudeen Associate Software Engineer @redhat @ IRC:ashiq on freenode, github: github.com/MohamedAshiqrh, mailto: ashiq333@gmail.com Humble Devassy Chirammal Senior Software Engineer @redhat @ IRC : hchiramm on freenode, github : github.com/humblec, mailto : humble.devassy@gmail.com
  • 3.
    Gluster Meetup June2016, Bangalore, India Agenda ● GlusterFS ● Docker ● Why containers? ● GlusterFS Container ● Q&A
  • 4.
    Gluster Meetup June2016, Bangalore, India GlusterFS ● Open-source scale-out distributed file system. ● Aggregates various storage servers over network interconnects into one large parallel network file system. ● User space filesystem with no dedicated metadata server that helps us achieve scalability.
  • 5.
    Gluster Meetup June2016, Bangalore, India Docker ● Docker offers a simple way to package an application and its runtime dependencies into a single container. ● Docker Container uses shared kernel concept and are much more efficient than hypervisors in system resource terms. ● Docker is both a development tool and a runtime environment.
  • 6.
    Gluster Meetup June2016, Bangalore, India Container Architecture Source: Docker Inc.
  • 7.
    Gluster Meetup June2016, Bangalore, India Docker Concepts ● Docker Image is a static specification of what the container should be in runtime, including the application code inside the container and runtime configuration settings. They are Read-only.
  • 8.
    Gluster Meetup June2016, Bangalore, India Docker Concepts ● Docker container always starts with an image and is considered an instantiation of that image.
  • 9.
    Gluster Meetup June2016, Bangalore, India Why Container? ● Shared Kernel ● Environment Regardless ● Isolation ● Minimizes System Drift ● Shipping ● Cleaner updates and Manageable state change ● Storage Hyper-converged Environment Support
  • 10.
    Gluster Meetup June2016, Bangalore, India GlusterFS Container ● Systemd is used in GlusterFS container because gluster needs to run more than one process in the container. ● GlusterFS configuration is made persistent as containers writable layer is not persistent in case of re- spawn. ● Even though GlusterFS can operate on any networking model, we prefer host networking (--net=host) of Docker for better performance on storage network.
  • 11.
    Gluster Meetup June2016, Bangalore, India Steps Breakdown ● Pull the image ● Prepare the node for GlusterFS configuration persistence(create directories /var/lib/glusterd, /var/log/glusterfs and /etc/glusterfs). ● /var/lib/glusterd – volume management files ● /var/log/glusterfs – gluster log files ● /etc/glusterfs – glusterd management files ● GlusterFS prerequisites(like opening the required port). ● Start GlusterFS container.
  • 12.
    Gluster Meetup June2016, Bangalore, India GlusterFS Container ● # docker pull gluster/gluster-centos:gluster3u7_centos7 ● # mkdir -p /etc/glusterfs /var/lib/glusterd /var/log/glusterfs ● # modprobe dm_snapshot (in case of snapshot) ● # docker run --name gluster -d -v /etc/glusterfs:/etc/glusterfs:z -v /var/lib/glusterd:/var/lib/glusterd:z -v /var/log/glusterfs:/var/log/glusterfs:z -v /sys/fs/cgroup:/sys/fs/cgroup:ro --net=host --privileged=true -v /dev:/dev gluster/gluster- centos:gluster3u7_centos7
  • 13.
    Gluster Meetup June2016, Bangalore, India GlusterFS docker run As seen before its a long command, lets check why all the options are required. ● As Gluster needs to run more than one service in a container, It needs systemd. For systemd to run in a container “--privileged=true”. ● Host Networking (--net=host) of Docker for better performance on storage network.
  • 14.
    Gluster Meetup June2016, Bangalore, India Network options ● Docker Network “none”, adds a container to a container-specific network stack. ● Docker Network ”bridge”, represents the bridge network (default docker0). ● Docker Network ”host”, adds a container on the hosts network stack.
  • 15.
    Gluster Meetup June2016, Bangalore, India Persistence of Gluster configuration ● -v /etc/glusterfs:/etc/glusterfs:z -v /var/lib/glusterd:/var/lib/glusterd:z -v /var/log/glusterfs:/var/log/glusterfs:z ● The above ensures that the gluster configuration are persistent, In the command. ● As we mention “:z” - SELinux labels are automatically reset to svirt_sandbox_file_t so that the container can interact with the Host directory.
  • 16.
    Gluster Meetup June2016, Bangalore, India Gluster Peer Probe
  • 17.
    Gluster Meetup June2016, Bangalore, India Trusted Storage Pool
  • 18.
    Gluster Meetup June2016, Bangalore, India References ● http://humblec.com/building-glusterfs-in-a-docker-container/ ● http://humblec.com/gluster_containers_in_kubernetes_clust ● http://humblec.com/glusterfs-containers-docker-kubernetes- ● https://hub.docker.com/r/gluster/ ● https://github.com/gluster/docker ● https://github.com/gluster/glusterfs-kubernetes-openshift
  • 19.
    Gluster Meetup June2016, Bangalore, India Q & A