GlusterFS is an open source distributed file system that aggregates storage from multiple servers into a single logical volume. It uses a distributed hash table architecture to distribute files across storage nodes. The document discusses GlusterFS integration with OpenStack, including using GlusterFS as the backend storage for Glance images, Cinder volumes, and Nova instance disks. It provides an overview of the libgfapi application programming interface that allows direct access to GlusterFS volumes without using FUSE.
While probably the most prominent, Docker is not the only tool for building and managing containers. Originally meant to be a "chroot on steroids" to help debug systemd, systemd-nspawn provides a fairly uncomplicated approach to work with containers. Being part of systemd, it is available on most recent distributions out-of-the-box and requires no additional dependencies.
This deck will introduce a few concepts involved in containers and will guide you through the steps of building a container from scratch. The payload will be a simple service, which will be automatically activated by systemd when the first request arrives.
Anatomy of a Container: Namespaces, cgroups & Some Filesystem Magic - LinuxConJérôme Petazzoni
Containers are everywhere. But what exactly is a container? What are they made from? What's the difference between LXC, butts-nspawn, Docker, and the other container systems out there? And why should we bother about specific filesystems?
In this talk, Jérôme will show the individual roles and behaviors of the components making up a container: namespaces, control groups, and copy-on-write systems. Then, he will use them to assemble a container from scratch, and highlight the differences (and likelinesses) with existing container systems.
The latest releases of today’s popular Linux distributions include all the tools needed to do interesting things with Linux containers.
For the Makefile MicroVPS project, I set out to build a minimal virtual private server-like environment in a Linux container from scratch.
These are my requirements for the MicroVPS:
Minimal init sequence
Most of what happens in a rc.sysinit file is not needed (or wanted) in a container. However, to work like a virtual private server, the MicroVPS will need some kind of init system. The absolute minimum would be enough to start the network and at least one service.
Native network namespace
The MicroVPS will have a dedicated network namespace. It should be easy to configure.
Native package management
The package set installed in the container image will be managed using native tools like deb or rpm.
Automated build
An automated repeatable build process is a must.
Fast iteration cycle
The building and testing cycle must be fast enough not to drive me insane.
Easy management
It should be easy to distribute, monitor, and run a MicroVPS container.
In this tutorial, I will show how to use the tools included with Linux to build a virtual private server in a Linux container from scratch, using GNU Make to automate the build process.
Cgroups, namespaces, and beyond: what are containers made from? (DockerCon Eu...Jérôme Petazzoni
Linux containers are different from Solaris Zones or BSD Jails: they use discrete kernel features like cgroups, namespaces, SELinux, and more. We will describe those mechanisms in depth, as well as demo how to put them together to produce a container. We will also highlight how different container runtimes compare to each other.
This talk was delivered at DockerCon Europe 2015 in Barcelona.
KVM and docker LXC Benchmarking with OpenStackBoden Russell
Passive benchmarking with docker LXC and KVM using OpenStack hosted in SoftLayer. These results provide initial incite as to why LXC as a technology choice offers benefits over traditional VMs and seek to provide answers as to the typical initial LXC question -- "why would I consider Linux Containers over VMs" from a performance perspective.
Results here provide insight as to:
- Cloudy ops times (start, stop, reboot) using OpenStack.
- Guest micro benchmark performance (I/O, network, memory, CPU).
- Guest micro benchmark performance of MySQL; OLTP read, read / write complex and indexed insertion.
- Compute node resource consumption; VM / Container density factors.
- Lessons learned during benchmarking.
The tests here were performed using OpenStack Rally to drive the OpenStack cloudy tests and various other linux tools to test the guest performance on a "micro level". The nova docker virt driver was used in the Cloud scenario to realize VMs as docker LXC containers and compared to the nova virt driver for libvirt KVM.
Please read the disclaimers in the presentation as this is only intended to be the "chip of the ice burg".
An introduction to Linux Container, Namespace & Cgroup.
Virtual Machine, Linux operating principles. Application constraint execution environment. Isolate application working environment.
Linux Containers(LXC) allow running multiple isolated Linux instances (containers) on the same host.
Containers share the same kernel with anything else that is running on it, but can be constrained to only use a defined amount of resources such as CPU, memory or I/O.
A container is a way to isolate a group of processes from the others on a running Linux system.
Docker storage drivers by Jérôme PetazzoniDocker, Inc.
The first release of Docker only supported AUFS, and AUFS was available (out of the box) only on Debian and Ubuntu kernel. Then Red Hat wanted Docker to run on its distros, and contributed the Device Mapper driver, and later the BTRFS driver, and recently the overlayfs driver.
Jérôme presents how those drivers compare from a high-level perspective, explaining their pros and cons.
Then he showed each driver in action, and look at low-level implementation details. We won't dive into the golang implementation code itself, but we will explain the concepts of each driver. This will help to better understand how they work, and give some hints when it comes to troubleshoot their behaviour.
While probably the most prominent, Docker is not the only tool for building and managing containers. Originally meant to be a "chroot on steroids" to help debug systemd, systemd-nspawn provides a fairly uncomplicated approach to work with containers. Being part of systemd, it is available on most recent distributions out-of-the-box and requires no additional dependencies.
This deck will introduce a few concepts involved in containers and will guide you through the steps of building a container from scratch. The payload will be a simple service, which will be automatically activated by systemd when the first request arrives.
Anatomy of a Container: Namespaces, cgroups & Some Filesystem Magic - LinuxConJérôme Petazzoni
Containers are everywhere. But what exactly is a container? What are they made from? What's the difference between LXC, butts-nspawn, Docker, and the other container systems out there? And why should we bother about specific filesystems?
In this talk, Jérôme will show the individual roles and behaviors of the components making up a container: namespaces, control groups, and copy-on-write systems. Then, he will use them to assemble a container from scratch, and highlight the differences (and likelinesses) with existing container systems.
The latest releases of today’s popular Linux distributions include all the tools needed to do interesting things with Linux containers.
For the Makefile MicroVPS project, I set out to build a minimal virtual private server-like environment in a Linux container from scratch.
These are my requirements for the MicroVPS:
Minimal init sequence
Most of what happens in a rc.sysinit file is not needed (or wanted) in a container. However, to work like a virtual private server, the MicroVPS will need some kind of init system. The absolute minimum would be enough to start the network and at least one service.
Native network namespace
The MicroVPS will have a dedicated network namespace. It should be easy to configure.
Native package management
The package set installed in the container image will be managed using native tools like deb or rpm.
Automated build
An automated repeatable build process is a must.
Fast iteration cycle
The building and testing cycle must be fast enough not to drive me insane.
Easy management
It should be easy to distribute, monitor, and run a MicroVPS container.
In this tutorial, I will show how to use the tools included with Linux to build a virtual private server in a Linux container from scratch, using GNU Make to automate the build process.
Cgroups, namespaces, and beyond: what are containers made from? (DockerCon Eu...Jérôme Petazzoni
Linux containers are different from Solaris Zones or BSD Jails: they use discrete kernel features like cgroups, namespaces, SELinux, and more. We will describe those mechanisms in depth, as well as demo how to put them together to produce a container. We will also highlight how different container runtimes compare to each other.
This talk was delivered at DockerCon Europe 2015 in Barcelona.
KVM and docker LXC Benchmarking with OpenStackBoden Russell
Passive benchmarking with docker LXC and KVM using OpenStack hosted in SoftLayer. These results provide initial incite as to why LXC as a technology choice offers benefits over traditional VMs and seek to provide answers as to the typical initial LXC question -- "why would I consider Linux Containers over VMs" from a performance perspective.
Results here provide insight as to:
- Cloudy ops times (start, stop, reboot) using OpenStack.
- Guest micro benchmark performance (I/O, network, memory, CPU).
- Guest micro benchmark performance of MySQL; OLTP read, read / write complex and indexed insertion.
- Compute node resource consumption; VM / Container density factors.
- Lessons learned during benchmarking.
The tests here were performed using OpenStack Rally to drive the OpenStack cloudy tests and various other linux tools to test the guest performance on a "micro level". The nova docker virt driver was used in the Cloud scenario to realize VMs as docker LXC containers and compared to the nova virt driver for libvirt KVM.
Please read the disclaimers in the presentation as this is only intended to be the "chip of the ice burg".
An introduction to Linux Container, Namespace & Cgroup.
Virtual Machine, Linux operating principles. Application constraint execution environment. Isolate application working environment.
Linux Containers(LXC) allow running multiple isolated Linux instances (containers) on the same host.
Containers share the same kernel with anything else that is running on it, but can be constrained to only use a defined amount of resources such as CPU, memory or I/O.
A container is a way to isolate a group of processes from the others on a running Linux system.
Docker storage drivers by Jérôme PetazzoniDocker, Inc.
The first release of Docker only supported AUFS, and AUFS was available (out of the box) only on Debian and Ubuntu kernel. Then Red Hat wanted Docker to run on its distros, and contributed the Device Mapper driver, and later the BTRFS driver, and recently the overlayfs driver.
Jérôme presents how those drivers compare from a high-level perspective, explaining their pros and cons.
Then he showed each driver in action, and look at low-level implementation details. We won't dive into the golang implementation code itself, but we will explain the concepts of each driver. This will help to better understand how they work, and give some hints when it comes to troubleshoot their behaviour.
Turning object storage into vm storagewim_provoost
Object Storage is today the standard to build scale-out storage. But due to technical hurdles it is impossible to run Virtual Machines directly from an Object Store. Open vStorage is the layer between the hypervisor and Object Store and turns the Object Store into a high performance, distributed, VM-centric storage platform.
This is to introduce the related components in SUSE Linux Enterprise High Availability Extension product to build High Available Storage (ha-lvm/drbd/iscsi/nfs, clvm, ocfs2, cluster-raid1).
GlusterFS : un file system open source per i big data di oggi e domani - Robe...Codemotion
GlusterFS (www.gluster.org) è un file system distribuito open source, scalabile fino ai petabytes. La presentazione ha lo scopo di mostrare le feature di questo FS e la nostra esperienza, che parte nel 2010 con un cluster da 4TB all'odierno da 30TB: perché è stato scelto, principali features, evoluzione, fallimenti (anche quelli), futuro. Alcune feature: accesso in user-space, protocollo nativo, NFS, SMB . Replicazione, distribuzione, striping dei file o una loro combinazione (e.g: distributed striped replicated). All'interno dell'ecosistema Hadoop può sostituire HDFS.
OSDC 2016 - Interesting things you can do with ZFS by Allan Jude&Benedict Reu...NETWAYS
ZFS is the next generation filesystem originally developed at Sun Microsystems. Available under the CDDL, it uniquely combines volume manager and filesystem into a powerful storage management solution for Unix systems. Regardless of big or small storage requirements. ZFS offers features, for free, that are usually found only in costly enterprise storage solutions. This talk will introduce ZFS and give an overview of its features like snapshots and rollback, compression, deduplication as well as replication. We will demonstrate how these features can make a difference in the datacenter, giving administrators the power and flexibility to adapt to changing storage requirements.
Real world examples of ZFS being used in production for video streaming, virtualization, archival, and research are shown to illustrate the concepts. The talk is intended for people considering ZFS for their data storage needs and those who are interested in the features ZFS provides.
Lukas Macura - Employing Zabbix to monitor OpenWrt (Beesip) devices with UciprovZabbix
Beesip (Bright Efficient Embedded Solution for IP Telephony) is platform based on OpenWrt, primarily made for IP telephony purposes. The aim of the project is the development and implementation of embedded SIP communication server with an easy integration into the computer network based on open-source solutions. Beesip uses uciprov for remote management and provisioning. Next to this, it uses Zabbix for inventory and monitoring utilizing auto-discovery and auto-registration. Even if it is used primarily for IP telephony, it is common to use Beesip build system for other purposes like monitoring probes distribution or Eduroam AP management. Talk will be practically oriented showing possibilities of provisioning and auto inventory in OpenWrt world.
Zabbix Conference 2015
[KubeCon NA 2020] containerd: Rootless Containers 2020Akihiro Suda
Rootless Containers means running the container runtimes (e.g. runc, containerd, and kubelet) as well as the containers without the host root privileges. The most significant advantage of Rootless Containers is that it can mitigate potential container-breakout vulnerability of the runtimes, but it is also useful for isolating multi-user environments on HPC hosts. This talk will contain the introduction to rootless containers and deep-dive topics about the recent updates such as Seccomp User Notification. The main focus will be on containerd (CNCF Graduated Project) and its consumer projects including Kubernetes and Docker/Moby, but topics about other runtimes will be discussed as well.
https://sched.co/fGWc
This document describes how XenServer provides and keeps track of the storage supplied to its guests. The first section
is a reminder of how Linux looks at storage and the second section builds on that to explain XenServer storage. Basic
knowledge of Linux is required, as some standard tools are used.
containerit at useR!2017 conference, BrusselsDaniel Nüst
**Webpage**
https://github.com/o2r-project/containerit/
**Abstract**
Reproducibility of computations is crucial in an era where data is born digital and analysed algorithmically. Most studies however only publish the results, often with figures as important interpreted outputs. But where do these figures come from? Scholarly articles must provide not only a description of the work but be accompanied by data and software. R offers excellent tools to create reproducible works, i.e. Sweave and RMarkdown. Several approaches to capture the workspace environment in R have been made, working around CRAN’s deliberate choice not to provide explicit versioning of packages and their dependencies. They preserve a collection of packages locally (packrat, pkgsnap, switchr/GRANBase) or remotely (MRAN timemachine/checkpoint), or install specific versions from CRAN or source (requireGitHub, devtools). Installers for old versions of R are archived on CRAN. A user can manually re-create a specific environment, but this is a cumbersome task.
We introduce a new possibility to preserve a runtime environment including both, packages and R, by adding an abstraction layer in the form of a container, which can execute a script or run an interactive session. The package containeRit automatically creates such containers based on Docker. Docker is a solution for packaging an application and its dependencies, but shows to be useful in the context of reproducible research (Boettiger 2015). The package creates a container manifest, the Dockerfile, which is usually written by hand, from sessionInfo(), R scripts, or RMarkdown documents. The Dockerfiles use the Rocker community images as base images. Docker can build an executable image from a Dockerfile. The image is executable anywhere a Docker runtime is present. containeRit uses harbor for building images and running containers, and sysreqs for installing system dependencies of R packages. Before the planned CRAN release we want to share our work, discuss open challenges such as handling linked libraries (see discussion on geospatial libraries in Rocker), and welcome community feedback.
containeRit is developed within the DFG-funded project Opening Reproducible Research to support the creation of Executable Research Compendia (ERC) (Nüst et al. 2017).
**References**
Boettiger, Carl. 2015. “An Introduction to Docker for Reproducible Research, with Examples from the R Environment.” ACM SIGOPS Operating Systems Review 49 (January): 71–79. doi:10.1145/2723872.2723882.
Nüst, Daniel, Markus Konkol, Edzer Pebesma, Christian Kray, Marc Schutzeichel, Holger Przibytzin, and Jörg Lorenz. 2017. “Opening the Publication Process with Executable Research Compendia.” D-Lib Magazine 23 (January). doi:10.1045/january2017-nuest.
Introducton to Convolutional Nerural Network with TensorFlowEtsuji Nakai
Explaining basic mechanism of the Convolutional Neural Network with sample TesnsorFlow codes.
Sample codes: https://github.com/enakai00/cnn_introduction
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Welocme to ViralQR, your best QR code generator.ViralQR
Welcome to ViralQR, your best QR code generator available on the market!
At ViralQR, we design static and dynamic QR codes. Our mission is to make business operations easier and customer engagement more powerful through the use of QR technology. Be it a small-scale business or a huge enterprise, our easy-to-use platform provides multiple choices that can be tailored according to your company's branding and marketing strategies.
Our Vision
We are here to make the process of creating QR codes easy and smooth, thus enhancing customer interaction and making business more fluid. We very strongly believe in the ability of QR codes to change the world for businesses in their interaction with customers and are set on making that technology accessible and usable far and wide.
Our Achievements
Ever since its inception, we have successfully served many clients by offering QR codes in their marketing, service delivery, and collection of feedback across various industries. Our platform has been recognized for its ease of use and amazing features, which helped a business to make QR codes.
Our Services
At ViralQR, here is a comprehensive suite of services that caters to your very needs:
Static QR Codes: Create free static QR codes. These QR codes are able to store significant information such as URLs, vCards, plain text, emails and SMS, Wi-Fi credentials, and Bitcoin addresses.
Dynamic QR codes: These also have all the advanced features but are subscription-based. They can directly link to PDF files, images, micro-landing pages, social accounts, review forms, business pages, and applications. In addition, they can be branded with CTAs, frames, patterns, colors, and logos to enhance your branding.
Pricing and Packages
Additionally, there is a 14-day free offer to ViralQR, which is an exceptional opportunity for new users to take a feel of this platform. One can easily subscribe from there and experience the full dynamic of using QR codes. The subscription plans are not only meant for business; they are priced very flexibly so that literally every business could afford to benefit from our service.
Why choose us?
ViralQR will provide services for marketing, advertising, catering, retail, and the like. The QR codes can be posted on fliers, packaging, merchandise, and banners, as well as to substitute for cash and cards in a restaurant or coffee shop. With QR codes integrated into your business, improve customer engagement and streamline operations.
Comprehensive Analytics
Subscribers of ViralQR receive detailed analytics and tracking tools in light of having a view of the core values of QR code performance. Our analytics dashboard shows aggregate views and unique views, as well as detailed information about each impression, including time, device, browser, and estimated location by city and country.
So, thank you for choosing ViralQR; we have an offer of nothing but the best in terms of QR code services to meet business diversity!
4. 4
What is GlusterFS?
GlusterFS is opensource software to create a scale-out distributed
filesystem on top of commodity x86_86 servers.
– It aggregates local storage of many servers into a single logical volume.
– You can extend the volume just by adding more servers.
GlusterFS runs on top of Linux. You can use it wherever you can use Linux.
– Physical/virtual machines in your data center.
– Linux VM on public clouds.
GlusterFS provides a wide variety of APIs.
– FUSE mount (using the native client.)
– NFSv3 (supporting the distributed NFS lock.)
– CIFS (using libgfapi native access from Samba.)
– REST API (compatible with OpenStack Swift.)
– Native application library (providing POSIX-like system calls.)
5. 5
Brief history of GlusterFS
2005 2011 2012 2013 2014
GlusterFS 3.3
GlusterFS 3.4
GlusterFS 3.5
http://www.slideshare.net/johnmarkorg/gluster-where-weve-been-a-history
Red Hat acquisition
of Gluster Inc.
The early days
of Gluster Inc.
6. 6
Architecture overview
The standard filesystem (typically xfs) of each storage node is used as a backend device
of the logical volume.
– Each file in the volume is physically stored in one of the storage nodes' filesystem, just as the
same plain file seen from the client.
Hash value of the file name is used to decide the node to store it.
– The metadata server storing the file location is not used in GlusterFS.
file01 file02 file03
・・・ Storage nodes
file01, file02, file03
GlusterFS client
The volume is seen as a single filesystem
mounted on a local directory tree.
Files are distributed across
local filesystem of storage nodes.
GlusterFS volume
7. 7
Hierarchy structure consisted of Node / Brick / Volume
・・・
Volume vol01
Filesystem mounted on /data
/data/brick02
/data/brick01 Brick
(Just a directory)
・・・
/data/brick02
/data/brick01
・・・
/data/brick02
/data/brick01
A volume is created as a
"bundle" of bricks which are
provided by storage nodes.
Node01
A single node can provide multiple bricks to create multiple volumes.
You don't need to use the same number of bricks nor the same directory name on each node.
You can add/remove bricks to extend/reduce the size of volumes.
Node02 Node03
10. 10
DHT: Distributed Hash Table
Distributed has table is:
– A rule for deciding a brick to store the file based on filename's hash value.
– More precisely, it's just a table of bricks and corresponding hash ranges.
file01
Brick1
Hash range 0〜99
Calculate the hash
value of filename.
Brick2
Hash range 100〜199
・・・
127
Stored in the brick which is
responsible for this hash value.
Brick3
Hash range 200〜299
The actual hash length is 32bit.
0x00000000 〜 0xFFFFFFFF
Brick1 Brick2 Brick3 ・・・
Hash
range
0〜99 100〜199 200〜299
DHT (Distributed Hash Table)
11. 11
DHT structure in GlusterFS
Hash tables are created for each directory in a single volume.
– Two files with the same name (in different directories) are placed in different bricks.
– By assigning different hash ranges for different directories, files are more evenly distributed.
The hash range of each brick (directory) is recorded in the extended attribute of the
directory.
Brick1
[root@gluster01 ~]# getfattr -d -m . /data/brick01/dir01
getfattr: Removing leading '/' from absolute path names
# file: data/brick01/dir01
trusted.gfid=0shk2IwdFdT0yI1K7xXGNSdA==
trusted.glusterfs.10d3504b-7111-467d-8d4f-d25f0b504df6.xtime=0sT+vTRwADqyI=
trusted.glusterfs.dht=0sAAAAAQAAAAB//////////w==
Brick1 Brick2 Brick3 ・・・
/dir01 0〜99 100〜199 200〜299 ・・・
/dir02 100〜199 400〜499 300〜399 ・・・
/dir03 500〜599 200〜299 100〜199 ・・・
・・・
Brick2 Brick3 ・・・
12. 12
How GlusterFS client recognizes the hash table
# mount -t glusterfs gluster01:/vol01
Volume "vol01" is provided
by gluster01〜gluster04
gluster01 gluster02 gluster03 gluster04
13. 13
How GlusterFS client recognizes the hash table
# cat /vol01/dir01/file01
gluster01 gluster02 gluster03 gluster04
The hash range of dir01
is xxx.
The hash range of dir01
is yyy.
14. 14
How GlusterFS client recognizes the hash table
# cat /vol01/dir01/file01
gluster01 gluster02 gluster03 gluster04
The hash range of dir01
is xxx.
The hash range of dir01
is yyy.
Construct the whole hash table
for dir01 on memory!
Brick1 Brick2 Brick3 ・・・
dir01 0〜99 100〜199 200〜299
15. 15
Translator modules
GlusterFS works with multiple translator modules.
– There are modules running on clients and modules running on servers.
Each module has its own role.
– Translator modules are built as shared library.
– Original modules can be added as a plug-in.
[root@gluster01 ~]# ls -l /usr/lib64/glusterfs/3.3.0/xlator/
total 48
drwxr-xr-x 2 root root 4096 Jun 16 15:25 cluster
drwxr-xr-x 2 root root 4096 Jun 16 15:25 debug
drwxr-xr-x 2 root root 4096 Jun 16 15:25 encryption
drwxr-xr-x 2 root root 4096 Jun 16 15:25 features
drwxr-xr-x 2 root root 4096 Jun 16 15:25 mgmt
drwxr-xr-x 2 root root 4096 Jun 16 15:25 mount
drwxr-xr-x 2 root root 4096 Jun 16 15:25 nfs
drwxr-xr-x 2 root root 4096 Jun 16 15:25 performance
drwxr-xr-x 2 root root 4096 Jun 16 15:25 protocol
drwxr-xr-x 2 root root 4096 Jun 16 15:25 storage
drwxr-xr-x 2 root root 4096 Jun 16 15:25 system
drwxr-xr-x 3 root root 4096 Jun 16 15:25 testing
DHT, replication, etc.
quota, file lock, etc.
caching, read ahead, etc.
physical I/O
16. 16
Typical combination of translator modules
io-stats
md-cache
quick-read
io-cache
read-ahead
write-behind
dht
replicate-1 replicate-2
server
brick
marker
index
io-threads
locks
access-control
posix
server
brick
marker
index
io-threads
locks
access-control
posix
server
brick
marker
index
io-threads
locks
access-control
posix
server
brick
marker
index
io-threads
locks
access-control
posix
client-1 client-2 client-3 client-4
Client modules(*1)
Server modules(*2)
Brick
Recording statistics information
Metadata caching
Data caching
Handling DHT
Replication
Communication with servers
Communication with clients
Activating I/O thereads
File locking
ACL management
Physical access to bricks
Brick Brick Brick
(*1) Defined in /var/lib/glusterd/vols/<Vol>/<Vol>-fuse.vol (*2) Defined in /var/lib/glusterd/vols/<Vol>/<Vol>.<Node>.<Brick>.vol
17. 17
The past wish list for GlusterFS
Volume Snapshot (master branch)
File Snapshot (GlusterFS3.5)
On-wire compression / decompression (GlusterFS3.4)
Disk Encryption (GlusterFS3.4)
Journal based distributed GeoReplication (GlusterFS3.5)
Erasure coding (Not yet...)
Integration with OpenStack
etc...
http://www.gluster.org/
19. 19
Four locations you need storage system in OpenStack
Swift
Nova Compute
Glance
Application
Data
OS
Cinder
Object Store
Template
Image
Typcally, original distributed
object store using commodity
x86_86 servers is used.
Typcally, external hardware
storege (iSCSI) is used
Typcally, local storage of
compute nodes is used.
Typcally, Swift or NFS
storage is used.
20. Using GlusterFS for Glance backend
GlusterFS Cluster
GlusterFS
Volume
GlusterFS manages scalability, redundancy
and consistency.
Glance Server
Just use GlusterFS volume instead of local storage. So simple.
This is actually being used in many production clusters.
21. 21
Nova Compute
Cinder
VM instance
/dev/vdb
Virtual disk
Linux KVM
/dev/sdX
iSCSI LUN
Storage box
Create LUNs
iSCSI SW
Initiator
iSCSI Target
In typical configuration, block volumes are created as LUNs in iSCSI storage boxes.
Cinder operates on the management interface of the storage through the
corresponding driver.
Nova Compute attaches it to the host Linux using the software initiator, then it's
attached to the VM instance through KVM hypervisor.
How Nova and Cinder works together
22. 22
Cinder also provides the NFS driver which uses NFS server as a storage backend.
– The driver simply mounts the NFS exported directly and create disk image files
in it. Compute nodes use NFS mount to access the image files.
Virtual disk
NFS server
NFS mount
・・・
NFS mount
・・・
Nova ComputeVM instance
/dev/vdb
Linux KVM
Cinder
Using NFS driver
23. 23
There is a driver for GlusterFS distributed filesystem, too.
– Currently it uses FUSE mount mechanism. This will be replaced with more optimized
mechanism (libgfapi) which bypasses the FUSE layer.
Cinder
GlusterFS cluster
FUSE mount
FUSE mount
・・・
Virtual disk
・・・
Nova ComputeVM instance
/dev/vdb
Linux KVM
Using GlusterFS driver for Cinder
24. 24
The same can work for Nova Compute. You can store running VM's OS
image on locally mounted GlusterFS volume.
GlusterFS cluster
FUSE mount
・・・
Virtual disk
・・・
Nova ComputeVM instance
/dev/vda
Linux KVM
GlusterFS shared volume for Nova Compute
Template
Image
25. 25
The FUSE mount/file based architecture is not well suited to workload
for VM disk images (small random I/O).
How can we imporve it?
The challenge in Cinder/Nova Compute integration
26. 26
The FUSE mount/file based architecture is not well suited to workload
for VM disk images (small random I/O).
How can we imporve it?
The challenge in Cinder/Nova Compute integration
http://www.inktank.com/
Using Ceph?
CENSORED
27. 27
"libgfapi" is an application library with which user applications can
directly access GlusterFS volume via native protocol.
– It reduces the overhead of FUSE architecture.
GlusterFS way for qemu integration
Now qemu is integrated with libgfapi
so that it can directly access disk
image files placed in GlusterFS
volume.
– This feature is available since Havana
release.
FUSE mount
libgfapi
28. Architecture of Swift Account Servers
Maintain mappings
between
accounts and containers
Container Servers
Object Servers
Maintain lists and ACLs
of objects
in each container.
Store object contents
in file system.
Proxy Servers
Handling REST
request from clients
Authentication Server
DB
DB
File System
29. Architecture of GlusterFS with Swift API
Proxy / Account / Container / Object
“all in one” server & GlusterFS client
Authentication Server
GlusterFS Cluster
One volume is used
for one account
Account/Container/Object Server
modules retrieve required information
directly from locally mounted volumes.
GlusterFS
Volume
Volume for each account is locally mounted at:
/mnt/gluster-object/AUTH_<account name>
GlusterFS manages scalability, redundancy
and consistency.
31. Using libgfapi with RHEL6/CentOS6
Install development tools, and libgfapi library from EPEL repository.
Build your application with libgfapi.
That's all!
Pseudo-Posix I/O system calls are listed in the header file.
– https://github.com/gluster/glusterfs/blob/release-3.5/api/src/glfs.h
– file stream and mmap are not there :-(
# yum install http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
# yum groupinstall "Development Tools"
# yum install glusterfs-api-devel
# gcc hellogluster.c -lgfapi
# ./a.out
32. "Hello, World!" with libgfapi
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <glusterfs/api/glfs.h>
int main (int argc, char** argv) {
const char *gfserver = "gluster01";
const char *gfvol = "testvol01";
int ret;
glfs_t *fs;
glfs_fd_t *fd;
fs = glfs_new(gfvol);
glfs_set_volfile_server(fs, "tcp", gfserver, 24007);
ret = glfs_init (fs);
if (ret) {
printf( "Failed to connect server/volume: %s/%sn", gfserver, gfvol );
exit(ret);
}
char *greet = "Hello, Gluster!n";
fd = glfs_creat(fs, "greeting.txt", O_RDWR, 0644);
glfs_write(fd, greet, strlen(greet), 0);
glfs_close(fd);
return 0;
}
type struct representing the volume (filesystem) "testvol01"
Connecting to the volume.
Opening a new file on the volume.
Write and close the file.