This document summarizes a presentation given by Masanori Itoh on OCDET activity and Glusterfs. OCDET is a non-profit organization in Japan that promotes open source cloud computing components through proof-of-concept activities. The presentation discussed OCDET's goals and organization, provided examples of using Glusterfs for VM image storage with OpenStack and CloudStack, and outlined future expectations and ideas for Glusterfs, including enhancing the OpenStack Cinder driver and improving InfiniBand support.
ABCI: AI Bridging Cloud Infrastructure for Scalable AI/Big DataHitoshi Sato
Presentation Slides for ExaComm2018, Fourth International Workshop on Communication Architectures for HPC, Big Data, Deep Learning and Clouds at Extreme Scale, in conjunction with International Supercomputing Conference (ISC 2018)
http://nowlab.cse.ohio-state.edu/exacomm/
Building and operating HPC-based AI computing environment inside Gwangju Institute of Science and Technology
For using the part of the slide, you need to cite "Narantuya Jargalsaikhan, GIST AI-X Computing Cluster, 2021".
Thank you!
ABCI: AI Bridging Cloud Infrastructure for Scalable AI/Big DataHitoshi Sato
Presentation Slides for ExaComm2018, Fourth International Workshop on Communication Architectures for HPC, Big Data, Deep Learning and Clouds at Extreme Scale, in conjunction with International Supercomputing Conference (ISC 2018)
http://nowlab.cse.ohio-state.edu/exacomm/
Building and operating HPC-based AI computing environment inside Gwangju Institute of Science and Technology
For using the part of the slide, you need to cite "Narantuya Jargalsaikhan, GIST AI-X Computing Cluster, 2021".
Thank you!
What's Next in OpenStack? A Glimpse At The RoadmapShamailXD
YouTube Recording: https://www.youtube.com/watch?v=cCdqOxD5G0M
Whether you are a newbie to OpenStack looking at building your first cloud or an experienced operator with years of OpenStack success behind you, you've probably spent some time wondering what to expect from the OpenStack project over the next several releases. Will it finally support that new capability you've been waiting for? Should you plan for an upgrade in the next 6 months? While the development community is always working and planning new features, its takes a lot of time on IRC to get a complete view across the different projects. The OpenStack Product WG spent time this cycle working with the project teams and PTLs to understand their priorities for the next several OpenStack releases. Where we have always had an understanding of what's to come in the next release, we're hoping to present a long-term view of the future landscape of OpenStack. In this session, we'll present our findings across the different projects in an effort to give users a glimpse into the OpenStack roadmap
Slides from our Q3 meetup held in Montreal on September 27th 2017 at the Cloud.ca Center.
Video recording can be seen at: https://www.youtube.com/watch?v=_1btwHW39ms&list=PLSsQodeQD6LPyqrvvczcC5mkOOnPt469o
It's been said that open source software is eating the world. In the observability space, the project making this possible is OpenTelemetry. It's quickly becoming the standard for instrumentation and data collection of observability data. Understanding what data to collect and how to collect it properly is fundamental to ensuring users can quickly address availability and performance issues. Steve Flanders, Director of Engineering at Splunk, discusses the components of the project, its current status, and how you can get started integrating it into your modern app infrastructure.
Speakers:
Steve Flanders
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storageMayaData Inc
Webinar Session - https://youtu.be/_5MfGMf8PG4
In this webinar, we share how the Container Attached Storage pattern makes performance tuning more tractable, by giving each workload its own storage system, thereby decreasing the variables needed to understand and tune performance.
We then introduce MayaStor, a breakthrough in the use of containers and Kubernetes as a data plane. MayaStor is the first containerized data engine available that delivers near the theoretical maximum performance of underlying systems. MayaStor performance scales with the underlying hardware and has been shown, for example, to deliver in excess of 10 million IOPS in a particular environment.
Rook: Storage for Containers in Containers – data://disrupted® 2020data://disrupted®
In this talk Kim-Norman Sahm and Alexander Trost dive into the challenges of storage for containerized applications on Kubernetes. We'll see how the current state is and how Rook can help with that. We are going to especially look at Ceph run through Rook here, but nonetheless trying not to lose sight of the whole picture. There is a lot to keep in mind storage as is, but everything gets more complex with storage for containers. From what type of storage to how much and how "safe" it should, all questions that should be asked and most of them which should be answered as well. Rook's project site https://rook.io/
Kim-Norman Sahm is CTO of Cloudical. He also works as Executive Cloud Architect at Cloudical. Previously, he was OpenStack Cloud Architect at T-Systems (operational services GmbH) and noris network AG. He is an expert of the technologies OpenStack, Ceph and Kubernetes (CKA).
Alexander Trost works as a DevOps Engineer at Cloudical Deutschlang GmbH and is a Certified Kubernetes Administrator (CKA). He is one of four maintainers of the Rook.io Project and is engaged in several more open source projects, for example a Prometheus exporter for Dell Hardware (Dell OMSA Metrics), k8s-vagrant-multi-node an easy local multi node Kubernetes environment, and others. Besides Containers and Kubernetes he is expert on Software Defined Storage, Golang and Continuous Integration (with GitLab CI). He passionately enjoys working on open source projects, such as Rook, Ancientt and other projects.
What's Next in OpenStack? A Glimpse At The RoadmapShamailXD
YouTube Recording: https://www.youtube.com/watch?v=cCdqOxD5G0M
Whether you are a newbie to OpenStack looking at building your first cloud or an experienced operator with years of OpenStack success behind you, you've probably spent some time wondering what to expect from the OpenStack project over the next several releases. Will it finally support that new capability you've been waiting for? Should you plan for an upgrade in the next 6 months? While the development community is always working and planning new features, its takes a lot of time on IRC to get a complete view across the different projects. The OpenStack Product WG spent time this cycle working with the project teams and PTLs to understand their priorities for the next several OpenStack releases. Where we have always had an understanding of what's to come in the next release, we're hoping to present a long-term view of the future landscape of OpenStack. In this session, we'll present our findings across the different projects in an effort to give users a glimpse into the OpenStack roadmap
Slides from our Q3 meetup held in Montreal on September 27th 2017 at the Cloud.ca Center.
Video recording can be seen at: https://www.youtube.com/watch?v=_1btwHW39ms&list=PLSsQodeQD6LPyqrvvczcC5mkOOnPt469o
It's been said that open source software is eating the world. In the observability space, the project making this possible is OpenTelemetry. It's quickly becoming the standard for instrumentation and data collection of observability data. Understanding what data to collect and how to collect it properly is fundamental to ensuring users can quickly address availability and performance issues. Steve Flanders, Director of Engineering at Splunk, discusses the components of the project, its current status, and how you can get started integrating it into your modern app infrastructure.
Speakers:
Steve Flanders
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storageMayaData Inc
Webinar Session - https://youtu.be/_5MfGMf8PG4
In this webinar, we share how the Container Attached Storage pattern makes performance tuning more tractable, by giving each workload its own storage system, thereby decreasing the variables needed to understand and tune performance.
We then introduce MayaStor, a breakthrough in the use of containers and Kubernetes as a data plane. MayaStor is the first containerized data engine available that delivers near the theoretical maximum performance of underlying systems. MayaStor performance scales with the underlying hardware and has been shown, for example, to deliver in excess of 10 million IOPS in a particular environment.
Rook: Storage for Containers in Containers – data://disrupted® 2020data://disrupted®
In this talk Kim-Norman Sahm and Alexander Trost dive into the challenges of storage for containerized applications on Kubernetes. We'll see how the current state is and how Rook can help with that. We are going to especially look at Ceph run through Rook here, but nonetheless trying not to lose sight of the whole picture. There is a lot to keep in mind storage as is, but everything gets more complex with storage for containers. From what type of storage to how much and how "safe" it should, all questions that should be asked and most of them which should be answered as well. Rook's project site https://rook.io/
Kim-Norman Sahm is CTO of Cloudical. He also works as Executive Cloud Architect at Cloudical. Previously, he was OpenStack Cloud Architect at T-Systems (operational services GmbH) and noris network AG. He is an expert of the technologies OpenStack, Ceph and Kubernetes (CKA).
Alexander Trost works as a DevOps Engineer at Cloudical Deutschlang GmbH and is a Certified Kubernetes Administrator (CKA). He is one of four maintainers of the Rook.io Project and is engaged in several more open source projects, for example a Prometheus exporter for Dell Hardware (Dell OMSA Metrics), k8s-vagrant-multi-node an easy local multi node Kubernetes environment, and others. Besides Containers and Kubernetes he is expert on Software Defined Storage, Golang and Continuous Integration (with GitLab CI). He passionately enjoys working on open source projects, such as Rook, Ancientt and other projects.
This presentation is part of Container Conference 2018 presented by Vaibhav Choudhary, Principal Staff Engineer, Oracle: www.containerconf.in
--
Container become the fundamental building block unit of modern application and for cloud. So, lets learn in this session that how you can containerize (dockerize in the demo) a java image. What are the effort we put to make Java, Container aware. How Java started understanding Container and start respecting the container boundary. We will also learn how you can leverage java features like AppCDS, AOT inside the container and make it faster than even.
--
Netflix Open Source: Building a Distributed and Automated Open Source Programaspyker
Netflix has been using and contributing to open source for several years. Over the years, Netflix has released over one hundred Netflix Open Source (aka NetflixOSS) libraries, servers, and technologies. Netflix engineers benefit by accepting contributions and gathering feedback with key collaborators around the world. Users of NetflixOSS from many industries benefit from our solutions including Big Data, Build and Delivery Tools, Runtime Services and Libraries, Data Persistence, Insight, Reliability and Performance, Security and User Interface. With such a large and mature open source program, Netflix has worked on approaches and tools that help manage and improve the NetflixOSS source offerings and communities. Netflix has taken a different approach to building support for open source as compared to other Internet scale companies. Come to this session to learn about the unique approaches Netflix has taken to both distribute and automate the responsibilities of building a world-class open source program.
Building a Distributed & Automated Open Source Program at NetflixAll Things Open
Andrew Spyker
Senior Software Engineer for Netflix
Find more by Andrew Spyker: http://www.slideshare.net/aspyker
All Things Open
October 26-27, 2016
Raleigh, North Carolina
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
OCDET Activity and Glusterfs
1. OCDET Activity and Glusterfs
May 31, 2013
Gluster Workshop @ LinuxCon / OpenCloud Japan 2013
Copyright (c) OCDET All Rights Reserved.
May 31, 2013
OCDET / NTT DATA CORPORATION
Masanori Itoh
2. Who I am
Masanori Itoh
– Professional Affiliation
• NTT DATA CORPORATION
– A Japanese System Integrator (NOT a Telco ! ☺)
– Activities
• OCDET (Steering Committee) Today, I’m on behalf of
Copyright (c) OCDET All Rights Reserved. 1
• OCDET (Steering Committee)
• Japan OpenStack Users Group
• etc.
– Speciality
• Operating System
– Filesystem, Networking, …
• Trouble Shooting … :o
Today, I’m on behalf of
OCDET !
3. Agenda
1. What is OCDET ?
2. OSS IaaS with Glusterfs
3. Future Expectations and Ideas for Glusterfs
4. Summary
Copyright (c) OCDET All Rights Reserved. 2
5. What is OCDET?
Home Page : http://ocdet.org/ (Sorry, No English page… )
Copyright (c) OCDET All Rights Reserved. 4
6. What is OCDET?
OCDET stands for :
Open Cloud Demonstration Experiment Taskforce
オープン クラウド 実証 実験 タスクフォース
BTW, “Demonstration Experiment” ?
Copyright (c) OCDET All Rights Reserved. 5
BTW, “Demonstration Experiment” ?
– In Japanese,
• “実証実験” (Jissho-Jikken)
– In Short,
• An Activity on POC : Proof-of-Concept (of Something)
7. What is OCDET? : the Goal
the Goal
To Promote “Open Source Cloud Computing Components”
• From the Lowest Layer to Upper Layer things.
• Even including Hardware (!?)
Prove that we can Build Mature Infrastructure Using Them
By doing various POC activities
Copyright (c) OCDET All Rights Reserved. 6
By doing various POC activities
Under Fully Volunteered and Non-Profit Organization
And Share the Results Publicly
• Documentation
– Installation/Configuration Procedures, System Design
Guidelines, Knowhows , … etc.
• Tools, Sample Configuration Files, …
8. What is OCDET? : Overview
Formed on:
Feb. 3, 2012
Participants:
24 Organization Members
Founding Organizations : Hokkaido University,
NTT Communications, Cloud Business Association, CUPA
Copyright (c) OCDET All Rights Reserved. 7
NTT Communications, Cloud Business Association, CUPA
68 Active Volunteer Members (May 2013)
Organization
Steering Committee
Study Groups / Working Groups
9. What is OCDET? : Press Coverage
Study Groups / Working Groups
http://ocdet.org/information.html#pageLink02
Copyright (c) OCDET All Rights Reserved. 8
10. What is OCDET? : Organization
Study Groups / Working Groups
http://ocdet.org/information.html#pageLink02
Copyright (c) OCDET All Rights Reserved. 9
11. What is OCDET? : Organization
Study Groups / Working Groups
Open IaaS Study Group
• OpenStack WG
• CloudStack WG
• System Management WG
Open Storage Study Group
Orchestration SIG
Copyright (c) OCDET All Rights Reserved. 10
Open Storage Study Group
• Swift, Glusterfs, etc.
Open Network Study Group
Open PaaS Study Group
• CloudFoundry <--> Japan CloudFoundry User Group
Open Facility Study Group
• OpenCompute WG <--> OCP Japan
13. IaaS Software and Glusterfs : IaaS at a Glance
Provide Infrastructure Level Resources on Demand
Compute Resource
CPU, Memory, …
Network Resource
L2 Network, IP address Management, Router, Firewall, Load
Balancer, VPN, …
Copyright (c) OCDET All Rights Reserved. 12
Balancer, VPN, …
Storage Resource
File Access Store
Block Access Store
Object Storage Store
14. IaaS Software and Glusterfs : Examples
Examples of Public IaaS Service
Amazon Web Services
RackSpace Cloud
HP Cloud Services
etc.
Copyright (c) OCDET All Rights Reserved. 13
Examples of Open Source Software for IaaS
OpenStack
CloudStack
Eucalyptus
OpenNebula
etc.
15. IaaS Software and Glusterfs : Typical Use Cases
VM Images Store (Not always Expected being Persistent)
Image Templates and Image Instances of Running VMs
Need to be Either of File or Block Access I/F
External(/Additional) Volumes
Typically, Expected as ‘Persistent Storage’ for Database
and to have Block Storage I/F (to the guest OS instances)
Copyright (c) OCDET All Rights Reserved. 14
and to have Block Storage I/F (to the guest OS instances)
VM Image/External Volumes Snapshot Store
(Tenant) Backup / Bulk Data Store
XXX as a backend of HDFS (gluster already has!)
Object Storage I/F Fits for this Purpose Best. (Swift API?)
etc.
16. Test Case 1 : OpenStack Configuration Example
Intranet
Internet /
Intranet
Zone #1
External Network
End User PC
Firewall
Amazon EC2 type Network Model
NW Gateway Node aggregates
tenant external traffic
(*1) Cloud Controller
API, Scheduler, RabbitMQ, RDBMS etc.
Courtesy of Internet Initiative Japan Inc.
Copyright (c) OCDET All Rights Reserved.
Quantum
agent
(L2/L3)
Cloud
Controller
Glance
(Swift)
VM
Image
Nova Compute
Quantum Agent
V
M
V
M
…
Internal Network (VLAN/GRE etc.)
Management/Storage Network
DB
(MySQL)
Nova Compute
Quantum Agent
V
M
V
M
ovs ovs
Management/Storage Network
NW Gateway
Hypervisors
gluser
gluser
gluster
. . .
gluster
17. Test Case 1 : VM Image Store w/OpenStack
Shared FS by
Gluster for VM
Images
volume: gvol1
V
M
V
M
V
M
V
M
VM Service Network
OpenStack Folsom + CentOS 6.3 (x86_64) + gluster 3.3
gluster
Copyright (c) OCDET All Rights Reserved.
Management
/mnt/sdb1 /mnt/sdb1 /mnt/sdb1
volume: gvol1
/mnt/gluster1 /mnt/gluster1 /mnt/gluster1
NFS mount
bricks
18. Test Case 2 : CloudStack Configuration Example
Intranet
Internet /
Intranet
Zone/Pod
External Network
End User PC
Traffic for
Virtual Routers
DNAT based Network Model
Per-tenant Virtual Router VM will
be created and it proxies tenant
traffic.
Courtesy of DataHotel Co., Ltd.
for Secondary
Stoage
Copyright (c) OCDET All Rights Reserved.
Mgmt.
Server.
NFS
VM
Image
Compute
V
M
…
DB
Compute
V
M
Compute
Virtual
Router
Compute
Virtual
Router
vnic0
vnic1
V
M
vnic0
vnic1
SS
VM
CP
VM
Management/Storage Network
Management/Storage Network
gluster
19. Test Case 2 : VM Image Store w/CloudStack
Shared FS by
Gluster for VM
Images
V
M
V
M
V
M
V
M
VM Service Network
CloudStack 4.0 + CentOS 6.4(x86_64) + gluster 3.3 (4nodes)
Copyright (c) OCDET All Rights Reserved.
Management / Storage Traffic
/mnt/sdb1 /mnt/sdb1 /mnt/sdb1
volume: gvol1 /mnt/gluster1 /mnt/gluster1 /mnt/gluster1
NFS mount
brick
. . .
20. Summary of our (current) POC Experiences
VM Image Store
Glusterfs is QUITE EASY to install/configure and WORKS
FAIRLY WELL
With both OpenStack and CloudStack
Without almost No Pitfalls except trivial configuration mistakes
Forgot to add iptables ACCEPT rule for NFS etc. :o
Copyright (c) OCDET All Rights Reserved. 19
Forgot to add iptables ACCEPT rule for NFS etc. :o
Bandwidth Bottleneck when Many VMs launched at once
etc.
Object Storage
TBD
Volume Backend
TBD
21. OCDET (Possible) Future Plan
Large Scale Evaluation Using STARBED
Reserved 140 nodes (Now, Swift evaluation is ongoing)
Gluster Swift API Evaluation
OpenStack BlockStorage (Cinder) driver Evaluation
InfiniBand Support Evaluation (?)
Copyright (c) OCDET All Rights Reserved. 20
InfiniBand Support Evaluation (?)
etc.
22. 3. Future Expectations and (Stupid?) Ideas3. Future Expectations and (Stupid?) Ideas
for Glusterfs
21
23. Future Expectations and (Stupid?) Ideas
Enhancements of OpenStack Cinder Backend Driver
Add Volume Snapshot Feature
Snapshot Support (versioning?)
Some Handoffs with Local Filesystems? (e.g. btrfs?)
libvirt support of qemu-gluster integration
Copyright (c) OCDET All Rights Reserved. 22
libvirt support of qemu-gluster integration
Improvement of InfiniBand Support Quality
I saw a gluster-rdma system hung up under heavy I/O
load (using iozone) using gluster 3.3beta2 formerly…
Linux Upstream Code Merge(?)
24. Future Expectations and (Stupid?) Ideas
Erasure Coding Support (Like the case of HDFS) ?
To Reduce Required Physical Capacity without Losing
Redundancy Level
To Reduce Storage I/O Traffic
Replica Locality Control ??
Copyright (c) OCDET All Rights Reserved. 23
Replica Locality Control ??
To Reduce Replication Traffic
NFS/RDMA on top of Glustefs ?
Geologically Distributed “Synchronous” Replication?
For DR. In this sense, Single Master makes sense …
27. Conclusion and Summary
Glusterfs is a Fairly Good Solution for IaaS
Deployments ☺
OCDET is Further Evaluating Glusterfs
and Sharing our Results Publicly
Join us ! and Let’s Play with Glusterfs together !☺
Copyright (c) OCDET All Rights Reserved. 26
Join us ! and Let’s Play with Glusterfs together !☺