Cloud computing and OpenStack basic introduction. This presentation was given on November 13, 2014 at Universitat Politecnica de Catalunya. Barcelona, Spain.
Cloud orchestration major tools comparisionRavi Kiran
Cloud Orchestration major tools comparison (including history, installation, market share, integration with other public cloud system for each tool) For any clarification contact kiran79@techgeek.co.in
Cloud computing and OpenStack basic introduction. This presentation was given on November 13, 2014 at Universitat Politecnica de Catalunya. Barcelona, Spain.
Cloud orchestration major tools comparisionRavi Kiran
Cloud Orchestration major tools comparison (including history, installation, market share, integration with other public cloud system for each tool) For any clarification contact kiran79@techgeek.co.in
OpenStack is an open source cloud project and community with broad commercial and developer support. OpenStack is currently developing two interrelated technologies: OpenStack Compute and OpenStack Object Storage. OpenStack Compute is the internal fabric of the cloud creating and managing large groups of virtual private servers and OpenStack Object Storage is software for creating redundant, scalable object storage using clusters of commodity servers to store terabytes or even petabytes of data. In this tutorial, Bret Piatt will explain how to deploy OpenStack Compute and Object Storage, including an overview of the architecture and technology requirements.
Introduction to OpenStack Architecture (Grizzly Edition)Ken Pepple
Presentation from OpenStack Summit in April 2013.
Building upon his popular blog posts and diagrams (http://ken.pepple.info), Ken will walk through the architecture of OpenStack Grizzly and describe its key software components and important interactions with a special focus on recent changes. After finishing with the software architecture, he will discuss common physical design patterns available for large scale deployments.
What is OpenStack? This presentation is an overview about the most fascinating projects out there today.
In this presentation, I cover the following topics:
- Quick introduction to OpenStack project
- Explain the OpenStack architecture and how its built
- Get you familiar with the different terminology and concepts
- Get you familiar with OpenStack services (components)
- Go over installation methods and tools
- Review risks
Verizon's Beth Cohen explains the process of creating the OpenStack Architecture Guide, as delivered to the Boston OpenStack Meetup September 10, 2014.
Policy Based SDN Solution for DC and Branch Office by Suresh Boddapatibuildacloud
In this talk Suresh will discuss how Nuage Networks Virtualized Services Platform (VSP) helps overcome the challenges that cloud service providers and large enterprises face delivering, and managing, large multi-tenant clouds. He will discuss how Nuage Networks delivers a massively scalable SDN solution that ensures that datacenters, and wide area networks, are able to respond instantly to demand, and are boundary-less. The talk will also provide an overview of the SDN capabilities that Nuage VSP adds to CloudStack.
Bio
Suresh is the VP of Engineering at Nuage Networks. He has over 19 years experience in software development, building great teams and delivering high quality software. As the first engineer at Nuage Networks, Suresh played a key role in shaping the architecture of the Nuage Virtualized Services Platform (VSP). Suresh’s experience includes extensive protocol development, having developed IP routing and multicast protocols from scratch and deploying them in large ISPs. Suresh was part of the original TiMetra team before becoming part of Alcatel Lucent as Principal Engineer. He then took a role as Director of Engineering at Juniper where he worked on their QFabric product. Earlier in his career, Suresh worked in software engineering at Shasta Networks (Nortel acquired) as well as Fore Systems (Marconi, Ericsson acquired).
Eager to learn more about OpenStack? This presentation provides an overview of OpenStack basics and an introduction to the types of storage in OpenStack. Choosing the right storage for your cloud can be the hardest part of building out your environment – this is a great primer to picking the right storage for your OpenStack deployment.
Are you overwhelmed by storage capacity requirements? Are you wondering how web giants are able to store large amounts of data at a fraction of your storage costs?
OpenStack is the fastest growing open-source project to date, and its community builds cloud software. Join us to learn about the two OpenStack storage projects and how your company can take advantage of them.
OpenStack storage allows the use of commodity hardware at massive scales that you can consume as a public, private, or hybrid cloud.
View the on-demand webinar. Special guest speaker Randy Bias, founder and CEO of Cloudscaling and member of the Board of Directors for OpenStack Foundation, and EVault big data expert Joey Yep will inform you about this fast-growing, open-source project: OpenStack.
• OpenStack Swift and Cinder storage projects
• High-level functionality and architecture
• Public, private, and hybrid use-cases
Do you think that Nova, Cinder, Heat, Ceilometer, and Neutron are all references to global warming and looming apocalypse? For all those who come to the OpenStack community and wonder what all the fuss is about, this quick introduction will answer your many questions. It includes a short history of the largest Open Source project in history and will touch on
the basic OpenStack components, so you will be prepared the next time someone mentions Keystone, Nova and Swift in the same sentence.
This session was presented by Beth Cohen at the OpenStack meetup on Feb 19th, 2014 in Boston. Beth works for Verizon developing cool Cloud based products that she can't talk about without a strict NDA. She is a technical leader with over 25 years of experience architecting leading-edge system infrastructures and managing complex projects in the telecom, manufacturing, financial services, government, and technology industries. She has been involved in building some of the world's largest OpenStack architectures and has way too much fun at OpenStack Summits!
Build clouds the way some of the world’s biggest public and private clouds are built—using CloudStack. This 60-minute webinar with the Cloudstack team will help you gain a better understanding of the CloudStack architecture and feature set.
Rightscale Webinar: Designing Private & Hybrid Clouds (Hosted by Citrix)RightScale
Do you want to turn your existing data center into a private cloud? Exploring how to integrate your private cloud with a public cloud? In this webinar, we will discuss key considerations when designing a private cloud from internal resources and best practices for architecture of private and hybrid clouds. The webinar will include a demonstration, plus real-world examples of customers running their private cloud implementations on Citrix CloudPlatform using RightScale.
Topics to be covered:
• When to use private clouds
• Hardware selection
• Reference architectures and design considerations
• Use cases and real-life scenarios
• Managing your cloud resources effectively
AWS compared to OpenStack, slides for OpenStack Summit session in Vancouver.
Blog here:
http://redhatstackblog.redhat.com/2015/05/13/public-vs-private-amazon-compared-to-openstack/
Demo:
www.youtube.com/watch?v=QK5pcowc5Pg
The road to Cloud Computing is not without a few bumps. This session will help to smooth out your journey by tackling some of the potential complications. We’ll examine whether standardization is a prerequisite for the Cloud. We’ll look at why refactoring isn’t just for application code. We’ll check out deployable entities and their simplification via higher levels of abstraction. And we’ll close out the session with a look at engineered systems and modular clouds.
(As presented by Dr. James Baty at Oracle Technology Network Architect Day in Chicago, October 24, 2011.)
OpenStack is an open source cloud project and community with broad commercial and developer support. OpenStack is currently developing two interrelated technologies: OpenStack Compute and OpenStack Object Storage. OpenStack Compute is the internal fabric of the cloud creating and managing large groups of virtual private servers and OpenStack Object Storage is software for creating redundant, scalable object storage using clusters of commodity servers to store terabytes or even petabytes of data. In this tutorial, Bret Piatt will explain how to deploy OpenStack Compute and Object Storage, including an overview of the architecture and technology requirements.
Introduction to OpenStack Architecture (Grizzly Edition)Ken Pepple
Presentation from OpenStack Summit in April 2013.
Building upon his popular blog posts and diagrams (http://ken.pepple.info), Ken will walk through the architecture of OpenStack Grizzly and describe its key software components and important interactions with a special focus on recent changes. After finishing with the software architecture, he will discuss common physical design patterns available for large scale deployments.
What is OpenStack? This presentation is an overview about the most fascinating projects out there today.
In this presentation, I cover the following topics:
- Quick introduction to OpenStack project
- Explain the OpenStack architecture and how its built
- Get you familiar with the different terminology and concepts
- Get you familiar with OpenStack services (components)
- Go over installation methods and tools
- Review risks
Verizon's Beth Cohen explains the process of creating the OpenStack Architecture Guide, as delivered to the Boston OpenStack Meetup September 10, 2014.
Policy Based SDN Solution for DC and Branch Office by Suresh Boddapatibuildacloud
In this talk Suresh will discuss how Nuage Networks Virtualized Services Platform (VSP) helps overcome the challenges that cloud service providers and large enterprises face delivering, and managing, large multi-tenant clouds. He will discuss how Nuage Networks delivers a massively scalable SDN solution that ensures that datacenters, and wide area networks, are able to respond instantly to demand, and are boundary-less. The talk will also provide an overview of the SDN capabilities that Nuage VSP adds to CloudStack.
Bio
Suresh is the VP of Engineering at Nuage Networks. He has over 19 years experience in software development, building great teams and delivering high quality software. As the first engineer at Nuage Networks, Suresh played a key role in shaping the architecture of the Nuage Virtualized Services Platform (VSP). Suresh’s experience includes extensive protocol development, having developed IP routing and multicast protocols from scratch and deploying them in large ISPs. Suresh was part of the original TiMetra team before becoming part of Alcatel Lucent as Principal Engineer. He then took a role as Director of Engineering at Juniper where he worked on their QFabric product. Earlier in his career, Suresh worked in software engineering at Shasta Networks (Nortel acquired) as well as Fore Systems (Marconi, Ericsson acquired).
Eager to learn more about OpenStack? This presentation provides an overview of OpenStack basics and an introduction to the types of storage in OpenStack. Choosing the right storage for your cloud can be the hardest part of building out your environment – this is a great primer to picking the right storage for your OpenStack deployment.
Are you overwhelmed by storage capacity requirements? Are you wondering how web giants are able to store large amounts of data at a fraction of your storage costs?
OpenStack is the fastest growing open-source project to date, and its community builds cloud software. Join us to learn about the two OpenStack storage projects and how your company can take advantage of them.
OpenStack storage allows the use of commodity hardware at massive scales that you can consume as a public, private, or hybrid cloud.
View the on-demand webinar. Special guest speaker Randy Bias, founder and CEO of Cloudscaling and member of the Board of Directors for OpenStack Foundation, and EVault big data expert Joey Yep will inform you about this fast-growing, open-source project: OpenStack.
• OpenStack Swift and Cinder storage projects
• High-level functionality and architecture
• Public, private, and hybrid use-cases
Do you think that Nova, Cinder, Heat, Ceilometer, and Neutron are all references to global warming and looming apocalypse? For all those who come to the OpenStack community and wonder what all the fuss is about, this quick introduction will answer your many questions. It includes a short history of the largest Open Source project in history and will touch on
the basic OpenStack components, so you will be prepared the next time someone mentions Keystone, Nova and Swift in the same sentence.
This session was presented by Beth Cohen at the OpenStack meetup on Feb 19th, 2014 in Boston. Beth works for Verizon developing cool Cloud based products that she can't talk about without a strict NDA. She is a technical leader with over 25 years of experience architecting leading-edge system infrastructures and managing complex projects in the telecom, manufacturing, financial services, government, and technology industries. She has been involved in building some of the world's largest OpenStack architectures and has way too much fun at OpenStack Summits!
Build clouds the way some of the world’s biggest public and private clouds are built—using CloudStack. This 60-minute webinar with the Cloudstack team will help you gain a better understanding of the CloudStack architecture and feature set.
Rightscale Webinar: Designing Private & Hybrid Clouds (Hosted by Citrix)RightScale
Do you want to turn your existing data center into a private cloud? Exploring how to integrate your private cloud with a public cloud? In this webinar, we will discuss key considerations when designing a private cloud from internal resources and best practices for architecture of private and hybrid clouds. The webinar will include a demonstration, plus real-world examples of customers running their private cloud implementations on Citrix CloudPlatform using RightScale.
Topics to be covered:
• When to use private clouds
• Hardware selection
• Reference architectures and design considerations
• Use cases and real-life scenarios
• Managing your cloud resources effectively
AWS compared to OpenStack, slides for OpenStack Summit session in Vancouver.
Blog here:
http://redhatstackblog.redhat.com/2015/05/13/public-vs-private-amazon-compared-to-openstack/
Demo:
www.youtube.com/watch?v=QK5pcowc5Pg
The road to Cloud Computing is not without a few bumps. This session will help to smooth out your journey by tackling some of the potential complications. We’ll examine whether standardization is a prerequisite for the Cloud. We’ll look at why refactoring isn’t just for application code. We’ll check out deployable entities and their simplification via higher levels of abstraction. And we’ll close out the session with a look at engineered systems and modular clouds.
(As presented by Dr. James Baty at Oracle Technology Network Architect Day in Chicago, October 24, 2011.)
Brocade: Storage Networking For the Virtual Enterprise EMC
This Session will review the latest in storage networking protocols for EMC storage platforms. Topics include new architecture for scaling SAN fabrics, new features for enhancing SAN metro connectivity, and new capabilities for SAN Management. We will also review Ethernet Fabric deployments for FCOE, iSCSI and NAS.
Presentation of Vincent Desveronnieres, Oracle at the TMT.CloudComputing'11 Warsaw conference organized in Warsaw, Poland on February 10th, 2011 by New Europe Events
S cv3179 spectrum-integration-openstack-edge2015-v5Tony Pearson
IBM is a platinum sponsor of OpenStack, and is the #1 ranked vendor of Software Defined Storage. This session explains how its Spectrum Storage family of products support Glance, Cinder, Manila, Swift and Keystone interfaces of OpenStack.
Similar to Distributed Block-level Storage Management for OpenStack, by Danile lee (20)
DevOps in a Public OpenStack Cloud - Hui ChengHui Cheng
Presented at OpenStack Summit Fall 2012, San Diego, by Hui Cheng. Detail notes see here: http://freedomhui.com/2012/10/devops-in-openstack-public-cloud/
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Distributed Block-level Storage Management for OpenStack, by Danile lee
1. Distributed Block-level Storage
Management for OpenStack
OpenStack APAC Conference 2012
Daniel Lee
CCMA/ITRI
Cloud Computing Center for Mobile Applications
Industrial Technology Research Institute
( 雲端運算行動運用科技中心 )
2012/8/11 OpenStack APAC Conference 1 1
2012
Copyright 2009 ITRI 工業技術研究院
2. Outline
• Overview
• Brief on ITRI Cloud OS
• Cloud storage system in ITRI Cloud OS
• Distributed Main Storage System
• Distributed Secondary Storage System
• Integration of Cloud OS Storage System with
OpenStack
• Summary
• Q&A
2012/8/11 OpenStack APAC Conference 2
2012
Copyright 2009 ITRI 工業技術研究院
3. Overview
• Data is growing exponentially and moving on to Could. It
needs to be available at all time, secured and protected.
• 911 in Twin Towers, 311 earthquake in Japan and 420 outage
of Amazon datacenters are disasters.
– Block-level and fully redundant
– Backup / restore + wide-area
– Thin provisioning
– De-duplication
– Others
• Cloud OS Storage System has these functionality and is ready
to integrate to OpenStack
2012/8/11 OpenStack APAC Conference 3
2012
Copyright 2009 ITRI 工業技術研究院
4. ITRI Cloud OS
(An All-in-one IaaS Total Solution)
Virtual Data Center Management
Primary/Secondary
Virtual Data Center
Storage Management Physical Data
Provisioning (VMWare)
(EMC) Center
Management
Physical (IBM)
Resource Security
Management (Checkpoint)
(HP/Dell)
Inter-VM Load
Balancing (F5)
Power Management
(IBM) Inter-PM Load Balancing
and VM Fail-over (VMWare)
*Blue name in bracket indicates
the leading company of the component
2012/8/11 OpenStack APAC Conference 4
2012
Copyright 2009 ITRI 工業技術研究院
5. ITRI Cloud OS
Service Interface
Multiplexing VDC’s in a Physical DC
Multiplexing VDC’s in a Physical DC
Virtual Data Center Management Physical Data Center Management
Virtual Data Center Management Physical Data Center Management
Photo Video Web
Sharing Streami Confere
ng nce
VDC VDC
Provision VDC
and Deploy
Physical
•Cloud Application
Cluster •Cloud Service
Infrastructure
Developer Administrator
•Cloud Service •Carrier
Provider
Monitor and Configure Monitor, Diagnose and
Virtual Resources Configure
Physical Resources
2012/8/11 OpenStack APAC Conference 5
2012
Copyright 2009 ITRI 工業技術研究院
9. Cloud Storage System in Cloud OS
• Cloud storage aims at cloud-scale data
centers, and is designed to be scalable,
available and low-cost.
• Main components
– Distributed Main Storage subsystem – DMS
• Provide current image I/O request processing
– Distributed Secondary Storage subsystem – DSS
• Block-level incremental metadata only backup system
• Take volume snapshot / restore, de-duplication engine,
garbage collection and WADB engine
2012/8/11 OpenStack APAC Conference 9
2012
Copyright 2009 ITRI 工業技術研究院
10. Key Features of Cloud Storage
System from Users’ Perspective
• Use like raw, unformatted block devices
• Support volume size from 1 GB to 2 TB (64TB architecture-wise)
• Can be attached by different instances (one at a time for now)
• Multiple volumes can be mounted to the same instance
• Data is automatically replicated on different physical storage
server (N=3)
• Ability to clone volumes and/or create point-in-time snapshots of
volumes
• Wide-area backup for disaster recovery
• Snapshot can be scheduled by assigning a backup policy with time,
frequency, retention, and WADB option
• Save system image for starting multiple VMs using the same image
2012/8/11 OpenStack APAC Conference 10
2012
Copyright 2009 ITRI 工業技術研究院
11. Key Features of Cloud Storage
System from Providers’ Perspective
• Use of commodity hardware – reducing cost
(JBOD)
• Disk space management for up to multiple
petabytes
• Add disk, remove disk without interruption
• Thin provisioning
• Enables you to provision a specific level of I/O
performance if desired, by setting I/O throttle
• Block-level de-duplication for reducing space
requirement
2012/8/11 OpenStack APAC Conference 11
2012
Copyright 2009 ITRI 工業技術研究院
12. DMS in Cloud OS
• DMS - Distributed Main Storage
• Goal: A cloud scale network storage solution that provide
high capacity, high reliability, high availability and high
performance
• Features:
– Volume operations: Create / Attach / Detach / Delete
– High Reliability: data replication (N=3)
– High Availability: No single point of failure
– High Performance: client-side metadata/data cache for performance
– Thin provisioning is utilized to optimize utilization of available storage.
– Fast Volume Cloning
– Save Image; FastBoot
– Disk I/O Throttle
2012/8/11 OpenStack APAC Conference 12
2012
Copyright 2009 ITRI 工業技術研究院
13. DMS System Architecture
Compute Node
…
VM VM VM Service Node
Volume Volume Volume
Metadata
DMS In-kernel client NameNode
Data RR
Storage Node
…
DMS three components:
1. Namenode
DataNode 2. DMS in-kernel client
3. Datanode
2012/8/11 OpenStack APAC Conference 13
2012
Copyright 2009 ITRI 工業技術研究院
14. DMS Compute Node
Architecture
Compute Node
Domain U User Space Domain U Kernel Space
Applications
RPM
Read/Write
Sectors /dev/hda1 Block Frontend
Driver
Attach/Detach Attach a disk to Dom U
Volume
Xen / KVM I/O Virtualization
Block Backend
User-space Daemon
Driver
NameNode IOCTL
Metadata
In-Kernel Client
Payload
DataNodes Block
Devices
/dev/dms01
/dev/dms02
.
..
..
Domain 0 User Space Domain 0 Kernel Space
2012/8/11 OpenStack APAC Conference 14
2012
Copyright 2009 ITRI 工業技術研究院
15. Read Flow
Compute Node
VM
Volume Service Node
6. Response
1. Read request
(payload) 2. query metadata: (VolID, LBID)
DMS In-kernel Client 3. HBID, N-locations
NameNode
7. Report failed locations
4. Read Data 5. Read Data from another datanode
if previous-try failed
Storage Node Storage Node Storage Node
DataNode DataNode DataNode …
OpenStack APAC Conference 2012
Copyright 2008 ITRI 工業技術研究院 2012/8/11 15
16. Write Flow – Memory-Ack Approach
Compute Node
VM
Volume Service Node
6. Response 1. Write request
2. Request for new space:(VolID, LBID)
DMS In-kernel Client 3. HBID, N-locations
NameNode
7. Commit (Success, or
4. Write Data report failed locations)
5. Memory Ack
6. Disk Ack
• Memory-ack approach:
Storage Node Storage Node Storage Node In client, I/O response to
VM(step6) is right after
DataNode DataNode DataNode … receiving memory ack from
Datanodes (step5) .
OpenStack APAC Conference 2012
Copyright 2008 ITRI 工業技術研究院 2012/8/11 16
17. DSS in Cloud OS
• Share the homogeneous space with DMS
• Block-level Incremental backup
– Copy-on-write (COW) and volume cloning techniques
– Backup is just a snapshot of meta-data
• Support volume-level restore
• Block-level de-duplication
• Garbage collection of expired, un-used and redundant disk
blocks
• Wide-area data backup / restore and near-realtime
replication
• The secondary storage itself is failure-tolerant (HA)
2012/8/11 OpenStack APAC Conference 17
2012
Copyright 2009 ITRI 工業技術研究院
20. DSS – WADB
Backing up the data to a remote location at
the time volume snapshot is taken…
snapshot
Network / restore
VPN
Beijing Shang-Hai
2012/8/11 OpenStack APAC Conference 20
2012
Copyright 2009 ITRI 工業技術研究院
25. Integration of Cloud OS Storage
System with OpenStack
User portal
Web Services
User (OpenStack VDCM LOG PDCM PDCM User
API enabled)
VMM L2 PRM
Nova API Glance
Nova Manager DB DMS
VM1 VM2… VMn Nova Volume volume
…
Nova Compute Nova Network
SECS
RS
SLB
2012/8/11 OpenStack APAC Conference 25
2012
Copyright 2009 ITRI 工業技術研究院
26. Summary of Advantages
Feature Benefits
Unlimited storage highly scalable read/write access
Leverages commodity hardware (JBOD) No lock-in, lower price/GB
Built-in replication Fully redundant (default: N=3)
Used as system volume Save Image, Fast Boot, etc.
Thin Provisioning Optimize utilization of available storage
Highly available (HA) storage servers Server failover for high available
(DMS/DSS)
Snapshot and backup for block volumes Data protection and recovery for VM data
Policy based backup Free of worry – set it and forget about it
Wide-area data backup For disaster recovering
Block-level data de-duplication Store more data in the limited space
Standalone volume API, CLI available Easy integration with other compute systems
2012/8/11 OpenStack APAC Conference 26
2012
Copyright 2009 ITRI 工業技術研究院
27. More Feature in V2.0
Feature Benefits
Volume Cloning Fast cloning without copying data
Share virtual disk within cluster Support sharing of volume and cluster file system
Disk bandwidth QoS guarantee Dynamic adjustment at I/O request level
Storage space optimization N-way replication for write-intensive blocks
N-parity-group for read-intensive blocks
Wide-area data replication Near real-time wide-area replication
Integration with Compute / Volume Fully integrated to Compute for attaching block
volumes and reporting on usage
2012/8/11 OpenStack APAC Conference 27
2012
Copyright 2009 ITRI 工業技術研究院
Why don’t we just use Hadoop as block level storage What are the advantages over iSCSI – RAID + SW for backup restore and WADB When will it be available for OpenStack – 4Q2012? The limitations? Volume / total space, size of metadata, network, … Performance? 12 MB/sec or 49 MB/sec (multiple concurrent clients) How does it compare to EMC, Amazon S3? S3: Object Store; AWS Import/Export Amazon Elastic Block Storage (EBS): 1GB~1TB; ss saved in S3 Cinder, NetApp, Nexenta and SolidFire CIFS, GlusterFS, OCSF2 XCP
CRITICAL CHANGES NO HDFS In-kernal driver “ Native” networking disk driver (not xen back-end driver) Key design Why we Why not use iscii protocol Datanode prioritization Datanode hardware dependency
Provides a global storage abstraction on a large number of distributed disks Each storage block remains available despite failure in switch, server, or disk drive Scales to a very large number of concurrent accesses Exploits distributed object caching Initial thought: Use HDFS as the base for global storage management Support concurrency control among accesses to common storage blocks Support QoS guarantee or performance isolation among virtual clusters