SlideShare a Scribd company logo
Sheepdog: Yet Another All-In-One Storage
For Openstack
Openstack Hong Kong Summit
Liu Yuan 2013.11.8
Who I Am
• Active contributor to various open source projects such as Sheepdog,
QEMU, Linux Kernel, Xen, Openstack, etc.
• Primary top contributor of Sheepdog project and co-maintains it with
Kazutaka Morita from NTT Japan since 2011.12
• Technically lead the storage projects based on Sheepdog for internal
uses of www.taobao.com
• Contacts
–Email: namei.unix@gmail.com
–Micro Blog: @ 淘泰来
Agenda
Introduction - Sheepdog Overview
Exploration - Sheepdog Internals
Openstack - Sheepdog Goal
Roadmap - Features From The Future
Industry - How Industry Use Sheepdog
Introduction

Sheepdog Overview
What Is Sheepdog
•

Distributed Object Storage System In User Space
–Manage Disks and Nodes
• Aggregate the capacity and the power (IOPS + throughput)
• Hide the failure of hardware
• Dynamically grow or shrink the scale

–Manage Data
• Provide redundancy mechanisms (replication and erasure code) for high-availability
• Secure the data with auto-healing and auto-rebalanced mechanisms

–Provide Services
• Virtual volume for QEMU VM, iSCSI TGT (Perfectly supported by upstream)
• RESTful container (Openstack Swift and Amazon S3 Compatible, in progress)
• Storage for Openstack Cinder, Glance, Nova (Available for Havana)
Sheepdog Architecture
No meta servers!

Zookeeper: membership management and message queue

Node event: global rebalance
Disk event: local rebalance

Global Consistent Hash Ring and P2P
sheep daemon
Gateway

Gateway

Gateway

Store

Store

Store

1TB

1TB

1TB

1TB

4TB

2TB

1TB
Hot-plugged

1TB

X

2TB

dog admin tool
Private Hash Ring

Auto unplugged on EIO
Why Sheepdog
•

Minimal assumptions of underlying kernel and file system
–Any type of file systems that support extended attribute(xattr)
–Only require kernel version >= 2.6.32

•

Full of features
–snapshot, clone, incremental backup, cluster-wide snapshot, discard, etc.
–User-defined replication/erasure code scheme on VDI(Virtual Disk Image) basis
–Auto node/disk management

•

Easy to set up the cluster with thousands of nodes
–Single daemon can manage unlimited number of disks in one node as efficient as RAID0
–as many as 6k+ for a single cluster

•

Small
–Fast and very small memory footprint (less than 50 MB even when busy)
–Easy to hack and maintain, 35K lines of code in C as of now
Exploration

Sheepdog Internals
Sheepdog In A Nutshell
Gateway
256G
SSD

Data
Replicated or
Erasure coded

Store
4G

/dev/vda
Net

Weighted
Nodes & Disks

/dev/vda
Shared
Persistent
Cache

Store
Journal

2TB

1TB

1TB

1TB

1TB
Sheepdog Volume
• Copy-On-Write Snapshot
–Disk-only snapshot, disk & memory snapshot
–Live snapshot, offline snapshot
–Rollback(tree structure), clone
–Incremental backup
–Instant operation, only create 4M inode object

• Push many logics into client -> simple and fast code !
–Only 4 opcodes for store, read/write/create/remove, snapshot is done by QEMU
block driver or dog
–Requests serialization is not handled by Sheepdog but client
–Inode object is treated the same as data object
Gateway - Request Engine
Return Only All
Succeed
Req Queue
Node Ring
Socket
Cache
Concurrent
Handling

Route
Retry

Net
Cache

Req Manager
Store

1TB

1TB

2TB

X
Retry On
Error(Timeout, EIO)

2TB

4TB

Sheep

Sheep

Sheep

Sheep

X

Degraded to
GW-only On EIO
Store - Data Engine
Sheep
Disk Ring
GW

Net

Sheep

Disk Manager
Sheep

Journal

1TB

1TB

2TB

X

2TB

Auto unplugged on EIO
1. Fake network err to ask GW retry
2. Update disk ring
3. Start local data rebalance

Sheep
Redundancy Scheme

Erasure Coding

Full Replication

Parity

Sheep

Sheep

Sheep

Sheep

Sheep

Sheep

Sheep

Sheep

Sheep
Erasure Coding Over Full Replication
• Advantages
–Far less storage overhead

6 N
odes w t h 1G N C
i
b I
100

–Rumors breaking

80

• Better R/W performance
• Can run VM Images !

• Disadvantages

s
/
B
M

• Support random R/W

60

R i cat i on ( 3 C
epl
opy)
Er asur e ( 4: 2)

40
20
0

R
ead

–Generate more traffic for recovery
• (X + Y)/(Y + 1) times data ( Suppose X data, Y parity strips)
–

2 times data for 4:2 compared to 3 full copies

Wi t e
r
Recovery - Redundancy Repair & Data Rebalance
Sleep Queue
Node Ring
Update Version
Schedule
Recovery
Manager

If lost, read or
rebuild from other
copy
Sheep
Net

Req Queue
Migrate to other node
for rebalance
X

Sheep
Recovery Cont.
• Eager Recovery As Default
–Allow users to stop it and do manual recovery temporarily

• Node/Disk Events Handling
–Disk event and node event share the same algorithm
• Handle mixed node and disk events nicely

–Subsequent event will supersede previous one
• Handle group join/leave of disks and nodes gracefully

• Recovery Handling Transparently to the Client
–Put requests for objects being recovered on sleep queue and wake it up later
–Serve the request directly if object is right there in the store
Farm - Cluster Wide Snapshot
Sheep
Meta Data
Generator

Slicer

Net

128K
Hash Dedupper

Dedicated Backup Storage

Sheep

Sheep
• Incremental backup
• Up to 50% dedup ratio
• Compression doesn't help
Think of Sheepdog On Sheepdog ? Yeah!

Sheep
Openstack

Sheepdog Goal
Openstack Storage Components
•

Cinder - Block Storage
–Support since day 1

•

Glance - Image Storage

Swift

Cinder

Nova

Glance

–Support merged at Havana version

•

Nova - Ephemeral Storage
–Not yet started

•

Unified Storage

Swift - Object Storage
–Swift API compatible In progress

•

Final Goal - Unified Storage
–Copy-On-Write anywhere ?
–Data dedup ?

Sheep

Sheep

Sheep

Sheep
Roadmap

Features From The Future
Look Into The Future
• RESTful Container
–Plans to be Openstack Swift API compatible first, coming soon

• Hyper Volume
–256PB Volume, coming soon

• Geo-Replication
• Sheepdog On Sheepdog
–Storage for cluster wide snapshot

• Slow Disk & Broken Disk Detecter
–Deal with dead D state process hang because of broken disk in massive
deployment
Industry

How Industry Use Sheepdog
Sheepdog In Taobao & NTT
SD VM SD VM

SD VM SD VM

SD

SD

SD

SD

SD

SD

SD

SD

SD

SD

SD

SD

HTTP
VM running inside
Sheepdog Cluster
for test & dev at
Taobao

Ongoing project with
10k+ ARM nodes
for cold data at
Taobao

LUN device pool
Sheepdog cluster run
as iSCSI TGT backend
storage at
NTT
Other Users In Production
Any more users I don't know ?
Q&A
Homepage
http://sheepdog.github.io/sheepdog/
Try me out
git clone git://github.com/sheepdog/sheepdog.git

Go Sheepdog !

More Related Content

What's hot

Divide and conquer: resource segregation in the OpenStack cloud
Divide and conquer: resource segregation in the OpenStack cloudDivide and conquer: resource segregation in the OpenStack cloud
Divide and conquer: resource segregation in the OpenStack cloud
Stephen Gordon
 
Ceph Tech Talk: Ceph at DigitalOcean
Ceph Tech Talk: Ceph at DigitalOceanCeph Tech Talk: Ceph at DigitalOcean
Ceph Tech Talk: Ceph at DigitalOcean
Ceph Community
 
Red Hat Ceph Storage Roadmap: January 2016
Red Hat Ceph Storage Roadmap: January 2016Red Hat Ceph Storage Roadmap: January 2016
Red Hat Ceph Storage Roadmap: January 2016
Red_Hat_Storage
 
Nick Fisk - low latency Ceph
Nick Fisk - low latency CephNick Fisk - low latency Ceph
Nick Fisk - low latency Ceph
ShapeBlue
 
How shit works: the CPU
How shit works: the CPUHow shit works: the CPU
How shit works: the CPU
Tomer Gabel
 
OpenNebula Conf 2014 | Lightning talk: OpenNebula at Etnetera by Jan Horacek
OpenNebula Conf 2014 | Lightning talk: OpenNebula at Etnetera by Jan HoracekOpenNebula Conf 2014 | Lightning talk: OpenNebula at Etnetera by Jan Horacek
OpenNebula Conf 2014 | Lightning talk: OpenNebula at Etnetera by Jan Horacek
NETWAYS
 
Introduction to Vacuum Freezing and XID
Introduction to Vacuum Freezing and XIDIntroduction to Vacuum Freezing and XID
Introduction to Vacuum Freezing and XID
PGConf APAC
 
ceph-barcelona-v-1.2
ceph-barcelona-v-1.2ceph-barcelona-v-1.2
ceph-barcelona-v-1.2
Ranga Swami Reddy Muthumula
 
M|18 Writing Stored Procedures in the Real World
M|18 Writing Stored Procedures in the Real WorldM|18 Writing Stored Procedures in the Real World
M|18 Writing Stored Procedures in the Real World
MariaDB plc
 
Managing ceph through_oVirt_using_Cinder
Managing ceph through_oVirt_using_CinderManaging ceph through_oVirt_using_Cinder
Managing ceph through_oVirt_using_Cinder
Maor Lipchuk
 
Ceph Day London 2014 - Deploying ceph in the wild
Ceph Day London 2014 - Deploying ceph in the wildCeph Day London 2014 - Deploying ceph in the wild
Ceph Day London 2014 - Deploying ceph in the wild
Ceph Community
 
Best practices for MySQL High Availability
Best practices for MySQL High AvailabilityBest practices for MySQL High Availability
Best practices for MySQL High Availability
Colin Charles
 
OpenNebulaConf 2016 - Measuring and tuning VM performance by Boyan Krosnov, S...
OpenNebulaConf 2016 - Measuring and tuning VM performance by Boyan Krosnov, S...OpenNebulaConf 2016 - Measuring and tuning VM performance by Boyan Krosnov, S...
OpenNebulaConf 2016 - Measuring and tuning VM performance by Boyan Krosnov, S...
OpenNebula Project
 
Optimizing VM images for OpenStack with KVM/QEMU
Optimizing VM images for OpenStack with KVM/QEMUOptimizing VM images for OpenStack with KVM/QEMU
Optimizing VM images for OpenStack with KVM/QEMU
OpenStack Foundation
 
Scaling Ceph at CERN - Ceph Day Frankfurt
Scaling Ceph at CERN - Ceph Day Frankfurt Scaling Ceph at CERN - Ceph Day Frankfurt
Scaling Ceph at CERN - Ceph Day Frankfurt
Ceph Community
 
Laying OpenStack Cinder Block Services
Laying OpenStack Cinder Block ServicesLaying OpenStack Cinder Block Services
Laying OpenStack Cinder Block Services
Kenneth Hui
 
RHEVM - Live Storage Migration
RHEVM - Live Storage MigrationRHEVM - Live Storage Migration
RHEVM - Live Storage Migration
Raz Tamir
 
OSv – The OS designed for the Cloud
OSv – The OS designed for the CloudOSv – The OS designed for the Cloud
OSv – The OS designed for the Cloud
Yandex
 
SLE12 SP2 : High Availability et Geo Cluster
SLE12 SP2 : High Availability et Geo ClusterSLE12 SP2 : High Availability et Geo Cluster
SLE12 SP2 : High Availability et Geo Cluster
SUSE
 
Protecting the Galaxy - Multi-Region Disaster Recovery with OpenStack and Ceph
Protecting the Galaxy - Multi-Region Disaster Recovery with OpenStack and CephProtecting the Galaxy - Multi-Region Disaster Recovery with OpenStack and Ceph
Protecting the Galaxy - Multi-Region Disaster Recovery with OpenStack and Ceph
Sean Cohen
 

What's hot (20)

Divide and conquer: resource segregation in the OpenStack cloud
Divide and conquer: resource segregation in the OpenStack cloudDivide and conquer: resource segregation in the OpenStack cloud
Divide and conquer: resource segregation in the OpenStack cloud
 
Ceph Tech Talk: Ceph at DigitalOcean
Ceph Tech Talk: Ceph at DigitalOceanCeph Tech Talk: Ceph at DigitalOcean
Ceph Tech Talk: Ceph at DigitalOcean
 
Red Hat Ceph Storage Roadmap: January 2016
Red Hat Ceph Storage Roadmap: January 2016Red Hat Ceph Storage Roadmap: January 2016
Red Hat Ceph Storage Roadmap: January 2016
 
Nick Fisk - low latency Ceph
Nick Fisk - low latency CephNick Fisk - low latency Ceph
Nick Fisk - low latency Ceph
 
How shit works: the CPU
How shit works: the CPUHow shit works: the CPU
How shit works: the CPU
 
OpenNebula Conf 2014 | Lightning talk: OpenNebula at Etnetera by Jan Horacek
OpenNebula Conf 2014 | Lightning talk: OpenNebula at Etnetera by Jan HoracekOpenNebula Conf 2014 | Lightning talk: OpenNebula at Etnetera by Jan Horacek
OpenNebula Conf 2014 | Lightning talk: OpenNebula at Etnetera by Jan Horacek
 
Introduction to Vacuum Freezing and XID
Introduction to Vacuum Freezing and XIDIntroduction to Vacuum Freezing and XID
Introduction to Vacuum Freezing and XID
 
ceph-barcelona-v-1.2
ceph-barcelona-v-1.2ceph-barcelona-v-1.2
ceph-barcelona-v-1.2
 
M|18 Writing Stored Procedures in the Real World
M|18 Writing Stored Procedures in the Real WorldM|18 Writing Stored Procedures in the Real World
M|18 Writing Stored Procedures in the Real World
 
Managing ceph through_oVirt_using_Cinder
Managing ceph through_oVirt_using_CinderManaging ceph through_oVirt_using_Cinder
Managing ceph through_oVirt_using_Cinder
 
Ceph Day London 2014 - Deploying ceph in the wild
Ceph Day London 2014 - Deploying ceph in the wildCeph Day London 2014 - Deploying ceph in the wild
Ceph Day London 2014 - Deploying ceph in the wild
 
Best practices for MySQL High Availability
Best practices for MySQL High AvailabilityBest practices for MySQL High Availability
Best practices for MySQL High Availability
 
OpenNebulaConf 2016 - Measuring and tuning VM performance by Boyan Krosnov, S...
OpenNebulaConf 2016 - Measuring and tuning VM performance by Boyan Krosnov, S...OpenNebulaConf 2016 - Measuring and tuning VM performance by Boyan Krosnov, S...
OpenNebulaConf 2016 - Measuring and tuning VM performance by Boyan Krosnov, S...
 
Optimizing VM images for OpenStack with KVM/QEMU
Optimizing VM images for OpenStack with KVM/QEMUOptimizing VM images for OpenStack with KVM/QEMU
Optimizing VM images for OpenStack with KVM/QEMU
 
Scaling Ceph at CERN - Ceph Day Frankfurt
Scaling Ceph at CERN - Ceph Day Frankfurt Scaling Ceph at CERN - Ceph Day Frankfurt
Scaling Ceph at CERN - Ceph Day Frankfurt
 
Laying OpenStack Cinder Block Services
Laying OpenStack Cinder Block ServicesLaying OpenStack Cinder Block Services
Laying OpenStack Cinder Block Services
 
RHEVM - Live Storage Migration
RHEVM - Live Storage MigrationRHEVM - Live Storage Migration
RHEVM - Live Storage Migration
 
OSv – The OS designed for the Cloud
OSv – The OS designed for the CloudOSv – The OS designed for the Cloud
OSv – The OS designed for the Cloud
 
SLE12 SP2 : High Availability et Geo Cluster
SLE12 SP2 : High Availability et Geo ClusterSLE12 SP2 : High Availability et Geo Cluster
SLE12 SP2 : High Availability et Geo Cluster
 
Protecting the Galaxy - Multi-Region Disaster Recovery with OpenStack and Ceph
Protecting the Galaxy - Multi-Region Disaster Recovery with OpenStack and CephProtecting the Galaxy - Multi-Region Disaster Recovery with OpenStack and Ceph
Protecting the Galaxy - Multi-Region Disaster Recovery with OpenStack and Ceph
 

Viewers also liked

RAID, Replication, and You
RAID, Replication, and YouRAID, Replication, and You
RAID, Replication, and You
Great Wide Open
 
Cities social issues
Cities social issuesCities social issues
Cities social issues
dwessler
 
I can\'t believe this is butter - A Tour of btrfs
I can\'t believe this is butter - A Tour of btrfsI can\'t believe this is butter - A Tour of btrfs
I can\'t believe this is butter - A Tour of btrfs
Avi Miller
 
Btrfs by Chris Mason
Btrfs by Chris MasonBtrfs by Chris Mason
Btrfs by Chris Mason
Terry Wang
 
Sheepdog- Google Webinar
Sheepdog- Google Webinar Sheepdog- Google Webinar
Sheepdog- Google Webinar
Sheepdog
 
Reduce Resource Consumption & Clone in Seconds your Oracle Virtual Environmen...
Reduce Resource Consumption & Clone in Seconds your Oracle Virtual Environmen...Reduce Resource Consumption & Clone in Seconds your Oracle Virtual Environmen...
Reduce Resource Consumption & Clone in Seconds your Oracle Virtual Environmen...
BertrandDrouvot
 
LUG-BG 2017 - Rangel Ivanov - Spread some butter - BTRFS
LUG-BG 2017 - Rangel Ivanov - Spread some butter - BTRFSLUG-BG 2017 - Rangel Ivanov - Spread some butter - BTRFS
LUG-BG 2017 - Rangel Ivanov - Spread some butter - BTRFS
Marian Marinov
 
Btrfs: Design, Implementation and the Current Status
Btrfs: Design, Implementation and the Current StatusBtrfs: Design, Implementation and the Current Status
Btrfs: Design, Implementation and the Current Status
Lukáš Czerner
 
Introduction to BTRFS and ZFS
Introduction to BTRFS and ZFSIntroduction to BTRFS and ZFS
Introduction to BTRFS and ZFS
Tsung-en Hsiao
 
Case study of BtrFS: A fault tolerant File system
Case study of BtrFS: A fault tolerant File systemCase study of BtrFS: A fault tolerant File system
Case study of BtrFS: A fault tolerant File system
Kumar Amit Mehta
 
B tree file system
B tree file systemB tree file system
B tree file system
Dinesh Gupta
 
File System Comparison on Linux Ubuntu
File System Comparison on Linux UbuntuFile System Comparison on Linux Ubuntu
File System Comparison on Linux Ubuntu
Jayesh Tambe
 
Btrfs current status and_future_prospects
Btrfs current status and_future_prospectsBtrfs current status and_future_prospects
Btrfs current status and_future_prospects
fj_staoru_takeuchi
 
Sheepdogを使ってみて分かったこと(第六回ストレージ研究会発表資料)
Sheepdogを使ってみて分かったこと(第六回ストレージ研究会発表資料)Sheepdogを使ってみて分かったこと(第六回ストレージ研究会発表資料)
Sheepdogを使ってみて分かったこと(第六回ストレージ研究会発表資料)
Masahiro Tsuji
 
分散ストレージ技術Cephの最新情報
分散ストレージ技術Cephの最新情報分散ストレージ技術Cephの最新情報
分散ストレージ技術Cephの最新情報
Emma Haruka Iwao
 
Erasure codes and storage tiers on gluster
Erasure codes and storage tiers on glusterErasure codes and storage tiers on gluster
Erasure codes and storage tiers on gluster
Red_Hat_Storage
 
Performance comparison of Distributed File Systems on 1Gbit networks
Performance comparison of Distributed File Systems on 1Gbit networksPerformance comparison of Distributed File Systems on 1Gbit networks
Performance comparison of Distributed File Systems on 1Gbit networks
Marian Marinov
 
Ceph アーキテクチャ概説
Ceph アーキテクチャ概説Ceph アーキテクチャ概説
Ceph アーキテクチャ概説
Emma Haruka Iwao
 
Feature rich BTRFS is Getting Richer with Encryption
Feature rich BTRFS is Getting Richer with EncryptionFeature rich BTRFS is Getting Richer with Encryption
Feature rich BTRFS is Getting Richer with Encryption
LF Events
 
Container Storage Best Practices in 2017
Container Storage Best Practices in 2017Container Storage Best Practices in 2017
Container Storage Best Practices in 2017
Keith Resar
 

Viewers also liked (20)

RAID, Replication, and You
RAID, Replication, and YouRAID, Replication, and You
RAID, Replication, and You
 
Cities social issues
Cities social issuesCities social issues
Cities social issues
 
I can\'t believe this is butter - A Tour of btrfs
I can\'t believe this is butter - A Tour of btrfsI can\'t believe this is butter - A Tour of btrfs
I can\'t believe this is butter - A Tour of btrfs
 
Btrfs by Chris Mason
Btrfs by Chris MasonBtrfs by Chris Mason
Btrfs by Chris Mason
 
Sheepdog- Google Webinar
Sheepdog- Google Webinar Sheepdog- Google Webinar
Sheepdog- Google Webinar
 
Reduce Resource Consumption & Clone in Seconds your Oracle Virtual Environmen...
Reduce Resource Consumption & Clone in Seconds your Oracle Virtual Environmen...Reduce Resource Consumption & Clone in Seconds your Oracle Virtual Environmen...
Reduce Resource Consumption & Clone in Seconds your Oracle Virtual Environmen...
 
LUG-BG 2017 - Rangel Ivanov - Spread some butter - BTRFS
LUG-BG 2017 - Rangel Ivanov - Spread some butter - BTRFSLUG-BG 2017 - Rangel Ivanov - Spread some butter - BTRFS
LUG-BG 2017 - Rangel Ivanov - Spread some butter - BTRFS
 
Btrfs: Design, Implementation and the Current Status
Btrfs: Design, Implementation and the Current StatusBtrfs: Design, Implementation and the Current Status
Btrfs: Design, Implementation and the Current Status
 
Introduction to BTRFS and ZFS
Introduction to BTRFS and ZFSIntroduction to BTRFS and ZFS
Introduction to BTRFS and ZFS
 
Case study of BtrFS: A fault tolerant File system
Case study of BtrFS: A fault tolerant File systemCase study of BtrFS: A fault tolerant File system
Case study of BtrFS: A fault tolerant File system
 
B tree file system
B tree file systemB tree file system
B tree file system
 
File System Comparison on Linux Ubuntu
File System Comparison on Linux UbuntuFile System Comparison on Linux Ubuntu
File System Comparison on Linux Ubuntu
 
Btrfs current status and_future_prospects
Btrfs current status and_future_prospectsBtrfs current status and_future_prospects
Btrfs current status and_future_prospects
 
Sheepdogを使ってみて分かったこと(第六回ストレージ研究会発表資料)
Sheepdogを使ってみて分かったこと(第六回ストレージ研究会発表資料)Sheepdogを使ってみて分かったこと(第六回ストレージ研究会発表資料)
Sheepdogを使ってみて分かったこと(第六回ストレージ研究会発表資料)
 
分散ストレージ技術Cephの最新情報
分散ストレージ技術Cephの最新情報分散ストレージ技術Cephの最新情報
分散ストレージ技術Cephの最新情報
 
Erasure codes and storage tiers on gluster
Erasure codes and storage tiers on glusterErasure codes and storage tiers on gluster
Erasure codes and storage tiers on gluster
 
Performance comparison of Distributed File Systems on 1Gbit networks
Performance comparison of Distributed File Systems on 1Gbit networksPerformance comparison of Distributed File Systems on 1Gbit networks
Performance comparison of Distributed File Systems on 1Gbit networks
 
Ceph アーキテクチャ概説
Ceph アーキテクチャ概説Ceph アーキテクチャ概説
Ceph アーキテクチャ概説
 
Feature rich BTRFS is Getting Richer with Encryption
Feature rich BTRFS is Getting Richer with EncryptionFeature rich BTRFS is Getting Richer with Encryption
Feature rich BTRFS is Getting Richer with Encryption
 
Container Storage Best Practices in 2017
Container Storage Best Practices in 2017Container Storage Best Practices in 2017
Container Storage Best Practices in 2017
 

Similar to Sheepdog: yet another all in-one storage for openstack

Webinar - DreamObjects/Ceph Case Study
Webinar - DreamObjects/Ceph Case StudyWebinar - DreamObjects/Ceph Case Study
Webinar - DreamObjects/Ceph Case Study
Ceph Community
 
Open vStorage Road show 2015 Q1
Open vStorage Road show 2015 Q1Open vStorage Road show 2015 Q1
Open vStorage Road show 2015 Q1
wim_provoost
 
Managing Security At 1M Events a Second using Elasticsearch
Managing Security At 1M Events a Second using ElasticsearchManaging Security At 1M Events a Second using Elasticsearch
Managing Security At 1M Events a Second using Elasticsearch
Joe Alex
 
Azug - successfully breeding rabits
Azug - successfully breeding rabitsAzug - successfully breeding rabits
Azug - successfully breeding rabits
Yves Goeleven
 
Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303...
Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303...Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303...
Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303...
Amazon Web Services
 
Open vStorage Meetup - Santa Clara 04/16
Open vStorage Meetup -  Santa Clara 04/16Open vStorage Meetup -  Santa Clara 04/16
Open vStorage Meetup - Santa Clara 04/16
wim_provoost
 
OpenStack Cinder, Implementation Today and New Trends for Tomorrow
OpenStack Cinder, Implementation Today and New Trends for TomorrowOpenStack Cinder, Implementation Today and New Trends for Tomorrow
OpenStack Cinder, Implementation Today and New Trends for Tomorrow
Ed Balduf
 
AWS re:Invent presentation: Unmeltable Infrastructure at Scale by Loggly
AWS re:Invent presentation: Unmeltable Infrastructure at Scale by Loggly AWS re:Invent presentation: Unmeltable Infrastructure at Scale by Loggly
AWS re:Invent presentation: Unmeltable Infrastructure at Scale by Loggly
SolarWinds Loggly
 
Cloud computing UNIT 2.1 presentation in
Cloud computing UNIT 2.1 presentation inCloud computing UNIT 2.1 presentation in
Cloud computing UNIT 2.1 presentation in
RahulBhole12
 
Drupal performance
Drupal performanceDrupal performance
Drupal performance
Piyuesh Kumar
 
SoftLayer Storage Services Overview
SoftLayer Storage Services OverviewSoftLayer Storage Services Overview
SoftLayer Storage Services Overview
Michael Fork
 
Webinar: Sizing Up Object Storage for the Enterprise
Webinar: Sizing Up Object Storage for the EnterpriseWebinar: Sizing Up Object Storage for the Enterprise
Webinar: Sizing Up Object Storage for the Enterprise
Storage Switzerland
 
Swift at Scale: The IBM SoftLayer Story
Swift at Scale: The IBM SoftLayer StorySwift at Scale: The IBM SoftLayer Story
Swift at Scale: The IBM SoftLayer Story
Brian Cline
 
Ruby and Distributed Storage Systems
Ruby and Distributed Storage SystemsRuby and Distributed Storage Systems
Ruby and Distributed Storage Systems
SATOSHI TAGOMORI
 
DatEngConf SF16 - Apache Kudu: Fast Analytics on Fast Data
DatEngConf SF16 - Apache Kudu: Fast Analytics on Fast DataDatEngConf SF16 - Apache Kudu: Fast Analytics on Fast Data
DatEngConf SF16 - Apache Kudu: Fast Analytics on Fast Data
Hakka Labs
 
Ceph Deployment at Target: Customer Spotlight
Ceph Deployment at Target: Customer SpotlightCeph Deployment at Target: Customer Spotlight
Ceph Deployment at Target: Customer Spotlight
Colleen Corrice
 
Ceph Deployment at Target: Customer Spotlight
Ceph Deployment at Target: Customer SpotlightCeph Deployment at Target: Customer Spotlight
Ceph Deployment at Target: Customer Spotlight
Red_Hat_Storage
 
Overview of the ehcache
Overview of the ehcacheOverview of the ehcache
Overview of the ehcache
HyeonSeok Choi
 
In-memory Data Management Trends & Techniques
In-memory Data Management Trends & TechniquesIn-memory Data Management Trends & Techniques
In-memory Data Management Trends & Techniques
Hazelcast
 
Optimizing Latency-sensitive queries for Presto at Facebook: A Collaboration ...
Optimizing Latency-sensitive queries for Presto at Facebook: A Collaboration ...Optimizing Latency-sensitive queries for Presto at Facebook: A Collaboration ...
Optimizing Latency-sensitive queries for Presto at Facebook: A Collaboration ...
Alluxio, Inc.
 

Similar to Sheepdog: yet another all in-one storage for openstack (20)

Webinar - DreamObjects/Ceph Case Study
Webinar - DreamObjects/Ceph Case StudyWebinar - DreamObjects/Ceph Case Study
Webinar - DreamObjects/Ceph Case Study
 
Open vStorage Road show 2015 Q1
Open vStorage Road show 2015 Q1Open vStorage Road show 2015 Q1
Open vStorage Road show 2015 Q1
 
Managing Security At 1M Events a Second using Elasticsearch
Managing Security At 1M Events a Second using ElasticsearchManaging Security At 1M Events a Second using Elasticsearch
Managing Security At 1M Events a Second using Elasticsearch
 
Azug - successfully breeding rabits
Azug - successfully breeding rabitsAzug - successfully breeding rabits
Azug - successfully breeding rabits
 
Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303...
Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303...Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303...
Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303...
 
Open vStorage Meetup - Santa Clara 04/16
Open vStorage Meetup -  Santa Clara 04/16Open vStorage Meetup -  Santa Clara 04/16
Open vStorage Meetup - Santa Clara 04/16
 
OpenStack Cinder, Implementation Today and New Trends for Tomorrow
OpenStack Cinder, Implementation Today and New Trends for TomorrowOpenStack Cinder, Implementation Today and New Trends for Tomorrow
OpenStack Cinder, Implementation Today and New Trends for Tomorrow
 
AWS re:Invent presentation: Unmeltable Infrastructure at Scale by Loggly
AWS re:Invent presentation: Unmeltable Infrastructure at Scale by Loggly AWS re:Invent presentation: Unmeltable Infrastructure at Scale by Loggly
AWS re:Invent presentation: Unmeltable Infrastructure at Scale by Loggly
 
Cloud computing UNIT 2.1 presentation in
Cloud computing UNIT 2.1 presentation inCloud computing UNIT 2.1 presentation in
Cloud computing UNIT 2.1 presentation in
 
Drupal performance
Drupal performanceDrupal performance
Drupal performance
 
SoftLayer Storage Services Overview
SoftLayer Storage Services OverviewSoftLayer Storage Services Overview
SoftLayer Storage Services Overview
 
Webinar: Sizing Up Object Storage for the Enterprise
Webinar: Sizing Up Object Storage for the EnterpriseWebinar: Sizing Up Object Storage for the Enterprise
Webinar: Sizing Up Object Storage for the Enterprise
 
Swift at Scale: The IBM SoftLayer Story
Swift at Scale: The IBM SoftLayer StorySwift at Scale: The IBM SoftLayer Story
Swift at Scale: The IBM SoftLayer Story
 
Ruby and Distributed Storage Systems
Ruby and Distributed Storage SystemsRuby and Distributed Storage Systems
Ruby and Distributed Storage Systems
 
DatEngConf SF16 - Apache Kudu: Fast Analytics on Fast Data
DatEngConf SF16 - Apache Kudu: Fast Analytics on Fast DataDatEngConf SF16 - Apache Kudu: Fast Analytics on Fast Data
DatEngConf SF16 - Apache Kudu: Fast Analytics on Fast Data
 
Ceph Deployment at Target: Customer Spotlight
Ceph Deployment at Target: Customer SpotlightCeph Deployment at Target: Customer Spotlight
Ceph Deployment at Target: Customer Spotlight
 
Ceph Deployment at Target: Customer Spotlight
Ceph Deployment at Target: Customer SpotlightCeph Deployment at Target: Customer Spotlight
Ceph Deployment at Target: Customer Spotlight
 
Overview of the ehcache
Overview of the ehcacheOverview of the ehcache
Overview of the ehcache
 
In-memory Data Management Trends & Techniques
In-memory Data Management Trends & TechniquesIn-memory Data Management Trends & Techniques
In-memory Data Management Trends & Techniques
 
Optimizing Latency-sensitive queries for Presto at Facebook: A Collaboration ...
Optimizing Latency-sensitive queries for Presto at Facebook: A Collaboration ...Optimizing Latency-sensitive queries for Presto at Facebook: A Collaboration ...
Optimizing Latency-sensitive queries for Presto at Facebook: A Collaboration ...
 

Recently uploaded

Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
saastr
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
Postman
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStrDeep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
saastr
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Tatiana Kojar
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
kumardaparthi1024
 
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Jeffrey Haguewood
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
Hiike
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
fredae14
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 

Recently uploaded (20)

Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStrDeep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
 
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 

Sheepdog: yet another all in-one storage for openstack

  • 1. Sheepdog: Yet Another All-In-One Storage For Openstack Openstack Hong Kong Summit Liu Yuan 2013.11.8
  • 2. Who I Am • Active contributor to various open source projects such as Sheepdog, QEMU, Linux Kernel, Xen, Openstack, etc. • Primary top contributor of Sheepdog project and co-maintains it with Kazutaka Morita from NTT Japan since 2011.12 • Technically lead the storage projects based on Sheepdog for internal uses of www.taobao.com • Contacts –Email: namei.unix@gmail.com –Micro Blog: @ 淘泰来
  • 3. Agenda Introduction - Sheepdog Overview Exploration - Sheepdog Internals Openstack - Sheepdog Goal Roadmap - Features From The Future Industry - How Industry Use Sheepdog
  • 5. What Is Sheepdog • Distributed Object Storage System In User Space –Manage Disks and Nodes • Aggregate the capacity and the power (IOPS + throughput) • Hide the failure of hardware • Dynamically grow or shrink the scale –Manage Data • Provide redundancy mechanisms (replication and erasure code) for high-availability • Secure the data with auto-healing and auto-rebalanced mechanisms –Provide Services • Virtual volume for QEMU VM, iSCSI TGT (Perfectly supported by upstream) • RESTful container (Openstack Swift and Amazon S3 Compatible, in progress) • Storage for Openstack Cinder, Glance, Nova (Available for Havana)
  • 6. Sheepdog Architecture No meta servers! Zookeeper: membership management and message queue Node event: global rebalance Disk event: local rebalance Global Consistent Hash Ring and P2P sheep daemon Gateway Gateway Gateway Store Store Store 1TB 1TB 1TB 1TB 4TB 2TB 1TB Hot-plugged 1TB X 2TB dog admin tool Private Hash Ring Auto unplugged on EIO
  • 7. Why Sheepdog • Minimal assumptions of underlying kernel and file system –Any type of file systems that support extended attribute(xattr) –Only require kernel version >= 2.6.32 • Full of features –snapshot, clone, incremental backup, cluster-wide snapshot, discard, etc. –User-defined replication/erasure code scheme on VDI(Virtual Disk Image) basis –Auto node/disk management • Easy to set up the cluster with thousands of nodes –Single daemon can manage unlimited number of disks in one node as efficient as RAID0 –as many as 6k+ for a single cluster • Small –Fast and very small memory footprint (less than 50 MB even when busy) –Easy to hack and maintain, 35K lines of code in C as of now
  • 9. Sheepdog In A Nutshell Gateway 256G SSD Data Replicated or Erasure coded Store 4G /dev/vda Net Weighted Nodes & Disks /dev/vda Shared Persistent Cache Store Journal 2TB 1TB 1TB 1TB 1TB
  • 10. Sheepdog Volume • Copy-On-Write Snapshot –Disk-only snapshot, disk & memory snapshot –Live snapshot, offline snapshot –Rollback(tree structure), clone –Incremental backup –Instant operation, only create 4M inode object • Push many logics into client -> simple and fast code ! –Only 4 opcodes for store, read/write/create/remove, snapshot is done by QEMU block driver or dog –Requests serialization is not handled by Sheepdog but client –Inode object is treated the same as data object
  • 11. Gateway - Request Engine Return Only All Succeed Req Queue Node Ring Socket Cache Concurrent Handling Route Retry Net Cache Req Manager Store 1TB 1TB 2TB X Retry On Error(Timeout, EIO) 2TB 4TB Sheep Sheep Sheep Sheep X Degraded to GW-only On EIO
  • 12. Store - Data Engine Sheep Disk Ring GW Net Sheep Disk Manager Sheep Journal 1TB 1TB 2TB X 2TB Auto unplugged on EIO 1. Fake network err to ask GW retry 2. Update disk ring 3. Start local data rebalance Sheep
  • 13. Redundancy Scheme Erasure Coding Full Replication Parity Sheep Sheep Sheep Sheep Sheep Sheep Sheep Sheep Sheep
  • 14. Erasure Coding Over Full Replication • Advantages –Far less storage overhead 6 N odes w t h 1G N C i b I 100 –Rumors breaking 80 • Better R/W performance • Can run VM Images ! • Disadvantages s / B M • Support random R/W 60 R i cat i on ( 3 C epl opy) Er asur e ( 4: 2) 40 20 0 R ead –Generate more traffic for recovery • (X + Y)/(Y + 1) times data ( Suppose X data, Y parity strips) – 2 times data for 4:2 compared to 3 full copies Wi t e r
  • 15. Recovery - Redundancy Repair & Data Rebalance Sleep Queue Node Ring Update Version Schedule Recovery Manager If lost, read or rebuild from other copy Sheep Net Req Queue Migrate to other node for rebalance X Sheep
  • 16. Recovery Cont. • Eager Recovery As Default –Allow users to stop it and do manual recovery temporarily • Node/Disk Events Handling –Disk event and node event share the same algorithm • Handle mixed node and disk events nicely –Subsequent event will supersede previous one • Handle group join/leave of disks and nodes gracefully • Recovery Handling Transparently to the Client –Put requests for objects being recovered on sleep queue and wake it up later –Serve the request directly if object is right there in the store
  • 17. Farm - Cluster Wide Snapshot Sheep Meta Data Generator Slicer Net 128K Hash Dedupper Dedicated Backup Storage Sheep Sheep • Incremental backup • Up to 50% dedup ratio • Compression doesn't help Think of Sheepdog On Sheepdog ? Yeah! Sheep
  • 19. Openstack Storage Components • Cinder - Block Storage –Support since day 1 • Glance - Image Storage Swift Cinder Nova Glance –Support merged at Havana version • Nova - Ephemeral Storage –Not yet started • Unified Storage Swift - Object Storage –Swift API compatible In progress • Final Goal - Unified Storage –Copy-On-Write anywhere ? –Data dedup ? Sheep Sheep Sheep Sheep
  • 21. Look Into The Future • RESTful Container –Plans to be Openstack Swift API compatible first, coming soon • Hyper Volume –256PB Volume, coming soon • Geo-Replication • Sheepdog On Sheepdog –Storage for cluster wide snapshot • Slow Disk & Broken Disk Detecter –Deal with dead D state process hang because of broken disk in massive deployment
  • 23. Sheepdog In Taobao & NTT SD VM SD VM SD VM SD VM SD SD SD SD SD SD SD SD SD SD SD SD HTTP VM running inside Sheepdog Cluster for test & dev at Taobao Ongoing project with 10k+ ARM nodes for cold data at Taobao LUN device pool Sheepdog cluster run as iSCSI TGT backend storage at NTT
  • 24. Other Users In Production Any more users I don't know ?
  • 25. Q&A Homepage http://sheepdog.github.io/sheepdog/ Try me out git clone git://github.com/sheepdog/sheepdog.git Go Sheepdog !

Editor's Notes

  1. {}