Your SlideShare is downloading. ×
0
TLUG Meeting 2008/09/13 Gosuke Miyashita
My company <ul><li>paperboy&co. </li></ul><ul><ul><li>Web hosting, blog, ec hosting and so on for indivisuals </li></ul></...
My goal of a scalable storage system <ul><li>Storage system for a web hosting service </li></ul><ul><ul><li>High resource ...
 
<ul><li>cman </li></ul><ul><li>CLVM </li></ul><ul><li>GFS2 </li></ul><ul><li>GNBD </li></ul><ul><li>DRBD </li></ul><ul><li...
cman <ul><li>Cluster Manager </li></ul><ul><li>A component of Red Hat Cluster Suit </li></ul><ul><li>Membership management...
CLVM <ul><li>Cluster Logical Volume Manager </li></ul><ul><li>Cluster-wide version of LVM2 </li></ul><ul><li>Automatically...
CLVM clvmd distributes metadata among cluster nodes Logical volumes presented to each cluster node Logical volume on share...
GNBD <ul><li>Global Network Block Device </li></ul><ul><li>Provides block-device access over TCP/IP  </li></ul><ul><li>Sim...
GNBD TCP/IP network GNBD client GNBD client GNBD client GNBD Server Exported block device
GFS2 <ul><li>Global File System 2 </li></ul><ul><li>One of cluster-aware file systems </li></ul><ul><li>Multiple nodes can...
GFS2 These nodes can access to the GFS2 file system simultaneously GNBD Server GFS2 GNBD client cman GNBD client cman GNBD...
DRBD <ul><li>Distributed Replicated Block Device </li></ul><ul><li>RAID1 over a network </li></ul><ul><li>Mirrors a whole ...
DRBD Replication Server Block Device Server Block Device
DM-MP <ul><li>Device-Mapper Multipath </li></ul><ul><li>Bundles I/O paths to one virtual I/O path </li></ul><ul><li>Can ch...
DM-MP with SAN storage /dev/sda1 /dev/sdb1 Seen as one device /dev/mapper/mpath0 active/passive or active/active Node HBA1...
 
A scalable storage system /dev/VG0/LV0 (CLVM) mount /dev/VG0/LV0 /mnt cman GNBD cman GNBD GNBD Server GFS2 GNBD Server GFS...
How to extend /dev/VG0/LV0 (CLVM) mount /dev/VG0/LV0 /mnt cman GNBD cman GNBD GNBD Server GFS2 GNBD Server GFS2 /dev/mappe...
 
I wonder ... <ul><li>Many components cause troubles? </li></ul><ul><li>How about overhead and performance? </li></ul><ul><...
 
Upcoming SlideShare
Loading in...5
×

How To Build A Scalable Storage System with OSS at TLUG Meeting 2008/09/13

3,509

Published on

Published in: Technology, Business

Transcript of "How To Build A Scalable Storage System with OSS at TLUG Meeting 2008/09/13"

  1. 1. TLUG Meeting 2008/09/13 Gosuke Miyashita
  2. 2. My company <ul><li>paperboy&co. </li></ul><ul><ul><li>Web hosting, blog, ec hosting and so on for indivisuals </li></ul></ul><ul><ul><li>About 1,000 Linux servers </li></ul></ul><ul><ul><li>Many single servers ... </li></ul></ul>
  3. 3. My goal of a scalable storage system <ul><li>Storage system for a web hosting service </li></ul><ul><ul><li>High resource availability </li></ul></ul><ul><ul><li>Flexible I/O distribution </li></ul></ul><ul><ul><li>Easy to extend </li></ul></ul><ul><ul><li>Mountable by multiple hosts </li></ul></ul><ul><ul><li>No SPoF </li></ul></ul><ul><ul><li>With OSS </li></ul></ul><ul><ul><li>Without expensive hardwares </li></ul></ul><ul><li>Now I’m trying technologies for these purposes </li></ul>
  4. 5. <ul><li>cman </li></ul><ul><li>CLVM </li></ul><ul><li>GFS2 </li></ul><ul><li>GNBD </li></ul><ul><li>DRBD </li></ul><ul><li>DM-MP </li></ul>Technologies
  5. 6. cman <ul><li>Cluster Manager </li></ul><ul><li>A component of Red Hat Cluster Suit </li></ul><ul><li>Membership management </li></ul><ul><li>Messaging among cluster nodes </li></ul><ul><li>Needed for CLVM and GFS2 </li></ul>
  6. 7. CLVM <ul><li>Cluster Logical Volume Manager </li></ul><ul><li>Cluster-wide version of LVM2 </li></ul><ul><li>Automatically share LVM2 metadata among all cluster nodes </li></ul><ul><li>So logical volumes with CLVM available to all cluster nodes </li></ul>
  7. 8. CLVM clvmd distributes metadata among cluster nodes Logical volumes presented to each cluster node Logical volume on shared storage LVM2 Metadata clvmd LVM2 Metadata clvmd LVM2 Metadata clvmd
  8. 9. GNBD <ul><li>Global Network Block Device </li></ul><ul><li>Provides block-device access over TCP/IP </li></ul><ul><li>Similar to iSCSI </li></ul><ul><li>Advantage over iSCSI is built-in fencing </li></ul>
  9. 10. GNBD TCP/IP network GNBD client GNBD client GNBD client GNBD Server Exported block device
  10. 11. GFS2 <ul><li>Global File System 2 </li></ul><ul><li>One of cluster-aware file systems </li></ul><ul><li>Multiple nodes can simultaneously access this filesystem </li></ul><ul><li>Uses DLM(Distributed Lock Manager) of cman to maintain file system integrity </li></ul><ul><li>OCFS is another cluster-aware file system </li></ul>
  11. 12. GFS2 These nodes can access to the GFS2 file system simultaneously GNBD Server GFS2 GNBD client cman GNBD client cman GNBD client cman
  12. 13. DRBD <ul><li>Distributed Replicated Block Device </li></ul><ul><li>RAID1 over a network </li></ul><ul><li>Mirrors a whole block device over TCP/IP </li></ul><ul><li>Available Active/Active with cluster file systems </li></ul>
  13. 14. DRBD Replication Server Block Device Server Block Device
  14. 15. DM-MP <ul><li>Device-Mapper Multipath </li></ul><ul><li>Bundles I/O paths to one virtual I/O path </li></ul><ul><li>Can choose active/passive or active/active </li></ul>
  15. 16. DM-MP with SAN storage /dev/sda1 /dev/sdb1 Seen as one device /dev/mapper/mpath0 active/passive or active/active Node HBA1 HBA2 SAN swtich 1 SAN swtich 2 Storage CNTRLR1 CNTRLR2
  16. 18. A scalable storage system /dev/VG0/LV0 (CLVM) mount /dev/VG0/LV0 /mnt cman GNBD cman GNBD GNBD Server GFS2 GNBD Server GFS2 Replication (DRBD) /dev/mapper/mpath0 (DM-MP) /dev/gnbd0 /dev/gnbd1 GNBD Server GFS2 GNBD Server GFS2 Replication (DRBD) /dev/mapper/mpath1 (DM-MP) /dev/gnbd2 /dev/gnbd3
  17. 19. How to extend /dev/VG0/LV0 (CLVM) mount /dev/VG0/LV0 /mnt cman GNBD cman GNBD GNBD Server GFS2 GNBD Server GFS2 /dev/mapper/mpath0 /dev/gnbd0 /dev/gnbd1 GNBD Server GFS2 GNBD Server GFS2 /dev/mapper/mpath1 /dev/gnbd2 /dev/gnbd3 GNBD Server GFS2 GNBD Server GFS2 /dev/mapper/mpath2 /dev/gnbd4 /dev/gnbd5
  18. 21. I wonder ... <ul><li>Many components cause troubles? </li></ul><ul><li>How about overhead and performance? </li></ul><ul><li>How about stability? </li></ul><ul><li>More better way? </li></ul><ul><li>How about other than Red Hat Linux? </li></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×