Recommended
More Related Content
What's hot
What's hot (11)
Viewers also liked
Viewers also liked (17)
Similar to GlusterFS Basics for OpenStack
Similar to GlusterFS Basics for OpenStack (20)
Recently uploaded
Recently uploaded (20)
GlusterFS Basics for OpenStack
- 1. GlusterFS Overview Cody Hill Lead Systems Engineer Platform9
- 2. http://platform9.com / @Platform9Sys Redundancy Considerations • How many failures can you tolerate? • Do you want redundancy during maintenance windows? • How do you prevent split-brain issues? • How much disk space are you willing to give up for data redundancy? 2
- 4. Preventing split-brain is pretty simple, but it can be expensive. Ensure there is always a quorum at all times. Meaning double parity.
- 5. Replication
- 6. Storage replication is a service in which stored data is duplicated in real time to another device.
- 7. http://platform9.com / @Platform9Sys Replication • The creation for data copies across different locations of the storage system Typically 2 or 3 copies, configurable based on accepted risk level If a drive fails, data is recreated on another drive from replica(s) • Pros • Less CPU intensive = faster write performance • Simple restore = faster rebuild performance • Cons Requires 2x or more of the original storage space
- 8. http://platform9.com / @Platform9Sys Disk Layout Gluster01 Gluster05 Gluster02 Gluster06 Gluster03 Gluster04 Total Disks: 36 - Redundant Disks: 24_ = Data Disks: 12 Usable Capacity: 33%
- 9. http://platform9.com / @Platform9Sys Commands: Create Volume # gluster volume create pf9 replica 3 transport tcp gluster01:/bricks/brick1/brick gluster02:/bricks/brick1/brick gluster03:/bricks/brick1/brick gluster04:/bricks/brick1/brick gluster05:/bricks/brick1/brick gluster06:/bricks/brick1/brick gluster01:/bricks/brick2/brick gluster02:/bricks/brick2/brick gluster03:/bricks/brick2/brick gluster04:/bricks/brick2/brick gluster05:/bricks/brick2/brick gluster06:/bricks/brick2/brick gluster01:/bricks/brick3/brick gluster02:/bricks/brick3/brick gluster03:/bricks/brick3/brick gluster04:/bricks/brick3/brick gluster05:/bricks/brick3/brick gluster06:/bricks/brick3/brick gluster01:/bricks/brick4/brick gluster02:/bricks/brick4/brick gluster03:/bricks/brick4/brick gluster04:/bricks/brick4/brick gluster05:/bricks/brick4/brick gluster06:/bricks/brick4/brick gluster01:/bricks/brick5/brick gluster02:/bricks/brick5/brick gluster03:/bricks/brick5/brick gluster04:/bricks/brick5/brick gluster05:/bricks/brick5/brick gluster06:/bricks/brick5/brick gluster01:/bricks/brick6/brick gluster02:/bricks/brick6/brick gluster03:/bricks/brick6/brick gluster04:/bricks/brick6/brick gluster05:/bricks/brick6/brick gluster06:/bricks/brick6/brick
- 10. http://platform9.com / @Platform9Sys Adding Nodes Gluster01 Gluster05 Gluster02 Gluster06 Gluster03 Gluster04 Total Disks: 42 - Unused Disks: 6 - Redundant Disks: 24_ = Data Disks: 12 Usable Capacity: 28% Gluster07
- 11. http://platform9.com / @Platform9Sys Relocating Bricks Gluster01 Gluster05 Gluster02 Gluster06 Gluster03 Gluster04 Total Disks: 42 - Unused Disks: 6 - Redundant Disks: 24_ = Data Disks: 12 Usable Capacity: 28% Gluster07
- 12. http://platform9.com / @Platform9Sys Expand Volume Gluster01 Gluster05 Gluster02 Gluster06 Gluster03 Gluster04 Total Disks: 42 - Redundant Disks: 28_ = Data Disks: 14 Usable Capacity: 33% Gluster07
- 13. http://platform9.com / @Platform9Sys Commands: Add node # gluster peer probe gluster07 Relocate bricks (Redundancy is degraded until this completes) # gluster volume replace-brick pf9 gluster03:/bricks/brick1/brick gluster07:/bricks/brick1/brick commit force # gluster volume replace-brick pf9 gluster04:/bricks/brick2/brick gluster07:/bricks/brick2/brick commit force # gluster volume replace-brick pf9 gluster05:/bricks/brick3/brick gluster07:/bricks/brick3/brick commit force # gluster volume replace-brick pf9 gluster06:/bricks/brick4/brick gluster07:/bricks/brick4/brick commit force Clean up old bricks gluster03:# rm -rf /bricks/brick1/brick gluster03:# mkdir –p /bricks/brick1/brick gluster04:# rm -rf /bricks/brick2/brick gluster04:# mkdir –p /bricks/brick2/brick gluster05:# rm -rf /bricks/brick3/brick gluster05:# mkdir –p /bricks/brick3/brick gluster06:# rm -rf /bricks/brick4/brick gluster06:# mkdir –p /bricks/brick4/brick Add new bricks # gluster volume add-brick pf9 replica 3 gluster03:/bricks/brick1/brick gluster04:/bricks/brick2/brick gluster07:/bricks/brick5/brick # gluster volume add-brick pf9 replica 3 gluster05:/bricks/brick3/brick gluster06:/bricks/brick4/brick gluster07:/bricks/brick6/brick Rebalance volume # gluster volume rebalance pf9 start
- 14. Erasure Coding
- 15. Erasure coding is a method of data protection in which data is broken into fragments, expanded and encoded with redundant data pieces and stored across a set of different locations or storage media.
- 16. http://platform9.com / @Platform9Sys Erasure Coding • A parity based protection technique Data is broken into fragments and encoded Data is stored across multiple different locations with a configurable number of redundant pieces • Pros • Consumes less storage than replication – good for cheap/deep • Allows for the failure of two or more elements of the storage system • Cons Parity calculation is CPU-intensive Increased latency can slow production writes and rebuilds
- 17. http://platform9.com / @Platform9Sys Disk Layout Gluster01 Gluster05 Gluster02 Gluster06 Gluster03 Gluster04 Total Disks: 36 - Parity Disks: 12_ = Data Disks: 24 Usable Capacity: 67%
- 18. http://platform9.com / @Platform9Sys Commands: Create Volume # gluster volume create pf9 disperse-data 4 redundancy 2 transport tcp gluster01:/bricks/brick1/brick gluster02:/bricks/brick1/brick gluster03:/bricks/brick1/brick gluster04:/bricks/brick1/brick gluster05:/bricks/brick1/brick gluster06:/bricks/brick1/brick gluster02:/bricks/brick2/brick gluster03:/bricks/brick2/brick gluster04:/bricks/brick2/brick gluster05:/bricks/brick2/brick gluster06:/bricks/brick2/brick gluster01:/bricks/brick2/brick gluster03:/bricks/brick3/brick gluster04:/bricks/brick3/brick gluster05:/bricks/brick3/brick gluster06:/bricks/brick3/brick gluster01:/bricks/brick3/brick gluster02:/bricks/brick3/brick gluster04:/bricks/brick4/brick gluster05:/bricks/brick4/brick gluster06:/bricks/brick4/brick gluster01:/bricks/brick4/brick gluster02:/bricks/brick4/brick gluster03:/bricks/brick4/brick gluster05:/bricks/brick5/brick gluster06:/bricks/brick5/brick gluster01:/bricks/brick5/brick gluster02:/bricks/brick5/brick gluster03:/bricks/brick5/brick gluster04:/bricks/brick5/brick gluster06:/bricks/brick6/brick gluster01:/bricks/brick6/brick gluster02:/bricks/brick6/brick gluster03:/bricks/brick6/brick gluster04:/bricks/brick6/brick gluster05:/bricks/brick6/brick
- 19. http://platform9.com / @Platform9Sys Adding Nodes Total Disks: 42 - Unused Disks: 6 - Parity Disks: 12_ = Data Disks: 24 Usable Capacity: 57% Gluster07 Gluster01 Gluster05 Gluster02 Gluster06 Gluster03 Gluster04
- 20. http://platform9.com / @Platform9Sys Relocating Bricks Total Disks: 42 - Unused Disks: 6 - Parity Disks: 12_ = Data Disks: 24 Usable Capacity: 57% Gluster07 Gluster01 Gluster05 Gluster02 Gluster06 Gluster03 Gluster04
- 21. http://platform9.com / @Platform9Sys Growing Volume Total Disks: 42 - Parity Disks: 14_ = Data Disks: 28 Usable Capacity: 67% Gluster07 Gluster01 Gluster05 Gluster02 Gluster06 Gluster03 Gluster04
- 22. http://platform9.com / @Platform9Sys Commands: Add node # gluster peer probe gluster07 Relocate bricks (Redundancy is degraded until this completes) # gluster volume replace-brick pf9 gluster01:/bricks/brick1/brick gluster07:/bricks/brick1/brick commit force # gluster volume replace-brick pf9 gluster02:/bricks/brick2/brick gluster07:/bricks/brick2/brick commit force # gluster volume replace-brick pf9 gluster03:/bricks/brick3/brick gluster07:/bricks/brick3/brick commit force # gluster volume replace-brick pf9 gluster04:/bricks/brick5/brick gluster07:/bricks/brick5/brick commit force # gluster volume replace-brick pf9 gluster06:/bricks/brick6/brick gluster07:/bricks/brick6/brick commit force Clean up old bricks gluster01:# rm -rf /bricks/brick1/brick gluster01:# mkdir –p /bricks/brick1/brick gluster02:# rm -rf /bricks/brick2/brick gluster02:# mkdir –p /bricks/brick2/brick gluster03:# rm -rf /bricks/brick3/brick gluster03:# mkdir –p /bricks/brick3/brick gluster04:# rm -rf /bricks/brick5/brick gluster04:# mkdir –p /bricks/brick5/brick gluster06:# rm -rf /bricks/brick6/brick gluster06:# mkdir –p /bricks/brick6/brick Add new bricks # gluster volume add-brick pf9 disperse-data 4 redundancy 2 gluster01:/bricks/brick1/brick gluster02:/bricks/brick2/brick gluster03:/bricks/brick3/brick gluster04:/bricks/brick5/brick gluster06:/bricks/brick6/brick gluster07:/bricks/brick4/brick Rebalance volume # gluster volume rebalance pf9 start
- 23. Striping
- 24. Disk striping is the process of dividing a body of data into blocks and spreading the data blocks across multiple storage devices, such as hard disks or solid- state drives (SSDs). A stripe consists of the data divided across the set of hard disks or SSDs, and a striped unit, or strip, that refers to the data slice on an individual drive.
- 25. Great…But how do I know it’s healthy?!
- 26. Monitoring
- 27. http://platform9.com / @Platform9Sys Things to monitor • Hosts are online • GlusterFS services are online • Blocks are online • Volumes are online • Volumes aren’t degraded • Remote connectivity • Host’s CPU • Host’s RAM • Network IO • Disk IO • Block’s remaining capacity • Volume’s remaining capacity 27