Your SlideShare is downloading. ×
Gluster: where weve been - a history
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Gluster: where weve been - a history

963
views

Published on

Delivered at LinuxCon Japan, as part of the Gluster workshop.

Delivered at LinuxCon Japan, as part of the Gluster workshop.

Published in: Technology, Education

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
963
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
24
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Add examples where complexity has been bad - EMC, Cisco, Brocade et al. certification made business out of complexity - if too complicated, doesn't scale
  • Discuss approach – how GlusterFS is unique and different from other approaches - Lessons form GNU Hurd - user space distributed storage operating system - overcome some parts of the OS: implemented scheduler, POSIX locking, RDMA, MM, cf. JVM, python, etc. - no metadata separation
  • If you have a bunch of files, should be as simple as an FTP server - in user space, required FUSE, POSIX translator, NAS protocol, cluster translator
  • Learned about missing features Found the largest problem and wanted to solve it - patterns emerged - scalable unstructured data storage was the #1 problem people wanted to solve Had a clearer idea where we wanted to go – clear direction
  • Standalone NFS replacement Active-active replicated storage Scalable, distributed storage .. And then scalable, replicated distributed storage + other combos
  • Elastic features driven by cloud and virt usage - shared storage for virtual guests - flexible, self-service storage - elastic volume management became requirement - automated provisioning of storage w/ CLI (native NFS server? Or 3.2?)
  • Marker famework: - story of why it's necessary - backup of data in other locales - don't need entire snapshot - users wanted to continuous, unlimited replication - don't want sysadmin intervention – on-demand - queries FS to find what files have changed - manages queue, telling rsync exactly which files to change Inotify – doesn't scale, if daemon crashes, stops tracking changes - would have to write journaling feature to maintain change queue Geo-replication – can work on high-latency, flaky networks
  • Transcript

    • 1. Gluster: Where Weve Been AB Periasamy Office of the CTO, Red Hat John Mark Walker Gluster Community Guy
    • 2. Topics The Big Idea Humble beginnings From Bangalore to Milpitas Scale-out + Open source == WINNING User-space, no metadata server, stackable Cloud and commoditization06/13/12
    • 3. A Data Explosion! 74% == Unstructured data annual growth 63,000 PB == Scale-out storage in 2015 40% == storage-related expense for cloud 44x == Unstructured data volume growth by 202006/13/12
    • 4. Conference Room US Head Office 06/13/12Bengaluru Office Bengaluru Office
    • 5. Gluster Community Deployments
    • 6. Gluster Production Deployments
    • 7. What Can You Store? Media – Docs, Photos, Video VM Filesystem – VM Disk Images Big Data – Log Files, RFID Data Objects – Long Tail Data06/13/12
    • 8. The big idea: Storage should be simple Simple, scalable, low-cost06/13/12
    • 9. What is GlusterFS, Really? Gluster is a unified, distributed storage system DHT, stackable, POSIX, Swift, HDFS06/13/12
    • 10. Phase 1: Lego Kit for Storage “People who think that userspace filesystems are realistic for anything but toys are just misguided" – Linus Torvalds Goal: create a global namespace06/13/12
    • 11. volume testvol-posix type storage/posix option directory /media/datastore option volume-id 329e31c1-04cc-4386-8bb8-xxxx end-volume volume testvol-access-control type features/access-control subvolumes testvol-posix end-volume volume testvol-locks type features/locks subvolumes testvol-access-control end-volume volume testvol-io-threads type performance/io-threads subvolumes testvol-locks end-volume06/13/12
    • 12. Versions 1.x – 2.x Hand-crafted volume definition files See examples Simple configuration files Faster than tape? Its good!06/13/12
    • 13. Phase 2: Repeatability of Use Cases06/13/12
    • 14. Community-led Learned from community Desired features Usage profiles All about scalable storage of unstructured data06/13/12
    • 15. GlusterFS 3.0: Putting it all together Adding, removing features Templates – recipes for common use cases06/13/12
    • 16. GlusterFS 3.1 - 2010 Elasticity: add and remove volumes w/ glusterd Automation: CLI, scriptable06/13/12
    • 17. CLI Magic $ gluster peer probe HOSTNAME $ gluster volume info $ gluster volume create VOLNAME [stripe COUNT] [replica COUNT] [transport tcp | rdma] BRICK $ gluster volume delete VOLNAME $ gluster volume add-brick VOLNAME NEW-BRICK ... $ gluster volume rebalance VOLNAME start06/13/12
    • 18. GlusterFS 3.2 - 2011 Native NFS server Marker framework Geo-replication Asynchronous06/13/12
    • 19. And now for something completely different Commoditization and the changing economics of storage Why were winning06/13/12
    • 20. Simple Economics Simplicity, scalability, less cost Virtualized Multi-Tenant Automated CommoditizedScale on Demand In the Cloud Scale Out Open Source 06/13/12
    • 21. Simplicity Bias FC, FCoE, iSCSI → HTTP, Sockets Modified BSD OS → Linux / User Space / C, Python & Java Appliance based → Application based06/13/12
    • 22. Scale-out Open Source is the winner06/13/12
    • 23. Thank you! AB Periasamy Office of the CTO, Red Hat ab@redhat.com John Mark Walker Gluster Community Guy johnmark@redhat.com