Gluster: Where Weve Been                               AB Periasamy                 Office of the CTO, Red Hat            ...
Topics The Big Idea Humble beginnings      From Bangalore to Milpitas Scale-out + Open source == WINNING      User-space, ...
A Data Explosion! 74% == Unstructured data annual growth 63,000 PB == Scale-out storage in 2015 40% == storage-related exp...
Conference Room    US Head Office  06/13/12Bengaluru Office   Bengaluru Office
Gluster Community Deployments
Gluster Production Deployments
What Can You Store?      Media – Docs, Photos, Video      VM Filesystem – VM Disk Images      Big Data – Log Files, RFID D...
The big idea:   Storage should be               simple Simple, scalable, low-cost06/13/12
What is GlusterFS, Really?           Gluster is a unified, distributed            storage system             DHT, stackabl...
Phase 1: Lego Kit for  Storage      “People who think that userspace filesystems        are realistic for anything but toy...
volume testvol-posix         type storage/posix         option directory /media/datastore         option volume-id 329e31c...
Versions 1.x – 2.x      Hand-crafted volume definition files        See examples      Simple configuration files      Fast...
Phase 2: Repeatability  of Use Cases06/13/12
Community-led      Learned from community         Desired features         Usage profiles      All about scalable storage ...
GlusterFS 3.0: Putting it  all together      Adding, removing features      Templates – recipes for common use       cases...
GlusterFS 3.1 - 2010      Elasticity: add and remove volumes w/       glusterd      Automation: CLI, scriptable06/13/12
CLI Magic $ gluster peer probe HOSTNAME $ gluster volume info $ gluster volume create VOLNAME [stripe COUNT]            [r...
GlusterFS 3.2 - 2011      Native NFS server      Marker framework      Geo-replication        Asynchronous06/13/12
And now for something  completely different           Commoditization and the changing            economics of storage    ...
Simple Economics        Simplicity, scalability, less cost  Virtualized     Multi-Tenant   Automated   CommoditizedScale o...
Simplicity Bias      FC, FCoE, iSCSI → HTTP, Sockets      Modified BSD OS → Linux / User Space /                        C,...
Scale-out Open Source  is the winner06/13/12
Thank you!                     AB Periasamy       Office of the CTO, Red Hat                   ab@redhat.com              ...
Upcoming SlideShare
Loading in …5
×

Gluster: where weve been - a history

1,273 views

Published on

Delivered at LinuxCon Japan, as part of the Gluster workshop.

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,273
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
25
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Add examples where complexity has been bad - EMC, Cisco, Brocade et al. certification made business out of complexity - if too complicated, doesn't scale
  • Discuss approach – how GlusterFS is unique and different from other approaches - Lessons form GNU Hurd - user space distributed storage operating system - overcome some parts of the OS: implemented scheduler, POSIX locking, RDMA, MM, cf. JVM, python, etc. - no metadata separation
  • If you have a bunch of files, should be as simple as an FTP server - in user space, required FUSE, POSIX translator, NAS protocol, cluster translator
  • Learned about missing features Found the largest problem and wanted to solve it - patterns emerged - scalable unstructured data storage was the #1 problem people wanted to solve Had a clearer idea where we wanted to go – clear direction
  • Standalone NFS replacement Active-active replicated storage Scalable, distributed storage .. And then scalable, replicated distributed storage + other combos
  • Elastic features driven by cloud and virt usage - shared storage for virtual guests - flexible, self-service storage - elastic volume management became requirement - automated provisioning of storage w/ CLI (native NFS server? Or 3.2?)
  • Marker famework: - story of why it's necessary - backup of data in other locales - don't need entire snapshot - users wanted to continuous, unlimited replication - don't want sysadmin intervention – on-demand - queries FS to find what files have changed - manages queue, telling rsync exactly which files to change Inotify – doesn't scale, if daemon crashes, stops tracking changes - would have to write journaling feature to maintain change queue Geo-replication – can work on high-latency, flaky networks
  • Gluster: where weve been - a history

    1. 1. Gluster: Where Weve Been AB Periasamy Office of the CTO, Red Hat John Mark Walker Gluster Community Guy
    2. 2. Topics The Big Idea Humble beginnings From Bangalore to Milpitas Scale-out + Open source == WINNING User-space, no metadata server, stackable Cloud and commoditization06/13/12
    3. 3. A Data Explosion! 74% == Unstructured data annual growth 63,000 PB == Scale-out storage in 2015 40% == storage-related expense for cloud 44x == Unstructured data volume growth by 202006/13/12
    4. 4. Conference Room US Head Office 06/13/12Bengaluru Office Bengaluru Office
    5. 5. Gluster Community Deployments
    6. 6. Gluster Production Deployments
    7. 7. What Can You Store? Media – Docs, Photos, Video VM Filesystem – VM Disk Images Big Data – Log Files, RFID Data Objects – Long Tail Data06/13/12
    8. 8. The big idea: Storage should be simple Simple, scalable, low-cost06/13/12
    9. 9. What is GlusterFS, Really? Gluster is a unified, distributed storage system DHT, stackable, POSIX, Swift, HDFS06/13/12
    10. 10. Phase 1: Lego Kit for Storage “People who think that userspace filesystems are realistic for anything but toys are just misguided" – Linus Torvalds Goal: create a global namespace06/13/12
    11. 11. volume testvol-posix type storage/posix option directory /media/datastore option volume-id 329e31c1-04cc-4386-8bb8-xxxx end-volume volume testvol-access-control type features/access-control subvolumes testvol-posix end-volume volume testvol-locks type features/locks subvolumes testvol-access-control end-volume volume testvol-io-threads type performance/io-threads subvolumes testvol-locks end-volume06/13/12
    12. 12. Versions 1.x – 2.x Hand-crafted volume definition files See examples Simple configuration files Faster than tape? Its good!06/13/12
    13. 13. Phase 2: Repeatability of Use Cases06/13/12
    14. 14. Community-led Learned from community Desired features Usage profiles All about scalable storage of unstructured data06/13/12
    15. 15. GlusterFS 3.0: Putting it all together Adding, removing features Templates – recipes for common use cases06/13/12
    16. 16. GlusterFS 3.1 - 2010 Elasticity: add and remove volumes w/ glusterd Automation: CLI, scriptable06/13/12
    17. 17. CLI Magic $ gluster peer probe HOSTNAME $ gluster volume info $ gluster volume create VOLNAME [stripe COUNT] [replica COUNT] [transport tcp | rdma] BRICK $ gluster volume delete VOLNAME $ gluster volume add-brick VOLNAME NEW-BRICK ... $ gluster volume rebalance VOLNAME start06/13/12
    18. 18. GlusterFS 3.2 - 2011 Native NFS server Marker framework Geo-replication Asynchronous06/13/12
    19. 19. And now for something completely different Commoditization and the changing economics of storage Why were winning06/13/12
    20. 20. Simple Economics Simplicity, scalability, less cost Virtualized Multi-Tenant Automated CommoditizedScale on Demand In the Cloud Scale Out Open Source 06/13/12
    21. 21. Simplicity Bias FC, FCoE, iSCSI → HTTP, Sockets Modified BSD OS → Linux / User Space / C, Python & Java Appliance based → Application based06/13/12
    22. 22. Scale-out Open Source is the winner06/13/12
    23. 23. Thank you! AB Periasamy Office of the CTO, Red Hat ab@redhat.com John Mark Walker Gluster Community Guy johnmark@redhat.com

    ×