Your SlideShare is downloading. ×
Clusters with GlusterFS
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Clusters with GlusterFS

1,162
views

Published on

Kosovo 2009

Kosovo 2009

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,162
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
15
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Kosova Sofware Freedom Conference 2009 Clusters with GlusterFS Marian Marinov - mm@yuhu.biz System Architect - Siteground.comPrishtina 29-30.Aug.2009
  • 2. Agenda Cluster Filesystems Some facts Gluster Design ➢ kernel ➢ gluster engine ➢ protocols ➢ translators ➢ storage ➢ performance ➢ others ➢ schedulers Some benchmarks 2Prishtina 29-30.Aug.2009 1/29
  • 3. Cluster Filesystems 3Prishtina 29-30.Aug.2009
  • 4. Cluster Filesystems 4Prishtina 29-30.Aug.2009 3/29
  • 5. Cluster Filesystems 5Prishtina 29-30.Aug.2009
  • 6. GFarm Desgin 6Prishtina 29-30.Aug.2009 6/29
  • 7. Facts GlusterFS project starts in August 2006 ➢ It is not actual Filesystem ➢ Server only for any POSIX compliant but mainly tested on Linux ➢ Client running on Linux, FreeBSD & MacOS X as they require FUSE ➢ Very scallable ➢ Very easy to install and maintain 7Prishtina 29-30.Aug.2009 4/29
  • 8. GlusterFS Desgin 8Prishtina 29-30.Aug.2009
  • 9. Gluster Filesystem Design In the kernel ➢ Requires FUSE ➢ FUSE as module ➢ GlusterFUSE The engine ➢ Server & Client ➢ Transport Modules ➢ Translators ➢ Scheduler Modules 9Prishtina 29-30.Aug.2009
  • 10. GlusterFS Desgin 10Prishtina 29-30.Aug.2009
  • 11. GlusterFS Desgin 11Prishtina 29-30.Aug.2009
  • 12. GlusterFS Desgin 12Prishtina 29-30.Aug.2009
  • 13. GlusterFS Desgin 13Prishtina 29-30.Aug.2009
  • 14. GlusterFS DesginThe picture explained:ClientX:volume serverX - defines a name for a remote serversubvolumes brick0 - defines in which of all exported volumes from the remote server we are interestedsome performance translatorsvolume unify - defines that we will use unify cluster translatorsubvolumes serverX serverY - defines which already connected storagevolumes will be used 14Prishtina 29-30.Aug.2009
  • 15. Gluster Filesystem Design Transport Modules: For TCP/IP transport transport-type tcp/server For Infiniband SDP transport transport-type ib-sdp/server For Infiniband Verbs transport transport-type ib-verbs/server 15Prishtina 29-30.Aug.2009
  • 16. Gluster Filesystem Design The idea – GNU/Hurd Translators ➢ Performance ➢ Clustering ➢ Features ➢ Storage ➢ Others 16Prishtina 29-30.Aug.2009 14/29
  • 17. Gluster Filesystem Design Performance translators ➢ Read Ahead ➢ Write Behind ➢ Threaded I/O ➢ IO-Cache ➢ Stat Pre-fetch – still not ported to the new versions ➢ Booster 17Prishtina 29-30.Aug.2009 15/29
  • 18. Gluster Filesystem Design Clustering translators ➢ Distributed Hash Table (DHT) ➢ Stripe ➢ Replicate (old AFR) ➢ Unify (new HA) ➢ NUFA 18Prishtina 29-30.Aug.2009 16/29
  • 19. Gluster Filesystem Design Distributed Hash Table (DHT) ➢ lookup-unhashed ➢ min-free-disk Replicate ➢ read-subvolume ➢ favorite-child ➢ data-self-heal, metadata-self-heal & entry-self-heal ➢ data-change-log, metadata-change-log & entry-change-log ➢ data-lock-server-count, metadata-lock-server-count & entry-lock-server-count Stripe & Unify 19Prishtina 29-30.Aug.2009
  • 20. Gluster Filesystem Design Scheduling systems ➢ Adaptive Least Usage (ALU) ➢ Non-uniform filesystem architecture (NUFA) ➢ Random ➢ Rand-Robin ➢ Switch 20Prishtina 29-30.Aug.2009 17/29
  • 21. Gluster Filesystem Design Adaptive Least Usage (ALU) ➢ disk-usage ➢ read-usage ➢ write-usage ➢ open-files-usage ➢ disk-speed-usage 21Prishtina 29-30.Aug.2009 18/29
  • 22. Gluster Filesystem Design Non-uniform filesystem architecture (NUFA) ➢ local-volume-name ➢ limits.min-free-disk Random ➢ limits.min-free-disk Round-Robin ➢ limits.min-free-disk ➢ read-only-subvolumes ➢ refresh-interval 22Prishtina 29-30.Aug.2009 19/29
  • 23. Gluster Filesystem Design Switch ➢ switch.case *jpg:brick1,brick2;*mp3:brick3;*:brick4,brick5 ➢ switch.read-only-subvolumes brick7 23Prishtina 29-30.Aug.2009 20/29
  • 24. Gluster Filesystem Design Other translators ➢ client ➢ server ➢ posix ➢ posix-locks ➢ bdb ➢ filter ➢ rot-13 ➢ trace 24Prishtina 29-30.Aug.2009 21/29
  • 25. Gluster Filesystem Design filter ➢ root-squashing ➢ read-only ➢ fixed-uid & fixed-gid ➢ translate-uid & translate-gid ➢ filter-uid & filter-gid 25Prishtina 29-30.Aug.2009 21/29
  • 26. Gluster Filesystem Design In the feature ➢ Live addition/removal of nodes ➢ Automatic File Reordering ➢ Web GUI ➢ mod_glusterfs 26Prishtina 29-30.Aug.2009 22/29
  • 27. Gluster Design 27Prishtina 29-30.Aug.2009 23/29
  • 28. Benchmarks 28Prishtina 29-30.Aug.2009 24/29
  • 29. Benchmarks 29Prishtina 29-30.Aug.2009
  • 30. BenchmarksAggregated Read Throughput BenchmarkMultiple dd utility were executed simultaneously with different block sizesto read from GlusterFS filesystem. 4KB 16KB 128KB 256KB 512KB 1024KBLustre 1,796 MB/s 5,782 MB/s 20,423 MB/s 21,582 MB/s 22,789 MB/s 23,731 MB/sGlusterFS 11,415 MB/s 11,424 MB/s 11,427 MB/s 11,419 MB/s 11,411 MB/s 11,409 MB/sAggregated Write Throughput BenchmarkMultiple dd utility were executed simultaneously with different block sizesto write to GlusterFS filesystem. 4KB 16KB 128KB 256KB 512KB 1024KBLustre 969 MB/s 1,613 MB/s 1,988 MB/s 1,989 MB/s 1,984 MB/s 1,983 MB/sGlusterFS 1,886 MB/s 2,191 MB/s 2,237 MB/s 2,231 MB/s 2,236 MB/s 2,223 MB/sNote: Higher means faster. 30Prishtina 29-30.Aug.2009 26/29
  • 31. BenchmarksApache Web Server BenchmarkApache served 12039 files (595 MB) over HTTP protocol.wget client fetched the files recursively. TimeLustre Failed after downloading 33 MB out of 585 MB in 11 mins.GlusterFS 3 mins 11 secsArchive Creationtar utility created an archive of 12039 files (595 MB) served through GlusterFS. TimeLustre 41 secsGlusterFS 25 secsArchive Extraction TimeLustre FAILED No space left on device.GlusterFS 43 secsNote: Lower means faster. 31Prishtina 29-30.Aug.2009 27/29
  • 32. Benchmarks 32Prishtina 29-30.Aug.2009
  • 33. Benchmarks Sequencial Write : 1KB x 1,000,000 times = 1GB # time dd if=/dev/zero of=/mnt/unify/file bs=1024 count=1000000 Test Case Local SATA Local NFS Single Unified 500G RAID0 GlusterFS GlusterFS 1 Worker - 11.836s 11.371s 23.162s 2m19.597s 3m39.279s 1st test (82.5MB/s) (85.9MB/s) (42.2MB/s) (7.0MB/s) (4.4MB/s) 1 Worker - 10.537s 10.777s 24.181s 2m24.623s 3m40.334s 2nd test (92.7MB/s) (90.6MB/s) (40.4MB/s) (6.7MB/s) (4.4MB/s) 33Prishtina 29-30.Aug.2009
  • 34. Benchmarks Sequencial Write : 64KB x 15,625 times = 1GB # time dd if=/dev/zero of=/mnt/unify/file bs=65536 count=15625 Test Case Local SATA Local NFS Single Unified 500G RAID0 GlusterFS GlusterFS 1 Worker - 6.390s 7.939s 22.766s 24.637s 22.436s 1st test (152.8MB/s) (123.0MB/s) (42.9MB/s) (39.6MB/s) (43.5MB/s) 1 Worker - 6.588s 7.542s 21.901s 22.001s 23.378s 2nd test (148.2MB/s) (129.5MB/s) (44.6MB/s) (44.4MB/s) (41.8MB/s) 34Prishtina 29-30.Aug.2009
  • 35. Sources of Information Projects site: http://www.gluster.com Official GlusterFS documentation wiki: http://www.gluster.org/docs/index.php/GlusterFS On IRC: irc.freenode.net #gluster The mailing list: gluster-devel@nongnu.org 35Prishtina 29-30.Aug.2009 28/29
  • 36. Clusters with GlusterFS ? ? ? ? ? ? ? ? ? ?? ? ? ? ? ? Questions ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 36Prishtina 29-30.Aug.2009

×