Clusters With Glusterfs

  • 3,243 views
Uploaded on

 

More in: Technology , Design
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
3,243
On Slideshare
0
From Embeds
0
Number of Embeds
13

Actions

Shares
Downloads
121
Comments
0
Likes
4

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. HighLoad++ 2008 Moscow Clusters with GlusterFS Marian Marinov - mm@yuhu.biz System Architect - Siteground.com Moscow 06-07.Oct.2008
  • 2. Agenda Cluster Filesystems Some facts Gluster Design ➢ kernel ➢ gluster engine ➢ protocols ➢ translators ➢ storage ➢ performance ➢ others ➢ schedulers Some benchmarks 2 Moscow 06-07.Oct.2008 1/22d
  • 3. Cluster Filesystems 3 Moscow 06-07.Oct.2008 2/22
  • 4. Facts GlusterFS project starts in August 2006 It is not actual Filesystem Server only for Linux Client running on Linux & FreeBSD Very scallable Very easy to install and maintain 4 Moscow 06-07.Oct.2008 3/22
  • 5. GlusterFS Desgin 5 Moscow 06-07.Oct.2008 4/22
  • 6. GFarm Desgin 6 Moscow 06-07.Oct.2008 5/22
  • 7. Gluster Filesystem Design In the kernel ➢ Requires FUSE ➢ FUSE as module ➢ GlusterFUSE The engine ➢ Server & Client ➢ Transport Modules ➢ Translators ➢ Scheduler Modules 7 Moscow 06-07.Oct.2008 6/22
  • 8. GlusterFS Desgin 8 Moscow 06-07.Oct.2008 7/22
  • 9. GlusterFS Desgin The picture explained: ClientX: volume serverX - defines a name for a remote server subvolumes brick0 - defines in which of all exported volumes from the remote server we are interested some performance translators volume unify - defines that we will use unify cluster translator subvolumes serverX serverY - defines which already connected storage volumes will be used 9 Moscow 06-07.Oct.2008 8/22
  • 10. Gluster Filesystem Design Transport Modules: For TCP/IP transport transport-type tcp/server For Infiniband SDP transport transport-type ib-sdp/server For Infiniband Verbs transport transport-type ib-verbs/server 10 Moscow 06-07.Oct.2008 9/22
  • 11. Gluster Filesystem Design The idea – GNU/Hurd Translators ➢ Performance ➢ Clustering ➢ Scheduling ➢ Storage ➢ Others 11 Moscow 06-07.Oct.2008 10/22
  • 12. Gluster Filesystem Design Performance translators ➢ Read Ahead ➢ Write Behind ➢ Threaded I/O ➢ IO-Cache ➢ Stat Pre-fetch – still not ported to the new versions ➢ Booster 12 Moscow 06-07.Oct.2008 11/22
  • 13. Gluster Filesystem Design Clustering translators ➢ Stripe ➢ Unify ➢ AFR 13 Moscow 06-07.Oct.2008 12/22
  • 14. Gluster Filesystem Design Scheduling translators ➢ Adaptive Least Usage (ALU) ➢ Non-uniform filesystem architecture (NUFA) ➢ Random ➢ Rand-Robin ➢ Switch 14 Moscow 06-07.Oct.2008 13/22
  • 15. Gluster Filesystem Design Adaptive Least Usage (ALU) ➢ disk-usage ➢ read-usage ➢ write-usage ➢ open-files-usage ➢ disk-speed-usage 15 Moscow 06-07.Oct.2008 14/22
  • 16. Gluster Filesystem Design Non-uniform filesystem architecture (NUFA) ➢ local-volume-name ➢ limits.min-free-disk Random ➢ limits.min-free-disk Round-Robin ➢ limits.min-free-disk ➢ read-only-subvolumes ➢ refresh-interval 16 Moscow 06-07.Oct.2008 15/22
  • 17. Gluster Filesystem Design Switch ➢ switch.case *jpg:brick1,brick2;*mp3:brick3;*:brick4,brick5 ➢ switch.read-only-subvolumes brick7 17 Moscow 06-07.Oct.2008 16/22
  • 18. Gluster Filesystem Design Other translators ➢ client ➢ server ➢ posix ➢ posix-locks ➢ bdb - very new ➢ rot-13 ➢ trace 18 Moscow 06-07.Oct.2008 17/22
  • 19. Gluster Filesystem Design In the feature ➢ Live addition/removal of nodes ➢ Automatic File Reordering ➢ Web GUI ➢ mod_glusterfs 19 Moscow 06-07.Oct.2008 18/22
  • 20. Gluster Design 20 Moscow 06-07.Oct.2008 19/22
  • 21. Benchmarks Aggregated Read Throughput Benchmark Multiple dd utility were executed simultaneously with different block sizes to read from GlusterFS filesystem. 4KB 16KB 128KB 256KB 512KB 1024KB Lustre 1,796 MB/s 5,782 MB/s 20,423 MB/s 21,582 MB/s 22,789 MB/s 23,731 MB/s GlusterFS 11,415 MB/s 11,424 MB/s 11,427 MB/s 11,419 MB/s 11,411 MB/s 11,409 MB/s Aggregated Write Throughput Benchmark Multiple dd utility were executed simultaneously with different block sizes to write to GlusterFS filesystem. 4KB 16KB 128KB 256KB 512KB 1024KB Lustre 969 MB/s 1,613 MB/s 1,988 MB/s 1,989 MB/s 1,984 MB/s 1,983 MB/s GlusterFS 1,886 MB/s 2,191 MB/s 2,237 MB/s 2,231 MB/s 2,236 MB/s 2,223 MB/s Note: Higher means faster. 21 Moscow 06-07.Oct.2008 20/22
  • 22. Benchmarks Apache Web Server Benchmark Apache served 12039 files (595 MB) over HTTP protocol. wget client fetched the files recursively. Time Lustre Failed after downloading 33 MB out of 585 MB in 11 mins. GlusterFS 3 mins 11 secs Archive Creation 'tar utility created an archive of 12039 files (595 MB) served through GlusterFS. Time Lustre 41 secs GlusterFS 25 secs Archive Extraction Time Lustre FAILED No space left on device. GlusterFS 43 secs Note: Lower means faster. 22 Moscow 06-07.Oct.2008 21/22
  • 23. Sources of Information Project's site: http://www.gluster.org Official GlusterFS documentation wiki: http://www.gluster.org/docs/index.php/GlusterFS On IRC: irc.freenode.net #gluster The mailing list: gluster-devel@nongnu.org 23 Moscow 06-07.Oct.2008 22/22
  • 24. Clusters with GlusterFS ? ? ? ? ? ? ? ? ? ?? ? ? ? ? ? Questions ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 24 Moscow 06-07.Oct.2008