• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
ZFS on the server and the desktop: N+1 ways to better store your data
 

ZFS on the server and the desktop: N+1 ways to better store your data

on

  • 441 views

Presented on 21 November 2013 at the NLUUG fall conference in Bunnik, The Netherlands

Presented on 21 November 2013 at the NLUUG fall conference in Bunnik, The Netherlands

Statistics

Views

Total Views
441
Views on SlideShare
441
Embed Views
0

Actions

Likes
1
Downloads
6
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    ZFS on the server and the desktop: N+1 ways to better store your data ZFS on the server and the desktop: N+1 ways to better store your data Presentation Transcript

    • ZFS on the server and the desktop N+1 ways to better store your data Presented at NLUUG 21-11-2013
    • About Matthias van der Heide Storage Network Engineer Inprove IT BV an EraStor member @matthiasvdh on Twitter
    • Contents • • • • • ZFS Intro Features N+1 ways Why use it in DevOps Where to get it
    • What is ZFS? • “Zettabyte File System” • Sun, now Oracle • Thank Sun for OpenSolaris and releasing ZFS
    • ZFS Features • • • • • • • • Yes, it is a file system It is also volume manager Virtually unlimited capacity Transactional Multiple types of redundancy Virtually unlimited snapshots, clones Smart caching Versioned on-disk format
    • Copy-on-write
    • Checksums NL 08 IN GB 00 00 00 05 55
    • ARC • Adaptive Replacement Cache • Combines MRU and MFU • Uses ghost lists • Consumes (a lot of) memory, extend to SSD
    • ZIL • ZFS Intent Log • In memory • On the pool • Optionally on a seperate log device (SSD)
    • Transaction Groups (TXG) Open Quiescing Syncing • Accept new writes • Wait for I/O to complete • Write the data to disk
    • N+1 ways • Designing your pool • Testing your pool • Tuning the filesystem • Maintenance
    • N=1: Designing your pool • Your performance and redundancy needs determine your pool design • You can change it, only slightly, afterwards • Consider a system with 4 HDDs • Now what do I do?
    • Redundancy types • A vdev is the building block of the zpool • Everything is striped over vdevs • Options: – – – – Leaf (1) Mirror (2..n) RAID-Z1 (n+1) RAID-Z2 (n+2)
    • Options • Stripe over all drives • Stripe over mirrors • RAID-Z1 • RAID-Z2
    • Fun options • Dual RAID-Z1 • 4-way mirror • HDD as slog device
    • N=2: Testing your pool • Simple: time dd • Iometer • Vdbench • FIO
    • N=3: Tuning your filesystem • • • • • Block size Log bias Access time Sync on/off Caching on/off
    • N=4: Maintenance • ZFS will tell you when stuff is broken • Check data on all reads • Regular scrubs
    • Scrubs • Scans the pool for data integrity • Checks checksums • Restores data if needed • Can take a while on large pools
    • Scrub
    • Scrub time Estimate accuracy 100,00 90,00 80,00 70,00 60,00 50,00 40,00 30,00 20,00 10,00 0,00 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 Estimate accuracy
    • Resilvers • When replacing a device • When adding a device back after removal
    • Simple detach/attach
    • After being offline
    • New device
    • Future work on ZFS • Feature flags enables development of features seperately • Persistent L2ARC • Larger block support • Dedupe tables on SSD
    • Why use it in DevOps • ZFS performs • ZFS scales – From your laptop to your workstation to your cloud • ZFS replicates – Using zfs send/receive you can deploy copies of your VMs to other hosts – Keep simple backups of your repos and machines • Use snapshots to boot into new versions of your operating system
    • Where to get it? • • • • • • FreeBSD Mac ZFS on Linux Illumos (EraStor OS) NexentaStor (based on OpenSolaris/Illumos) OpenZFS - http://open-zfs.org
    • Links • Open ZFS Project http://open-zfs.org • ARC Paper http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.1 3.5210 • NOSIG http://www.nosig.nl/ • Erastor http://www.erastor.eu