Your SlideShare is downloading. ×
ZFS on the server and the desktop: N+1 ways to better store your data
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

ZFS on the server and the desktop: N+1 ways to better store your data

353
views

Published on

Presented on 21 November 2013 at the NLUUG fall conference in Bunnik, The Netherlands

Presented on 21 November 2013 at the NLUUG fall conference in Bunnik, The Netherlands

Published in: Technology

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
353
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
9
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. ZFS on the server and the desktop N+1 ways to better store your data Presented at NLUUG 21-11-2013
  • 2. About Matthias van der Heide Storage Network Engineer Inprove IT BV an EraStor member @matthiasvdh on Twitter
  • 3. Contents • • • • • ZFS Intro Features N+1 ways Why use it in DevOps Where to get it
  • 4. What is ZFS? • “Zettabyte File System” • Sun, now Oracle • Thank Sun for OpenSolaris and releasing ZFS
  • 5. ZFS Features • • • • • • • • Yes, it is a file system It is also volume manager Virtually unlimited capacity Transactional Multiple types of redundancy Virtually unlimited snapshots, clones Smart caching Versioned on-disk format
  • 6. Copy-on-write
  • 7. Checksums NL 08 IN GB 00 00 00 05 55
  • 8. ARC • Adaptive Replacement Cache • Combines MRU and MFU • Uses ghost lists • Consumes (a lot of) memory, extend to SSD
  • 9. ZIL • ZFS Intent Log • In memory • On the pool • Optionally on a seperate log device (SSD)
  • 10. Transaction Groups (TXG) Open Quiescing Syncing • Accept new writes • Wait for I/O to complete • Write the data to disk
  • 11. N+1 ways • Designing your pool • Testing your pool • Tuning the filesystem • Maintenance
  • 12. N=1: Designing your pool • Your performance and redundancy needs determine your pool design • You can change it, only slightly, afterwards • Consider a system with 4 HDDs • Now what do I do?
  • 13. Redundancy types • A vdev is the building block of the zpool • Everything is striped over vdevs • Options: – – – – Leaf (1) Mirror (2..n) RAID-Z1 (n+1) RAID-Z2 (n+2)
  • 14. Options • Stripe over all drives • Stripe over mirrors • RAID-Z1 • RAID-Z2
  • 15. Fun options • Dual RAID-Z1 • 4-way mirror • HDD as slog device
  • 16. N=2: Testing your pool • Simple: time dd • Iometer • Vdbench • FIO
  • 17. N=3: Tuning your filesystem • • • • • Block size Log bias Access time Sync on/off Caching on/off
  • 18. N=4: Maintenance • ZFS will tell you when stuff is broken • Check data on all reads • Regular scrubs
  • 19. Scrubs • Scans the pool for data integrity • Checks checksums • Restores data if needed • Can take a while on large pools
  • 20. Scrub
  • 21. Scrub time Estimate accuracy 100,00 90,00 80,00 70,00 60,00 50,00 40,00 30,00 20,00 10,00 0,00 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 Estimate accuracy
  • 22. Resilvers • When replacing a device • When adding a device back after removal
  • 23. Simple detach/attach
  • 24. After being offline
  • 25. New device
  • 26. Future work on ZFS • Feature flags enables development of features seperately • Persistent L2ARC • Larger block support • Dedupe tables on SSD
  • 27. Why use it in DevOps • ZFS performs • ZFS scales – From your laptop to your workstation to your cloud • ZFS replicates – Using zfs send/receive you can deploy copies of your VMs to other hosts – Keep simple backups of your repos and machines • Use snapshots to boot into new versions of your operating system
  • 28. Where to get it? • • • • • • FreeBSD Mac ZFS on Linux Illumos (EraStor OS) NexentaStor (based on OpenSolaris/Illumos) OpenZFS - http://open-zfs.org
  • 29. Links • Open ZFS Project http://open-zfs.org • ARC Paper http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.1 3.5210 • NOSIG http://www.nosig.nl/ • Erastor http://www.erastor.eu