This document provides an overview of the features and management utilities of the ZFS filesystem for FreeNAS and PC-BSD operating systems. It describes key ZFS concepts like pools, RAIDZ levels, datasets, snapshots, scrubs, deduplication, and boot environments. It also outlines how to perform tasks like creating pools and datasets, adding disks, taking and restoring snapshots, and scheduling scrubs on both FreeNAS and PC-BSD. Additional resources for learning more about best practices and advanced ZFS topics are also referenced.
In computing, ZFS is a combined file system and logical volume manager designed by Sun Microsystems, a subsidiary of Oracle Corporation. The features of ZFS include support for high storage capacities, integration of the concepts of file system and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z and native NFSv4 ACLs. Unlike traditional file systems, which reside on single devices and thus require a volume manager to use more than one device, ZFS file systems are built on top of virtual storage pools called zpools. A zpool is constructed of virtual devices (vdevs), which are themselves constructed of block devices: files, hard drive partitions, or entire drives, with the last being the recommended usage.[7] Thus, a vdev can be viewed as a group of hard drives. This means a zpool consists of one or more groups of drives.
In addition, pools can have hot spares to compensate for failing disks. In addition, ZFS supports both read and write caching, for which special devices can be used. Solid State Devices can be used for the L2ARC, or Level 2 ARC, speeding up read operations, while NVRAM buffered SLC memory can be boosted with supercapacitors to implement a fast, non-volatile write cache, improving synchronous writes. Finally, when mirroring, block devices can be grouped according to physical chassis, so that the filesystem can continue in the face of the failure of an entire chassis. Storage pool composition is not limited to similar devices but can consist of ad-hoc, heterogeneous collections of devices, which ZFS seamlessly pools together, subsequently doling out space to diverse file systems as needed. Arbitrary storage device types can be added to existing pools to expand their size at any time. The storage capacity of all vdevs is available to all of the file system instances in the zpool. A quota can be set to limit the amount of space a file system instance can occupy, and a reservation can be set to guarantee that space will be available to a file system instance.
In computing, ZFS is a combined file system and logical volume manager designed by Sun Microsystems, a subsidiary of Oracle Corporation. The features of ZFS include support for high storage capacities, integration of the concepts of file system and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z and native NFSv4 ACLs. Unlike traditional file systems, which reside on single devices and thus require a volume manager to use more than one device, ZFS file systems are built on top of virtual storage pools called zpools. A zpool is constructed of virtual devices (vdevs), which are themselves constructed of block devices: files, hard drive partitions, or entire drives, with the last being the recommended usage.[7] Thus, a vdev can be viewed as a group of hard drives. This means a zpool consists of one or more groups of drives.
In addition, pools can have hot spares to compensate for failing disks. In addition, ZFS supports both read and write caching, for which special devices can be used. Solid State Devices can be used for the L2ARC, or Level 2 ARC, speeding up read operations, while NVRAM buffered SLC memory can be boosted with supercapacitors to implement a fast, non-volatile write cache, improving synchronous writes. Finally, when mirroring, block devices can be grouped according to physical chassis, so that the filesystem can continue in the face of the failure of an entire chassis. Storage pool composition is not limited to similar devices but can consist of ad-hoc, heterogeneous collections of devices, which ZFS seamlessly pools together, subsequently doling out space to diverse file systems as needed. Arbitrary storage device types can be added to existing pools to expand their size at any time. The storage capacity of all vdevs is available to all of the file system instances in the zpool. A quota can be set to limit the amount of space a file system instance can occupy, and a reservation can be set to guarantee that space will be available to a file system instance.
An Introduction to the Implementation of ZFS by Kirk McKusickeurobsdcon
Abstract
Much has been documented about how to use ZFS, but little has been written about how it is implemented. This talk pulls back the covers to describe the design and implementation of ZFS. The content of this talk was developed by scouring through blog posts, tracking down unpublished papers, hours of reading through the quarter-million lines of code that implement ZFS, and endless email with the ZFS developers themselves. The result is a concise description of an elegant and powerful system.
Speaker bio
Dr. Marshall Kirk McKusick's work with Unix and BSD development spans over four decades. It begins with his first paper on the implementation of Berkeley Pascal in 1979, goes on to his pioneering work in the eighties on the BSD Fast File System, the BSD virtual memory system, the final release of 4.4BSD-Lite from the UC Berkeley Computer Systems Research Group, and carries on with his work on FreeBSD. A key figure in Unix and BSD development, his experiences chronicle not only the innovative technical achievements but also the interesting personalities and philosophical debates in Unix over the past thirty-five years.
PostgreSQL and ZFS were made for each other. This talk dives downstack into the internals and way that PostgreSQL consumes disk resources and tricks that are available if you run PostgreSQL on ZFS (ZFS on Linux, ZFS on FreeBSD, or ZFS on Illumos). Topics covered will include:
* Performance and sizing considerations
* Workload estimation heuristics
* Standard administrative practices that leverage ZFS
* Recovery using ZFS
* Performing database migrations using ZFS
JetStor NAS 724UXD Dual Controller Active-Active ZFS BasedGene Leyzarovich
The JetStor NAS 724UXD is a unified / hybrid NAS storage system that consolidates NAS and IP-based iSCSI SAN in one chassis. Featuring the newest Intel Haswell platform to lower power consumption and 7x 1Gb Ethernet host ports per controller, all encompassed in a small 4U enclosure. The JetStor NAS 724UXD offers SSD Caching to boost random I/O intensive application, Snapshot, Thin Provisioning, Online Capacity Expansion and Controller-based cable-less design for excellent manageability.
Btrfs and Snapper - The Next Steps from Pure Filesystem Features to Integrati...Gábor Nyers
These are the slides of our SUSECon 2013 presentation with Arvin (the inventor of Snapper)
Btrfs as technology has been getting a lot of attention over the past few years. While interesting for its feature set alone, like checksums, copy on write, snapshots and built-in device management, without proper management tooling and integration with other parts of the operating system, it is difficult for the average user to use Btrfs to its full potential.
This session will help you understand the features of Btrfs and how Snapper can be used for snapshot management in SUSE Linux Enterprise. We also will provide use cases and an outlook for future functionality.
Presentation slides for running MySQL (InnoDB) on ZFS. Since most databases have analogues to optimisation targets mentioned, it is more broadly applicable.
Open Source Backup Conference 2014: Rear, by Ralf DannertNETWAYS
ReaR(Relax and Recover) is delivered as part of the SUSE Linux High Availability Extension.
We show -by way of example- how corporations integrate ReaR during Preparation, Testing and Recovery as buildingblock of their disaster recovery strategy.In the technical part we will highlight the AutoYaST/YaST integration with rear-suse.
We will also investigate some of the adaptations, that had to be done to make ReaR work with upcoming SLES12, that will include systemd and grub2 to be able to automatically recover btrfs subvolumes.
Introduction to Btrfs - FLOSS UK Spring Conference York 2015Richard Melville
With the availability of relatively cheap, large capacity hard disks, the likelihood of running out of disk space is relatively low. However, we may be in the process of re-organising our data storage by increasing the number, or capacity, of our disk array. Or, we may wish to substitute SSDs for our conventional HDDs.
Anybody who has run out of disk capacity will know just how frustrating it is. Equally, the re-organisation of our data is also time consuming and can be fraught with difficulty. If we have a RAID we can, of course, remove and replace drives on the fly as they become full, or even fail. With LVM we can extend, or shrink, file systems on the fly by changing the capacity of the data pool.
Wouldn't it be great, though, if we could combine all these functions, and more besides, into one set of tools? Furthermore, what if we could have these tools working at the file system level, rather than on a virtual layer above the file system?
Well, Sun Microsystems thought so too and developed the ZFS file system for use in its Solaris OS. Sun is now owned by Oracle but ZFS is still very much alive. Unfortunately, for GNU/Linux users, the CDDL licence that came with ZFS was not compatible with the GPL, so inclusion in the Linux kernel was just not possible. ZFS can be built as a module, but this is not very convenient for a file system, hence, the birth of Btrfs.
Btrfs could be seen as a ZFS for GNU/Linux. It is younger, and therefore not as complete as ZFS, but it is getting there. Linus Torvalds claims that it will be the default GNU/Linux file system when it is ready for production use, so it is worth becoming familiar with it now. The Btrfs file system is built into recent kernels, and additional userland tools, btrfs-progs, are provided for manipulating the file system.
This talk will be a basic introduction for those not yet familiar with the Btrfs file system.
An Introduction to the Implementation of ZFS by Kirk McKusickeurobsdcon
Abstract
Much has been documented about how to use ZFS, but little has been written about how it is implemented. This talk pulls back the covers to describe the design and implementation of ZFS. The content of this talk was developed by scouring through blog posts, tracking down unpublished papers, hours of reading through the quarter-million lines of code that implement ZFS, and endless email with the ZFS developers themselves. The result is a concise description of an elegant and powerful system.
Speaker bio
Dr. Marshall Kirk McKusick's work with Unix and BSD development spans over four decades. It begins with his first paper on the implementation of Berkeley Pascal in 1979, goes on to his pioneering work in the eighties on the BSD Fast File System, the BSD virtual memory system, the final release of 4.4BSD-Lite from the UC Berkeley Computer Systems Research Group, and carries on with his work on FreeBSD. A key figure in Unix and BSD development, his experiences chronicle not only the innovative technical achievements but also the interesting personalities and philosophical debates in Unix over the past thirty-five years.
PostgreSQL and ZFS were made for each other. This talk dives downstack into the internals and way that PostgreSQL consumes disk resources and tricks that are available if you run PostgreSQL on ZFS (ZFS on Linux, ZFS on FreeBSD, or ZFS on Illumos). Topics covered will include:
* Performance and sizing considerations
* Workload estimation heuristics
* Standard administrative practices that leverage ZFS
* Recovery using ZFS
* Performing database migrations using ZFS
JetStor NAS 724UXD Dual Controller Active-Active ZFS BasedGene Leyzarovich
The JetStor NAS 724UXD is a unified / hybrid NAS storage system that consolidates NAS and IP-based iSCSI SAN in one chassis. Featuring the newest Intel Haswell platform to lower power consumption and 7x 1Gb Ethernet host ports per controller, all encompassed in a small 4U enclosure. The JetStor NAS 724UXD offers SSD Caching to boost random I/O intensive application, Snapshot, Thin Provisioning, Online Capacity Expansion and Controller-based cable-less design for excellent manageability.
Btrfs and Snapper - The Next Steps from Pure Filesystem Features to Integrati...Gábor Nyers
These are the slides of our SUSECon 2013 presentation with Arvin (the inventor of Snapper)
Btrfs as technology has been getting a lot of attention over the past few years. While interesting for its feature set alone, like checksums, copy on write, snapshots and built-in device management, without proper management tooling and integration with other parts of the operating system, it is difficult for the average user to use Btrfs to its full potential.
This session will help you understand the features of Btrfs and how Snapper can be used for snapshot management in SUSE Linux Enterprise. We also will provide use cases and an outlook for future functionality.
Presentation slides for running MySQL (InnoDB) on ZFS. Since most databases have analogues to optimisation targets mentioned, it is more broadly applicable.
Open Source Backup Conference 2014: Rear, by Ralf DannertNETWAYS
ReaR(Relax and Recover) is delivered as part of the SUSE Linux High Availability Extension.
We show -by way of example- how corporations integrate ReaR during Preparation, Testing and Recovery as buildingblock of their disaster recovery strategy.In the technical part we will highlight the AutoYaST/YaST integration with rear-suse.
We will also investigate some of the adaptations, that had to be done to make ReaR work with upcoming SLES12, that will include systemd and grub2 to be able to automatically recover btrfs subvolumes.
Introduction to Btrfs - FLOSS UK Spring Conference York 2015Richard Melville
With the availability of relatively cheap, large capacity hard disks, the likelihood of running out of disk space is relatively low. However, we may be in the process of re-organising our data storage by increasing the number, or capacity, of our disk array. Or, we may wish to substitute SSDs for our conventional HDDs.
Anybody who has run out of disk capacity will know just how frustrating it is. Equally, the re-organisation of our data is also time consuming and can be fraught with difficulty. If we have a RAID we can, of course, remove and replace drives on the fly as they become full, or even fail. With LVM we can extend, or shrink, file systems on the fly by changing the capacity of the data pool.
Wouldn't it be great, though, if we could combine all these functions, and more besides, into one set of tools? Furthermore, what if we could have these tools working at the file system level, rather than on a virtual layer above the file system?
Well, Sun Microsystems thought so too and developed the ZFS file system for use in its Solaris OS. Sun is now owned by Oracle but ZFS is still very much alive. Unfortunately, for GNU/Linux users, the CDDL licence that came with ZFS was not compatible with the GPL, so inclusion in the Linux kernel was just not possible. ZFS can be built as a module, but this is not very convenient for a file system, hence, the birth of Btrfs.
Btrfs could be seen as a ZFS for GNU/Linux. It is younger, and therefore not as complete as ZFS, but it is getting there. Linus Torvalds claims that it will be the default GNU/Linux file system when it is ready for production use, so it is worth becoming familiar with it now. The Btrfs file system is built into recent kernels, and additional userland tools, btrfs-progs, are provided for manipulating the file system.
This talk will be a basic introduction for those not yet familiar with the Btrfs file system.
OSDC 2016 - Interesting things you can do with ZFS by Allan Jude&Benedict Reu...NETWAYS
ZFS is the next generation filesystem originally developed at Sun Microsystems. Available under the CDDL, it uniquely combines volume manager and filesystem into a powerful storage management solution for Unix systems. Regardless of big or small storage requirements. ZFS offers features, for free, that are usually found only in costly enterprise storage solutions. This talk will introduce ZFS and give an overview of its features like snapshots and rollback, compression, deduplication as well as replication. We will demonstrate how these features can make a difference in the datacenter, giving administrators the power and flexibility to adapt to changing storage requirements.
Real world examples of ZFS being used in production for video streaming, virtualization, archival, and research are shown to illustrate the concepts. The talk is intended for people considering ZFS for their data storage needs and those who are interested in the features ZFS provides.
This presentation will provide the information about the Linux Root File systems and its hierarchy. So any technocrate who is willing to gain info about root files of Linux can easily understand . preffered for Embedded system design Students who are pursuing diploma courses in various CDAC centers.
This document describes how XenServer provides and keeps track of the storage supplied to its guests. The first section
is a reminder of how Linux looks at storage and the second section builds on that to explain XenServer storage. Basic
knowledge of Linux is required, as some standard tools are used.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
2. Outline
Discuss ZFS features and describe the available
management utilities for the following FreeBSD-based
operating systems:
- FreeNAS 9.2.1.7: open source NAS (Network
Attached Storage)
- PC-BSD 10.0.3: open source desktop (GUI) or
server (CLI)
3. History of ZFS
Modern filesystem specifically designed to add
features not available in traditional filesystems
Originally developed at Sun with the intent to
open source
After the Oracle acquisition, open source
development continued and the original engineers
founded OpenZFS (open-zfs.org) which is under
active development
OpenZFS uses feature flags instead of versions
4. What is ZFS?
128-bit COW (Copy on Write) filesystem and
logical volume manager with a maximum pool/file
size of 16 exabytes
In a traditional Unix filesystem, you need to define
the partition size and mount point at filesystem
creation time
In ZFS, you instead feed disks to a “pool” and
create filesystems from the pool as needed
5. Pool
Root (parent) volume which can be logically sub-divided
as needed
The number of disks added at a time is known as
a “vdev”
To optimize performance and resilvering time,
number of disks per vdev is limited
As more capacity is needed, add identical vdevs--
these will be striped into the pool
6. RAIDZ
RAIDZ* levels designed to overcome hardware
RAID limitations such as the write-hole and
corrupt data written over time before the controller
provides an alert
Designed for commodity disks so no RAID
controller is needed
Can also be used with a RAID controller, but it
typically should be put into JBOD mode
7. RAIDZ1
Parity blocks are distributed across all disks
Up to one disk can fail per vdev without losing
pool
Pool can be lost if second disk in a vdev fails
before resilver completes
8. RAIDZ2
Double-parity solution similar to RAID6
Parity blocks are distributed across all disks
Up to two disks can fail per vdev without losing
pool, with no restrictions on which disks can fail
9. RAIDZ3
Triple-parity solution
Parity blocks are distributed across all disks
Up to three disks can fail per vdev without losing
pool, with no restrictions on which disks can fail
12. ZIL
ZFS Intent Log
Effectively a filesystem journal that stores sync
writes until they are committed to the pool
A dedicated SSD as a secondary log device
(SLOG) can increase synchronous write
performance, will have no effect on asynchronous
writes
FreeNAS includes the zilstat CLI utility to help
determine if system would benefit from a SLOG
13. ARC and L2ARC
ARC refers to read cache in RAM. Takes time for
ARC to populate with hits; if high misses continue
for cached reads, the system needs to be tuned.
Freenas adds ARC stats to top(1) and includes
arc_summary.py and arcstat.py tools for ARC
monitoring
Optional, secondary ARC can be installed on SSD
or disk in order to increase random read
performance. Always add as much RAM as
possible first.
17. Datasets
As needed, pool can be divided into additional,
dynamically sized filesystems known as datasets
Permissions and properties such as quotas and
compression can be set on a per-dataset level
A well thought out design can optimize storage for
the type of data being stored
18. Properties
Dozens of configurable properties such as: atime
(access time), canmount, compression, copies,
dedup, exec, quota, userquota, groupquota,
readonly, recordsize, reservation, setuid, etc.
Descriptions can be found at
http://www.freebsd.org/cgi/man.cgi?query=zfs
22. Zvols
Pool can also be divided into zvols
Essentially, a virtual, raw block device
Ideal for iSCSI device extents or for hosting
foreign file systems
Regardless of the filesytem the zvol is formatted
with by the iSCSI initiator, the underlying disk
blocks still benefit from all of the features provided
by ZFS
24. Snapshots
Provide low cost, instantaneous, read-only, point-in-
time image of the specified pool, dataset, or
zvol
Snapshots can be recursive (atomic inclusion of
all child datasets)
Initial size (of local copy) is 0 bytes as COW,
snapshot increases in size as changes are written
to disk
Can be replicated to another system
28. Snapshot Restore
In PC-BSD, the Life Preserver utility provides a
snapshot browser for finding and restoring copies
of earlier versions of files
It can also automate the replication of local
snapshots to another system or to a FreeNAS
system over SSH
A remote snapshot can be used to perform an
operating system restore from a PC-BSD install
media, should the system become unusable
31. Scrubs
ZFS was designed to be self-healing; it creates
and verifies checksums as data is written to disk
A scrub verifies the checksum in each disk block
and attempts to correct data as necessary
I/O intensive, so should be scheduled
appropriately
Reading the scrub results can provide an early
indication of possible disk failure
32. Scrubs
In FreeNAS, a scrub is automatically scheduled to
run every Sunday at midnight whenever a
pool/volume is created (this can be edited)
The results of the last scrub can be viewed from
Volume Status or by typing “zpool status”, and a
scrub can be started now from View Volumes
In PC-BSD, a scrub can be started from Disk
Manager or Life Preserver
35. Deduplication
ZFS property which avoids writing duplicate data
Can improve storage efficiency at the price of
performance—compression is often the better
choice
Dedup tables must fit into L2ARC, rule of thumb
is at least 5 GB RAM/L2ARC per TB of storage to
be deduplicated
36. Boot Environments
A snapshot of the dataset the operating system
resides on is automatically taken before an
operating system or software upgrades
This saved “boot environment” is automatically
added to the GRUB boot manager
Should the upgrade fail, simply reboot and select
the previous boot environment from the boot
menu (essentially go back in time to before the
upgrade occurred)