ZFS Storage

pg zfs turns a Linux host into a ZFS storage server: install the packages, inspect disks, create redundancy-aware pools, and manage datasets and snapshots. It talks directly to the kernel — no state is written to pg.yaml, and it does not need a PostgreSQL instance.

The command family is deliberately resource-oriented: pg zfs pool create, pg zfs dataset create, pg zfs snapshot rollback. Anything not wrapped is reachable through pg zfs raw.

Supported systems

pg zfs install detects the distribution from /etc/os-release and installs everything needed, including package repositories:

These are the versions the upstream repositories shipped when the installer was tested; the repositories decide which release you get. pg zfs status always prints the installed userland and kernel-module versions.

Other distributions are detected and given manual instructions rather than installed automatically. Only Linux is supported.

Install

pg zfs install

The command is zero-configuration and idempotent:

1. Verifies privileges (root, or working sudo -n).
2. Enables the required repository (contrib on Debian, universe on Ubuntu, the OpenZFS release repository on Fedora and Rocky).
3. Installs ZFS, kernel headers/module and the ZED daemon.
4. Loads the module and enables the ZFS systemd units for boot.
5. Prints userland and kernel-module versions.

Running it again on a healthy host is a no-op. There are no --version, --backports or --force switches — use the distribution package manager when you need a specific ZFS build, then let pg zfs manage it.

Verify the result at any time:

pg zfs status

OS:            Debian GNU/Linux 12 (bookworm)
Family:        debian
ZFS:           installed
Userland:      zfs-2.1.11-1+deb12u1
Kernel module: 2.1.11-1+deb12u1 (loaded)
Kernel:        6.1.0-54-cloud-amd64
Privilege:     root
Pools:         none
Secure Boot:   disabled

RAID levels and ZFS

ZFS has no standard RAID2/RAID3 levels. The pool types map as follows:

The aliases raid0/raid1/raid2/raid3 are accepted for convenience.

Quick start

# 1. Install ZFS
pg zfs install

# 2. See which disks are usable
pg zfs disk list

# 3. Two whole disks -> mirror automatically
pg zfs pool create tank /dev/vdb /dev/vdc

# 4. Confirm the pool is online
pg zfs pool list

# 5. A dataset for backups
pg zfs dataset create tank/backups

# 6. Snapshot before an upgrade
pg zfs snapshot create tank/backups@before-upgrade

# 7. Roll back if the upgrade went wrong
pg zfs snapshot rollback tank/backups@before-upgrade

The pool is mounted at /tank, the dataset at /tank/backups. Rollback discards every change made after the snapshot and asks for confirmation.

Reusing disks from an old pool

ZFS keeps its vdev partitions and labels on the disks, and export does not remove them, so the safety check refuses such disks. Destroy the old pool and clear the leftovers before creating a new one:

# 1. Import and destroy the old pool if its data is no longer needed
pg zfs pool import fast
pg zfs pool destroy fast --force

# 2. Clear the ZFS labels and the partition table
pg zfs raw zpool labelclear -f /dev/vdd
pg zfs raw zpool labelclear -f /dev/vde
wipefs -a /dev/vdd /dev/vde

# 3. Create the new pool
pg zfs pool create tank /dev/vdd /dev/vde

If labelclear fails on the whole disk, clear the member partitions instead (/dev/vdd1, /dev/vde1). Skip step 1 when the old pool was already destroyed.

Disk safety

pg zfs disk list reports every whole disk with its mountpoints, current ZFS membership and whether pgcli considers it safe to hand to ZFS.

Pool creation refuses:

• the root disk, or a disk with a mounted partition;
• partitions, loop, ROM and CD-ROM devices;
• disks carrying a filesystem (ext4, xfs, swap, LVM, LUKS, mdraid, …);
• disks with an existing ZFS label;
• read-only disks;
• duplicate devices, including duplicates reached through different paths;
• disks already used by another pool.

--force skips only the confirmation prompt. It is never passed to ZFS as -f, and it does not bypass any safety check.

Creating pools

Automatic layouts

Positional devices select the layout:

pg zfs pool create tank /dev/vdb /dev/vdc              # mirror
pg zfs pool create data /dev/vdb /dev/vdc /dev/vdd     # raidz1
pg zfs pool create bulk /dev/vdb /dev/vdc /dev/vdd /dev/vde   # raidz2

Override with --type:

pg zfs pool create fast /dev/vdb /dev/vdc /dev/vdd /dev/vde --type mirror
pg zfs pool create safe /dev/vdb /dev/vdc /dev/vdd /dev/vde /dev/vdf --type raidz3

Use disks of the same capacity (and similar performance) within a vdev. ZFS does not reject mixed sizes, but a vdev’s usable capacity is limited by its smallest member, so the extra space on larger disks is wasted. Different vdevs in one pool may have different capacities.

Multiple vdevs

Use repeatable --data groups together with an explicit --type. Each group is one vdev; the pool stripes across them:

# Two mirror vdevs (striped mirrors)
pg zfs pool create fast --type mirror \
  --data /dev/vdb,/dev/vdc --data /dev/vdd,/dev/vde

# Two raidz2 vdevs
pg zfs pool create bulk --type raidz2 \
  --data /dev/vdb,/dev/vdc,/dev/vdd,/dev/vde \
  --data /dev/vdf,/dev/vdg,/dev/vdh,/dev/vdi

--type applies inside each --data group, not to the pool as a whole: every group becomes one vdev of that layout, and the pool stripes across the vdevs. All groups share the same --type; pg zfs cannot mix, for example, one mirror vdev and one raidz2 vdev in a single create. Use pg zfs raw zpool create for a mixed topology.

Capacity is the sum of the vdev capacities. Redundancy is per vdev: losing more disks than one vdev can tolerate takes down the whole pool.

SLOG, special, cache and spares

pg zfs pool create tank /dev/vdb /dev/vdc \
  --type mirror \
  --special /dev/vdd,/dev/vde \
  --log /dev/vdf,/dev/vdg \
  --cache /dev/vdh \
  --spare /dev/vdi

--special follows --type (a mirror here). --log with two or more devices is a mirror; a single device is a stripe and is warned about. --cache and --spare take independent device lists.

A device may appear only once across data, special, log, cache and spare.

Add a spare to an existing pool with pool add:

pg zfs pool add tank --class spare /dev/vdi

Default properties

Unless --no-defaults is given:

Override pool properties with --prop/-p (maps to zpool -o) and root dataset properties with --fs-prop/-P (maps to zpool -O):

pg zfs pool create tank /dev/vdb /dev/vdc \
  --prop ashift=13 --fs-prop compression=lz4 --fs-prop atime=on

-o is pgcli’s global output flag, which is why the pool/dataset properties use -p/-P. Inside pg zfs raw the native flags work normally.

Managing pools

pg zfs pool list
pg zfs pool status tank
pg zfs pool status --brief          # only pools with problems
pg zfs pool scrub tank
pg zfs pool scrub tank --stop

Growth and repair:

# Add another mirror vdev
pg zfs pool add tank /dev/vdd /dev/vde --type mirror

# Attach /dev/vdd to the mirror holding /dev/vdc (2-way to 3-way)
pg zfs pool attach tank /dev/vdc /dev/vdd

# Shrink it back
pg zfs pool detach tank /dev/vdd

# Replace the failing /dev/vdc with the new /dev/vde
pg zfs pool replace tank /dev/vdc /dev/vde

# Temporarily take a disk out
pg zfs pool offline tank /dev/vdc
pg zfs pool online tank /dev/vdc

attach targets mirror vdevs. Adding disks to a raidz vdev (raidz expansion) requires OpenZFS 2.3 or newer; Debian 12 (2.1) and Ubuntu 24.04 (2.2) cannot do it. Use pg zfs raw zpool attach on a recent release if you need it.

Unmounting and destroying a pool

pg zfs pool export unmounts every dataset and detaches the pool from the host, but keeps all data on the member disks:

pg zfs pool export tank

An exported pool is not imported at boot. Attach it again by listing the importable pools and importing it:

pg zfs pool import                  # list importable pools
pg zfs pool import tank
pg zfs pool import tank --dir /dev/disk/by-id --no-mount

pg zfs pool destroy deletes the pool, every dataset and all data:

pg zfs pool destroy tank

destroy only works on an imported pool. If the pool was exported, import it first:

pg zfs pool import tank
pg zfs pool destroy tank

Without --force both commands ask for confirmation; EOF and the default answer mean “no”. Non-interactive automation must pass --force. --force only skips the prompt — it is never passed to ZFS as -f and does not bypass safety checks. Export is reversible; destroy is not.

Persistence and boot

Pools and datasets survive reboots without any extra steps:

• pg zfs install enables zfs-load-module, zfs-import-cache, zfs-mount and zfs-zed (when the unit exists).
• zpool create records the pool in /etc/zfs/zpool.cache; on boot zfs-import-cache imports it.
• ZFS does not use /etc/fstab. Each dataset’s mountpoint is a ZFS property, and zfs-mount mounts them at boot. A pool’s root dataset mounts at /<pool>, and child datasets mount under it (for example /fast/data).
• The pool state lives on the member disks, so data survives a crash or power loss even if the host did not shut down cleanly.

Verify the setup:

systemctl is-enabled zfs-load-module zfs-import-cache zfs-mount zfs-zed
pg zfs raw zpool get cachefile fast
pg zfs raw zfs get mountpoint fast

After a reboot pg zfs pool list should list the pool and findmnt /fast should show it mounted.

On real hardware, create pools from stable paths such as /dev/disk/by-id/.... /dev/sdX names can change across boots, and a pool whose devices are missing will not import. Run pg zfs pool export tank before moving disks to another host; the destination can then adopt them with pg zfs pool import tank.

Datasets and snapshots

pg zfs dataset create tank/data
pg zfs dataset create tank/a/b/c --parents
pg zfs dataset create tank/logs --prop quota=50G --prop recordsize=128K

pg zfs dataset list
pg zfs dataset list tank/data --props quota,recordsize,compression

pg zfs dataset set tank/data compression=lz4 quota=100G

pg zfs snapshot create tank/data@snap1
pg zfs snapshot create tank/data@weekly --recursive
pg zfs snapshot list tank/data
pg zfs snapshot rollback tank/data@snap1
pg zfs snapshot destroy tank/data@snap1

pg zfs dataset clone tank/data@snap1 tank/clone --prop compression=off
pg zfs dataset destroy tank/clone
pg zfs dataset destroy tank/a --recursive

A dataset with children requires --recursive to destroy. Rollback discards all changes made after the snapshot.

Rollback

pg zfs snapshot rollback tank/data@snap1 returns the dataset to the state captured by the snapshot; every change made after it is discarded. If newer snapshots exist, ZFS refuses the rollback unless --recursive is given, which deletes those newer snapshots. Clones that depend on the snapshot also block the rollback — remove them first, or use pg zfs raw zfs rollback -R (which destroys the dependent clones). Non-interactive runs need --force to skip the confirmation prompt.

# Snapshot before an upgrade
pg zfs snapshot create tank/data@before-upgrade

# Roll back (asks for confirmation)
pg zfs snapshot rollback tank/data@before-upgrade

# Non-interactive automation
pg zfs snapshot rollback tank/data@before-upgrade --force

# Newer snapshots exist: --recursive also destroys them
pg zfs snapshot rollback tank/data@before-upgrade --recursive --force

# Clones block the rollback: -R also destroys dependent clones
pg zfs raw zfs rollback -R tank/data@before-upgrade

Clone

pg zfs dataset clone tank/data@snap1 tank/clone creates a new, writable dataset from the snapshot. Unchanged blocks are shared copy-on-write, so the clone does not duplicate the full data. Pass --prop to set properties on the new dataset:

# Create a writable copy of a snapshot
pg zfs dataset clone tank/data@snap1 tank/clone

# Set properties on the clone
pg zfs dataset clone tank/data@snap1 tank/clone \
  --prop compression=off --prop mountpoint=/tank/clone

# Detach the clone; the original snapshot can then be destroyed
pg zfs raw zfs promote tank/clone
pg zfs snapshot destroy tank/data@snap1
pg zfs dataset destroy tank/clone --force

While a clone depends on a snapshot, the snapshot cannot be destroyed. Promote the clone (pg zfs raw zfs promote tank/clone) to make it independent of the original dataset before removing the old snapshot.

Raw passthrough

pg zfs raw zpool status -v
pg zfs raw zpool set comment="backup pool" tank
pg zfs raw zfs create -o compression=off tank/scratch
pg zfs raw zpool create -f -O compression=gzip temp /dev/vdb

raw bypasses every safety check and confirmation. It is the escape hatch for options pgcli does not wrap — treat it exactly like running zpool/zfs as root. Binary pipelines work too, for example:

pg zfs snapshot create tank/data@send
pg zfs raw zfs send tank/data@send | pg zfs raw zfs receive tank/recv

Permissions

Every command that invokes zpool or zfs first checks privileges:

• root: runs directly;

• non-root with working sudo -n: all ZFS commands run through sudo -n;

• otherwise: fails immediately with

  ZFS management requires root: run "sudo pg zfs ..." or configure passwordless sudo for this user

pgcli never prompts for a sudo password. pg zfs status and pg zfs disk list still show static information without privileges, marking pool details as unavailable.

Command reference

Troubleshooting

pg zfs install cannot find packages. Check that the archive repository is reachable and that contrib/universe is enabled. The installer edits only distribution archive sources (never third-party repositories) and leaves a *.pgcli.bak next to each modified file.

DKMS build fails. Confirm headers for the running kernel are installed (linux-headers-$(uname -r) on Debian/Ubuntu, kernel-devel-$(uname -r) on Fedora/Rocky). The installer reports the build log location; it never reboots or changes the kernel. On very small VMs, add swap before installing.

Secure Boot is enabled. DKMS modules must be signed. pgcli reports the condition but does not enroll keys; sign the module with your MOK key or disable Secure Boot for the test host.

Module is not loaded. modprobe zfs then pg zfs status. The installer enables zfs-load-module, zfs-import-cache, zfs-mount and zfs-zed so pools import automatically on boot.

A disk is rejected because it has partitions or an old label. If it really is free, clear the leftovers:

pg zfs raw zpool labelclear -f /dev/vdb
wipefs -a /dev/vdb

On OpenZFS 2.2 and newer, clearing the whole disk can fail with failed to clear label for /dev/vdb when the label lives in the ZFS member partition. In that case clear the partition instead, then wipe the table:

pg zfs raw zpool labelclear -f /dev/vdb1
wipefs -a /dev/vdb

Never do this to a disk that belongs to an imported pool — pg zfs disk list shows the owning pool, and the safety check refuses such disks.

SELinux (Fedora/Rocky). ZFS mountpoints use standard file contexts. If a service cannot access a dataset, check ausearch/journalctl for AVC denials instead of disabling SELinux globally.

Pools do not come back after reboot. systemctl status zfs-import-cache and zpool import as root. Pools created from /dev/sdX names follow device renames; use stable paths (/dev/disk/by-id/...) when creating pools on real hardware.