ZeroFS

ZeroFS is an S3-backed filesystem that keeps everything inside the bucket: instead of JuiceFS’s external metadata engine, it runs its own LSM tree directly on object storage, and every object is encrypted with a per-volume data key wrapped by your encryption password. Snapshots and consistency are handled by background compaction — there is no database to back up, no Redis to keep alive. You lose neither property than a normal filesystem gives you (it mounts via FUSE through a local 9P hop), and the bucket alone is a complete, encrypted copy of the volume.

Like JuiceFS, ZeroFS is a host-binary addon — no container. pg addon install zerofs downloads the official release binary (sha256-verified; a zerofs already on PATH is reused), writes the volume’s config.toml itself, and runs two host processes:

• the server (zerofs run -c config.toml) — owns the S3 + encryption side and speaks 9P on a loopback port (default 5564). 9P has no authentication, so pgcli hard-rejects a non-loopback --listen;
• the client (zerofs mount 127.0.0.1:<port> <mount-point>) — the FUSE mount you actually use, auto-reconnecting if the server bounces.

Platform: Linux only (host FUSE; macOS refuses, like BeeGFS). The mount is a plain host directory — it does not propagate into PG instance containers.

Everything is explicit

The object store is an EXTERNAL S3 service (cloud, or a MinIO on another machine), named on the command line — pgcli never auto-detects or inherits a same-host addon’s credentials:

pg addon install zerofs --name z1 \
    --s3-endpoint http://minio.example.com:9000 \
    --access-key AK --secret-key SK \
    --bucket zerofs-data

• --s3-endpoint is a bare http(s)://host[:port] (no bucket path, no embedded credentials); pgcli probes it before starting the server, so an unreachable host fails fast instead of hanging the server’s retry loop.
• --access-key/--secret-key are required, always together.
• The bucket is never created by pgcli — provision it yourself with the pg mc wrapper; the install prints the exact two lines.
• ZeroFS signs requests, so --s3-region is honoured (optional — zerofs itself defaults to us-east-1; most MinIO-compatible stores accept the default).

The mount point is yours to choose — by default it lives under pgcli’s addon dir; point it anywhere with --mount-point. Create the directory yourself (mkdir -p) at a path the user running pg has read-write access to — pgcli never creates it:

mkdir -p /mnt/cloudfs
pg addon install zerofs --name z1 \
    --s3-endpoint http://minio.example.com:9000 \
    --access-key AK --secret-key SK \
    --bucket zerofs-data --mount-point /mnt/cloudfs

A missing or unwritable directory fails the install (and every later mount) with the exact mkdir -p remedy before anything is downloaded or started.

Both backends can be pgcli’s own services — created with pgcli, then wired in explicitly as ordinary external URLs (the install never looks them up by itself; it could equally well point at a cloud bucket or a remote Redis):

# MinIO as the object store (a pgcli minio addon — or silo/rustfs/garage/
# seaweedfs; create the bucket by hand, pgcli never does; the root user is
# admin unless --root-user said otherwise, and `pg addon password minio`
# returns the secret key bare — safe to inline):
pg addon install minio --name ms3
pg mc mb ms3/zerofs-data

pg addon install zerofs --name z1 \
    --s3-endpoint http://127.0.0.1:9000 --bucket zerofs-data \
    --access-key admin --secret-key "$(pg addon password minio --name ms3)"

# Redis as the fence backend (--conditional-put redis — for stores lacking
# conditional put; pgcli's MinIO needs none, this shows the wiring). `pg
# addon password redis` returns the generated password bare, paste it into
# the URL, /N picks the logical database:
pg addon install redis --name fence
pg addon password redis --name fence

pg addon install zerofs --name z2 \
    --s3-endpoint http://silo.example.com:9000 \
    --access-key AK --secret-key SK --bucket zerofs-fenced \
    --conditional-put redis --redis-dsn "redis://:PASS@127.0.0.1:6379/0"

These examples run pg on the same machine as the store, so loopback (default 127.0.0.1) is enough. When the zerofs client lives on another host, publish the store on the network and point --s3-endpoint at its address — the endpoint goes into config.toml, so re-render with --force if you change it later:

pg addon install minio --name ms3 --listen 0.0.0.0   # --force if already installed
# from the other host:
pg addon install zerofs --name z1 --s3-endpoint http://10.0.0.5:9000 …

(Redis already binds 0.0.0.0 by default — its password is the gate. Note this is about the S3/redis backends only: zerofs’s OWN 9P --listen must stay loopback — 9P has no authentication.)

The password is the data

⚠️ 🔑 pg addon remove DESTROYS your only copy of the password — save it first. remove keeps the bucket but deletes the config entry, and the password lives only in that entry and its config.toml. Without the original password the kept bucket is permanently unreadable — no recovery path exists. Before any remove you might want to undo later: pg addon password zerofs --name z1 > /somewhere/safe (outside pgcli’s dirs).

The encryption password wraps the bucket’s data-encryption key (DEK). It is generated on first install — retrieve it any time with:

pg addon password zerofs --name z1

Lose it and the bucket’s contents are unrecoverable. This also matters on pg addon remove, which deliberately keeps the bucket: the config entry (and its copy of the password) is gone, so a plain re-install generates a new password that cannot unwrap the old DEK — the server fails to start with Invalid password or corrupted key. Re-attach the kept bucket by passing the original password back in:

pg addon install zerofs --name z1 --encryption-password '<the stored one>'

(To truly start over: empty the bucket, or pick a new --bucket.)

Fencing: conditional put

ZeroFS fences concurrent writers with S3 conditional puts (If-None-Match). --conditional-put selects how:

If a server start fails on a store that lacks conditional-put, the install recognises the log signature and tells you to retry with --conditional-put redis --redis-dsn ….

--redis-dsn is a plain redis:// URL — the DSN is written verbatim into config.toml, and any Redis works, including a pgcli redis addon (see the wiring example above; pg addon password redis returns its generated password bare, /N picks the logical database). TLS endpoints use the same URL shape as everywhere else in pgcli: rediss://user:pass@redis-host:6380/1.

One 9P port per instance: a zerofs server that loses the port bind starts silently and listens on nothing, so install and start refuse an already- answering --port outright — run several volumes by giving each its own port.

Parameters

FUSE (/dev/fuse + fusermount3) is probed, never installed.

Daily operations

pg addon zerofs status --name z1     # server process, mount state, files, fence mode
pg addon zerofs umount --name z1     # detach the FUSE client
pg addon zerofs mount  --name z1     # attach it again (server must run)
pg addon zerofs mount  --name z1 --mount-point /mnt/cloudfs
                                     # custom path (mkdir -p it first); while
                                     # unmounted this relocates the volume too
pg addon zerofs stop   --name z1     # unmount + kill server
pg addon zerofs start  --name z1
pg addon start/stop zerofs --name z1 # top-level aliases (start=server, stop=full)
pg logs addon zerofs --name z1 -f                      # the server log
pg logs addon zerofs --name z1 --service mount -f      # the FUSE client log
pg addon remove zerofs --name z1     # stop + forget; bucket + encrypted data stay
                                     # 🔑 save the password FIRST: pg addon password zerofs --name z1
pg addon remove zerofs --name z1 --clean-data   # also deletes the addon host dir

The config.toml, server/client logs and pid files live under <base>/addon/zerofs/<name>/; the secrets (encryption password + S3 keys) are kept in the 0600 config.toml and — like every pgcli addon credential — in plain view in your pg.yaml.

Relocating the mount point works exactly like JuiceFS: umount, mkdir -p the new directory (writable — same contract as install), then mount with the override; the new path is persisted to pg.yaml, so plain mounts and autostart use it from then on:

pg addon zerofs umount --name z1
mkdir -p /srv/cloudfs
pg addon zerofs mount --name z1 --mount-point /srv/cloudfs
#   [i] mount point moved to /srv/cloudfs (saved)
# ✓ zerofs "z1" mounted at /srv/cloudfs

Attempting the move while still mounted refuses with an “umount first” hint; the FUSE mount point is fully decoupled from pgcli’s bookkeeping (the 9P server keeps running on its port, config.toml/logs/pid stay under <base>/addon/zerofs/<name>/), so the move never touches the volume.

Autostart

pg autostart enable --zerofs --name z1

At boot, pg start --autostart brings up the 9P server (waiting for the port to listen) and then the FUSE client. The backend is external — boot succeeds only when the S3 endpoint is reachable again. Nothing is re-rendered or regenerated: the existing config.toml and stored password are reused verbatim.