Addons

• 1: PgBouncer
• 2: Etcd
• 3: PgDog
• 4: PostgREST
• 5: RustFS
• 6: HAProxy
• 7: Silo
• 8: MinIO

pg addon install <addon> [flags]   # install / reconfigure (idempotent)
pg addon list                       # show all installed addons and status
pg addon remove <addon> [flags]     # remove the addon, container, and data

1 - PgBouncer

PgBouncer is a lightweight connection pooler for PostgreSQL. As a pgcli addon it runs as a container in front of one or more PostgreSQL instances, reusing server connections across many short-lived client sessions — transaction-level pooling by default.

PgBouncer supports two deployment modes:

• Local: pg addon install pgbouncer -i <instance> — stored under instances.<name>.addons
• Remote: pg addon install pgbouncer --dsn <dsn> --pg-name <name> — stored in top-level addons.pgbouncer

Platform support

PgBouncer works on Linux (host networking, including cross-host pools) and on macOS for single-host dev/test (the container joins the pgcli-net bridge and publishes its port, the same way a PG instance does under podman machine). On macOS:

• Local mode just works: PgBouncer reaches the managed instance by its container name automatically — no address to configure.
• Remote mode: the --dsn host must be reachable from the Mac — do not point it at 127.0.0.1, which is the Mac itself, not the podman machine VM.
• Client connections still target 127.0.0.1:<port>; gvproxy forwards the published port to the Mac’s loopback, exactly as for a PG instance.

How It Works

1. pg addon install pgbouncer generates configuration files and starts the container
2. Config files live in <base-dir>/addon/pgbouncer/<instance>/
3. The container reaches PostgreSQL over the host network (Linux) or the pgcli-net bridge by container name (macOS)
4. Config file updates restart the container automatically

Namespace isolation: PgBouncer respects the config’s namespace setting. Container names and auth users include the namespace prefix (e.g. pgb_<namespace>_<instance>), so different configs with different namespaces can create independent poolers for the same PostgreSQL instance without conflicting.

# Config with namespace "prod"
pg -c prod-pg.yaml addon install pgbouncer -i mypg
# → container: pgcli-pgbouncer-prod-mypg, auth user: pgb_prod_mypg

# Config with namespace "staging"
pg -c staging-pg.yaml addon install pgbouncer -i mypg
# → container: pgcli-pgbouncer-staging-mypg, auth user: pgb_staging_mypg

Commands

Install

# Local mode: pool a managed instance
pg addon install pgbouncer -i mypg

# Remote mode: pool a remote PG instance
pg addon install pgbouncer \
  --dsn "postgres://admin:pass@10.0.0.20:35432/mypg_db" \
  --pg-name my-remote-pool

# Tune pool parameters
pg addon install pgbouncer -i mypg \
  --max-client-conn 200 \
  --default-pool-size 30 \
  --min-pool-size 5 \
  --reserve-pool-size 10 \
  --max-db-connections 50 \
  --query-timeout 60 \
  --admin-users admin \
  --log-connections 1

Parameters:

List

pg addon list

PgBouncer appears under Local add-ons / Remote add-ons:

Local add-ons:
  pgbouncer (instance: mypg)
    Status:    running
    Host:      127.0.0.1:56432
    Port:      56432
    Pool mode: transaction
    Container: pgcli-pgbouncer-default-mypg

Remote add-ons:
  pgbouncer (pg-name: my-remote-pool)
    Status:    running
    Host:      10.0.0.20:56433
    Port:      56433
    Pool mode: transaction
    Container: pgcli-pgbouncer-default-my-remote-pool

Start and stop

After a reboot or manual stop, bring the pooler back up without re-running install (config and auth users are untouched):

# Local
pg addon start pgbouncer -i mypg
pg addon stop pgbouncer -i mypg

# Remote
pg addon start pgbouncer --pg-name my-remote-pool
pg addon stop pgbouncer --pg-name my-remote-pool

start is idempotent — a running pooler is left alone. stop keeps the container and its config; use pg addon remove to tear it down.

Remove

# Remove local PgBouncer
pg addon remove pgbouncer -i mypg

# Remove remote PgBouncer
pg addon remove pgbouncer --pg-name my-remote-pool

Workflow:

1. Stop and remove the addon container
2. Delete the <base-dir>/addon/pgbouncer/<instance>/ directory and files
3. Remove the addon entry from pg.yaml

Configuration

Local mode (under instances.<name>.addons):

instances:
  mypg:
    addons:
      pgbouncer:
        enabled: true
        max_client_conn: 200
        default_pool_size: 30
        min_pool_size: 5
        reserve_pool_size: 10
        max_db_connections: 50
        query_timeout: 60
        admin_users: admin
        log_connections: 1

Remote mode (under top-level addons):

addons:
  pgbouncer:
    my-remote-pool:
      container_name: pgcli-pgbouncer-default-my-remote-pool
      host_port: 56433
      pool_mode: transaction
      dsn: "postgres://admin:pass@10.0.0.20:35432/mypg_db"
      max_client_conn: 200
      default_pool_size: 30

Generated files, per instance, under <base-dir>/addon/pgbouncer/:

<base-dir>/addon/pgbouncer/
├── mypg/
│   ├── pgbouncer.ini    # PgBouncer main configuration
│   └── userlist.txt     # auth user credentials (auto-regenerated)
└── my-remote-pool/
    ├── pgbouncer.ini
    └── userlist.txt

Authentication

PgBouncer uses the auth_query method for dynamic password lookup:

1. A per-pooler auth user is created on PostgreSQL with a random password, named pgb_<namespace>_<instance> (e.g. pgb_default_mypg, pgb_test-ns_my-remote)
2. A shared SECURITY DEFINER function pgbouncer_lookup() is installed to query pg_authid
3. When a client connects, PgBouncer uses its own auth user to run the auth_query and fetch the real user’s password hash
4. The password is cached in PgBouncer’s memory for subsequent connections

userlist.txt only contains the pooler’s auth user (plaintext password). All other users are authenticated dynamically via auth_query — no password sync needed.

Each pooler gets its own PG auth user, so multiple poolers (local or cross-host) targeting the same PG instance do not conflict.

After changing a PostgreSQL user’s password, re-run pg addon install pgbouncer to reset the auth cache, or connect to the admin console and run RECONNECT.

Connecting

Clients connect through the addon port:

# Direct connection to PostgreSQL
pg exec -i mypg "SELECT version()"

# Connection through PgBouncer
pg exec --dsn "postgres://user:pass@127.0.0.1:56432/mypg_db" "SELECT version()"

Port allocation: PgBouncer defaults to port 56432; if occupied, pgcli assigns the next free port. View the current port with pg addon list.

Use Cases

High Concurrency

pg addon install pgbouncer -i mypg \
  --max-client-conn 1000 \
  --default-pool-size 50 \
  --reserve-pool-size 20 \
  --max-db-connections 100

Short-Lived Connections

pg addon install pgbouncer -i mypg \
  --pool-mode transaction \
  --server-idle-timeout 60 \
  --server-lifetime 600

Long-Lived Connections

pg addon install pgbouncer -i mypg \
  --pool-mode session \
  --server-lifetime 86400

Read Replicas

pg addon install pgbouncer -i mypg-replica \
  --pool-mode transaction \
  --max-db-connections 30 \
  --query-timeout 30

Monitoring

PgBouncer exposes an admin console for pool and runtime status.

Connecting to the Admin Console

Connect to the pgbouncer virtual database as an admin user:

pg exec --dsn "postgres://<admin-user>:<password>@127.0.0.1:<pgbouncer-port>/pgbouncer" "SHOW pools"

Example:

pg exec --dsn "postgres://admin:secret@127.0.0.1:56432/pgbouncer" "SHOW pools"

Note: Only users listed in admin_users can access the admin console.

Common SHOW Commands

Examples

# Check pool status
pg exec --dsn "postgres://admin:secret@127.0.0.1:56432/pgbouncer" "SHOW pools"

# View active client connections
pg exec --dsn "postgres://admin:secret@127.0.0.1:56432/pgbouncer" "SHOW clients"

# View PostgreSQL backend connections
pg exec --dsn "postgres://admin:secret@127.0.0.1:56432/pgbouncer" "SHOW servers"

# Check current configuration
pg exec --dsn "postgres://admin:secret@127.0.0.1:56432/pgbouncer" "SHOW config"

# View traffic statistics
pg exec --dsn "postgres://admin:secret@127.0.0.1:56432/pgbouncer" "SHOW stats"

Other Admin Commands

Troubleshooting

Connection Pool Full

ERROR: no more connections allowed

Cause: Reached max_client_conn limit.

# Increase maximum connections
pg addon install pgbouncer -i mypg --max-client-conn 500

# Or reduce pool size
pg addon install pgbouncer -i mypg --default-pool-size 10

Authentication Failure

FATAL: password authentication failed

Cause: Auth cache contains a stale password hash.

# Re-run install to reset the auth cache
pg addon install pgbouncer -i mypg

Container Won’t Start

# View container logs
podman logs pgcli-pgbouncer-default-mypg

# Check the configuration file
cat <base-dir>/addon/pgbouncer/mypg/pgbouncer.ini

Common causes: config syntax error, malformed user list, or port already in use.

Query Timeout

ERROR: query timeout

Cause: Query exceeded query_timeout.

# Increase the timeout, or disable it
pg addon install pgbouncer -i mypg --query-timeout 300
# or
pg addon install pgbouncer -i mypg --query-timeout 0

Notes

• Port: PgBouncer defaults to 56432; ensure firewall rules allow access
• Passwords: After changing a PostgreSQL user password, re-run pg addon install to reset the auth cache
• Config edits: After editing config files manually, restart to apply:

  podman restart pgcli-pgbouncer-default-mypg

• Transaction mode: transaction mode does not support session-level features (e.g. temporary tables); use session mode instead

2 - Etcd

etcd is a distributed key-value store. pgcli can run one or more etcd members as a standalone, top-level addon — shared infrastructure rather than a per-instance sidecar. This is useful as the DCS (Distributed Concurrent Store) backing a PostgreSQL HA stack, or as a general config/lock service.

Security caveat: pgcli-managed etcd members currently run without TLS/CA certificates and without authentication/RBAC — any client that can reach a client port has full read/write access. Plan a deployment around network isolation (loopback binds by default; keep advertised ports inside a trusted network). CA/TLS and auth/RBAC support is on the roadmap for a later release.

Members are managed one pg addon install etcd at a time: the first member bootstraps a cluster, and later members join the same named cluster on the fly — from the same host or from another machine.

How It Works

• Shared infrastructure: etcd lives in the top-level addons.etcd map in pg.yaml, keyed by member name — not under any single instance.
• Host network, configurable bind: each member runs with --network host. By default its client/peer URLs bind to 127.0.0.1 — etcd ships without authentication, so loopback-only exposure is the intended posture for a single-host cluster. For cross-host clusters each member advertises a reachable address (--advertise-host) while also listening on loopback, so local etcdctl and remote peers both work.
• Dynamic membership: the first member starts with --initial-cluster-state new; each subsequent member is registered with etcdctl member add against a running peer, then started with --initial-cluster-state existing. pgcli does this automatically — same-host via a member container, cross-host via a temporary etcdctl container.
• Cluster identity: members sharing the same --cluster value join the same etcd cluster (it maps to etcd’s --initial-cluster-token). The default is pgcli-etcd.
• Baked-in tuning: every member launches with periodic compaction (--auto-compaction-mode periodic, --auto-compaction-retention 24h) and an 8 GiB backend quota (--quota-backend-bytes 8589934592) — sane defaults for a small HA metadata store.

Deployment Topologies

pgcli supports two cluster shapes. Both use the same install commands — the difference is whether members share a host.

Single host — all members on one machine (testing / dev)

The classic layout for a dev box or CI: every member listens on loopback with a different auto-assigned port. No --advertise-host needed (default 127.0.0.1), no firewall changes, nothing reachable from outside the host.

pg addon install etcd --name m1              # 127.0.0.1:2379/2380
pg addon install etcd --name m2              # 127.0.0.1:2381/2382
pg addon install etcd --name m3              # 127.0.0.1:2383/2384

All three belong to cluster pgcli-etcd; m1 bootstraps, m2/m3 join on the fly. This gives you real 3-node raft semantics but zero host isolation — the whole cluster dies with one machine, which is exactly why it is a testing topology.

Cross host — one member per machine (production HA)

The production shape: spread members across machines (ideally 3 or 5, an odd count — see Topology & Quorum). Each member advertises its host’s LAN address; the first member must bootstrap with --advertise-host so remote peers can dial it. Ports may repeat on every host since each binds its own interface.

# host A (10.0.0.1) — bootstrap
pg addon install etcd --name m1 --cluster prod \
  --advertise-host 10.0.0.1 --client-port 2379 --peer-port 2380

# host B (10.0.0.2) — join
pg addon install etcd --name m2 --cluster prod \
  --advertise-host 10.0.0.2 --client-port 2379 --peer-port 2380 \
  --join http://10.0.0.1:2379

# host C (10.0.0.3) — join
pg addon install etcd --name m3 --cluster prod \
  --advertise-host 10.0.0.3 --client-port 2379 --peer-port 2380 \
  --join http://10.0.0.1:2379

Requirements: every member’s --advertise-host is a mutual-LAN-reachable address, the firewall on each host lets the other members through on every member’s client and peer ports (bidirectional peer traffic — e.g. open 2379-2386/tcp for a 4-member cluster on default ports), and all members share the same --cluster token. Each host survives losing any one member; the cluster only needs a majority of machines.

Commands

Create the first member

pg addon install etcd --name m1

-> Bootstrapping etcd cluster...
  [OK] etcd container started

✓ etcd installed: "m1"
  Container:    pgcli-etcd-m1
  Cluster:      pgcli-etcd
  Data dir:     ~/.pgcli/addon/etcd/m1/data
  Client port:  2379
  Peer port:    2380
  Advertise:    127.0.0.1

  Client URL: http://127.0.0.1:2379
  Connect (etcdctl): ETCDCTL_ENDPOINTS=http://127.0.0.1:2379

The client port starts at 2379 and the peer port takes the next free port (2380), auto-allocated from etcd_start_port.

Add members to the same cluster

With m1 running, installing another member of the same --cluster joins it:

pg addon install etcd --name m2
pg addon install etcd --name m3

pgcli finds a running peer to act as coordinator, registers m2/m3 via etcdctl member add, and starts them with --initial-cluster-state existing. Ports continue to auto-assign without collisions (m2 → 2381/2382, m3 → 2383/2384). A freshly-grown cluster briefly lacks quorum during leader re-election; pgcli retries the membership operations automatically, so you do not need to wait between installs.

Use a different --cluster to keep members in a separate etcd cluster.

Join a member on another machine

Members on different hosts form one cluster too. Each cross-host member must advertise an address the other members can reach, set with --advertise-host. On the machine that already hosts a member, bootstrap as usual, passing its LAN address:

# host A (10.0.0.1)
pg addon install etcd --name m1 --cluster prod \
  --advertise-host 10.0.0.1 --client-port 2379 --peer-port 2380

Then on the new host, point --join at any existing member’s client endpoint. pgcli registers the member through that endpoint (from a temporary etcdctl container — no local member container or image needed; the image is pulled on demand), takes the authoritative ETCD_INITIAL_CLUSTER from the response, and starts the member:

# host B (10.0.0.2)
pg addon install etcd --name m2 --cluster prod \
  --advertise-host 10.0.0.2 --client-port 2379 --peer-port 2380 \
  --join http://10.0.0.1:2379

-> Registering member with the cluster at http://10.0.0.1:2379...
-> Starting etcd container (joining cluster)...
  [OK] etcd container started

Notes for cross-host clusters:

• Every member needs a reachable --advertise-host — including the first. Its advertised peer URL propagates into the cluster’s membership list, so a first member started on the default 127.0.0.1 can never be joined from another host. If you plan a cross-host cluster, pass the LAN address at bootstrap time.
• --cluster is required with --join and must match the remote cluster’s token — membership is checked by that cluster, not by the local config.
• --advertise-host is required with --join; without it the other members could register a peer URL they can’t dial.
• Ports may repeat across hosts (2379/2380 on both) since each host binds its own interfaces.
• A member’s client/peer ports must be reachable between hosts — open your firewall for every member’s ports, in both directions. Peers dial each other’s peer ports bidirectionally, and clients (and --join/pg etcdctl) reach the client ports, so allow both. Auto-assignment means the exact numbers vary per member — read them from the install summary or pg addon list (or pin them with --client-port/--peer-port) and open that range, e.g. 2379-2386/tcp for a 4-member cluster on default ports. Skipping this is the usual cause of a member hanging with etcdserver: no leader or a --join that times out.
• Re-running the same --join install for an already-registered name fails with a clear error; deregister first via pg etcdctl member remove <hex-id> against the cluster.

Inspect with pg etcdctl

pg etcdctl runs etcd’s client from a short-lived container — no need to exec into a member (and it works even on a host with no member installed, pulling the image on demand). The target comes from ETCDCTL_ENDPOINTS, falling back to the first configured member:

export ETCDCTL_ENDPOINTS=http://10.0.0.2:2379
pg etcdctl member list
pg etcdctl endpoint health
pg etcdctl endpoint status -- -w table

etcdctl flags that pg’s own parser would reject (-w table, --hex, …) go after --.

List

pg addon list

etcd members appear under the Infra add-ons (etcd) section:

Infra add-ons (etcd):
  etcd (name: m1)
    Status:      running
    Cluster:     pgcli-etcd
    Client URL:  http://127.0.0.1:2379
    Client port: 2379
    Peer port:   2380
    Image:       quay.io/coreos/etcd:v3.5.30
    Container:   pgcli-etcd-m1

Start and stop

After a host reboot or a manual stop, bring a member back up without re-running install — no member add, the existing config and on-disk data are reused:

pg addon start etcd --name m1
pg addon stop etcd --name m1

start is safe to repeat: an already-running member is a no-op, and a member whose container was removed is recreated from its data with --initial-cluster-state existing. For a cross-host cluster, confirm the member’s peers are reachable (quorum intact) before starting it.

Remove a member

pg addon remove etcd --name m2

Removal first deregisters the member from the cluster (resolving its hex member ID — etcd v3.5 member remove takes an ID, not a name), then deletes the container and the member’s data directory. Remaining members stay healthy as long as quorum holds.

Cross-host members must be deregistered separately. The automatic deregistration above only works when another running member of the same cluster is present in the local pg.yaml — pgcli has no view of members that live on other hosts. So on a cross-host cluster, removing a member from the host that runs it deletes the container and data but leaves its entry in the cluster’s membership list (a “stale” member) — the surviving members keep trying to peer with the now-deleted host.

Remove it in two steps instead:

# 1. on the host that runs the member — stop and clean up local state
pg addon remove etcd --name m4

# 2. from any surviving host's member, deregister it from the cluster
export ETCDCTL_ENDPOINTS=http://10.0.0.1:2379   # a surviving member's client URL
pg etcdctl member list                            # find m4's hex ID
pg etcdctl member remove <hex-id>                 # e.g. 5c7048c8f7521ec7

pg etcdctl needs a reachable peer to talk to — point ETCDCTL_ENDPOINTS at a member of that cluster that is still running (on a host that still has a running member, this falls back automatically; otherwise set it explicitly), then remove by ID (see Inspect with pg etcdctl). Verify with pg etcdctl member list afterward: the removed name should be gone.

Parameters

Container name follows the namespace convention: pgcli-etcd-<namespace>-<name> (namespace omitted when unset).

Data Directory Layout

Data is always laid out as <root>/<name>/data, so multiple members on one host never share a directory:

<base_dir>/addon/etcd/
├── m1/data/     # member m1
├── m2/data/     # member m2
└── m3/data/     # member m3

--data-dir sets the root. A relative value resolves against the config’s base_dir; the resolved root is persisted, so the config stays portable:

# base_dir: /data/pgcli  →  root /data/pgcli/etcd, member d1 → /data/pgcli/etcd/d1/data
pg addon install etcd --name d1 --data-dir ./etcd

# absolute root
pg addon install etcd --name d2 --data-dir /mnt/ssd/etcd

pg addon remove deletes only that member’s <name>/data dir and leaves the shared root in place.

Connecting

Use pg etcdctl, which runs etcd’s v3 client from a short-lived container against a member’s client URL — the target comes from ETCDCTL_ENDPOINTS, falling back to the first configured member. etcdctl’s native flags go after --:

pg etcdctl member list
pg etcdctl endpoint status -- -w table
pg etcdctl endpoint health
pg etcdctl put foo bar
pg etcdctl get foo -- --hex

pg etcdctl always speaks the v3 API (ETCDCTL_API=3, etcdctl’s default); v2 is not supported.

Topology & Quorum

etcd requires a quorum (majority) of members to accept writes:

• Use an odd number of members — 3 or 5 for production HA.
• Removing members can drop the cluster below quorum (e.g. 3 → 1 surviving); it will then be unable to commit writes. Keep a majority running.

Configuration

After install, pg.yaml records each member under the top-level addons.etcd:

addons:
  etcd:
    m1:
      container_name: pgcli-etcd-m1
      name: m1
      cluster_name: pgcli-etcd
      image_tag: quay.io/coreos/etcd:v3.5.30
      data_dir: /home/user/.pgcli/addon/etcd
      client_port: 2379
      peer_port: 2380
      autostart: true
    m2:
      container_name: pgcli-etcd-m2
      name: m2
      cluster_name: pgcli-etcd
      client_port: 2381
      peer_port: 2382

Port base is configurable via top-level etcd_start_port (default 2379). A cross-host member records the address it advertises:

addons:
  etcd:
    m1:
      name: m1
      cluster_name: prod
      advertise_host: 10.0.0.1
      client_port: 2379
      peer_port: 2380

Auto-start on Boot

Containers also carry a --restart unless-stopped policy, which a per-container conmon monitor enforces even without a podman daemon — it covers crashes but not host reboots. To bring members up after a reboot, enable autostart per member:

pg autostart enable --etcd --name m1
pg autostart enable --etcd --name m2
pg autostart enable --etcd --name m3

This sets autostart: true on the member in pg.yaml and installs/refreshes the boot service (see Auto-start on Boot). Boot is start-only: it starts the member’s existing container and never re-runs member add — an initialized member reloads its cluster from disk and rejoins. On a cross-host cluster, run the command on each host for that host’s member(s). pg autostart status lists every member’s state.

Notes

• Single-node vs cluster: --name m1 alone gives a one-member cluster; install more members with the same --cluster to grow it.
• Production-ready defaults: every member already launches with periodic compaction and an 8 GiB backend quota (see How It Works), suitable for a production DCS without further tuning.
• Reinstall is idempotent: re-running pg addon install etcd --name <m> on a running member recreates its container with updated flags; a member already registered in the cluster is not re-added.

3 - PgDog

PgDog is a high-performance Postgres proxy written in Rust (the successor to PgCat). It provides connection pooling, read/write load balancing across replicas, and horizontal sharding — in front of one or more PostgreSQL backends. pgcli can run PgDog as a standalone, top-level addon: shared infrastructure rather than a per-instance sidecar.

Security caveat: PgDog authenticates clients against plaintext passwords stored in users.toml. pgcli writes that file mode 0600 and mounts it read-only into the container, but the passwords still sit in clear text on disk and in pg.yaml. Keep the listen port on loopback (the default) or inside a trusted network, and treat the config file as a secret.

PgDog is configured with two TOML files — pgdog.toml (the proxy, its backends, and sharding rules) and users.toml (the client credentials). pgcli generates both entirely from the install flags: there is no config file to hand-edit, so pg addon install pgdog is the source of truth and is idempotent — re-running it re-renders the files from the flags given.

Platform support

PgDog runs on Linux (host networking) and on macOS for single-host dev/test, joining the same pgcli-net bridge a PG instance uses under podman machine and publishing its client + openmetrics ports. On macOS:

• The listen bind is widened to 0.0.0.0 inside the container so the published port is reachable (a loopback-only bind can’t be forwarded to); the client address you connect to is still 127.0.0.1:<port>.
• Backends must be reachable from the bridge. PgDog has no automatic name-resolution for --backend, so a 127.0.0.1 backend would point at the Mac, not the podman machine VM. Give each --backend either the target instance’s container name (run pg status -i <instance> and read the Container: line, e.g. app=pgcli-pg-mypg:5432:...) or an address the Mac can route to — never 127.0.0.1 for a managed instance.

How It Works

• Shared infrastructure: PgDog lives in the top-level addons.pgdog map in pg.yaml, keyed by proxy name — not under any single instance.
• Host network (Linux) / bridge (macOS): the proxy listens on its client port (auto-assigned from pgdog_start_port, default 7432) plus a Prometheus-style openmetrics port on the next free number. On Linux it runs with --network host; on macOS it joins pgcli-net and publishes both ports.
• Generated config: pgdog.toml + users.toml are written under <base-dir>/addon/pgdog/<name>/ and bind-mounted into the container at /pgdog.
• Image pinned: ghcr.io/pgdogdev/pgdog:v0.1.57 by default; override with --image.

Install

A minimal single-backend proxy (pooling only — see the note below):

pg addon install pgdog \
  --backend "app=127.0.0.1:35432:default_db" \
  --user "appuser:secret:app" \
  --sharded-table "app:users:id:bigint"

Declare a --sharded-table so the example is a complete, self-consistent config (it mirrors the sharding example below). Be aware that with a single --backend there is nothing to shard across — every row lands on that one backend regardless of the declaration, so this minimal example only demonstrates connection pooling. For real sharding you need two or more --backends with distinct shard numbers (see Sharding and replicas).

• --backend NAME=HOST:PORT:DBNAME[:SHARD[:ROLE]] — one [[databases]] entry. NAME is the logical database clients connect to; DBNAME is the real database on the backend. SHARD defaults to 0; ROLE to PgDog’s default (primary, or set replica for read routing). Repeatable.
• --user NAME:PASSWORD[:DBNAME] — one [[users]] entry. The DBNAME is the logical database (a --backend name) the user may reach; it defaults to the first backend’s name. Repeatable.
• At least one --backend and one --user are required.

Other flags:

Sharding and replicas

Repeat --backend across shards and roles, and declare the shard key with --sharded-table DBNAME:TABLE:COLUMN:DATA_TYPE:

pg addon install pgdog \
  --backend "app=10.0.0.1:5432:shard0:0" \
  --backend "app=10.0.0.2:5432:shard1:1" \
  --backend "app=10.0.0.1:5433:shard0:0:replica" \
  --backend "app=10.0.0.2:5433:shard1:1:replica" \
  --user "appuser:secret:app" \
  --sharded-table "app:users:id:bigint"

This puts two shards (each with a primary + replica) behind the logical database app and routes the users table by its id column.

Provisioning the backends

PgDog routes to databases, tables, and roles that must already exist on the backends — installing the proxy does not create them. For a local pgcli instance, use pg exec to create each shard database, its sharded table, and the login role. With the default instance on a single host (shards as separate databases on one port), the example below provisions the backends (steps 1–4), installs the proxy (step 5), and writes through it to show the routing (steps 6–7):

# 1. create the shard databases
pg exec -i default -- psql -U admin -d default_db -c "CREATE DATABASE shard0;"
pg exec -i default -- psql -U admin -d default_db -c "CREATE DATABASE shard1;"

# 2. create the sharded table (same schema) in each shard
for db in shard0 shard1; do
  pg exec -i default -- psql -U admin -d "$db" \
    -c "CREATE TABLE users (id bigint PRIMARY KEY, name text, email text);"
done

# 3. create the login role PgDog uses to reach the backends — its name must
#    match a --user, and its password must match that user's users.toml entry
pg exec -i default -- psql -U admin -d default_db \
  -c "CREATE ROLE appuser LOGIN PASSWORD 'secret';"

# 4. grant that role access to the sharded table in every shard
for db in shard0 shard1; do
  pg exec -i default -- psql -U admin -d "$db" \
    -c "GRANT SELECT, INSERT, UPDATE, DELETE ON TABLE users TO appuser;"
done

# 5. install the proxy over both shards (see "Sharding and replicas" above)
pg addon install pgdog \
  --backend "app=127.0.0.1:35432:shard0:0" \
  --backend "app=127.0.0.1:35432:shard1:1" \
  --user "appuser:secret:app" \
  --sharded-table "app:users:id:bigint"

# 6. write rows through the proxy with pgcli (pg exec --dsn), ONE row per
#    statement so PgDog routes each by its id (a multi-row INSERT is broadcast
#    to every shard instead)
for id in 1 2 3 4 5 6; do
  pg exec --dsn "postgres://appuser:secret@127.0.0.1:7432/app" \
    "INSERT INTO users (id, name, email) VALUES ($id, 'n$id', 'e$id@x');"
done

# 7. read each shard directly to confirm the rows landed where PgDog routed
#    them — pg psql -- -d <db> targets one database
pg psql -i default -- -d shard0 -tAc "SELECT id, name FROM users ORDER BY id;"
pg psql -i default -- -d shard1 -tAc "SELECT id, name FROM users ORDER BY id;"

pg exec -i default -- psql -d <db> runs a command against a specific database in the named instance’s container. The id bigint column matches the --sharded-table "app:users:id:bigint" declaration above, so PgDog can route users rows by id. Steps 6–7 make that routing visible: the two shard queries return different, non-overlapping subsets of the ids you inserted (the exact split depends on PgDog’s hash), and together they account for every row — no row lands in both shards. For remote backends, run the equivalent SQL against each host directly.

The backend role is not optional. By default PgDog authenticates to the backends using the client’s --user name and its users.toml password. So every backend must have a login role named appuser (etc.), or connections fail with password for user "..." is wrong, or the database does not exist, even on a loopback trust setup where the password is ignored but the role must still exist. Under password auth (the default for non-loopback scram-sha-256), the users.toml password must match the role’s.

Decoupling client and backend users. PgDog itself supports connecting to the backends with a different user than the client uses: a [[users]] entry can carry server_user / server_password, and a [[databases]] entry can carry user / password (which take priority). pgcli’s install flags do not currently emit these fields, so through pg addon install pgdog the backend role name is always the --user name. If you need a shared backend role (a common postgres superuser, or a name that differs from every client user), hand-edit the generated users.toml/pgdog.toml and restart the container — but note a re-install regenerates both files from the flags and will discard those edits.

Connecting

The install summary prints the endpoint. Clients connect through the proxy’s client port, using the logical database name and a --user credential — with pgcli, that’s a --dsn pointing at the proxy:

# interactive psql through the proxy
pg psql --dsn "postgres://appuser:secret@127.0.0.1:7432/app"

# one-shot SQL
pg exec --dsn "postgres://appuser:secret@127.0.0.1:7432/app" "SELECT version();"

Prometheus metrics are served on the openmetrics port:

curl -s http://127.0.0.1:7433/metrics

Writing to shards

How a write lands on the shards depends on the form of the statement. Observed with PgDog v0.1.57:

Single-row INSERT — routed by the shard key. One VALUES tuple per statement; PgDog hashes the id and sends that row to exactly one shard:

pg exec --dsn "postgres://appuser:secret@127.0.0.1:7432/app" \
  "INSERT INTO users (id, name, email) VALUES (4, 'n4', 'e4@x');"

Inserting ids 1–6 as six single-row statements split them across the two shards (in one run: shard0 got 1,2, shard1 got 3,4,5,6 — the exact split is PgDog’s hash, not something to rely on). No row lands in both shards.

Multi-row INSERT — broadcast, not split. A single statement carrying several VALUES tuples is sent verbatim to every shard, so each shard ends up with every row:

pg exec --dsn "postgres://appuser:secret@127.0.0.1:7432/app" \
  "INSERT INTO users (id, name, email) VALUES (1,'a','a@x'),(2,'b','b@x'),(3,'c','c@x'),(4,'d','d@x');"

This reported INSERT 0 8 — 4 rows in each shard. The per-shard PRIMARY KEY still holds (the duplicate ids live in different shards, never colliding), and there is no error or warning: every shard silently grows a full copy of the batch. Reads through the proxy will then see each id multiple times.

Bulk-loading correctly: loop one statement per row through the proxy (each routed individually), or skip the proxy and run the load against each shard directly (COPY via pg exec -i <inst> -- psql -d <shard>).

Configuration

After install, pg.yaml records each proxy under the top-level addons.pgdog:

addons:
  pgdog:
    pgdog:
      container_name: pgcli-pgdog-pgdog
      name: pgdog
      image_tag: ghcr.io/pgdogdev/pgdog:v0.1.57
      host: 127.0.0.1
      host_port: 7432
      openmetrics_port: 7433
      pooler_mode: transaction
      workers: 2
      default_pool_size: 10
      backends:
        - name: app
          host: 127.0.0.1
          port: 35432
          database_name: default_db
          shard: 0
      users:
        - name: appuser
          password: secret
          database: app
      autostart: false

Port base is configurable via the top-level pgdog_start_port (default 7432); the openmetrics port is always allocated just above it.

List

pg addon list

Every PgDog proxy appears under the Infra add-ons (pgdog) section with its runtime status, ports, backend/user counts, image, and container name:

Infra add-ons (pgdog):
  pgdog (name: pgdog)
    Status:      running
    Listen:      127.0.0.1:7432
    Client port: 7432
    Metrics:     http://127.0.0.1:7433/metrics
    Pool mode:   transaction
    Backends:    2
    Users:       1
    Image:       ghcr.io/pgdogdev/pgdog:v0.1.57
    Container:   pgcli-pgdog-pgdog

Backends / Users are the counts from pgdog.toml / users.toml; Status reflects the live container (a stopped proxy shows stopped).

Start and stop

After a reboot or a manual stop, bring the proxy up without re-running install (config and users are untouched):

pg addon start pgdog            # default name "pgdog"
pg addon start pgdog --name proxy
pg addon stop pgdog --name proxy

start is idempotent — a running proxy is left alone; a proxy whose container was removed is recreated from its config directory.

Auto-start on Boot

Containers carry a --restart unless-stopped policy (crashes, not reboots). To bring a proxy up after a host reboot:

pg autostart enable --pgdog --name pgdog

This sets autostart: true and installs/refreshes the boot service (see Auto-start on Boot). Boot is start-only: it starts the existing container reading the pgdog.toml/users.toml already on disk, and never re-generates config or auth. Install the proxy first. pg autostart status lists every target’s state.

Logs

pg logs addon pgdog --name pgdog       # last 50 lines
pg logs addon pgdog --name pgdog -f    # follow

PgDog logs connection-pool events (new server connections, auth, client connect/disconnect) as structured INFO lines. A repeatedly failing pool with password for user "..." is wrong, or the database does not exist means step 3 or 4 above was skipped — the backend has no such login role, or its password does not match users.toml.

Notes

• Reinstall is idempotent: re-running pg addon install pgdog --name <p> re-renders the config from the given flags and recreates the container. The backend/user/shard lists are fully replaced by this command’s flags — an omitted --backend is removed, not kept.
• Plaintext credentials: passwords live in users.toml and pg.yaml. Prefer a dedicated low-privilege role for proxy users and restrict network access to the client port.
• Write forms differ: single-row INSERT is routed by the shard key; a multi-row INSERT VALUES is broadcast to every shard. See Writing to shards.
• Not a per-instance sidecar: unlike PgBouncer, PgDog frontends backends by address in --backend — it does not have to point at a pgcli-managed instance, and one proxy can span many databases/shards.

4 - PostgREST

PostgREST is a single-process, stateless web server that turns a PostgreSQL schema into a RESTful API. As a pgcli addon it runs as one container in front of any PG endpoint — no data directory, no rendered config file, everything configured through PGRST_* environment variables.

PostgREST is a proxy-type addon like PgBouncer and PgDog: it holds no state of its own. Two deployment modes mirror PgBouncer’s:

• Local: pg addon install postgrest -i <instance> — stored as a sidecar under instances.<name>.addons.postgrest, DSN built from the instance.
• Remote: pg addon install postgrest --dsn <dsn> --pg-name <name> — stored in top-level addons.postgrest.

The --dsn in either mode is passed verbatim into the container’s PGRST_DB_URI, so it works against any PG endpoint: a direct managed instance, a PgBouncer pool, or a Patroni cluster behind its HAProxy listener.

Platform support

PostgREST works on Linux (host networking, --network host) and macOS (container joins the pgcli-net bridge and publishes its port, the same shape as a PG instance under podman machine).

• Local mode on macOS just works — the container reaches the managed instance over the bridge.
• Remote mode on macOS: --dsn must be reachable from the Mac itself, not from the podman machine VM. Point it at an address the Mac can resolve (a real host IP, or host.containers.internal for a service on the VM).
• Clients always reach the REST API at 127.0.0.1:<port> (or --listen); gvproxy forwards the published port to the Mac’s loopback.

How it works

1. pg addon install postgrest pulls the image (if missing) and starts a container wired entirely from PGRST_* env vars.
2. The container reaches the backend PG over the host network (Linux) or the pgcli-net bridge (macOS).
3. PostgREST introspects the exposed schema on startup, serves REST requests, and listens on the pgrst channel for a schema-cache reload signal.

PostgREST holds no data on the host — pg addon remove postgrest only stops and removes the container.

Container name: pgcli-postgrest<ns>-<name> (local mode uses the instance name as <name>; remote mode uses --pg-name). Namespace isolation applies: two configs with different namespaces can run independent APIs without colliding.

Commands

Install

# Local mode: expose a managed instance's schema
pg addon install postgrest -i mypg --schema api --anon-role web_anon

# Remote mode: any PG endpoint (direct instance, pgbouncer, or haproxy LB)
pg addon install postgrest \
  --dsn "postgres://api:pass@127.0.0.1:5000/appdb" \
  --pg-name app-api --schema api --anon-role web_anon

# Tune the connection pool PostgREST keeps toward its backend
pg addon install postgrest -i mypg --db-pool 20

# Enable JWT auth (unauthenticated requests still fall back to --anon-role)
pg addon install postgrest -i mypg --schema api --anon-role web_anon --jwt-secret "$(openssl rand -hex 32)"

Re-running install is idempotent: an existing container is reused (a stopped one is started). Pass --force to recreate it after changing ports, listen address, DSN, db-pool, schema, anon-role, or jwt-secret.

Parameters:

--db-pool and the backend’s max_connections. Total PG connections a PostgREST deployment opens is roughly number of PostgREST instances × --db-pool. If you run several replicas behind the same endpoint, budget max_connections accordingly.

List

pg addon list

PostgREST appears under Local add-ons / Remote add-ons with Status, REST URL, Backend, Schema, DB pool, Anon role, JWT auth, and Container.

Start / stop / remove / logs

pg addon start postgrest -i mypg
pg addon stop postgrest --pg-name app-api
pg addon remove postgrest -i mypg          # stateless: only the container goes away
pg addon remove postgrest --pg-name app-api
pg logs addon postgrest -i mypg            # local container logs
pg logs addon postgrest --pg-name app-api  # remote container logs

Autostart

pg autostart enable --postgrest -i mypg
pg autostart enable --postgrest --pg-name app-api
pg autostart status

Autostart is start-only: at boot the container is brought up reading the PGRST_* env already in the config (recreated from config if it was removed). Install the addon first.

Configuration

PostgREST settings live under addons.postgrest (remote) or as an instances.<name>.addons.postgrest sidecar (local):

postgrest_start_port: 3500      # base of the HTTP port pool

addons:
  postgrest:
    app-api:
      container_name: pgcli-postgrest-default-app-api
      name: app-api
      image_tag: docker.io/postgrest/postgrest:v16.3
      host_port: 3501
      listen: 127.0.0.1
      dsn: postgres://api:pass@10.0.0.20:35432/appdb
      backend_host: 10.0.0.20:35432
      db_pool: 4
      schemas: api
      anon_role: web_anon
      jwt_secret: <HS256-shared-secret>   # only present when --jwt-secret was given
      autostart: false

Ports are auto-assigned from the postgrest_start_port pool (base 3500) when --port is omitted; the pool is independent of the PgBouncer / etcd / pgdog / minio pools.

Recommended: front a Patroni cluster with HAProxy

For a Patroni cluster the recommended shape is PostgREST → HAProxy rw listener → cluster leader, not PostgREST pointed straight at a member. Pass the listener as --dsn (remote mode):

# Recommended: writes always follow the leader through the LB's rw listener
pg addon install postgrest --dsn "postgres://api:pass@<lb-host>:5000/appdb" \
  --pg-name app-api --schema api --anon-role web_anon

Why the listener, not a member’s direct port: a pinned member loses writes after a failover (the old leader stops accepting them, but the DSN keeps pointing there). pgcli detects a member’s direct port at install time and warns, suggesting the HAProxy listener instead.

• Failover self-heal, no restart. When the leader changes, PostgREST just reconnects through the LB and re-selects the new leader — service resumes without restarting the container or re-running install. Verified: after a pg ha switchover, the one in-flight write that hit the demoted connection failed 503 (SQLSTATE 57P01, connection terminated), and the very next retried request succeeded 201 against the new leader. Treat that single transient 503/57P01 during a leader change as expected and retry it.
• HAProxy routes by health check, decoupled from who is leader. The rw listener only admits the member passing the /primary check and the ro listener only the replicas passing /replica, so a failover needs no haproxy reconfiguration — the traffic moves on its own.
• Scale-out. Run several PostgREST replicas behind a load balancer; each adds its own --db-pool worth of backend connections.

Database-side setup (not managed by pgcli)

pgcli installs and runs the PostgREST container; it does not touch your database. Before the API serves anything, the database needs the unauthenticated role PostgREST SET ROLEs to, plus grants on the exposed schema — usually owned by your migrations, not pgcli. This block is re-runnable:

-- The exposed schema (--schema value; skip for public, which already exists):
CREATE SCHEMA IF NOT EXISTS api;

-- The NOINHERIT role unauthenticated requests run as (the --anon-role value).
DO $$ BEGIN
  IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'web_anon') THEN
    CREATE ROLE web_anon NOINHERIT NOLOGIN;
  END IF;
END $$;
GRANT USAGE ON SCHEMA api TO web_anon;
GRANT SELECT ON ALL TABLES IN SCHEMA api TO web_anon;
ALTER DEFAULT PRIVILEGES IN SCHEMA api GRANT SELECT ON TABLES TO web_anon;

-- SET ROLE requires the login user to be a member of the target role
-- (skip if the DSN user is a superuser):
GRANT web_anon TO <dsn-user>;

The role view’s column is rolname, not rolename — a typo fails the whole batch. ALTER DEFAULT PRIVILEGES covers only tables created after it runs; existing ones are handled by the GRANT ... ON ALL TABLES line.

PostgREST connects as the DSN user and SET ROLEs to web_anon per request, so the API carries exactly that role’s privileges: web_anon sees SELECT-only until you GRANT INSERT/UPDATE/DELETE on the tables you want writable. The connecting login role itself needs nothing beyond membership of the roles requests will assume. Then install with --anon-role web_anon. The two ways to authenticate a request — anonymous (--anon-role) and JWT (--jwt-secret) — are covered next; without either, every request is refused with 401.

PostgREST caches the schema it introspected. After a schema change, reload the cache:

NOTIFY pgrst, 'reload schema';

LISTEN and transaction pooling. PostgREST relies on a persistent LISTEN pgrst session to receive reload notifications. Behind a pooler in transaction pooling mode (PgBouncer’s default) that session-level LISTEN is broken — the notification is never delivered, and neither is NOTIFY pgrst reaching PostgREST via a pooled connection. Verified behaviour: a reload sent through a transaction-pooled PgBouncer does not refresh PostgREST’s cache (a fresh table stays 404), and even a NOTIFY sent directly to the backend is lost because PostgREST’s own listener has no stable connection. Either point PostgREST at a session-pooled or direct connection, or restart the PostgREST container (it re-introspects on boot) to pick up schema changes.

Verifying the API

The install prints the REST URL (http://127.0.0.1:<port>). Create a table in the exposed schema, then hit the API to confirm the server is live and serves its rows:

CREATE TABLE api.widgets (id integer PRIMARY KEY, name text);
INSERT INTO api.widgets VALUES (1, 'bolt'), (2, 'nut');
GRANT SELECT ON api.widgets TO web_anon;
NOTIFY pgrst, 'reload schema';   -- otherwise the new table stays 404

# The OpenAPI root — answers as soon as PostgREST has connected to the backend.
curl -s http://127.0.0.1:3500/ | head -c 120

# Which tables/relations are exposed (from the OpenAPI paths):
curl -s http://127.0.0.1:3500/ | grep -o '"/[a-z_]*"'

# Read rows from the table. The schema is implicit in the path — it is
# /widgets, NOT /api.widgets:
curl -s "http://127.0.0.1:3500/widgets"
curl -s "http://127.0.0.1:3500/widgets?id=eq.1"   # a filter

A few things to expect on a fresh install:

• The root can take a moment to answer — schema introspection runs after boot, so the first requests may 503 until PostgREST has connected.
• A 404 on a just-created table or a 401 on unauthenticated requests means the schema cache is stale or --anon-role is unset — see the Troubleshooting table below for the fix.

JWT authentication

--anon-role is the simplest model: every unauthenticated request runs as one fixed role. --jwt-secret turns on per-request identity. Pass it at install time (a HS256 shared secret, or a JSON Web Key for RS256); pgcli passes it to the container as PGRST_JWT_SECRET.

pg addon install postgrest -i mypg --schema api --anon-role web_anon \
  --jwt-secret "$(openssl rand -hex 32)"

The secret is not written to your shell history if you inline a command substitution as shown. It does land in pg.yaml (so pgcli can recreate the container) — treat that file as secret-bearing, and re-run install with --force after changing it.

Length: for HS256 the secret must be at least 32 characters (256-bit key material; 48 for HS384, 64 for HS512). PostgREST refuses to start with a shorter one — openssl rand -hex 32 (64 chars) is a safe default. A JWK (for RS256/ECDSA) has no such minimum; the key strength comes from the JWK itself.

With a secret set, PostgREST verifies any request that carries Authorization: Bearer <token> and runs it as the role named in the token’s role claim:

• The token must be signed with the same secret; a tampered one is rejected 401.
• The role claim must name a database role with grants on the exposed schema (create it like web_anon above, NOINHERIT).
• The login role in the DSN must be a member of that role, because serving the request means SET ROLE to it. So for a JWT role like web_user you also need GRANT web_user TO <dsn-user>; — same requirement as the anon role above, and likewise skippable when the DSN user is a superuser. Without the membership, requests fail with permission denied to set role.
• Change the claim key from role via PGRST_JWT_ROLE_CLAIM_KEY if your issuer uses another field — pgcli does not surface that flag; set it on the container directly if needed.

--anon-role and --jwt-secret compose: requests with a valid JWT run as their role claim, requests without one fall back to --anon-role. With neither flag, every request is refused (401). A typical progression is anon for public reads plus JWT roles for authenticated writes.

In production your application’s auth service signs the tokens. To hand-craft a test token with the same HS256 secret:

b64url() { openssl base64 -A | tr '+/' '-_' | tr -d '='; }
SECRET='<the --jwt-secret value>'
HEADER=$(printf '{"alg":"HS256","typ":"JWT"}' | b64url)
PAYLOAD=$(printf '{"role":"web_anon","exp":%d}' $(( $(date +%s) + 3600 )) | b64url)
SIG=$(printf '%s.%s' "$HEADER" "$PAYLOAD" | openssl dgst -sha256 -hmac "$SECRET" -binary | b64url)
curl -s -H "Authorization: Bearer $HEADER.$PAYLOAD.$SIG" http://127.0.0.1:3500/widgets

Troubleshooting

Related

• PgBouncer — connection pooler (watch the transaction-pooling caveat above when pooling PostgREST’s backend).
• PgDog — pooling / sharding proxy, same dual-mode shape.
• HAProxy — the listener PostgREST’s DSN should target for a Patroni cluster.
• HA Cluster — Patroni topology that --dsn points at.

5 - RustFS

rustfs is a Rust reimplementation of S3-compatible object storage: the same S3 API, a web console, erasure-coded multi-drive and multi-node layouts — and a completely different runtime model from MinIO/silo. pgcli runs it as a standalone, top-level addon with the same CLI surface as minio and silo (install, TLS, BYO certs, drives, logs, autostart), sharing the one port pool, but with three differences that shape this page:

1. A fixed container user. Upstream rustfs bakes User=rustfs (uid/gid 10001) into the image. pgcli works around this entirely inside its own wrapper image — see Privileges and ownership below — so a rustfs store needs no host-side ownership dance.
2. Three topologies, no multi-node single-drive. rustfs speaks SNSD / SNMD / MNMD only — see Deployment modes.
3. Its own TLS filenames. rustfs reads rustfs_cert.pem / rustfs_key.pem from RUSTFS_TLS_PATH, not MinIO’s public.crt / private.key.

Pick rustfs when you want a lean, Rust-native S3 endpoint; it can coexist with minio and silo on one host (all three draw from the same port pool).

Platform support: the rustfs addon is Linux only, in practice. The runtime model (a fixed container uid, drives on distinct block devices, host networking) is a Linux container story and is only exercised on Linux; the macOS bridge path is code-complete and the wrapper image is dual-arch, but it has not been tested on a Mac yet.

How It Works

One container per instance. The data lives in a bind-mounted host directory (default <base-dir>/addon/rustfs/<name>/data, or one directory per --drive), so it outlives the container. Unlike the minio/silo addons, pgcli does not drive the rustfs binary directly — it runs the image’s own /entrypoint.sh (via the wrapper below) because that entrypoint is what expands the multi-drive brace range in RUSTFS_VOLUMES, creates the per-drive directories, and assembles the server argv.

The image pgcli pulls is not the bare upstream one — it is ghcr.io/mars-base/pgcli/pgcli-rustfs:1.0.0, a thin pgcli wrapper built on top of docker.io/rustfs/rustfs:1.0.0 (the first GA release). pg addon install rustfs just pulls the wrapper; pgcli never builds it at run time. The tag tracks the pinned upstream version.

Credentials are handled like minio/silo:

• root_user defaults to admin (rustfs imposes no minimum key length; anything non-empty works, and admin sidesteps the rustfsadmin warning);
• root_password is generated on first install (or supplied via --root-password), stored in pg.yaml (addons.rustfs.<name>.root_password) and printed once in the install summary.

They are rustfs’s root access key / secret key, passed via the RUSTFS_ACCESS_KEY / RUSTFS_SECRET_KEY env vars — what every S3 client (including pg mc alias set) calls the access key and secret key.

Privileges and ownership

This is the one place rustfs genuinely differs from minio/silo, and pgcli has absorbed it into the image so it is invisible in practice.

Upstream rustfs runs as a fixed, non-configurable uid/gid (10001). Under rootless podman a host user has no claim on that uid, so the naive way to make a bind-mounted data directory writable by the process is a host-side chown dance (podman unshare chown 10001:10001 …) — fragile, and it fights with the fact that the operator may not be root. pgcli sidesteps all of it:

• The wrapper image’s entrypoint starts as container root, chowns its own bind-mounted data directories to 10001, then su-drops to the rustfs user and execs the unmodified upstream /entrypoint.sh. pgcli passes no --user flag and does no host-side ownership work at all.
• The result is the same on both daemon modes; only the host-visible numeric uid differs, because that is a property of podman’s user namespace, not of pgcli:
  • rootful podman → the directories land on the real host uid 10001;
  • rootless podman → they land on a subordinate host uid (subuid_start + 10001, e.g. 110001), with 10001 only visible inside the container.

Neither case needs the operator to be root or to run chown by hand. The one convention still worth knowing is that when you mount drives yourself (--drive), the mount point should be owned by the user who runs pg — under rootless podman a root-owned mount point is outside the namespace’s uid range and cannot be claimed by the in-container chown, exactly as with any bind mount.

TLS is copied, never re-owned. rustfs must read its key+cert as 10001, but the cert directory pgcli generated (0700, pgcli-owned) has to stay pgcli-owned so pg cert, CA refresh and pg backup fetch-ca keep working on it. So the wrapper does not chown that directory: it mounts it read-only and, as container root, copies the two required files into a fresh container-local directory (owned by 10001) that RUSTFS_TLS_PATH points at. No host file is ever mutated, and the BYO path uses the same copy mechanism, so BYO key/cert are mounted read-only at the required names and never chowned either.

Install

# default instance name "rustfs", ports from the shared pool, loopback bind
pg addon install rustfs

# a named instance with an explicit data directory
pg addon install rustfs --name store --data-dir /srv/rustfs

# fixed ports and a different root user
pg addon install rustfs --name store --api-port 9000 --console-port 9001 --root-user admin

# expose the store on the network instead of loopback only
pg addon install rustfs --name store --listen 0.0.0.0

# HTTPS via pgcli's self-signed CA (the prerequisite for a pgBackRest S3 repo)
pg addon install rustfs --name store --tls

# HTTPS with a certificate you already have (public-CA or private-CA cert)
pg addon install rustfs --name store --tls-cert /etc/ssl/rustfs.test.crt --tls-key /etc/ssl/rustfs.test.key

The output reports endpoints and the root credentials:

✓ rustfs installed: "store"
  Container:    pgcli-rustfs-default-store
  Image:        ghcr.io/mars-base/pgcli/pgcli-rustfs:1.0.0
  Data:         ~/pg/addon/rustfs/store/data
  S3 API:       http://127.0.0.1:9000
  Console:      http://127.0.0.1:9001

  Root user:     admin
  Root password: <generated>

Sign in to the console at the Console: URL with the printed root user and password. Point S3 clients (including pgBackRest) at the S3 API: URL, or drive them from the terminal with pg mc.

Re-running install against a live instance is a no-op: the container is not recreated (a stopped one is simply started, with a notice), the flags are merged into the stored config, and the existing root password is kept. Pass --force to recreate the container so changed ports, listen address, endpoint list, or credentials take effect — without losing the data directory.

Bind address: the default 127.0.0.1 keeps the store local. --listen 0.0.0.0 (or the listen key in pg.yaml) exposes it on the network. Anyone who can reach the port can then attempt the root credentials, so only do this behind a firewall or with TLS (--tls).

TLS (--tls)

--tls makes rustfs serve HTTPS. pgcli generates a self-signed CA and a leaf cert with the stdlib — SANs cover the loopback names, localhost, and every NIC IP of the host — into <base_dir>/tls/rustfs/<name>/: rustfs_cert.pem / rustfs_key.pem (rustfs’s required names, also mirrored as public.crt / private.key for the CA-friendly tooling) and ca.crt for distribution. The cert dir is mounted read-only and copied into the container as described in Privileges and ownership; the endpoint URL becomes https://.

Why you need it: pgBackRest forces HTTPS for S3 repositories, so a rustfs meant to receive Patroni archive-push must speak TLS. Point backup.repo.s3.ca_file at ca.crt and pgBackRest connects with full certificate verification — see Backup → S3 object storage repository. Everything that page says about a MinIO repo applies to a rustfs one: the S3 contract is identical (path-style, repo1-s3-uri-style=path).

Getting the CA onto a remote host — no scp needed. The server cert is served as a leaf+CA chain, so a consumer on another machine can pull the root straight out of a TLS handshake:

pg backup fetch-ca <store-host>:9000
#   [OK] CA fetched from <store-host>:9000
#        saved:    ~/.pgcli/backup/repo-ca/ca-<store-host>-9000.crt
#        SHA-256:  c0f0…fe2e
pg backup setup --s3-ca-file ~/.pgcli/backup/repo-ca/ca-<store-host>-9000.crt

The fetch is trust-on-first-use — compare the printed SHA-256 against the store host’s sha256sum ~/.pgcli/tls/rustfs/<name>/ca.crt before trusting it.

Bring your own certificate (--tls-cert / --tls-key)

--tls only ever serves pgcli’s own self-signed pair. To serve a certificate you already hold, pass --tls-cert <leaf(+chain).pem> --tls-key <key.pem>. Together with --tls (implied), this replaces the generated certs: the two files are mounted read-only and copied into the container at the names rustfs requires (/opt/rustfs/certs/rustfs_cert.pem, …/rustfs_key.pem); pgcli never re-owns them and never writes to their source directory.

pg addon install rustfs --name store \
  --tls-cert /etc/ssl/wildcard.example.com.crt \
  --tls-key  /etc/ssl/wildcard.example.com.key

No certificate yet? pg cert mints a self-signed leaf with whatever SANs you name, and rustfs serves it as-is (verified on a real install: the leaf rustfs presents matches the pg cert output off the live handshake, and pg mc does a full put/get round-trip against it over TLS). See Self-Cert for the mint flags, client trust-anchor, and renewal.

Everything else about BYO mode — why renewal needs --force (a single-file mount pins the source inode), how clients pick their trust anchor, and pg cert as a test-cert mint — is identical to the MinIO addon; see MinIO → Bring your own certificate. Turning BYO off: remove cert_file/key_file under this addon in pg.yaml and recreate with pg addon install rustfs --name store --force.

Deployment modes

rustfs has three layouts — there is no multi-node single-drive mode:

There is deliberately no MNSD (multi-node single-drive) in rustfs. Passing --endpoint without --drive is rejected at install time: rustfs derives its per-drive volume range itself, so a distributed node must say how many drives it has.

rustfs hard-requires distinct physical disks. Under SNMD/MNMD, every --drive must sit on its own block device. If two drives share a device (st_dev), the rustfs process [FATAL]s at startup. pgcli launches detached, so pg addon install still exits 0 and the failure surfaces as a container that never becomes healthy — check pg logs addon rustfs --name store if a multi-drive install won’t come up. (This is stricter than MinIO/silo, which only warn.)

Single-Node Multi-Drive (SNMD)

One rustfs process, several host directories, erasure-coded across them. Pass --drive once per drive instead of --data-dir:

pg addon install rustfs --name store \
  --drive /mnt/rustfs/disk1 --drive /mnt/rustfs/disk2 \
  --drive /mnt/rustfs/disk3 --drive /mnt/rustfs/disk4 --tls

Each --drive is a host directory on its own device; drive N (0-indexed) is bind-mounted at container path /data/rustfsN, and RUSTFS_VOLUMES is set to the brace range /data/rustfs{0...3} that the image’s /entrypoint.sh expands into the individual directories. --drive is mutually exclusive with --data-dir; combining --drive with --endpoint is MNMD.

pg addon remove rustfs --name store --clean-data deletes each drive directory — but refuses any drive that is still a mount point, so an accidental --clean-data can never rm -rf through a live mount into the disk below. Unmount first if the data below is really disposable.

Distributed / Cluster Mode (MNMD)

Every node runs its own pgcli with its own pg.yaml; each pg.yaml carries the same full endpoint list and the same root credentials. For rustfs the --endpoint list is one scheme://host:port per node (no path — rustfs derives the /data/rustfsN volume range from --drive), and each node passes its own drives:

# on node 1 (10.0.0.11), four dedicated data disks:
pg addon install rustfs --name store \
  --listen 0.0.0.0 \
  --root-password '<shared-secret>' \
  --drive /mnt/rustfs/d1 --drive /mnt/rustfs/d2 --drive /mnt/rustfs/d3 --drive /mnt/rustfs/d4 \
  --endpoint http://10.0.0.11:9000 --endpoint http://10.0.0.12:9000 \
  --endpoint http://10.0.0.20:9000 --endpoint http://10.0.0.21:9000

# nodes 2-4: same command, own --drive, and the SAME --endpoint list
# AND the SAME --root-password value.

Two things matter beyond the identical config: --listen must be 0.0.0.0 (or the node’s own reachable address), not the loopback default — under podman host networking the endpoints advertise each node’s real IP, and a node listening only on 127.0.0.1 can never be joined. And every node must carry the same endpoint list and root credentials: rustfs derives one shared erasure set from them, so a mismatch splits the ring into independent standalones.

--endpoint values are scheme://host:port with no path, repeated once per node (a comma-joined --endpoint a,b,c is equivalent — it is a string-slice flag). pgcli appends the per-node /data/rustfs{0...N-1} drive range and joins the four URLs with spaces into a single RUSTFS_VOLUMES. (The RUSTFS_VOLUMES value itself must be space-joined, not comma-joined: the rustfs binary mis-splits it, collapsing http:// to http:/ so a node can’t resolve its own disks — the first-listed node aborts with VolumeNotFound and the peers hang at waiting for storage_quorum, never forming a writable cluster. Space-separated literals parse exactly like the documented compact http://node{1...4}:9000/data/rustfs{0...3} brace form, but let you use plain per-node IPs with no /etc/hosts or DNS.)

Cross-host MNMD is verified on a real 4-node × 4-drive cluster (Linux, rootful podman): all four serve /health 200, and an object written once reads back byte-identical from every node — the erasure shards spread across all 16 drives.

Using the mc Client

pg mc runs MinIO’s mc client in a throwaway container and speaks rustfs’s S3 API like any other store:

pg mc alias set store http://127.0.0.1:9000 admin <password>    # Linux
pg mc mb store/backups
pg mc ls store
pg mc cp ./dump.pglz store/backups/

The S3 data plane (PUT/GET) is proven end-to-end against rustfs via pgBackRest and a basic mc round-trip, including a full PITR sequence — base backup, archive-push of the WAL, and a --time restore that replayed the archived WAL out of rustfs and stopped exactly at the target (Linux, rootless podman). Note honestly: the administrative mc surface (alias-set validation, bucket policies, admin commands) has not been exhaustively exercised against rustfs. Use it for plain object put/get with the understanding that some MinIO-specific admin commands may not map onto rustfs.

Ports

Each instance takes two consecutive ports from one pool, starting at minio_start_port (default 9000): the S3 API first, the console second. The pool is shared across minio, silo, and rustfs — all three are assigned from one cursor, so they coexist on a host without collision (minio first by name, then silo, then rustfs):

pg addon install minio  --name store   # 9000 / 9001
pg addon install silo   --name lake    # 9002 / 9003
pg addon install rustfs --name archive # 9004 / 9005

Configuration

Instances live under the top-level addons.rustfs map in pg.yaml:

namespace: default
minio_start_port: 9000       # shared pool: minio, silo AND rustfs draw from this
addons:
  rustfs:
    store:
      container_name: pgcli-rustfs-default-store
      name: store
      image_tag: ghcr.io/mars-base/pgcli/pgcli-rustfs:1.0.0
      # data_dir: /srv/rustfs    # omit for <base-dir>/addon/rustfs/store/data
      # drives:                  # multi-drive (SNMD/MNMD): one host dir per
      #   - /mnt/rustfs/d1       # drive, mounted at /data/rustfs0../rustfsN
      #   - /mnt/rustfs/d2
      listen: 127.0.0.1
      api_port: 9000
      console_port: 9001
      root_user: admin
      root_password: <generated>   # written on first install
      autostart: false             # pg autostart enable --rustfs --name store
      # tls: true                  # serve HTTPS (self-signed CA, or BYO below)
      # cert_file: /etc/ssl/rustfs.test.crt   # BYO leaf(+chain), implies tls
      # key_file:  /etc/ssl/rustfs.test.key   # BYO private key, must pair with cert_file
      # endpoints:                 # omit for single-node; one per node for MNMD:
      #   - http://10.0.0.11:9000
      #   - http://10.0.0.12:9000

Edits to listen, ports, root_user, root_password, image_tag, data_dir, tls/cert_file/key_file, or endpoints take effect after the next pg addon install rustfs --name store --force.

List

pg addon list

Infra add-ons (rustfs):
  rustfs (name: store)
    Status:      running
    Listen:      127.0.0.1
    API port:    9000
    Console port: 9001
    Console URL: http://127.0.0.1:9001/
    Data:        ~/pg/addon/rustfs/store/data
    Root user:   admin
    Image:       ghcr.io/mars-base/pgcli/pgcli-rustfs:1.0.0
    Container:   pgcli-rustfs-default-store

pg addon list never prints the password — read it from pg.yaml. Health is on GET /health (returns 200), unlike MinIO’s /minio/health/live.

Start and stop

pg addon start rustfs --name store
pg addon stop  rustfs --name store

install skips a still-present container (starting it if stopped); start only starts an existing one (and self-heals an improper state by recreating it from the config). In TLS generated mode start also re-validates and, if needed, re-signs the leaf.

Auto-start on Boot

Containers carry a --restart unless-stopped policy (crashes, not reboots). To bring instances up after a host reboot:

pg autostart enable --rustfs --name store

rustfs’s fixed container uid is handled inside pgcli’s wrapper image, so a boot-time start needs no special host privileges beyond running podman itself. Boot is start-only; rustfs is independent of the PostgreSQL stack, so it is started last.

Remove

pg addon remove rustfs --name store            # container gone, data kept
pg addon remove rustfs --name store --clean-data   # also delete the data directory

The data directory is the object storage — losing it means losing every bucket in it — so remove keeps it by default. --clean-data deletes it (and prunes the now-empty default-layout parent directory). Under rootless podman the objects are owned by a subordinate uid that plain rm cannot unlink, so --clean-data transparently falls back to podman unshare rm to reclaim them.

Logs

pg logs addon rustfs --name store      # last 50 lines
pg logs addon rustfs --name store -f   # follow

Troubleshooting

• pulling rustfs image ... : ... on install. The wrapper tag ghcr.io/mars-base/pgcli/pgcli-rustfs:1.0.0 is not reachable — check network access to ghcr.io, or pre-pull it with podman pull.
• A multi-drive install won’t become healthy. rustfs [FATAL]s when two drives share a physical device. Check pg logs addon rustfs --name store for the disk check firing, and make sure each --drive is on its own device (a separate disk or its own loop mount).
• Rootless podman can’t write a drive. A drive’s mount point is owned by root, outside the namespace’s uid range, so the container can’t claim it. chown the mount point to the user who runs pg.
• Port collision on manual --api-port. The pool is shared across minio/silo/rustfs; an explicit port must be dodged by the auto-assigner for all three. Each store needs a consecutive pair.
• --clean-data leaves a directory behind. Under rootless podman a subordinate-uid tree is removed via the podman unshare rm fallback; if that path is unavailable, remove it yourself with podman unshare rm -rf <dir>.

6 - HAProxy

HAProxy is the load balancer the Patroni docs recommend putting in front of a cluster: clients connect to one stable address, HAProxy health-checks each member’s REST API, and routes writes to the current leader (and, in read/write-split mode, reads to the replicas). pgcli runs it as a standalone, top-level addon — shared routing infrastructure, not a per-instance sidecar — pinned to the official haproxy:3.2.23-alpine image.

Platform support: the HAProxy addon is Linux-only for now. It reaches Patroni members over the host network, which the macOS podman machine does not expose to containers. On macOS NewHAProxyManager fails fast with a clear message; pg addon list still shows installed instances without live status.

How It Works

Routing follows the official Patroni haproxy.cfg pattern. Each backend is a Patroni member; two ports matter per member:

• the PostgreSQL port — the TCP port HAProxy actually forwards connections to;
• the REST API port — where HAProxy sends its httpchk health check.

Patroni’s REST API answers these GETs (unauthenticated by design):

So the leader is the sole UP server for a write listener, and the replicas are the sole UP servers for a read listener — failover is picked up automatically as the health checks flip. pgcli renders this into <base-dir>/addon/haproxy/<name>/haproxy.cfg and bind-mounts it read-only into the container.

Two modes

The mode is chosen with --mode:

• unified (default) — a single listener sends all traffic to the current leader. Simple; every connection can read and write. On leader change the write listener uses on-marked-down shutdown-sessions to drop stale connections so clients reconnect to the new leader.
• split — read/write separation: a _<name>_rw listener routes writes to the leader, and a separate _<name>_ro listener spreads reads across the replicas (round-robin, lag-filtered). Two client ports instead of one.

Stats page

Every instance also gets an HTTP stats listener so you can watch server up/down state in a browser: http://<listen>:<stats-port>/.

Install

Backend members are supplied one of two ways (mutually exclusive):

• --node NAME=HOST:PGPORT:RESTPORT (repeatable) — list them explicitly;
• --ha <scope> — auto-derive every local member of a Patroni scope that pg ha manages, using each member’s PostgreSQL and REST API ports.

What each --node field means — for node1=10.0.0.11:35532:8008:

With --ha, all four come from pg.yaml: the Patroni member name, its advertise-host (default 127.0.0.1), and its host_port / restapi_port.

# unified: one port, everything to the leader, backends auto-derived from scope "app"
pg addon install haproxy --name lb --ha app

# split: rw + ro listeners, reads capped at 1MB replica lag
pg addon install haproxy --name lb --mode split --ha app --max-lag 1MB

# explicit backends (e.g. members on other hosts, or a scope pgcli doesn't manage)
pg addon install haproxy --name lb --mode split \
  --node node1=10.0.0.11:35532:8008 \
  --node node2=10.0.0.11:35533:8009 \
  --node node3=10.0.0.11:35534:8010

The output reports the assigned ports and a ready-to-use connect URL:

✓ haproxy installed: "lb"
  Container:    pgcli-haproxy-default-lb
  Image:        docker.io/library/haproxy:3.2.23-alpine
  Mode:         split
  Patroni:      app
  Backends:     3
  Read-write:   127.0.0.1:5000
  Read-only:    127.0.0.1:5001 (lag ≤ 1MB)
  Stats:        http://127.0.0.1:5002/
  Config:       ~/pg/addon/haproxy/lb/haproxy.cfg

  Connect via HAProxy:
    postgres://<user>@127.0.0.1:5000/<database>

Adding or removing members later

The backend list follows the scope’s local members recorded in pg.yaml, so any topology change — or removal — is picked up by re-running the same install command: it re-derives the full backend list and recreates the container.

Added with pg ha create <scope> --member <m>, the new member is not in the running HAProxy config yet:

pg ha create app --member node3 --etcd m1
pg addon install haproxy --name lb --mode split --ha app --max-lag 1MB
# Backends: 2 -> 3, and haproxy.cfg gains: server node3 127.0.0.1:35534 ... check port 8011

Symmetrically, after pg ha remove <scope> --member <m> the removed member would otherwise linger in the config as a stale backend — re-running install drops it:

pg ha remove app --member node3
pg addon install haproxy --name lb --mode split --ha app --max-lag 1MB
# Backends: 3 -> 2, and the server node3 line is gone

For an explicit --node install there is no auto-derivation: edit the --node list itself (add or drop a spec) and re-run — the target list is replaced wholesale each time, so the command always reflects the full set you want.

Cross-host clusters

--ha derives backends from this host’s pg.yaml, which lists only the members this host’s pg ha create manages. A member living on another host was never a backend here — adding or removing one there requires no re-sync on this host (and leaves nothing stale). To front the whole cluster through one HAProxy, use an explicit --node list, with each member’s advertised host:

pg addon install haproxy --name lb --mode split \
  --node node1=10.0.0.11:35532:8008 \
  --node node2=10.0.0.12:35532:8008 \
  --node node3=10.0.0.13:35532:8008
# later: add one more --node spec (or drop one) and re-run — the list replaces wholesale

Caveat: an instance installed with --ha on one host routes writes only to a local leader — if failover promotes a remote member, the rw listener has no UP server until the leader moves back. For cross-host clusters where the leader can move, prefer the explicit all-members --node list (or run one HAProxy per host and front it with a higher-level VIP/DNS).

Read/Write Split in Practice

After installing with --mode split, two client ports are exposed:

# Write (always hits the current leader)
psql "host=127.0.0.1 port=5000 user=postgres dbname=postgres"

# Read (load-balanced across replicas)
psql "host=127.0.0.1 port=5001 user=postgres dbname=postgres"

After a failover, HAProxy picks up the new leader automatically via health checks — no connection string change needed. A browser at http://127.0.0.1:5002/ shows real-time backend state.

Ports

Ports are auto-assigned from a per-host pool, three consecutive free ports per instance (rw, then ro if split, then stats), starting at haproxy_start_port (default 5000). Override any of them explicitly:

pg addon install haproxy --name lb --ha app \
  --rw-port 6000 --ro-port 6001 --stats-port 6002

Because the pool scans for free ports and tracks what every HAProxy instance has already taken, you can run many instances on one machine without collisions. A split instance consumes 3 ports (rw / ro / stats), a unified instance 2 (rw / stats):

pg addon install haproxy --name lb1 --mode split --ha app     # 5000 / 5001 / 5002
pg addon install haproxy --name lb2 --ha report              # 5003 / 5004

The bind address defaults to 127.0.0.1; set --listen 0.0.0.0 — or the listen key in pg.yaml — to expose the listeners on the network. Keep it on loopback unless the backends require otherwise; there is no authentication in front of PostgreSQL here.

Configuration

Everything lives under the top-level addons.haproxy map in pg.yaml, keyed by instance name:

namespace: default
haproxy_start_port: 5000
addons:
  haproxy:
    lb:
      mode: split            # unified | split
      ha_scope: app          # scope the backends came from (for --ha re-derivation)
      listen: 127.0.0.1
      write_port: 5000
      read_port: 5001
      stats_port: 5002
      replica_max_lag: 1MB   # split mode only
      autostart: false       # pg autostart enable --haproxy --name lb
      image_tag: docker.io/library/haproxy:3.2.23-alpine
      targets:
        - { name: node1, host: 127.0.0.1, pg_port: 35532, rest_port: 8008 }
        - { name: node2, host: 127.0.0.1, pg_port: 35533, rest_port: 8009 }

targets is rendered from and re-derived by install; edit mode, ports, replica_max_lag, or image_tag here for fine control, then pg addon install haproxy --name lb ... to apply. The rendered haproxy.cfg carries no secrets — only hosts, ports, and health-check URIs.

List

pg addon list

Infra add-ons (haproxy):
  haproxy (name: lb)
    Status:      running
    Mode:        split
    Patroni:     app
    Read-write:  127.0.0.1:5000
    Read-only:   127.0.0.1:5001
    Stats:       http://127.0.0.1:5002/
    Backends:    3
      - node1        127.0.0.1:35532 (check :8009)
      - node2        127.0.0.1:35533 (check :8010)
      - node3        127.0.0.1:35534 (check :8011)

Start and stop

After a host reboot, bring an instance back without re-rendering config:

pg addon start haproxy --name lb
pg addon stop  haproxy --name lb

install always recreates the container so config changes take effect; start only starts an existing one (and self-heals an improper state by recreating from the haproxy.cfg on disk).

Auto-start on Boot

Containers carry a --restart unless-stopped policy (crashes, not reboots). To bring an instance up after a host reboot:

pg autostart enable --haproxy --name lb

This sets autostart: true and installs/refreshes the boot service (see Auto-start on Boot). Boot is start-only: it starts the existing container reading the haproxy.cfg already on disk, and never re-renders config — install the instance first. The boot service starts HAProxy after the Patroni members, so its backends are already coming up. pg autostart status lists every target’s state.

Remove

pg addon remove haproxy --name lb

Stops and removes the container and deletes its config directory.

Logs

pg logs addon haproxy --name lb       # last 50 lines
pg logs addon haproxy --name lb -f    # follow

The container logs to stdout (log stdout format raw local0 info), so health-check transitions and connection events show up here.

Troubleshooting

• All backends DOWN in the stats page. The health check hits each member’s REST API port. Confirm it is reachable — curl http://<host>:<rest_port>/ returns 200 on the leader, 503 elsewhere; /replica is the mirror image. A 000 means the REST API is down or the port is wrong.
• --ha finds no members. --ha only derives local members that pg ha manages in pg.yaml. For remote members, or a scope pgcli doesn’t manage, list them with --node.
• Port collision on manual --rw-port. Pick a port outside the auto pool (haproxy_start_port and up) or the auto-assigner will treat it as taken.
• macOS. Not supported yet — HAProxy needs host networking to reach Patroni members, which the podman machine VM does not provide to containers.

7 - Silo

silo is Pigsty’s maintained fork of MinIO: S3-compatible object storage with the web console bundled in, keeping MinIO’s wire contract end to end — the S3 API, the MINIO_* environment variables, the server /data --address :9000 --console-address :9001 command line, the --certs-dir layout, and erasure-coded cluster mode. pgcli runs it as a standalone, top-level addon with exactly the same surface as the minio addon — install, TLS, BYO certs, distributed mode, logs, autostart. Pick silo over MinIO if you follow Pigsty’s releases; the two can coexist on one host (they share one port pool, assigned without collision).

Platform support: the silo addon works on both platforms. Linux serves over host networking; macOS joins the pgcli-net bridge with its two ports published, so the Mac reaches the API and console on 127.0.0.1:<port> like any other addon. The public image is dual-arch (amd64 + arm64). Both clients — pg mcli (silo’s own) and pg mc (MinIO’s) — work on both platforms and against a silo store alike.

How It Works

One container, one directory: silo server /data --address <listen>:<api-port> --console-address <listen>:<console-port>, where /data is a bind mount of the instance’s host data directory (default <base-dir>/addon/silo/<name>/data). Everything silo stores lives there, so it outlives the container.

The image is the public upstream tag docker.io/pgsty/silo:RELEASE.2026-09-16T00-00-00Z — dual-arch, console included, and it ships the mcli client too — so pg addon install silo just pulls it; pgcli never builds the image at run time (unlike pgcli-minio, which is a self-maintained tag because MinIO’s official image dropped the console). pgcli drives the silo binary directly via an explicit --entrypoint silo, bypassing the image’s entrypoint wrapper, exactly as it does for MinIO.

Credentials are handled like Patroni’s:

• root_user defaults to admin;
• root_password is generated on first install (or supplied explicitly via --root-password) and stored in pg.yaml (addons.silo.<name>.root_password) and printed once in the install summary for convenience.

They are silo’s root access key / secret key — passed through the MINIO_ROOT_USER / MINIO_ROOT_PASSWORD env vars silo inherits from MinIO — what every S3 client (and pg mcli alias set <name> <url> <root_user> <root_password>) calls the access key and secret key.

The container runs --ulimit nofile=1048576:1048576 and --stop-timeout 60 as MinIO’s deployment docs recommend (the contract carries over). On macOS the podman machine VM caps RLIMIT_NOFILE lower, so the value used there is 65536 — plenty for a single-host dev/test store. Root credentials passed as -e are visible in podman inspect — the same exposure as any hand-run container; fine for a rootless single-host deployment, which is what this is.

Install

# default instance name "silo", ports from the shared pool (base 9000), loopback bind
pg addon install silo

# a named instance with an explicit data directory
pg addon install silo --name store --data-dir /srv/silo

# fixed ports and a different root user
pg addon install silo --name store --api-port 9000 --console-port 9001 --root-user admin

# expose the store on the network instead of loopback only
pg addon install silo --name store --listen 0.0.0.0

# HTTPS via pgcli's self-signed CA (the prerequisite for a pgBackRest S3 repo)
pg addon install silo --name store --tls

# HTTPS with a certificate you already have (public-CA or private-CA domain cert)
pg addon install silo --name store --tls-cert /etc/ssl/silo.test.crt --tls-key /etc/ssl/silo.test.key

The output reports endpoints and the root credentials:

✓ silo installed: "store"
  Container:    pgcli-silo-default-store
  Image:        docker.io/pgsty/silo:RELEASE.2026-09-16T00-00-00Z
  Data:         ~/pg/addon/silo/store/data
  S3 API:       http://127.0.0.1:9000
  Console:      http://127.0.0.1:9001

  Root user:     admin
  Root password: <generated>

Sign in to the console at the Console: URL with the printed root user and password. Point S3 clients (including pgBackRest) at the S3 API: URL, or drive them from the terminal with pg mcli below.

Re-running install against a live instance is a no-op: the container is not recreated (a stopped one is simply started, with a notice), the flags are merged into the stored config, and the existing root password is kept. Pass --force to recreate the container so changed ports, listen address, endpoint list, or credentials take effect — without losing the data directory.

Bind address: the default 127.0.0.1 keeps the store local. --listen 0.0.0.0 (or the listen key in pg.yaml) exposes it on the network. Anyone who can reach the port can then attempt the root credentials, so only do this behind a firewall or with TLS (--tls, below).

TLS (--tls)

--tls makes silo serve HTTPS. pgcli generates a self-signed CA and a leaf cert with the stdlib — SANs cover the loopback names, localhost, and every NIC IP of the host — into <base_dir>/tls/silo/<name>/: public.crt / private.key for silo’s --certs-dir, and ca.crt for distribution. The cert dir is mounted read-only and the endpoint URL becomes https://.

Why you need it: pgBackRest forces HTTPS for S3 repositories (plaintext is an upstream-rejected option), so a silo meant to receive Patroni archive-push must speak TLS. Point backup.repo.s3.ca_file at ca.crt and pgBackRest connects with full certificate verification — see Backup → S3 object storage repository. Everything that page says about a MinIO repo applies verbatim to a silo one: the client contract is identical.

One note on pairing stores with clusters: backup.repo.s3 is a single, global repo per config file, so the first TLS store you wire up — minio or silo — receives every stanza in that environment. If you truly need two destinations, run a second config (see Backup → Shared Backup Container, and the worked Example: HA Cluster with a Self-CA MinIO).

pg mcli adds --insecure automatically when a command targets a TLS store via a loopback alias (mcli persists no CA trust per alias; same-host loopback makes this acceptable). An alias to a LAN-IP endpoint is not loopback — append -- --insecure yourself, or set up the CA properly. External https:// endpoints keep full verification. Toggling --tls on an existing instance needs --force to take effect.

Lifetimes and access from other hosts. CA and leaf are both long-lived; pgcli re-signs the leaf on --force, or whenever a host address changes (silo watches its cert files and hot-reloads a re-signed pair, which is how a deployment picks up the new chain without a recreate), so handing ca.crt to clients is a one-time act per store. A TLS client validates the address it dials against the leaf’s SANs — the client’s own address never matters. Remote hosts therefore point backup.repo.s3.endpoint at one of the store host’s IPs (loopback names and every NIC IP, virtual bridges included, are in the SAN; raw hostnames are not — unless you set MINIO_SERVER_URL, whose host is added).

Getting the CA onto a remote host — no scp needed. The server cert is served as a leaf+CA chain, so a consumer on another machine can pull the root straight out of a TLS handshake:

pg backup fetch-ca <store-host>:9002
#   [OK] CA fetched from <store-host>:9002
#        saved:    ~/.pgcli/backup/repo-ca/ca-<store-host>-9002.crt
#        SHA-256:  c0f0…fe2e
pg backup setup --s3-ca-file ~/.pgcli/backup/repo-ca/ca-<store-host>-9002.crt

The fetch is trust-on-first-use — compare the printed SHA-256 against the store host’s sha256sum ~/.pgcli/tls/silo/<name>/ca.crt before trusting it. One setup then republishes the CA into the Patroni cluster’s etcd registry, so every other cluster host gets it with no manual step at all (see Backup → S3 object storage repository).

Bring your own certificate (--tls-cert / --tls-key)

--tls only ever serves pgcli’s own self-signed pair. To serve a certificate you already hold — one signed by a public CA for a real domain, or one issued by your private CA — pass --tls-cert <leaf(+chain).pem> --tls-key <key.pem> instead. Together with --tls (which it implies, so you don’t need to also pass it), this replaces the generated certs: the two files are mounted read-only straight at the names silo’s --certs-dir requires (/opt/silo/certs/public.crt, /opt/silo/certs/private.key), and pgcli never copies or re-signs them — the private key stays in exactly the one place you put it.

pg addon install silo --name store \
  --tls-cert /etc/ssl/wildcard.example.com.crt \
  --tls-key  /etc/ssl/wildcard.example.com.key

Everything else about BYO mode — what install checks, why renewal needs --force (a single-file mount pins the source inode), how clients pick their trust anchor, and pg cert as a test-cert mint — is identical to the MinIO addon; see MinIO → Bring your own certificate. The pg cert example, adapted:

pg cert --host "silo.test,127.0.0.1,10.0.0.9" \
  --cert-file silo.crt --key-file silo.key
pg addon install silo --name store --tls-cert silo.crt --tls-key silo.key

Turning BYO off. There is no off flag — the config merge across re-runs is one-way, matching how --tls itself works. To go back to generated certs, remove cert_file/key_file under this addon in pg.yaml and recreate with pg addon install silo --name store --force.

Deployment modes

silo/MinIO classifies its layouts; the addon supports all four:

SNSD is what pg addon install silo gives you out of the box. To get MNSD, pass the cluster’s endpoint list (at least four nodes) — see Distributed / Cluster Mode below.

MNMD combines both flags: give this node’s drives with --drive (as under SNMD) and the whole cluster’s host×drive endpoint matrix with --endpoint (one URL per drive on every node) — see Multi-Node Multi-Drive (MNMD) below. Disk redundancy under SNSD/MNSD is also available the other way — a ZFS pool under --data-dir — which keeps the layout changeable underneath and can be more space-efficient. S3 Storage High Availability compares native MNMD with the ZFS approach for the 4-hosts-each-with-several-disks hybrid that survives both a disk and a node.

Single-Node Multi-Drive (SNMD)

One silo process, several host directories, erasure-coded across them. Pass --drive once per drive instead of --data-dir:

pg addon install silo --name store \
  --drive /mnt/minio/disk1 --drive /mnt/minio/disk2 \
  --drive /mnt/minio/disk3 --drive /mnt/minio/disk4 --tls

Each --drive is a host directory on its own device; drive N is bind-mounted at container path /dataN and the server starts as silo server /data1 /data2 ... /dataN. --drive is mutually exclusive with --data-dir — multi-drive mode takes its data locations from --drive only. Combining --drive with --endpoint is MNMD, the multi-node version of this mode — see Multi-Node Multi-Drive (MNMD) below. Like every other silo/MinIO drive, one that shares the host’s root device is rejected at startup; pgcli lists the offending drives and warns at install time.

What EC buys you (measured on a live 4-drive silo set): a 4-drive set defaults to 2 parity shards — it tolerates 2 drive failures. With 1 drive down, reads and writes both continue; with 2 down, reads still succeed and writes are refused — that is the quorum boundary, the same arithmetic MNSD uses, just over drives instead of nodes. Usable capacity is roughly half the raw total. A drive that returns is healed by silo itself; pgcli does not need to do anything. The parity default scales with drive count — only the 4-drive shape is tested here.

pg addon remove silo --name store --clean-data deletes each drive directory — but refuses any drive that is still a mount point, so an accidental --clean-data can never rm -rf through a live mount into the disk below. Unmount first if the data below is really disposable.

Distributed / Cluster Mode

silo’s erasure-coded (EC) cluster mode works exactly like MinIO’s — same command shape, same rules. It requires at least four distinct host:port endpoints and, unlike the other addons, no central coordinator: every node runs its own pgcli with its own pg.yaml, and every one of those pg.yaml files carries the same full endpoint list and the same root credentials. pgcli only ever starts the container for the node it is running on — silo itself does the handshake to form the ring across hosts.

# on node 1 (10.0.0.11), with a dedicated data disk mounted at /data:
pg addon install silo --name store \
  --listen 10.0.0.11 \
  --data-dir /data \
  --root-password '<shared-secret>' \
  --endpoint http://10.0.0.11:9000/data \
  --endpoint http://10.0.0.12:9000/data \
  --endpoint http://10.0.0.20:9000/data \
  --endpoint http://10.0.0.21:9000/data

# nodes 2-4: same command, own --listen, and the SAME --endpoint list AND the
# SAME --root-password value.

The three strict rules (routable distinct hosts, a data directory on a disk separate from root — pgcli warns when it isn’t — and no per-node MINIO_SERVER_URL in cluster mode) and the quorum arithmetic are the same as MinIO → Distributed / Cluster Mode; silo enforces the same checks because it inherited them.

Multi-Node Multi-Drive (MNMD)

MNMD works exactly like it does under MinIO: give this node’s drives with --drive and the whole cluster’s host×drive endpoint matrix with --endpoint, one URL per drive on every node. Each endpoint must address one of that node’s /data1../dataN drive slots, the matrix must be a multiple of the per-node drive count and must name at least one remote node, and a --tls node’s endpoints must all be https:// — pgcli checks all four before starting the container. See MinIO → Multi-Node Multi-Drive (MNMD) for the full walkthrough.

On a live 4-node × 4-drive silo set (16 × 2 GiB) the measured shape matches MinIO’s: 16 drives online, EC:4 in a single erasure set of stripe size 16; losing one whole node (12/16 online) keeps reads and writes working, a 64 MiB round-trip byte-identical; losing a second node (8/16 online) refuses writes (Resource requested is unwritable) and fails reads too, and the set self-heals back to 16/16 once the nodes restart. EC:4 over 16 drives keeps 12/16 of raw bytes.

For a TLS grid the same recommendation applies as under MinIO — one pg cert leaf whose SANs cover every node address, the identical --tls-cert/--tls-key pair on every node — and it was verified on silo exactly as on minio: the ring forms with no per-node CA to reconcile. See MinIO → MNMD for the setup.

Using the mcli Client

pg mcli runs silo’s own mcli client from the pgsty/silo image in a throwaway container (selected via --entrypoint mcli — the image’s default entrypoint runs the server) — no local install:

# pick the URL for your platform — see "Endpoints by platform" in the mc page:
pg mcli alias set store http://127.0.0.1:9000 admin <password>    # Linux, local addon
pg mcli alias set store http://host.containers.internal:9000 admin <password>  # macOS, local addon
pg mcli mb store/backups
pg mcli ls store
pg mcli cp ./dump.pglz store/backups/

mcli speaks the same alias contract as mc, so it interoperates freely — against a silo store, a MinIO store, or any S3 endpoint. pgcli points both clients at one file: aliases persist on the host at ~/.mc/config.json (mc’s native default path), so an alias registered with pg mc is visible to pg mcli and vice versa — set it once, not once per client. (The stateless MC_HOST_<name> form works for both too.) A native mcli install left to its own defaults reads ~/.mcli/config.json instead, so it will not see the shared file unless MC_CONFIG_DIR is pointed at it.

alias set validates the credentials against the endpoint before writing, so a wrong password leaves the config untouched. Local file operands of cp / mirror / diff are resolved and mounted at their real absolute paths, exactly as pg mc does (on macOS, keep them under your home directory). Any mcli flag that pg’s own parser would reject goes after --:

pg mcli ls store -- --all
MC_HOST_store="http://admin:<password>@127.0.0.1:9000" pg mcli ls store

Both clients share the same command table as documented on the MinIO page — Common commands — with pg mcli in place of pg mc.

Ports

Each instance takes two consecutive ports from one pool, starting at minio_start_port (default 9000): the S3 API first, the console second. The pool is shared with the minio addon — minio and silo instances are assigned from one cursor, so they coexist on a host without collision, minio instances first (by name), then silo:

pg addon install minio --name store    # 9000 / 9001
pg addon install silo  --name lake     # 9002 / 9003

Configuration

Instances live under the top-level addons.silo map in pg.yaml, keyed by instance name:

namespace: default
minio_start_port: 9000       # shared pool: minio AND silo draw from this
addons:
  silo:
    store:
      container_name: pgcli-silo-default-store
      name: store
      image_tag: docker.io/pgsty/silo:RELEASE.2026-09-16T00-00-00Z
      # data_dir: /srv/silo     # omit for <base-dir>/addon/silo/store/data
      # drives:                  # multi-drive (SNMD/MNMD): one host dir per drive,
      #   - /mnt/minio/disk1     # each mounted at its own /dataN — see SNMD/MNMD below
      #   - /mnt/minio/disk2
      listen: 127.0.0.1
      api_port: 9000
      console_port: 9001
      root_user: admin
      root_password: <generated>   # written on first install
      autostart: false             # pg autostart enable --silo --name store
      # tls: true                  # serve HTTPS (self-signed CA, or BYO below)
      # cert_file: /etc/ssl/silo.test.crt   # BYO leaf(+chain), implies tls
      # key_file:  /etc/ssl/silo.test.key   # BYO private key, must pair with cert_file
      # endpoints:                 # omit for single-node; see "Distributed / Cluster Mode"
      #   - http://10.0.0.11:9000/data
      #   - http://10.0.0.12:9000/data
      #   - http://10.0.0.20:9000/data
      #   - http://10.0.0.21:9000/data
      #   (with `drives` also set this is MNMD — one /dataN endpoint per drive on
      #    every node, and every node's list identical; see "Multi-Node Multi-Drive")

Edits to listen, ports, root_user, root_password, image_tag, data_dir, tls/cert_file/key_file, or endpoints take effect after the next pg addon install silo --name store --force — the plain install skips a still-present container, --force recreates it (the data directory is never touched).

List

pg addon list

Infra add-ons (silo):
  silo (name: store)
    Status:      running
    Listen:      127.0.0.1
    API port:    9000
    Console port: 9001
    Console URL: http://127.0.0.1:9001/
    Data:        ~/pg/addon/silo/store/data
    Root user:   admin
    Image:       docker.io/pgsty/silo:RELEASE.2026-09-16T00-00-00Z
    Container:   pgcli-silo-default-store

pg addon list never prints the password — read it from pg.yaml.

Start and stop

After a host reboot, bring an instance back without re-applying config:

pg addon start silo --name store
pg addon stop  silo --name store

install skips a still-present container (starting it if stopped); start only starts an existing one (and self-heals an improper state by recreating it from the config). In TLS generated mode start also re-validates and, if needed, re-signs the leaf (silo hot-reloads its mounted cert).

Auto-start on Boot

Containers carry a --restart unless-stopped policy (crashes, not reboots). To bring instances up after a host reboot:

pg autostart enable --silo --name store

This sets autostart: true and installs/refreshes the boot service (see Auto-start on Boot). Boot is start-only. silo is independent of the PostgreSQL stack, so it is started last; there is no ordering constraint. pg autostart status lists every target’s state.

Remove

pg addon remove silo --name store            # container gone, data kept
pg addon remove silo --name store --clean-data   # also delete the data directory

The data directory is the object storage — losing it means losing every bucket in it — so remove keeps it by default and says where it is. --clean-data deletes it (and prunes the now-empty default-layout parent directory; a data_dir override and its parent are never touched).

Logs

pg logs addon silo --name store      # last 50 lines
pg logs addon silo --name store -f   # follow

silo logs to stdout: startup lines (API:/Console: addresses, Documentation:) and request errors. An API: http://... block confirms the listeners came up.

Troubleshooting

• pulling silo image ... : ... on install. The public tag docker.io/pgsty/silo:RELEASE.2026-09-16T00-00-00Z is not reachable — check network/registry access to docker.io, or pre-pull it with podman pull.
• Port collision on manual --api-port. Pick ports outside the auto pool (minio_start_port and up) or the auto-assigner will treat them as taken; the pool is shared with the minio addon, so a silo instance must also dodge the minio instances’ explicit ports and vice versa. Remember each store needs a consecutive pair.
• pg addon start after host reboot does nothing / fails. Check pg logs addon silo --name store -f — most often the data directory was deleted (with --clean-data or by hand) and silo refuses to start on an empty dir it previously formatted, or the bind port moved.
• Console reachable but S3 clients time out. The MINIO_SERVER_URL is built from listen + API port; if you serve on 127.0.0.1 but access from another host, clients get redirected to the loopback URL. Set listen to the address clients can actually reach. (Cluster mode does not set MINIO_SERVER_URL at all.)
• An alias set with pg mc isn’t visible to pg mcli (or vice versa). They share ~/.mc/config.json by design, so this means the alias really isn’t there — check pg mc alias list (same file), or pass MC_HOST_<name> in the environment. A native mcli reads ~/.mcli instead and won’t see the shared file unless MC_CONFIG_DIR points there.
• macOS. Supported: the addon serves on the pgcli-net bridge with both ports published, so the Mac’s 127.0.0.1:<port> reaches them (the container binds 0.0.0.0 internally and MINIO_SERVER_URL advertises the loopback the Mac uses). After changing ports or credentials, --force recreate the container. For pg mcli, the container cannot use a 127.0.0.1 alias — see the endpoint note on the MinIO mc page.

8 - MinIO

MinIO is S3-compatible object storage. pgcli runs it as a standalone, top-level addon — shared infrastructure, not a per-instance sidecar — with the web console included. It defaults to single-node mode and supports a genuinely distributed cluster mode across hosts (see below). Either way it is a general-purpose object store. silo — Pigsty’s MinIO fork, kept wire-compatible — is available as a sibling addon with the identical feature surface; the two share one port pool and can coexist on a host.

Platform support: the MinIO addon works on both platforms. Linux serves over host networking; macOS joins the pgcli-net bridge with its two ports published, so the Mac reaches the API and console on 127.0.0.1:<port> like any other addon. The public image is dual-arch (amd64 + arm64), so any host architecture works. pg mc, the client, works on both platforms too.

How It Works

One container, one directory: minio server /data --address <listen>:<api-port> --console-address <listen>:<console-port>, where /data is a bind mount of the instance’s host data directory (default <base-dir>/addon/minio/<name>/data). Everything MinIO stores lives there, so it outlives the container.

The image is the public pre-built tag ghcr.io/mars-base/pgcli/pgcli-minio:20250422221226 — the upstream static binaries (dual-arch amd64 + arm64, from the minio/minio GitHub releases) on Alpine, because MinIO’s official image dropped the bundled web console. pg addon install pulls it; pgcli never builds the image at run time.

Credentials are handled like Patroni’s:

• root_user defaults to admin;
• root_password is generated on first install (or supplied explicitly via --root-password) and stored in pg.yaml (addons.minio.<name>.root_password) and printed once in the install summary for convenience.

They are exactly MinIO’s root access key / secret key — what every S3 client (and pg mc alias set <name> <url> <root_user> <root_password>) calls the access key and secret key.

The container runs --ulimit nofile=1048576:1048576 and --stop-timeout 60 as MinIO’s deployment docs recommend. On macOS the podman machine VM caps RLIMIT_NOFILE lower, so the value used there is 65536 — plenty for a single-host dev/test store. Note that root credentials passed as -e are visible in podman inspect — the same exposure as any hand-run container; fine for a rootless single-host deployment, which is what this is.

Install

# default instance name "minio", ports from the pool (base 9000), loopback bind
pg addon install minio

# a named instance with an explicit data directory
pg addon install minio --name store --data-dir /srv/minio

# fixed ports and a different root user
pg addon install minio --name store --api-port 9000 --console-port 9001 --root-user admin

# expose the store on the network instead of loopback only
pg addon install minio --name store --listen 0.0.0.0

# HTTPS via pgcli's self-signed CA (the prerequisite for a pgBackRest S3 repo)
pg addon install minio --name store --tls

# HTTPS with a certificate you already have (public-CA or private-CA domain cert)
pg addon install minio --name store --tls-cert /etc/ssl/minio.test.crt --tls-key /etc/ssl/minio.test.key

The output reports endpoints and the root credentials:

✓ minio installed: "store"
  Container:    pgcli-minio-default-store
  Image:        ghcr.io/mars-base/pgcli/pgcli-minio:20250422221226
  Data:         ~/pg/addon/minio/store/data
  S3 API:       http://127.0.0.1:9000
  Console:      http://127.0.0.1:9001

  Root user:     admin
  Root password: <generated>

Sign in to the console at the Console: URL with the printed root user and password. Point S3 clients (including pgBackRest) at the S3 API: URL, or drive them from the terminal with pg mc below.

Re-running install against a live instance is a no-op: the container is not recreated (a stopped one is simply started, with a notice), the flags are merged into the stored config, and the existing root password is kept. Pass --force to recreate the container so changed ports, listen address, endpoint list, or credentials take effect — without losing the data directory.

Bind address: the default 127.0.0.1 keeps the store local. --listen 0.0.0.0 (or the listen key in pg.yaml) exposes it on the network. Anyone who can reach the port can then attempt the root credentials, so only do this behind a firewall or with TLS (--tls, below).

TLS (--tls)

--tls makes MinIO serve HTTPS. pgcli generates a self-signed CA and a leaf cert with the stdlib — SANs cover the loopback names, localhost, and every NIC IP of the host — into <base_dir>/tls/minio/<name>/: public.crt / private.key for MinIO’s --certs-dir, and ca.crt for distribution. The cert dir is mounted read-only and the endpoint URL becomes https://.

Why you need it: pgBackRest forces HTTPS for S3 repositories (plaintext is an upstream-rejected option), so a MinIO meant to receive Patroni archive-push must speak TLS. Point backup.repo.s3.ca_file at ca.crt and pgBackRest connects with full certificate verification — see Backup → S3 object storage repository.

One note on pairing stores with clusters: backup.repo.s3 is a single, global repo per config file, so the first TLS MinIO you wire up receives every stanza in that environment — installing a second MinIO does not give clusters a per-cluster store to choose. If you truly need two destinations, run a second config (see Backup → Shared Backup Container, and the worked Example: HA Cluster with a Self-CA MinIO).

pg mc adds --insecure automatically when a command targets a TLS store via a loopback alias (mc persists no CA trust per alias; same-host loopback makes this acceptable). An alias to a LAN-IP endpoint is not loopback — append -- --insecure yourself, or set up the CA properly. External https:// endpoints keep full verification. Toggling --tls on an existing instance needs --force to take effect.

Lifetimes and access from other hosts. CA and leaf are both valid for 100 years (the 825-day cap public CAs observe is a browser policy, not something Go’s verifier enforces for a private root). pgcli re-signs the leaf on --force, or whenever a host address changes, so handing ca.crt to clients is a one-time act per store. A TLS client validates the address it dials against the leaf’s SANs — the client’s own address never matters. Remote hosts (a VM’s Patroni member, another machine’s backup container) therefore point backup.repo.s3.endpoint at one of the store host’s IPs (loopback names and every NIC IP, virtual bridges included, are in the SAN; raw hostnames are not — unless you set MINIO_SERVER_URL, whose host is added).

Getting the CA onto a remote host — no scp needed. The server cert is served as a leaf+CA chain, so a consumer on another machine can pull the root straight out of a TLS handshake:

pg backup fetch-ca <store-host>:9002
#   [OK] CA fetched from <store-host>:9002
#        saved:    ~/.pgcli/backup/repo-ca/ca-<store-host>-9002.crt
#        SHA-256:  c0f0…fe2e
pg backup setup --s3-ca-file ~/.pgcli/backup/repo-ca/ca-<store-host>-9002.crt

The fetch is trust-on-first-use — the CA is the thing you cannot verify before you hold it — so compare the printed SHA-256 against the store host’s sha256sum ~/.pgcli/tls/minio/<name>/ca.crt (the way you would an SSH host key) before trusting it. One setup then republishes the CA into the Patroni cluster’s etcd registry, so every other cluster host gets it with no manual step at all (see Backup → S3 object storage repository). Copying ca.crt over by hand still works as the fallback, and is what you must do if the store host runs a pgcli from before chain distribution (the fetch-ca error says so, and re-running pg addon install minio --tls there upgrades it without a restart). A pgBackRest stanza never requires the MinIO to share its host.

Bring your own certificate (--tls-cert / --tls-key)

--tls only ever serves pgcli’s own self-signed pair. To serve a certificate you already hold — one signed by a public CA for a real domain, or one issued by your private CA — pass --tls-cert <leaf(+chain).pem> --tls-key <key.pem> instead. Together with --tls (which it implies, so you don’t need to also pass it), this replaces the generated certs: the two files are mounted read-only straight at the names MinIO’s --certs-dir requires (/opt/minio/certs/public.crt, /opt/minio/certs/private.key), and pgcli never copies or re-signs them — the private key stays in exactly the one place you put it.

pg addon install minio --name store \
  --tls-cert /etc/ssl/wildcard.hi.163.com.crt \
  --tls-key  /etc/ssl/wildcard.hi.163.com.key

What install checks, and what it does not. --tls-cert/--tls-key are paired through Go’s own TLS loader before anything starts, so a key that doesn’t match the cert, a cert that is really just a CA certificate, an expired cert, or one restricted to a use other than server auth all fail the install outright rather than surfacing later as a crash-looping container. Whether the cert’s SANs cover the configured --listen address is checked too, but only warned about — a domain cert is routinely dialed through a name behind DNS or a load balancer that has nothing to do with the host it runs on, so a mismatch here is informational, not fatal.

Renewing. Replace the cert/key files and recreate the container:

pg addon install minio --name store --tls-cert <new.crt> --tls-key <new.key> --force

Recreate is required, not optional: with the files bind-mounted read-only into the container, a running MinIO does not pick up a replaced certificate — not a new file moved over the old one (that swaps the inode a single-file mount pins), and not even an in-place rewrite of the same file. Verified: after overwriting the host file the container keeps serving the old pair until --force recreates it. pgcli never re-signs a BYO certificate — the operator owns its lifetime — so there is no automatic refresh to rely on.

pg addon start/stop re-validate the pair on every start but never regenerate it (there is nothing to regenerate — pgcli does not own your certificate’s lifetime). A start-time validation failure is reported as a warning, not a blocker, matching the existing tolerance around a cert that is very likely still fine: the fix is always to correct the files (or swap to a new pair) and recreate with --force.

Clients. With a certificate from a publicly-trusted CA, S3 clients (the mc family, aws CLI, pgBackRest via repo*-s3-ca-file) need no extra trust material at all — the chain is already rooted in a CA they trust. For any other cert you hand the client the trust anchor as backup.repo.s3.ca_file / --s3-ca-file: the issuing CA (or the full leaf+intermediate bundle) for a private-CA cert, or — since a self-signed cert is its own anchor — the served .crt itself when you minted it with pg cert. A remote host that never saw that .crt can pull it out of the TLS handshake automatically instead of an scp — fetch-ca recognizes a self-signed leaf and saves it for you:

pg backup fetch-ca <store-host>:9010
#   [OK] CA fetched from <store-host>:9010
#        saved:    ~/.pgcli/backup/repo-ca/ca-<store-host>-9010.crt
#        SHA-256:  d4df…81e2
pg backup setup --s3-ca-file ~/.pgcli/backup/repo-ca/ca-<store-host>-9010.crt

Cross-check the SHA-256 against the store host’s sha256sum of the .crt you gave --tls-cert (trust-on-first-use, same as the generated-CA fetch above). For a public-CA cert the command stays pointless — there is nothing to fetch that isn’t already trusted.

Turning BYO off. There is no --tls-cert=/off flag — the config merge across re-runs is one-way, matching how --tls itself works. To go back to generated certs, remove cert_file/key_file under this addon in pg.yaml and recreate with pg addon install minio --name store --force.

Generating a test certificate: pg cert

You don’t need a real CA to try BYO — pg cert mints a self-signed cert whose SANs cover any mix of DNS names and IPs you ask for:

# A leaf cert valid for a hostname AND two IPs, ECDSA P-256 (the default),
# 825 days (the default validity) — the shape most BYO installs want:
pg cert --host "minio.test,127.0.0.1,10.0.0.9" \
  --cert-file minio.crt --key-file minio.key

# Then serve it:
pg addon install minio --name store --tls-cert minio.crt --tls-key minio.key

Nothing it generates touches pg.yaml or a container — it just writes two PEM files wherever you point it and prints the SANs. Flags:

The same generator is also built as a standalone binary for use outside pg — make gencert → bin/gencert, with the identical flag set in single-dash form (-host, -cert-file, …). Both front the one internal/certgen package, so their output is byte-for-byte the same kind of certificate.

What pg cert writes is one self-signed leaf, not a chain. The PEM in --cert-file holds exactly one CERTIFICATE block — the cert signs itself (IsCA: false, serverAuth EKU, your SANs). It is deliberately not a leaf+intermediate+root bundle: there is no issuing CA above it, so there is nothing to chain, and ValidateBYOCert only inspects the first certificate in the file and rejects one that is a CA (leaf.IsCA), which is precisely why the --ca output is not something to point --tls-cert at — it is the trust anchor itself, not a server cert.

Since the result is self-signed, treat the generated minio.crt exactly like pgcli’s own --tls-generated ca.crt on the client side — the served leaf is its own trust anchor, so point backup.repo.s3.ca_file / pg backup setup --s3-ca-file at the same file (see Backup → S3 Object Storage Repository). This is not a workaround pgBackRest merely tolerates: OpenSSL’s trust store treats whatever you hand it via -CAfile/SSL_CTX as an anchor, CA:TRUE not required, and pgBackRest’s S3 TLS path (curl over OpenSSL) is that same mechanism — verified directly: openssl verify -CAfile <pg cert's cert> <pg cert's cert> on the self-signed, CA:FALSE leaf returns OK.

Deployment modes

MinIO/silo classifies its layouts; the addon supports all four:

SNSD is what pg addon install minio gives you out of the box. To get MNSD, pass the cluster’s endpoint list (at least four nodes) — see Distributed / Cluster Mode below.

MNMD combines both flags: give this node’s drives with --drive (as under SNMD) and the whole cluster’s host×drive endpoint matrix with --endpoint (one URL per drive on every node) — see Multi-Node Multi-Drive (MNMD) below. Disk redundancy under SNSD/MNSD is also available the other way — a ZFS pool under --data-dir — which keeps the layout changeable underneath and can be more space-efficient. S3 Storage High Availability compares native MNMD with the ZFS approach for the 4-hosts-each-with-several-disks hybrid that survives both a disk and a node.

Single-Node Multi-Drive (SNMD)

One MinIO process, several host directories, erasure-coded across them. Pass --drive once per drive instead of --data-dir:

pg addon install minio --name store \
  --drive /mnt/minio/disk1 --drive /mnt/minio/disk2 \
  --drive /mnt/minio/disk3 --drive /mnt/minio/disk4 --tls

Each --drive is a host directory on its own device; drive N is bind-mounted at container path /dataN and the server starts as minio server /data1 /data2 ... /dataN. --drive is mutually exclusive with --data-dir — multi-drive mode takes its data locations from --drive only. Combining --drive with --endpoint is MNMD, the multi-node version of this mode — see Multi-Node Multi-Drive (MNMD) below. Like every other MinIO drive, one that shares the host’s root device is rejected by MinIO at startup; pgcli lists the offending drives and warns at install time.

What EC buys you (measured on a live 4-drive set): MinIO splits each object into data + parity shards, and a 4-drive set defaults to 2 parity shards — it tolerates 2 drive failures. With 1 drive down, reads and writes both continue; with 2 down, reads still succeed and writes are refused — that is the quorum boundary, and it is the same arithmetic MNSD uses, just over drives instead of nodes. Usable capacity is roughly half the raw total (4 × 2 GiB drives → 3.6 GiB usable reported by mc admin info, EC:2). A drive that returns is healed by MinIO itself; pgcli does not need to do anything. The parity default scales with drive count — only the 4-drive shape is tested here.

pg addon remove minio --name store --clean-data deletes each drive directory — but refuses any drive that is still a mount point, so an accidental --clean-data can never rm -rf through a live mount into the disk below. Unmount first if the data below is really disposable.

Distributed / Cluster Mode

MinIO’s erasure-coded (EC) cluster mode is available too. It requires at least four distinct host:port endpoints and, unlike the other addons, no central coordinator: every node runs its own pgcli with its own pg.yaml, and every one of those pg.yaml files carries the same full endpoint list and the same root credentials. pgcli only ever starts the container for the node it is running on — MinIO itself does the handshake to form the ring across hosts.

The endpoint list is the mode switch: an empty list is single-node (unchanged), a non-empty list starts minio server <ep1> <ep2> ... in distributed mode.

Each endpoint is http://<host>:<port><path>. The host:port is how the nodes reach each other to form the ring. The trailing <path> is not an HTTP route — clients never see it — it is the export path, the directory inside the container where that node keeps its own slice of the erasure-coded data. Under MNSD (no --drive) pgcli bind-mounts the --data-dir at /data, so the path must start at /data — the simplest choice is literally /data. Keep the same path string across all four endpoints. (Under MNMD a node also passes --drive, and each endpoint instead names one of that node’s /data1../dataN drive slots — see MNMD below.)

Note the endpoint path is a container path, independent of the host layout: if /data is a shared disk and you want to keep other things on it, point --data-dir at a subdirectory of it — --data-dir /data/minio stores this cluster’s data under the host’s /data/minio while the endpoints stay http://<host>:9000/data (the endpoint cannot name that subdirectory; it sees the mount root). Only if you deliberately extend into the volume with a path below /data (e.g. /data/mystore) does the data land one level deeper on the host: <data-dir>/mystore — avoid pairing --data-dir /data/minio with endpoint .../data/minio, which nests as /data/minio/minio.

# on node 1 (10.0.0.11), with a dedicated data disk mounted at /data:
pg addon install minio --name store \
  --listen 10.0.0.11 \
  --data-dir /data \
  --root-password '<shared-secret>' \
  --endpoint http://10.0.0.11:9000/data \
  --endpoint http://10.0.0.12:9000/data \
  --endpoint http://10.0.0.20:9000/data \
  --endpoint http://10.0.0.21:9000/data

# nodes 2-4: same command, own --listen, and the SAME --endpoint list AND the
# SAME --root-password value.

--root-password is optional: omit it and the first install generates one (printed once, stored in pg.yaml). In cluster mode, pass the same value on every node so the stored credential is identical across the cluster without any manual copy.

The install summary lists the members and reminds you of the consistency requirement:

✓ minio installed: "store"
  ...
  Distributed mode: 4 endpoints
    - http://10.0.0.11:9000/data
    - http://10.0.0.12:9000/data
    - http://10.0.0.20:9000/data
    - http://10.0.0.21:9000/data
  NOTE: every node's pg.yaml must carry the identical endpoint list AND
  identical root credentials, or the cluster will not form.

Three things are strict in cluster mode, each learned the hard way:

• Endpoints must be routable, distinct hosts. Same-host folds into single-node-multi-drive and is rejected (use path style endpoint for single node setup); the 127.0.0.0/8 loopback range is rejected outright (resolves to localhost). Use each node’s real LAN address.
• The data directory must be on a disk separate from the root filesystem. MinIO refuses a drive that shares the OS disk (drive is part of root drive, will not be used). pgcli detects this with a stat of the data dir versus / and warns at install time when --data-dir is on the root device — the warning is advisory; point --data-dir at a mounted data disk.
• MINIO_SERVER_URL is not set in cluster mode. It would differ per node (each advertises its own IP), and MinIO requires every MINIO_* env var to be byte-identical across nodes, so a per-node MINIO_SERVER_URL makes each node loop on Waiting for at least 1 remote servers with valid configuration. The endpoint list defines the addresses instead. Single-node still sets MINIO_SERVER_URL from listen.

Quorum (EC): writes need ⌈N/2⌉+1 nodes up, reads need ⌈N/2⌉. A 4-node set therefore keeps serving reads with 2 nodes down but rejects writes; restart the downed nodes and the cluster self-heals. Under MNMD the members being counted are drives, not nodes — see MNMD for the measured drive-level boundary on a live 4×4 set.

Cross-host only. This is a genuinely distributed deployment — you run pgcli on each host yourself. pgcli does not SSH between nodes or register members; keeping the N pg.yaml files consistent is the operator’s job.

Multi-Node Multi-Drive (MNMD)

MNMD is MNSD with several drives per node instead of one: MinIO erasure-codes across the whole host×drive matrix, so the set survives losing individual disks and whole nodes without any filesystem layer. Give this node’s drives with --drive (exactly as under SNMD) and the entire cluster’s endpoint list with --endpoint — one URL per drive on every node, not just this one.

The one difference from MNSD is the export path. Each node bind-mounts its drives at /data1../dataN, so each endpoint must address one of those slots: http://<host>:<port>/data<k>. pgcli does not parse the URLs — it hands the matrix to minio server verbatim, the same way MinIO documents it — so every node carries the identical full list. The list must be a multiple of the per-node drive count (MinIO requires every node contribute the same number of drives) and must name at least one remote node; a matrix that folds onto a single host is rejected before the container starts, as is a --tls node whose endpoints are plaintext http://.

Serving the grid over TLS — one certificate, not a CA per node. Let each node’s bare --tls run and every host mints its own self-signed CA, so the first cross-node handshake dies with x509: certificate signed by unknown authority and you are left seeding one CA into every node’s cert store by hand. The clean path is a single certificate shared by the whole grid: pg cert writes one self-signed leaf whose SANs cover every node address, you copy that one .crt/.key pair to all nodes byte-identically, and serve it with --tls-cert / --tls-key (there is no per-node CA to reconcile). This is the recommended TLS setup — pgBackRest forces HTTPS for an S3 repo anyway, so a backup store is a TLS grid. The endpoint matrix must then be all https://, and every address a node dials must be in the leaf’s SANs. (The same shared leaf serves an MNSD grid just as well — the handshake is between the same servers.)

# once, anywhere — one leaf for the whole grid (list every node address; add any
# VIP or hostname clients will dial):
pg cert --host 10.0.0.11,10.0.0.12,10.0.0.20,10.0.0.21 \
        --cert-file grid.crt --key-file grid.key
# then copy grid.crt + grid.key to every node — identical files everywhere

# on node 1 (10.0.0.11), four data disks mounted and passed with --drive:
pg addon install minio --name store \
  --tls-cert grid.crt --tls-key grid.key \
  --listen 0.0.0.0 \
  --drive /mnt/minio/disk1 --drive /mnt/minio/disk2 \
  --drive /mnt/minio/disk3 --drive /mnt/minio/disk4 \
  --root-password '<shared-secret>' \
  --endpoint https://10.0.0.11:9000/data1 --endpoint https://10.0.0.11:9000/data2 \
  --endpoint https://10.0.0.11:9000/data3 --endpoint https://10.0.0.11:9000/data4 \
  --endpoint https://10.0.0.12:9000/data1 --endpoint https://10.0.0.12:9000/data2 \
  --endpoint https://10.0.0.12:9000/data3 --endpoint https://10.0.0.12:9000/data4 \
  --endpoint https://10.0.0.20:9000/data1 --endpoint https://10.0.0.20:9000/data2 \
  --endpoint https://10.0.0.20:9000/data3 --endpoint https://10.0.0.20:9000/data4 \
  --endpoint https://10.0.0.21:9000/data1 --endpoint https://10.0.0.21:9000/data2 \
  --endpoint https://10.0.0.21:9000/data3 --endpoint https://10.0.0.21:9000/data4

# nodes 2-4: same command, the SAME grid.crt/grid.key pair, own --drive paths,
# and the SAME 16-endpoint matrix AND the SAME --root-password value.

The --listen 0.0.0.0 is deliberate: each node must answer on the address the other 15 endpoints name. --root-password must be identical everywhere, exactly as under MNSD.

On a live 4-node × 4-drive set (16 × 2 GiB) — formed over exactly this shared-pg cert TLS setup, the ring coming up with Network: 4/4 OK and no per-node CA to reconcile — the measured shape, via mc admin info:

• 16 drives online, EC:4 in a single erasure set of stripe size 16 — the set reports 4 parity shards, so losing 4 drives of 16 stays healthy.
• Losing one whole node (4 drives → 12/16 online) keeps reads and writes working; a 64 MiB upload/download round-trips byte-identical.
• Losing a second node (8/16 online): writes are refused (Resource requested is unwritable) and reads fail too — the set is no longer safe to serve. Restart the nodes and it self-heals back to 16/16.
• Usable capacity follows the parity ratio: EC:4 over 16 drives keeps 12/16 of raw bytes. --clean-data deletes each drive dir but refuses one still mounted, and a drive that returns is healed by MinIO itself.

The pg.yaml files on all nodes carry the identical 16-line matrix and one shared root password; keeping them consistent is, as under MNSD, the operator’s job.

Using the mc Client

pg mc runs MinIO’s own mc client from a throwaway container — no local install, no manual podman run invocation:

# pick the URL for your platform — see "Endpoints by platform" below:
pg mc alias set store http://127.0.0.1:9000 admin <password>    # Linux, local addon
pg mc alias set store http://host.containers.internal:9000 admin <password>  # macOS, local addon
pg mc alias set store http://10.0.5.7:9000 admin <password>     # either, remote store
pg mc mb store/backups
pg mc ls store
pg mc cp ./dump.pglz store/backups/

Aliases persist on the host at ~/.mc/config.json — mc’s own default path, the same on Linux and macOS — so alias set once and every later pg mc invocation (and a native mc install, on Linux) sees the same aliases. pg mc alias list reads them back and pg mc alias remove drops one. alias set validates the credentials against the endpoint before writing, so a wrong password leaves ~/.mc/config.json untouched rather than storing a dead alias. Under the hood pg mc mounts your ~/.mc into the container and runs ghcr.io/mars-base/pgcli/pgcli-mc (the static upstream binary on scratch, pulled on first use); nothing else about it is magic.

Local file operands of cp / mirror / diff work too: pg mc resolves each one (like realpath) and mounts that exact absolute path into the container, so

pg mc cp ./dump.pglz store/backups/        # upload a file from the cwd
pg mc cp store/backups/dump.pglz ./        # download into the cwd
pg mc mirror ./repo store/backups/         # or mirror a whole tree

just does what it looks like. A download target that does not exist yet (a new filename, or a ./newdir/ not created) is handled too — its nearest existing parent directory is what gets mounted, so the file mc writes lands on the host. On macOS the path must live under your home directory — that is the only tree the podman machine VM shares; pg mc says so and skips any mount outside it.

Any mc flag that pg’s own flag parser would reject goes after --, exactly like pg etcdctl — this covers essentially every mc long flag, since none are registered on pg: --all, --json, --recursive, --force, --newer-than, and so on. Put them all after --:

pg mc ls store -- --all
pg mc cp ./dir store/backups/ -- --recursive
pg mc rm store/old -- --recursive --force

Passing one before -- fails in pg, not in mc, with a message like unknown flag: --recursive — a giveaway that the separator is missing.

One caveat from native mc is worth knowing, because it interacts with the local mounts above: in a file command (cp / mirror / diff) an operand whose first segment is not a known alias is treated as a local path. A typo in an alias name therefore uploads or downloads to a local directory named after the typo instead of erroring — pg mc faithfully mounts what mc decides to read, so the behaviour is native, not a pg mc quirk. Confirm the alias with pg mc alias list before a file operation.

MC_HOST_<name> environment aliases — mc’s stateless form, no config file touched — work through pg mc too, since the variable is forwarded into the container:

MC_HOST_store="http://admin:<password>@127.0.0.1:9000" pg mc ls store

Endpoints by platform: on Linux pg mc runs on the host network, so a 127.0.0.1 alias reaches a local addon instance directly. On macOS the container sits on the bridge network, where 127.0.0.1 is the container’s own loopback — point an alias at a local Mac addon via http://admin:<password>@host.containers.internal:9000, or at a remote store via its routable address. pg mc itself works identically on both.

A Mac browser reaches the console on 127.0.0.1:<console-port> (the addon publishes it); that is host-side and unrelated to what the pg mc container can address.

Common commands

The same root credentials work for any S3 SDK — including pgBackRest against a repo1-type=s3 repository.

Ports

Each instance takes two consecutive ports from one pool, starting at minio_start_port (default 9000): the S3 API first, the console second. Like every other auto-assigned pool, it skips ports in use and the explicit ports of sibling instances:

pg addon install minio --name store    # 9000 / 9001
pg addon install minio --name archive  # 9002 / 9003

Configuration

Instances live under the top-level addons.minio map in pg.yaml, keyed by instance name:

namespace: default
minio_start_port: 9000
addons:
  minio:
    store:
      container_name: pgcli-minio-default-store
      name: store
      image_tag: ghcr.io/mars-base/pgcli/pgcli-minio:20250422221226
      # data_dir: /srv/minio     # omit for <base-dir>/addon/minio/store/data
      # drives:                  # multi-drive (SNMD/MNMD): one host dir per drive,
      #   - /mnt/minio/disk1     # each mounted at its own /dataN — see SNMD/MNMD below
      #   - /mnt/minio/disk2
      listen: 127.0.0.1
      api_port: 9000
      console_port: 9001
      root_user: admin
      root_password: <generated>   # written on first install
      autostart: false             # pg autostart enable --minio --name store
      # tls: true                  # serve HTTPS (self-signed CA, or BYO below)
      # cert_file: /etc/ssl/minio.test.crt   # BYO leaf(+chain), implies tls; see "Bring your own certificate"
      # key_file:  /etc/ssl/minio.test.key   # BYO private key, must pair with cert_file
      # endpoints:                 # omit for single-node; see "Distributed / Cluster Mode"
      #   - http://10.0.0.11:9000/data
      #   - http://10.0.0.12:9000/data
      #   - http://10.0.0.20:9000/data
      #   - http://10.0.0.21:9000/data
      #   (with `drives` also set this is MNMD — one /dataN endpoint per drive on
      #    every node, and every node's list identical; see "Multi-Node Multi-Drive")

Edits to listen, ports, root_user, root_password, image_tag, data_dir, tls/cert_file/key_file, or endpoints take effect after the next pg addon install minio --name store --force — the plain install skips a still-present container, --force recreates it (the data directory is never touched).

List

pg addon list

Infra add-ons (minio):
  minio (name: store)
    Status:      running
    Listen:      127.0.0.1
    API port:    9000
    Console port: 9001
    Console URL: http://127.0.0.1:9001/
    Data:        ~/pg/addon/minio/store/data
    Root user:   admin
    Image:       ghcr.io/mars-base/pgcli/pgcli-minio:20250422221226
    Container:   pgcli-minio-default-store

pg addon list never prints the password — read it from pg.yaml.

Start and stop

After a host reboot, bring an instance back without re-applying config:

pg addon start minio --name store
pg addon stop  minio --name store

install skips a still-present container (starting it if stopped); start only starts an existing one (and self-heals an improper state by recreating it from the config).

Auto-start on Boot

Containers carry a --restart unless-stopped policy (crashes, not reboots). To bring instances up after a host reboot:

pg autostart enable --minio --name store

This sets autostart: true and installs/refreshes the boot service (see Auto-start on Boot). Boot is start-only. MinIO is independent of the PostgreSQL stack, so it is started last; there is no ordering constraint. pg autostart status lists every target’s state.

Remove

pg addon remove minio --name store            # container gone, data kept
pg addon remove minio --name store --clean-data   # also delete the data directory

The data directory is the object storage — losing it means losing every bucket in it — so remove keeps it by default and says where it is. --clean-data deletes it (and prunes the now-empty default-layout parent directory; a data_dir override and its parent are never touched).

Logs

pg logs addon minio --name store      # last 50 lines
pg logs addon minio --name store -f   # follow

MinIO logs to stdout: startup lines (API:/Console: addresses, Documentation:) and request errors. An API: http://... block confirms the listeners came up.

Troubleshooting

• pulling minio image ... : ... on install. The public tag ghcr.io/mars-base/pgcli/pgcli-minio:20250422221226 is not reachable — check network/registry access, or pre-pull it with podman pull.
• Port collision on manual --api-port. Pick ports outside the auto pool (minio_start_port and up) or the auto-assigner will treat them as taken; also remember MinIO needs a consecutive pair.
• pg addon start after host reboot does nothing / fails. Check pg logs addon minio --name store -f — most often the data directory was deleted (with --clean-data or by hand) and MinIO refuses to start on an empty dir it previously formatted, or the bind port moved.
• Console reachable but S3 clients time out. The MINIO_SERVER_URL is built from listen + API port; if you serve on 127.0.0.1 but access from another host, clients get redirected to the loopback URL. Set listen to the address clients can actually reach. (Cluster mode does not set MINIO_SERVER_URL at all — see Distributed / Cluster Mode.)
• Cluster stuck on Waiting for at least 1 remote servers with valid configuration. The nodes disagree. Check that every pg.yaml has the byte-identical endpoints list and root_password (podman inspect pgcli-minio-<ns>-<name> --format '{{json .Config.Env}}' on each node), and that no per-node MINIO_SERVER_URL crept back in.
• macOS. Supported: the addon serves on the pgcli-net bridge with both ports published, so the Mac’s 127.0.0.1:<port> reaches them (the container binds 0.0.0.0 internally and MINIO_SERVER_URL advertises the loopback the Mac uses). After changing ports or credentials, --force recreate the container. For pg mc, the container cannot use a 127.0.0.1 alias — see the endpoint note above.