RocketMQ

pg addon install rocketmq runs Apache RocketMQ 5.x as a managed addon from the official docker.io/apache/rocketmq image. Like Kafka, ClickHouse and RabbitMQ it is a top-level addon that does not front a PostgreSQL instance.

Each instance is three containers on the host network: a NameServer, a broker and a proxy. The same shape is used in both modes, so ports, commands, logs and lifecycle behave identically:

• Standalone — one NameServer with no controller, one async master broker with no replicas, and a proxy. This is the quickest way to send and receive messages. The 4.x master/slave and DLedger broker deployment modes are not used.
• Multi-host cluster — every NameServer embeds a DLedger (Raft) controller, the brokers run in controller mode, and one broker is elected master while another takes over automatically when it fails. The proxy is the client entry point. This is RocketMQ 5.x’s auto-failover deployment mode.

There is no authentication and no TLS. RocketMQ listens on host networking, and cluster ports are fixed. Anyone who can reach the NameServer, broker or proxy ports can read and write every topic. Run it only on a trusted private network or behind a firewall that restricts 9876, 9877, 10911, 10912, 8080 and 8081. There is also no web UI and no ACL in this addon.

Parameters

--image, --listen, --tls, --tls-pem, --tls-cert, --tls-key, --node-id, --cluster-id and --join do not apply to RocketMQ and are rejected. Use --version to select the image.

Install a standalone instance

pg addon install rocketmq --name rmq

The install pulls the image if needed, renders the configs, creates the directories, starts namesrv → broker → proxy, and waits for the NameServer, broker registration and both proxy listeners:

✓ RocketMQ "rmq" installed
  Version:     5.5.0
  Image:       docker.io/apache/rocketmq:5.5.0
  Containers:  pgcli-rocketmq-rmq-namesrv, pgcli-rocketmq-rmq-broker, pgcli-rocketmq-rmq-proxy
  Namesrv:     127.0.0.1:9876
  Broker:      127.0.0.1:10911 (HA 10912, VIP 10909)
  Proxy:       127.0.0.1:8080 (remoting), 127.0.0.1:8081 (gRPC)
  Mode:        single-node (no controller, no replicas)
  JVM heap:    broker 512m, namesrv 256m, proxy 256m
  Data dir:    /home/you/.pgcli/addon/rocketmq/rmq/data
  Config:      /home/you/.pgcli/addon/rocketmq/rmq/config

To let clients on other hosts connect, advertise the host’s LAN address (the proxy config and broker both use it):

pg addon install rocketmq --name rmq --advertise-host 10.0.0.11

A second standalone instance takes the next free port from each pool; it does not reuse the first instance’s ports:

pg addon install rocketmq --name rmq2 --data-dir /srv/rocketmq-data

On a small VM, lower all three heaps to keep the three JVMs inside memory:

pg addon install rocketmq --name rmq \
  --heap 384m --namesrv-heap 192m --proxy-heap 192m

Re-running install without --force is a no-op that reports the current state. With --force, pgcli rewrites namesrv.conf, broker.conf and rmq-proxy.json, recreates the three containers and keeps the broker and controller stores.

Multi-host cluster

Cluster mode uses RocketMQ 5.x’s controller mode: the controller is embedded in every NameServer as a DLedger (Raft) group, and controller-mode brokers elect a master and fail over automatically. Every host runs its own pg addon install rocketmq; pgcli never SSHes to other hosts.

Requirements:

• The same --cluster name on every host.
• A unique --controller-id per host (n0, n1, n2, …).
• That host’s --advertise-host as a concrete non-loopback IPv4 address.
• --peer ID@HOST entries for the other controller members. Pass the full peer set on every host; the local peer is added automatically.
• The fixed cluster ports free on every host: 9876, 9877, 10911, 10912, 8080 and 8081. One clustered RocketMQ instance per host.
• At least 3 members for fault tolerance. The DLedger quorum is the majority, so 2 members tolerate no failure; pgcli prints a warning for a 2-member cluster.

Install the members one host at a time. For three hosts 10.0.0.11, 10.0.0.12 and 10.0.0.13:

# on 10.0.0.11 (controller n0)
pg addon install rocketmq --name rmq --cluster prod --controller-id n0 \
  --advertise-host 10.0.0.11 \
  --peer n1@10.0.0.12 --peer n2@10.0.0.13

# on 10.0.0.12 (controller n1)
pg addon install rocketmq --name rmq --cluster prod --controller-id n1 \
  --advertise-host 10.0.0.12 \
  --peer n0@10.0.0.11 --peer n2@10.0.0.13

# on 10.0.0.13 (controller n2)
pg addon install rocketmq --name rmq --cluster prod --controller-id n2 \
  --advertise-host 10.0.0.13 \
  --peer n0@10.0.0.11 --peer n1@10.0.0.12

The first two installs are expected to finish with a warning such as:

[!] RocketMQ broker is running, but it has not registered with the controller yet
(broker not listed in clusterList) — install the remaining nodes with the same
--cluster and --peer values ...

This is normal: the controller quorum only forms once a majority of NameServers is up. The install still exits successfully and leaves the containers running. After the third member is installed, the controller elects the broker master and the broker registers.

Check every member:

pg addon rocketmq status --name rmq
pg addon rocketmq cluster status --name rmq

status shows the three container states, the controller id, the full peer list and — when the cluster is reachable — how many brokers are registered and how many are masters. cluster status prints the raw clusterList table; the controller’s BID column (labelled BrokerId in older clients) is 0 for the current master and non-zero for replicas:

rocketmq "rmq" — cluster prod: 3 broker(s), 1 master(s)
#Cluster Name  #Broker Name  #BID  #Addr             #Version  #InTPS(LOAD)  #OutTPS(LOAD)
prod           prod          0     10.0.0.13:10911   V5_5_0    0.00(0,0ms)   0.00(0,0ms)
prod           prod          1     10.0.0.11:10911   V5_5_0    0.00(0,0ms)   0.00(0,0ms)
prod           prod          2     10.0.0.12:10911   V5_5_0    0.00(0,0ms)   0.00(0,0ms)

The column layout and row order are produced by mqadmin and may differ slightly between 5.x releases; pgcli only relies on the cluster name and broker id columns.

Failover

When the master broker stops, the controller elects another in-sync replica after a heartbeat timeout. Clients that connect through the proxy or NameServer retry automatically. To watch a failover:

# find the master (BID 0) in cluster status, then on that host:
pg addon stop rocketmq --name rmq

# on any member, poll until another BID 0 appears:
pg addon rocketmq cluster status --name rmq

# start the old broker again; it rejoins as a replica:
pg addon start rocketmq --name rmq

enableElectUncleanMaster=false is rendered, so a replica that is not in the in-sync state set is not elected. If the controller quorum itself loses its majority (for example two of three NameServers are down), no master can be elected and writes stop until the quorum returns. Stopping or removing a member does not rewrite the peer lists on the remaining hosts; pgcli never contacts the other nodes.

Ports

Standalone instances can coexist on one host because every port is drawn from a pool and reserved together with the derived HA/VIP ports. In cluster mode all of these ports are fixed because the controller membership and broker addresses are baked into every node’s config.

Clients

The proxy is the single client entry point in both modes and runs in CLUSTER mode:

• 5.x gRPC SDK (the recommended 5.x client): connect to <advertise-host>:8081.
• Remoting SDK (4.x-style clients): connect to <advertise-host>:8080, or directly to the broker on <advertise-host>:10911.
• NameServer addresses (9876, controller 9877) are internal cluster plumbing. Do not expose them to remote clients.

A standalone instance defaults to 127.0.0.1, so a client on another host needs an install with --advertise-host set to the host’s reachable IP. In cluster mode every member requires a non-loopback IPv4 --advertise-host and the proxy on each host advertises that host.

Topics and messages

Topic auto-creation is enabled (autoCreateTopicEnable=true) for compatibility, but explicit creation is recommended so queue counts are deterministic:

pg addon rocketmq topic create surge --queues 3 --name rmq
pg addon rocketmq topic list --name rmq
pg addon rocketmq topic describe surge --name rmq

topic create creates the topic across the cluster (-c <cluster-or-name>) with --queues read and write queues (default 3). topic list hides RocketMQ’s internal topics (%RETRY%…, %DLQ%…, TBW102, SELF_TEST_TOPIC, transaction/schedule/trace topics, the 5.x rmq_sys_* / RMQ_SYS_* topics, *_REPLY_TOPIC and the topic named after the cluster); pass --system to show everything. topic describe prints the per-broker queue and offset table from topicStatus.

Send a message and look it up by key:

pg addon rocketmq send surge "hello" --key k1 --tags demo --name rmq

pg addon rocketmq shell --service namesrv --name rmq -- \
  sh mqadmin queryMsgByKey -t surge -k k1 -n 127.0.0.1:9876

send prints the sendMessage result, including the message id, send status and queue. Pull messages back with a bounded debug consumer:

pg addon rocketmq consume surge --count 10 --timeout 30s --name rmq

# one queue, from the start
pg addon rocketmq consume surge --queue 0 --offset 0 --count 50 --name rmq

# messages in a time window (epoch milliseconds or yyyy-MM-dd#HH:mm:ss:SSS)
pg addon rocketmq consume surge --begin 1793264528000 --end 1793264535000 \
  --count 0 --name rmq

consume runs mqadmin consumeMessage in a throwaway container and prints messages to the terminal. It drains the topic’s queues and exits; --timeout (default 30s) is a hard safety limit enforced inside the container, and Ctrl-C stops early. It is a pull-only debug viewer: it does not delete messages and does not commit consumer offsets.

--count is mqadmin’s message budget (default 10); mqadmin accounts it per queue, so a run can print slightly fewer messages than requested. --count 0 removes the budget. --queue/--offset read one queue and use the instance’s broker name (--broker overrides it). --begin/--end accept epoch milliseconds or yyyy-MM-dd#HH:mm:ss:SSS; mqadmin converts the boundaries to queue offsets, so the end of the window can include the first message at or after it.

Deleting a topic removes all of its messages and asks for confirmation unless --yes is passed:

pg addon rocketmq topic delete surge --name rmq --yes

Message retention

Consumption never deletes messages. The broker removes whole commitlog segments (the storage unit shared by all topics on that broker) when a segment is older than fileReservedTime hours and the clock reaches one of the deleteWhen hours, or earlier when disk usage exceeds diskMaxUsedSpaceRatio percent. Retention is broker-wide, not per topic.

Show the running broker’s effective values and change them:

pg addon rocketmq retention show --name rmq

pg addon rocketmq retention set --hours 24 --delete-when "04;16" \
  --disk-ratio 80 --name rmq

Defaults are 72 hours, 04 o’clock and 75 percent. set requires at least one of --hours (1–8760), --delete-when (one or more hours 0–23 separated by ;) and --disk-ratio (1–100). It applies the change to the running broker with updateBrokerConfig and writes the same values to pg.yaml and the rendered broker.conf, so the setting survives pg addon stop/start without a recreate. An expired segment is not deleted at the exact hour; the broker removes it during the next scheduled cleanup. Raising the ratio above the current disk usage stops disk-pressure deletions.

Retention is stored per broker. In a cluster, run the command on every member host; each broker cleans only its own commitlog.

Delivery and durability

Standalone. The broker is the only node (brokerId=0, brokerRole=ASYNC_MASTER) with no replicas. Messages are written to the broker store under data/broker, so a graceful stop/start keeps them. The rendered disk flush policy is RocketMQ’s default ASYNC_FLUSH; a host power loss can therefore lose the most recent writes. For stronger durability, render SYNC_FLUSH (at the cost of throughput):

pg addon install rocketmq --name rmq --force \
  --config-option flushDiskType=SYNC_FLUSH

Cluster. Brokers replicate in controller mode. pgcli renders allAckInSyncStateSet=true and minInSyncReplicas=2 for 3 or more members (1 for a 2-member cluster), so a produce succeeds only when every in-sync replica has accepted the message. Disk flush is still ASYNC by default; use flushDiskType=SYNC_FLUSH when a power loss must not lose acknowledged writes across all replicas.

Replication protects against a broker failure while the controller quorum is healthy. It does not make the controller store disposable: data/controller holds the DLedger state and epoch files. Deleting it, wiping a member, or reinstalling with --clean-data destroys that member’s controller state and can break the quorum. A member that is removed and rejoined must start with a fresh data directory and the same cluster/controller/peer identity.

pg addon remove rocketmq --clean-data deletes the instance directory (configs, broker store, controller store and logs) and a distinct custom data directory. Without --clean-data, the data and config directories are kept.

Configuration, upgrades and immutability

• Mutable with --force: version, JVM heaps, broker.conf overrides and, for standalone instances, the listener ports. --config-option replaces the stored override map, so pass every override again on a forced reinstall.
• Immutable even with --force: in cluster mode --cluster, --controller-id, --advertise-host, --peer and the fixed ports. These values are embedded in every peer’s configuration; changing them in place is rejected.
• Topology changes require a rebuild: standalone ↔ cluster is not an in-place change. Remove the instance with --clean-data and install it again in the desired mode.
• Custom data directory: pointing --data-dir at a new path makes the instance use a new, empty store; pgcli does not migrate existing data.

Managed broker.conf keys (rejected in --config-option) include brokerClusterName, brokerName, brokerId, brokerRole, brokerIP1/brokerIP2, listenPort, haListenPort, namesrvAddr, enableControllerMode, controllerAddr, every controller* key, storePathRootDir, storePathCommitLog, storePathEpochFile, autoCreateTopicEnable, allAckInSyncStateSet, minInSyncReplicas, enableControllerInNamesrv, enableElectUncleanMaster and notifyBrokerRoleChanged.

Lifecycle, logs and autostart

pg addon start rocketmq --name rmq
pg addon stop rocketmq --name rmq
pg addon remove rocketmq --name rmq --clean-data

pg logs addon rocketmq --name rmq --service broker -f
pg logs addon rocketmq --name rmq --service namesrv
pg logs addon rocketmq --name rmq --service proxy

Stopping walks the containers in reverse (proxy → broker → namesrv); starting walks them forward (namesrv → broker → proxy). In cluster mode, start does not wait for the full controller quorum: when a majority is not yet reachable the broker stays in its waiting state and the command still succeeds. Check pg addon rocketmq status and cluster status after the other members are up.

Autostart is available like the other top-level addons:

pg autostart enable --rocketmq --name rmq
pg autostart status

It re-renders the three configs and starts the containers in order. For a cluster member, autostart does not wait for the controller quorum, so members can boot in any order. pg addon start (the global autostart starter) behaves the same way.

Limitations

• No authentication, TLS or ACL. Restrict the network; the proxy and NameServer are open to anyone who can reach them.
• No web UI. Use pg addon rocketmq status, cluster status, topic commands and the logs.
• No online scaling or rolling upgrade. Cluster membership and fixed ports are immutable; member changes require a planned rebuild. The controller store must not be deleted while the member participates in a quorum.
• 2-member clusters have no fault tolerance. Use 3 or more members for production.
• Linux only. The addon relies on host networking and the official image’s uid/gid 3000.
• No message migration, consumer-group management or broker/storage rebalancing. Use the standard RocketMQ administration tools through pg addon rocketmq shell.