Cluster configuration
This page lists every setting that shapes a cluster node and its replication,
with its default, its name in hotkv.toml and whether CONFIG SET can change
it. Before the tables, it says which few settings you will actually touch.
Settings for TLS, memory and the rest are in the
configuration reference.
How to give a setting
Section titled “How to give a setting”A setting can be given in the configuration file (hotkv.conf, or the same
settings in hotkv.toml), on the command line as --setting value, or at
runtime with CONFIG SET. The command line wins over the file. A node reads the
file when it starts. A runtime change is lost at the next restart unless you also
change the file, or run CONFIG REWRITE, which writes the running values back.
For example, hotkv /etc/hotkv/7000.conf --cluster-node-timeout 20000 starts a
node with that timeout whatever its file says, and CONFIG SET cluster-node-timeout 20000 changes it on a running node.
A CONFIG SET that cannot change a setting says why. These are the replies of
a running cluster node:
127.0.0.1:7950> CONFIG GET cluster-node-timeout1) "cluster-node-timeout"2) "15000"127.0.0.1:7950> CONFIG SET cluster-node-timeout 20000OK127.0.0.1:7950> CONFIG GET cluster-node-timeout1) "cluster-node-timeout"2) "20000"127.0.0.1:7950> CONFIG SET cluster-node-timeout abc(error) ERR CONFIG SET failed (possibly related to argument 'cluster-node-timeout') - argument couldn't be parsed into an integer127.0.0.1:7950> CONFIG SET cluster-enabled no(error) ERR CONFIG SET failed (possibly related to argument 'cluster-enabled') - can't set immutable config127.0.0.1:7950> CONFIG SET cluster-port 17951(error) ERR CONFIG SET failed (possibly related to argument 'cluster-port') - can't set immutable config127.0.0.1:7950> CONFIG SET cluster-databases 2(error) ERR CONFIG SET failed (possibly related to argument 'cluster-databases') - can't set immutable config127.0.0.1:7950> CONFIG SET tls-replication yes(error) ERR CONFIG SET failed (possibly related to argument 'tls-replication') - parameter 'tls-replication' cannot be changed at runtime (requires restart)127.0.0.1:7950> CONFIG SET replicaof "127.0.0.1 7000"(error) ERR CONFIG SET failed (possibly related to argument 'replicaof') - can't set immutable config127.0.0.1:7950> CONFIG SET cluster-preferred-endpoint-type nothing(error) ERR CONFIG SET failed (possibly related to argument 'cluster-preferred-endpoint-type') - argument(s) must be one of the following: ip, hostname, unknown-endpoint“Can’t set immutable config” and “requires restart” mean the setting is read at
startup only: change the file and restart the node. In the tables below, the
CONFIG SET column says Yes or No for each.
In hotkv.toml, a setting sits in a table named for its group, and its key is
the directive’s name without the group prefix and with underscores instead of
dashes: cluster-node-timeout is node_timeout under [cluster], and
tls-cluster is cluster under [tls]. The tables give the exact key for each
setting. A few settings have no TOML key; give those in a .conf file or on the
command line.
What you will actually change
Section titled “What you will actually change”Most clusters run on defaults except for these:
| Setting | Why you change it |
|---|---|
cluster-enabled |
yes on every node. Startup only. |
bind, port |
The addresses and port the node serves on. See Deploy on VMs and cloud. |
requirepass |
The same password on every node. It is also what replicas use to prove themselves to their primary. |
cluster-node-timeout |
How fast a failure is noticed. See Failure detection. |
cluster-announce-ip and the other announce settings |
When clients or nodes must use another address: NAT, containers, names. |
cluster-require-full-coverage |
Whether one lost shard stops the whole cluster. |
cluster-replica-no-failover |
A replica that must never take over by itself, for example one used for reporting. |
cluster-databases |
More than one database on a cluster node. Startup only. |
tls-cluster, tls-replication |
Encrypt the bus and replication. See Cluster over TLS. |
replica-full-sync-buffer-limit |
On the node that imports slots with a Redis 8 style migration: how many bytes of the writes made during the copy it keeps itself. See Scaling a cluster. |
appendonly, appendfsync |
How much a crash can lose. See Backup and restore. |
Replication needs no setting in a cluster: replicas are made with
redis-cli --cluster create or CLUSTER REPLICATE, and they copy from their
primary’s client port.
Per node, or the same everywhere
Section titled “Per node, or the same everywhere”| Per node (differs between nodes) | The same on every node (recommended) |
|---|---|
bind, port, tls-port, cluster-port |
cluster-enabled |
dir, cluster-config-file, logfile, pidfile |
requirepass, and masterauth when you use it |
The announce settings and availability-zone |
tls-cluster, tls-replication, and the CA in tls-ca-cert-file |
cluster-replica-no-failover (it is a replica’s own choice) |
cluster-node-timeout, cluster-require-full-coverage, cluster-allow-* |
| The certificate and key files, the license file | cluster-databases, compat-redis-version |
The TLS settings must match as a whole: a node with tls-cluster and a node
without it do not talk to each other. For the others, a node applies its own
value, so differing values make the nodes behave differently, which makes the
cluster hard to reason about. ACL users are not a setting, but they are per node
too: create them on every node.
Cluster settings
Section titled “Cluster settings”Names with a dash are for hotkv.conf. cluster-databases is read at startup
only. CONFIG GET shows cluster-slot-migration-sync-buffer-drain-timeout and
cluster-slot-migration-max-archived-tasks only when you ask for them by their
exact name, not with a pattern such as cluster-slot-migration-*.
| Directive | Default | CONFIG SET | Description |
|---|---|---|---|
cluster-enabled | no | No | Run as a cluster node. |
cluster-config-file | nodes.conf | No | File where the node keeps its view of the cluster. The node writes it; do not edit it. |
cluster-port | 0 (port + 10000) | No | Port of the cluster bus. |
cluster-node-timeout | 15000 | Yes | How long a node may be unreachable before it is flagged as failing. |
cluster-announce-ip | (empty) | Yes | The address this node gives the other nodes (and clients, unless cluster-announce-client-ipv4 is set), when it differs from the one they see (NAT, containers). An IP address, or a name the other nodes resolve. Empty gives back the first bind address. |
cluster-announce-port | 0 (the client port) | Yes | Client port this node announces. |
cluster-announce-tls-port | 0 (the TLS port) | Yes | TLS client port this node announces. |
cluster-announce-bus-port | 0 (the bus port) | Yes | Cluster bus port this node announces. |
cluster-announce-client-ipv4 | (empty) | Yes | The address clients connected over IPv4 are given for this node in CLUSTER replies and redirects, in place of its own. Nodes keep using the real address among themselves. For a NAT or a load balancer. |
cluster-announce-client-ipv6 | (empty) | Yes | The same as cluster-announce-client-ipv4, for clients connected over IPv6. |
cluster-announce-client-port | 0 (the client port) | Yes | The plain client port clients are given for this node. 0 is the port it listens on. |
cluster-announce-client-tls-port | 0 (the TLS port) | Yes | The TLS client port clients are given for this node. 0 is the TLS port it listens on. |
availability-zone | (empty) | Yes | A name for where this node runs, given to clients by HELLO and INFO and, in a cluster, to the other nodes for CLUSTER SLOTS and CLUSTER SHARDS. Works on any node, cluster or not. At most 255 characters. |
cluster-announce-hostname | (empty) | Yes | Host name clients may be given for this node (see cluster-preferred-endpoint-type). Letters, digits, - and ., shorter than 256 characters. |
cluster-announce-human-nodename | (empty) | Yes | Friendly node name shown in other nodes' logs. |
cluster-preferred-endpoint-type | ip | Yes | What clients are given as a node's endpoint in redirects and CLUSTER SLOTS / CLUSTER SHARDS. |
cluster-require-full-coverage | yes | Yes | Stop serving when any hash slot is not covered by a node. |
cluster-allow-reads-when-down | no | Yes | Keep serving reads while the cluster is down. |
cluster-allow-pubsubshard-when-down | yes | Yes | Keep serving sharded pub/sub while the cluster is down. |
cluster-allow-replica-migration | yes | Yes | Let spare replicas move to primaries that have none. |
cluster-migration-barrier | 1 | Yes | Replicas a primary keeps before one of them may migrate to an orphaned primary. |
cluster-replica-validity-factor | 10 | Yes | A replica whose data is older than this limit does not try to fail over: the node timeout times this factor, plus the replication heartbeat period (repl-ping-replica-period, 10 seconds unless set). 0 turns off the age check only; a replica that never finished a sync never fails over by itself. |
cluster-replica-no-failover | no | Yes | Never start an automatic failover from this replica. CLUSTER FAILOVER still works. |
cluster-link-sendbuf-limit | 256mb | Yes | Unsent bytes a cluster bus link may hold before it is dropped. 0 means no limit. |
cluster-blacklist-ttl | 60 | Yes | How long a node removed with CLUSTER FORGET stays out: what other nodes say about it is ignored, so every node can be told to forget it. 0 keeps it out for no time. |
cluster-config-save-behavior | sync | Yes | What a node does when it cannot write its cluster config file: sync exits, best-effort keeps serving, tries again and sends no failover vote meanwhile. |
cluster-bus-port-protected-mode | no | Yes | With yes a cluster node refuses to start, and refuses CONFIG SET tls-cluster no, without tls-cluster, since only TLS tells a node who is at the other end of a bus link. With no it starts and logs a warning. See Cluster over TLS. |
cluster-manual-failover-timeout | 5000 | Yes | Time CLUSTER FAILOVER has to win its election. The primary holds writes for the same time. A CLUSTER MIGRATESLOTS holds writes for at most twice this. |
cluster-slot-migration-log-max-len | 1000 | Yes | Finished atomic slot migrations CLUSTER GETSLOTMIGRATIONS keeps. |
slot-migration-max-failover-repl-bytes | 0 | Yes | For CLUSTER MIGRATESLOTS: how much a source may still have to send to the target when it pauses writes to hand the slots over. -1 allows any amount. |
cluster-slot-migration-handoff-max-lag-bytes | 1mb | Yes | For CLUSTER MIGRATION: the source pauses writes once the target is within this many bytes of the stream. |
cluster-slot-migration-write-pause-timeout | 10000 | Yes | For CLUSTER MIGRATION: how long the source holds client writes waiting for the target to take the slots. Past it the task fails. |
cluster-slot-migration-sync-buffer-drain-timeout | 60000 | Yes | For CLUSTER MIGRATION: how long a source waits, once asked to pause, for the target to catch up enough. Past it the task fails. Shown by CONFIG GET only by its exact name. |
cluster-slot-migration-max-archived-tasks | 32 | Yes | Finished tasks CLUSTER MIGRATION STATUS keeps. Shown by CONFIG GET only by its exact name. |
cluster-slot-stats-enabled | no | Yes | What CLUSTER SLOT-STATS counts besides keys: yes for all, or any of cpu, net, mem. Memory counting cannot be turned on at runtime. |
cluster-databases | 1 | No | Number of databases of a cluster node (databases is the number for a standalone server). With 1, SELECT of another database, MOVE and COPY ... DB are refused. Fixed at startup. |
Replication settings
Section titled “Replication settings”| Directive | Default | CONFIG SET | Description |
|---|---|---|---|
replicaof | (empty) | No | Start this standalone node as a replica of the node at that host and client port. Not allowed on a cluster node, where replicas are made with CLUSTER REPLICATE. Change it at runtime with REPLICAOF, not CONFIG SET. CONFIG GET shows the primary as host port, and an empty string on a primary. |
replication.enabled | no | No | Turn on replication for this node. |
replication.role | primary | No | Role of this node. |
replication.primary-addr | (empty) | No | On a replica, the primary's address as host:port: its client port, or its replication port when replication.dedicated-port is on. |
replication.dedicated-port | no | No | Also listen for replicas on the replication port (replication.port; a cluster node's client port plus 20000), and have this node's replicas connect there. Without it replicas connect to the primary's client port. |
replication.bind | 0.0.0.0 | No | Address the replication port binds to. Used only with replication.dedicated-port. |
replication.port | 6389 | No | The replication port, used only with replication.dedicated-port. In cluster mode the replication port is always the client port plus 20000 and this setting does not change it. |
replication.password | (empty: masterauth, then requirepass) | No | Shared secret the primary and replicas prove to each other. |
replication.sync-timeout | 5000 | No | Timeout for the replication handshake and sync steps. Also accepted as replication.sync-timeout-ms. |
replication.heartbeat-interval | 1000 | No | Interval between heartbeats on a replication link. Also accepted as replication.heartbeat-interval-ms. repl-ping-replica-period wins when set. |
replication.backlog-size | 64mb | No | Bytes of recent writes kept so a replica that reconnects can catch up without a full sync. The backlog counts toward maxmemory as it fills. When maxmemory is set at start and no size is given, the default is a sixteenth of maxmemory if that is less than 64mb. repl-backlog-size wins when set. |
replication.full-sync-buffer-limit | 1gb | No | Bytes of writes buffered for a replica while its full sync is under way and the writes made meanwhile are being sent to it. 0 uses replication.replica-buffer-limit. |
replication.apply-threads | auto | No | How much parallelism a replica uses to apply the writes of its primary. auto follows the server's threads setting. 0 turns it off. |
replication.replica-buffer-limit | 256mb | No | Bytes of writes buffered for a replica that has not received them yet. 0 means no limit. The replica class of client-output-buffer-limit wins when given. |
replica-read-only | yes | Yes | Refuse writes from clients on a replica. |
repl-backlog-size | (not set: replication.backlog-size) | Yes | Redis name for the replication backlog size, at least 16kb. When set it overrides replication.backlog-size, and the default above does not apply. CONFIG SET resizes the backlog at once. The backlog counts toward maxmemory up to this size. |
replica-full-sync-buffer-limit | 0 (keep none) | Yes | Used by the target of a Redis 8 style atomic slot migration (CLUSTER MIGRATION IMPORT): how many bytes of the writes the source makes while it copies the slots the target keeps for itself, instead of the source holding them until the copy is out. A write that crosses the limit is kept whole. See Scaling a cluster. It does not change how a replica syncs from its primary: that is replication.full-sync-buffer-limit. |
repl-timeout | (not set: five heartbeats, at least 1 s) | Yes | How long a replication link may stay silent before it is treated as dead. |
repl-ping-replica-period | (not set: replication.heartbeat-interval) | Yes | Redis name for the heartbeat interval. When set it overrides replication.heartbeat-interval. |
repl-diskless-load | disabled | Yes | Accepted and reported for compatibility. A full sync always replaces the replica's data as it arrives, whatever this says. |
masterauth | (empty) | Yes | The password a replica logs in with (AUTH) at its primary's client port, and the replication secret when replication.password is not set. CONFIG GET shows it as (hidden). |
masteruser | (empty) | Yes | The ACL user a replica logs in as with masterauth at its primary's client port. That user must be allowed PSYNC. |
min-replicas-to-write | 0 | Yes | A primary refuses writes with NOREPLICAS while fewer than this many replicas have acknowledged within min-replicas-max-lag seconds. 0 turns it off. |
min-replicas-max-lag | 10 | Yes | How recently a replica must have acknowledged to count for min-replicas-to-write. 0 turns the rule off. |
repl-backlog-ttl | 3600 | Yes | How long a primary with no replica keeps its replication backlog before dropping it. 0 keeps it. A replica that comes later gets a full sync. |
propagation-error-behavior | ignore | Yes | What happens when a write that came from the primary (or the AOF) fails on this node. It is always logged. panic stops the process, panic-on-replicas stops a replica only. |
See also
Section titled “See also”- Deploy on VMs and cloud: the settings in use.
- Cluster over TLS: the TLS settings in use.
- Replicas and failover: what the failover settings do.
- Configuration reference: every other setting.
Was this page helpful?
Thanks for the feedback.