# Cluster configuration

> Every cluster and replication setting of a HotKV node, which ones you will change, which change at runtime, and which must match across nodes.

URL: https://docs.hotkv.com/clustering/configuration/


**Availability:** Not on HotKV Cloud. On-Premise: no license needed.

This page lists every setting that shapes a cluster node and its replication,
with its default, its name in `hotkv.toml` and whether `CONFIG SET` can change
it. Before the tables, it says which few settings you will actually touch.
Settings for TLS, memory and the rest are in the
[configuration reference](https://docs.hotkv.com/reference/configuration/index.md).

## How to give a setting

A setting can be given in the configuration file (`hotkv.conf`, or the same
settings in `hotkv.toml`), on the command line as `--setting value`, or at
runtime with `CONFIG SET`. The command line wins over the file. A node reads the
file when it starts. A runtime change is lost at the next restart unless you also
change the file, or run `CONFIG REWRITE`, which writes the running values back.

For example, `hotkv /etc/hotkv/7000.conf --cluster-node-timeout 20000` starts a
node with that timeout whatever its file says, and `CONFIG SET
cluster-node-timeout 20000` changes it on a running node.

A `CONFIG SET` that cannot change a setting says why. These are the replies of
a running cluster node:

```text
127.0.0.1:7950> CONFIG GET cluster-node-timeout
1) "cluster-node-timeout"
2) "15000"
127.0.0.1:7950> CONFIG SET cluster-node-timeout 20000
OK
127.0.0.1:7950> CONFIG GET cluster-node-timeout
1) "cluster-node-timeout"
2) "20000"
127.0.0.1:7950> CONFIG SET cluster-node-timeout abc
(error) ERR CONFIG SET failed (possibly related to argument 'cluster-node-timeout') - argument couldn't be parsed into an integer
127.0.0.1:7950> CONFIG SET cluster-enabled no
(error) ERR CONFIG SET failed (possibly related to argument 'cluster-enabled') - can't set immutable config
127.0.0.1:7950> CONFIG SET cluster-port 17951
(error) ERR CONFIG SET failed (possibly related to argument 'cluster-port') - can't set immutable config
127.0.0.1:7950> CONFIG SET cluster-databases 2
(error) ERR CONFIG SET failed (possibly related to argument 'cluster-databases') - can't set immutable config
127.0.0.1:7950> CONFIG SET tls-replication yes
(error) ERR CONFIG SET failed (possibly related to argument 'tls-replication') - parameter 'tls-replication' cannot be changed at runtime (requires restart)
127.0.0.1:7950> CONFIG SET replicaof "127.0.0.1 7000"
(error) ERR CONFIG SET failed (possibly related to argument 'replicaof') - can't set immutable config
127.0.0.1:7950> CONFIG SET cluster-preferred-endpoint-type nothing
(error) ERR CONFIG SET failed (possibly related to argument 'cluster-preferred-endpoint-type') - argument(s) must be one of the following: ip, hostname, unknown-endpoint
```

"Can't set immutable config" and "requires restart" mean the setting is read at
startup only: change the file and restart the node. In the tables below, the
**CONFIG SET** column says `Yes` or `No` for each.

In `hotkv.toml`, a setting sits in a table named for its group, and its key is
the directive's name without the group prefix and with underscores instead of
dashes: `cluster-node-timeout` is `node_timeout` under `[cluster]`, and
`tls-cluster` is `cluster` under `[tls]`. The tables give the exact key for each
setting. A few settings have no TOML key; give those in a `.conf` file or on the
command line.

## What you will actually change

Most clusters run on defaults except for these:

| Setting | Why you change it |
| --- | --- |
| [`cluster-enabled`](#cluster-enabled) | `yes` on every node. Startup only. |
| `bind`, `port` | The addresses and port the node serves on. See [Deploy on VMs and cloud](https://docs.hotkv.com/clustering/production/index.md). |
| `requirepass` | The same password on every node. It is also what replicas use to prove themselves to their primary. |
| [`cluster-node-timeout`](#cluster-node-timeout) | How fast a failure is noticed. See [Failure detection](https://docs.hotkv.com/clustering/production/index.md#failure-detection). |
| [`cluster-announce-ip`](#cluster-announce-ip) and the other announce settings | When clients or nodes must use another address: NAT, containers, names. |
| [`cluster-require-full-coverage`](#cluster-require-full-coverage) | Whether one lost shard stops the whole cluster. |
| [`cluster-replica-no-failover`](#cluster-replica-no-failover) | A replica that must never take over by itself, for example one used for reporting. |
| [`cluster-databases`](#cluster-databases) | More than one database on a cluster node. Startup only. |
| `tls-cluster`, `tls-replication` | Encrypt the bus and replication. See [Cluster over TLS](https://docs.hotkv.com/clustering/tls/index.md). |
| [`replica-full-sync-buffer-limit`](https://docs.hotkv.com/reference/configuration/index.md#replica-full-sync-buffer-limit) | On the node that imports slots with a Redis 8 style migration: how many bytes of the writes made during the copy it keeps itself. See [Scaling a cluster](https://docs.hotkv.com/clustering/scaling/index.md#writes-made-during-the-copy). |
| `appendonly`, `appendfsync` | How much a crash can lose. See [Backup and restore](https://docs.hotkv.com/clustering/backup-and-restore/index.md). |

Replication needs no setting in a cluster: replicas are made with
`redis-cli --cluster create` or `CLUSTER REPLICATE`, and they copy from their
primary's client port.

## Per node, or the same everywhere

| Per node (differs between nodes) | The same on every node (recommended) |
| --- | --- |
| `bind`, `port`, `tls-port`, `cluster-port` | `cluster-enabled` |
| `dir`, `cluster-config-file`, `logfile`, `pidfile` | `requirepass`, and `masterauth` when you use it |
| The announce settings and `availability-zone` | `tls-cluster`, `tls-replication`, and the CA in `tls-ca-cert-file` |
| `cluster-replica-no-failover` (it is a replica's own choice) | `cluster-node-timeout`, `cluster-require-full-coverage`, `cluster-allow-*` |
| The certificate and key files, the license file | `cluster-databases`, `compat-redis-version` |

The TLS settings must match as a whole: a node with `tls-cluster` and a node
without it do not talk to each other. For the others, a node applies its own
value, so differing values make the nodes behave differently, which makes the
cluster hard to reason about. ACL users are not a setting, but they are per node
too: create them on every node.

## Cluster settings

Names with a dash are for `hotkv.conf`. `cluster-databases` is read at startup
only. `CONFIG GET` shows `cluster-slot-migration-sync-buffer-drain-timeout` and
`cluster-slot-migration-max-archived-tasks` only when you ask for them by their
exact name, not with a pattern such as `cluster-slot-migration-*`.

| Directive | Default | CONFIG SET | Description |
| --- | --- | --- | --- |
| `cluster-enabled` (TOML `[cluster] enabled`; type yes/no) | `no` | No | Run as a cluster node. |
| `cluster-config-file` (TOML `[cluster] config_file`; type path) | `nodes.conf` | No | File where the node keeps its view of the cluster. The node writes it; do not edit it. |
| `cluster-port` (TOML `[cluster] port`; type integer) | `0 (port + 10000)` | No | Port of the cluster bus. |
| `cluster-node-timeout` (TOML `[cluster] node_timeout`; type milliseconds) | `15000` | Yes | How long a node may be unreachable before it is flagged as failing. |
| `cluster-announce-ip` (TOML `[cluster] announce_ip`; type IP address or name) | `(empty)` | Yes | The address this node gives the other nodes (and clients, unless `cluster-announce-client-ipv4` is set), when it differs from the one they see (NAT, containers). An IP address, or a name the other nodes resolve. Empty gives back the first `bind` address. |
| `cluster-announce-port` (TOML `[cluster] announce_port`; type integer) | `0 (the client port)` | Yes | Client port this node announces. |
| `cluster-announce-tls-port` (TOML `[cluster] announce_tls_port`; type integer) | `0 (the TLS port)` | Yes | TLS client port this node announces. |
| `cluster-announce-bus-port` (TOML `[cluster] announce_bus_port`; type integer) | `0 (the bus port)` | Yes | Cluster bus port this node announces. |
| `cluster-announce-client-ipv4` (TOML `[cluster] announce_client_ipv4`; type IPv4 address) | `(empty)` | Yes | The address clients connected over IPv4 are given for this node in `CLUSTER` replies and redirects, in place of its own. Nodes keep using the real address among themselves. For a NAT or a load balancer. |
| `cluster-announce-client-ipv6` (TOML `[cluster] announce_client_ipv6`; type IPv6 address) | `(empty)` | Yes | The same as `cluster-announce-client-ipv4`, for clients connected over IPv6. |
| `cluster-announce-client-port` (TOML `[cluster] announce_client_port`; type integer) | `0 (the client port)` | Yes | The plain client port clients are given for this node. `0` is the port it listens on. |
| `cluster-announce-client-tls-port` (TOML `[cluster] announce_client_tls_port`; type integer) | `0 (the TLS port)` | Yes | The TLS client port clients are given for this node. `0` is the TLS port it listens on. |
| `availability-zone` (TOML `[general] availability_zone`; type string) | `(empty)` | Yes | A name for where this node runs, given to clients by `HELLO` and `INFO` and, in a cluster, to the other nodes for `CLUSTER SLOTS` and `CLUSTER SHARDS`. Works on any node, cluster or not. At most 255 characters. |
| `cluster-announce-hostname` (TOML `[cluster] announce_hostname`; type string) | `(empty)` | Yes | Host name clients may be given for this node (see `cluster-preferred-endpoint-type`). Letters, digits, `-` and `.`, shorter than 256 characters. |
| `cluster-announce-human-nodename` (TOML `[cluster] announce_human_nodename`; type string) | `(empty)` | Yes | Friendly node name shown in other nodes' logs. |
| `cluster-preferred-endpoint-type` (TOML `[cluster] preferred_endpoint_type`; type ip, hostname, unknown-endpoint) | `ip` | Yes | What clients are given as a node's endpoint in redirects and `CLUSTER SLOTS` / `CLUSTER SHARDS`. |
| `cluster-require-full-coverage` (TOML `[cluster] require_full_coverage`; type yes/no) | `yes` | Yes | Stop serving when any hash slot is not covered by a node. |
| `cluster-allow-reads-when-down` (TOML `[cluster] allow_reads_when_down`; type yes/no) | `no` | Yes | Keep serving reads while the cluster is down. |
| `cluster-allow-pubsubshard-when-down` (TOML `[cluster] allow_pubsubshard_when_down`; type yes/no) | `yes` | Yes | Keep serving sharded pub/sub while the cluster is down. |
| `cluster-allow-replica-migration` (TOML `[cluster] allow_replica_migration`; type yes/no) | `yes` | Yes | Let spare replicas move to primaries that have none. |
| `cluster-migration-barrier` (TOML `[cluster] migration_barrier`; type integer) | `1` | Yes | Replicas a primary keeps before one of them may migrate to an orphaned primary. |
| `cluster-replica-validity-factor` (TOML `[cluster] replica_validity_factor`; type integer; alias `cluster-slave-validity-factor`) | `10` | Yes | A replica whose data is older than this limit does not try to fail over: the node timeout times this factor, plus the replication heartbeat period ([`repl-ping-replica-period`](#repl-ping-replica-period), 10 seconds unless set). `0` turns off the age check only; a replica that never finished a sync never fails over by itself. |
| `cluster-replica-no-failover` (TOML `[cluster] replica_no_failover`; type yes/no; alias `cluster-slave-no-failover`) | `no` | Yes | Never start an automatic failover from this replica. `CLUSTER FAILOVER` still works. |
| `cluster-link-sendbuf-limit` (TOML `[cluster] link_sendbuf_limit`; type memory size) | `256mb` | Yes | Unsent bytes a cluster bus link may hold before it is dropped. `0` means no limit. |
| `cluster-blacklist-ttl` (TOML `[cluster] blacklist_ttl`; type seconds) | `60` | Yes | How long a node removed with `CLUSTER FORGET` stays out: what other nodes say about it is ignored, so every node can be told to forget it. `0` keeps it out for no time. |
| `cluster-config-save-behavior` (TOML `[cluster] config_save_behavior`; type sync, best-effort) | `sync` | Yes | What a node does when it cannot write its cluster config file: `sync` exits, `best-effort` keeps serving, tries again and sends no failover vote meanwhile. |
| `cluster-bus-port-protected-mode` (TOML `[cluster] bus_port_protected_mode`; type yes/no) | `no` | Yes | With `yes` a cluster node refuses to start, and refuses `CONFIG SET tls-cluster no`, without `tls-cluster`, since only TLS tells a node who is at the other end of a bus link. With `no` it starts and logs a warning. See [Cluster over TLS](https://docs.hotkv.com/clustering/tls/index.md#bus-protection). |
| `cluster-manual-failover-timeout` (TOML `[cluster] manual_failover_timeout`; type milliseconds) | `5000` | Yes | Time `CLUSTER FAILOVER` has to win its election. The primary holds writes for the same time. A `CLUSTER MIGRATESLOTS` holds writes for at most twice this. |
| `cluster-slot-migration-log-max-len` (TOML `[cluster] slot_migration_log_max_len`; type integer) | `1000` | Yes | Finished atomic slot migrations `CLUSTER GETSLOTMIGRATIONS` keeps. |
| `slot-migration-max-failover-repl-bytes` (TOML `[cluster] slot_migration_max_failover_repl_bytes`; type bytes, or -1) | `0` | Yes | For `CLUSTER MIGRATESLOTS`: how much a source may still have to send to the target when it pauses writes to hand the slots over. `-1` allows any amount. |
| `cluster-slot-migration-handoff-max-lag-bytes` (TOML `[cluster] slot_migration_handoff_max_lag_bytes`; type memory size) | `1mb` | Yes | For `CLUSTER MIGRATION`: the source pauses writes once the target is within this many bytes of the stream. |
| `cluster-slot-migration-write-pause-timeout` (TOML `[cluster] slot_migration_write_pause_timeout`; type milliseconds) | `10000` | Yes | For `CLUSTER MIGRATION`: how long the source holds client writes waiting for the target to take the slots. Past it the task fails. |
| `cluster-slot-migration-sync-buffer-drain-timeout` (TOML `[cluster] slot_migration_sync_buffer_drain_timeout`; type milliseconds) | `60000` | Yes | For `CLUSTER MIGRATION`: how long a source waits, once asked to pause, for the target to catch up enough. Past it the task fails. Shown by `CONFIG GET` only by its exact name. |
| `cluster-slot-migration-max-archived-tasks` (TOML `[cluster] slot_migration_max_archived_tasks`; type integer) | `32` | Yes | Finished tasks `CLUSTER MIGRATION STATUS` keeps. Shown by `CONFIG GET` only by its exact name. |
| `cluster-slot-stats-enabled` (TOML `[cluster] slot_stats_enabled`; type no, yes, or cpu net mem) | `no` | Yes | What `CLUSTER SLOT-STATS` counts besides keys: `yes` for all, or any of `cpu`, `net`, `mem`. Memory counting cannot be turned on at runtime. |
| `cluster-databases` (TOML `[cluster] databases`; type integer, 1 to 128) | `1` | No | Number of databases of a cluster node (`databases` is the number for a standalone server). With `1`, `SELECT` of another database, `MOVE` and `COPY ... DB` are refused. Fixed at startup. |

## Replication settings

| Directive | Default | CONFIG SET | Description |
| --- | --- | --- | --- |
| `replicaof` (TOML not available; type host port; alias `slaveof`) | `(empty)` | No | Start this standalone node as a replica of the node at that host and client port. Not allowed on a cluster node, where replicas are made with `CLUSTER REPLICATE`. Change it at runtime with `REPLICAOF`, not `CONFIG SET`. `CONFIG GET` shows the primary as `host port`, and an empty string on a primary. |
| `replication.enabled` (TOML `[replication] enabled`; type yes/no) | `no` | No | Turn on replication for this node. |
| `replication.role` (TOML `[replication] role`; type primary, replica) | `primary` | No | Role of this node. |
| `replication.primary-addr` (TOML `[replication] primary_addr`; type host:port) | `(empty)` | No | On a replica, the primary's address as `host:port`: its client port, or its replication port when `replication.dedicated-port` is on. |
| `replication.dedicated-port` (TOML `[replication] dedicated_port`; type yes/no) | `no` | No | Also listen for replicas on the replication port (`replication.port`; a cluster node's client port plus 20000), and have this node's replicas connect there. Without it replicas connect to the primary's client port. |
| `replication.bind` (TOML `[replication] bind`; type string) | `0.0.0.0` | No | Address the replication port binds to. Used only with `replication.dedicated-port`. |
| `replication.port` (TOML `[replication] port`; type integer) | `6389` | No | The replication port, used only with `replication.dedicated-port`. In cluster mode the replication port is always the client port plus 20000 and this setting does not change it. |
| `replication.password` (TOML `[replication] password`; type string) | `(empty: masterauth, then requirepass)` | No | Shared secret the primary and replicas prove to each other. |
| `replication.sync-timeout` (TOML `[replication] sync_timeout`; type milliseconds; alias `replication.sync-timeout-ms`) | `5000` | No | Timeout for the replication handshake and sync steps. Also accepted as `replication.sync-timeout-ms`. |
| `replication.heartbeat-interval` (TOML `[replication] heartbeat_interval`; type milliseconds; alias `replication.heartbeat-interval-ms`) | `1000` | No | Interval between heartbeats on a replication link. Also accepted as `replication.heartbeat-interval-ms`. [`repl-ping-replica-period`](#repl-ping-replica-period) wins when set. |
| `replication.backlog-size` (TOML `[replication] backlog_size`; type memory size) | `64mb` | No | Bytes of recent writes kept so a replica that reconnects can catch up without a full sync. The backlog counts toward [`maxmemory`](https://docs.hotkv.com/reference/configuration/index.md#maxmemory) as it fills. When `maxmemory` is set at start and no size is given, the default is a sixteenth of `maxmemory` if that is less than 64mb. [`repl-backlog-size`](#repl-backlog-size) wins when set. |
| `replication.full-sync-buffer-limit` (TOML `[replication] full_sync_buffer_limit`; type memory size) | `1gb` | No | Bytes of writes buffered for a replica while its full sync is under way and the writes made meanwhile are being sent to it. `0` uses [`replication.replica-buffer-limit`](#replication.replica-buffer-limit). |
| `replication.apply-threads` (TOML `[replication] apply_threads`; type integer, or auto) | `auto` | No | How much parallelism a replica uses to apply the writes of its primary. `auto` follows the server's `threads` setting. `0` turns it off. |
| `replication.replica-buffer-limit` (TOML `[replication] replica_buffer_limit`; type memory size) | `256mb` | No | Bytes of writes buffered for a replica that has not received them yet. `0` means no limit. The replica class of [`client-output-buffer-limit`](https://docs.hotkv.com/reference/configuration/index.md#client-output-buffer-limit) wins when given. |
| `replica-read-only` (TOML `[replication] read_only`; type yes/no; alias `slave-read-only`) | `yes` | Yes | Refuse writes from clients on a replica. |
| `repl-backlog-size` (TOML not available; type memory size) | `(not set: replication.backlog-size)` | Yes | Redis name for the replication backlog size, at least 16kb. When set it overrides `replication.backlog-size`, and the default above does not apply. `CONFIG SET` resizes the backlog at once. The backlog counts toward [`maxmemory`](https://docs.hotkv.com/reference/configuration/index.md#maxmemory) up to this size. |
| `replica-full-sync-buffer-limit` (TOML not available; type memory size) | `0 (keep none)` | Yes | Used by the target of a Redis 8 style atomic slot migration (`CLUSTER MIGRATION IMPORT`): how many bytes of the writes the source makes while it copies the slots the target keeps for itself, instead of the source holding them until the copy is out. A write that crosses the limit is kept whole. See [Scaling a cluster](https://docs.hotkv.com/clustering/scaling/index.md#memory-on-the-target). It does not change how a replica syncs from its primary: that is [`replication.full-sync-buffer-limit`](#replication.full-sync-buffer-limit). |
| `repl-timeout` (TOML not available; type seconds) | `(not set: five heartbeats, at least 1 s)` | Yes | How long a replication link may stay silent before it is treated as dead. |
| `repl-ping-replica-period` (TOML not available; type seconds; alias `repl-ping-slave-period`) | `(not set: replication.heartbeat-interval)` | Yes | Redis name for the heartbeat interval. When set it overrides `replication.heartbeat-interval`. |
| `repl-diskless-load` (TOML not available; type disabled, on-empty-db, swapdb, flushdb, flush-before-load) | `disabled` | Yes | Accepted and reported for compatibility. A full sync always replaces the replica's data as it arrives, whatever this says. |
| `masterauth` (TOML not available; type string; alias `primaryauth`) | `(empty)` | Yes | The password a replica logs in with (`AUTH`) at its primary's client port, and the replication secret when `replication.password` is not set. `CONFIG GET` shows it as `(hidden)`. |
| `masteruser` (TOML not available; type string; alias `primaryuser`) | `(empty)` | Yes | The ACL user a replica logs in as with `masterauth` at its primary's client port. That user must be allowed `PSYNC`. |
| `min-replicas-to-write` (TOML not available; type integer; alias `min-slaves-to-write`) | `0` | Yes | A primary refuses writes with `NOREPLICAS` while fewer than this many replicas have acknowledged within `min-replicas-max-lag` seconds. `0` turns it off. |
| `min-replicas-max-lag` (TOML not available; type seconds; alias `min-slaves-max-lag`) | `10` | Yes | How recently a replica must have acknowledged to count for `min-replicas-to-write`. `0` turns the rule off. |
| `repl-backlog-ttl` (TOML not available; type seconds) | `3600` | Yes | How long a primary with no replica keeps its replication backlog before dropping it. `0` keeps it. A replica that comes later gets a full sync. |
| `propagation-error-behavior` (TOML not available; type ignore, panic, panic-on-replicas) | `ignore` | Yes | What happens when a write that came from the primary (or the AOF) fails on this node. It is always logged. `panic` stops the process, `panic-on-replicas` stops a replica only. |

## See also

- [Deploy on VMs and cloud](https://docs.hotkv.com/clustering/production/index.md): the settings in use.
- [Cluster over TLS](https://docs.hotkv.com/clustering/tls/index.md): the TLS settings in use.
- [Replicas and failover](https://docs.hotkv.com/clustering/replicas-and-failover/index.md): what the failover
  settings do.
- [Configuration reference](https://docs.hotkv.com/reference/configuration/index.md): every other setting.