Skip to content

Ask the HotKV docs

Answers come only from these docs and can be wrong; check the linked pages before you rely on them.

CLUSTER FAILOVER

Last updated
CLUSTER FAILOVER [FORCE | TAKEOVER]
Group
Cluster
Complexity
O(1)
Arity
-2 (at least 2 words, counting the command name)
HotKV Cloud
Not available
On-Premise
no license needed

CLUSTER FAILOVER starts a manual failover. Send it to a replica: the replica becomes the primary of its shard, takes over the slots, and its old primary becomes its replica. Use it for planned maintenance, or to recover a shard whose primary is gone when the automatic failover cannot run.

The command replies OK as soon as the failover has started. The switch itself happens shortly after; watch CLUSTER NODES or CLUSTER SHARDS to see the roles change. A failover that has not finished within cluster-manual-failover-timeout (default 5000 milliseconds) is given up: the replica logs Manual failover timed out. and stays a replica, and you can ask again.

Option Effect
(none) Coordinated failover. The replica asks its primary to pause writes, waits until it has every write the primary acknowledged, then wins a normal election among the primaries. No acknowledged write is lost. While the primary is paused, clients wait; they do not get an error. The primary must answer.
FORCE Skips the handshake with the primary, for when it is unreachable, but still needs a majority of the primaries to vote for the replica.
TAKEOVER Skips the election too: the replica takes a configuration epoch above every epoch it knows and takes over at once. Use it only when a majority of primaries cannot be reached. The old primary, when it comes back, finds it has lost its slots and becomes a replica.

A replica that never finished its first sync serves the shard without the missing data after a TAKEOVER, so check INFO replication first. Both FORCE and the default mode fail on a replica that is still copying its data from the primary (see below).

Simple string OK in RESP2 and RESP3 once the failover has started, or one of these errors:

Case Error
The node is a primary, or its primary is unknown ERR You should send CLUSTER FAILOVER to a replica
The replica is still copying its data from the primary LOADING HotKV is loading the dataset in memory
An argument other than FORCE or TAKEOVER ERR syntax error
Cluster mode is off ERR This instance has cluster support disabled

A coordinated failover while a client writes. The roles swap, and no write is lost:

127.0.0.1:7003> CLUSTER FAILOVER
OK
$ redis-cli -p 7001 CLUSTER NODES | cut -d" " -f2,3,4,8,9 | sort
127.0.0.1:7000@17000 slave 3bfee53e12421ac6c914bf03042dc2a2fde68fb4 connected
127.0.0.1:7001@17001 myself,master - connected 5461-10922
127.0.0.1:7002@17002 master - connected 10923-16383
127.0.0.1:7003@17003 master - connected 0-5460
127.0.0.1:7004@17004 slave 147c5200dfb75e7fcbe095c22dc93cc590cb7cac connected
127.0.0.1:7005@17005 slave f90cb48df290ae2ef2328925012abcebd48204e1 connected

With the primary frozen (kill -STOP), the default mode times out and FORCE works:

127.0.0.1:7004> CLUSTER FAILOVER
OK
$ redis-cli -p 7004 ROLE | head -n 1
slave
127.0.0.1:7004> CLUSTER FAILOVER FORCE
OK
$ redis-cli -p 7004 ROLE | head -n 1
master

Sent to the wrong node, to a replica that is copying its data, and with an unknown word:

127.0.0.1:7000> CLUSTER FAILOVER
(error) ERR You should send CLUSTER FAILOVER to a replica
127.0.0.1:7005> CLUSTER FAILOVER FORCE
(error) LOADING HotKV is loading the dataset in memory
127.0.0.1:7004> CLUSTER FAILOVER FOO
(error) ERR syntax error

Replicas and failover shows the three modes on a running cluster, with the log lines.

The three modes match Redis 7 and Valkey 8, and the replies and error texts are Redis’s, with LOADING while a replica copies its data. FORCE and TAKEOVER given together are accepted and act as TAKEOVER; Redis rejects that. Outside cluster mode, promote a replica with REPLICAOF NO ONE or REPLICATION.FAILOVER instead.

CLUSTER REPLICATE, CLUSTER NODES, SHUTDOWN (FAILOVER), REPLICATION.FAILOVER, Replicas and failover