Cluster in Docker Compose

Last updated 12 Oct 2026

Not on HotKV CloudOn-Premise: no license needed

This page builds a cluster of three primaries and three replicas in Docker Compose: one compose.yaml, one small init script that creates the cluster, a password, a data volume per node, and ports published on 127.0.0.1 only. It is a way to run a real multi-node cluster on a laptop or a CI machine without installing anything but Docker. It is one machine, so it is a good place to learn, develop and test failover, and not a production layout: a production cluster needs its nodes on separate hosts (Deploy on VMs and cloud).

What you need

Docker with Compose v2 (docker compose).
The HotKV image, called hotkv here as on Docker and Kubernetes: use the name and tag you were given.
redis-cli or valkey-cli on the host, to look at the cluster from outside. The init service needs nothing else: it uses the hotkv-cli that is in the image.
Free ports 7000 to 7005 on 127.0.0.1. Change them in the file if they are taken. The cluster bus and replication use the container network and are not published.

The files

Make a directory and put three files in it. The first holds the password, in the .env file that Compose reads. Keep it out of version control:

printf "HOTKV_PASSWORD=%s\n" "$(openssl rand -hex 16)" > .env

compose.yaml runs six identical nodes that differ only in their names and the port they publish, and an init service that runs once:

name: hotkv-cluster

x-node: &node
  image: hotkv
  restart: unless-stopped
  stop_grace_period: 2m
  environment:
    HOTKVCLI_AUTH: ${HOTKV_PASSWORD:?set HOTKV_PASSWORD in the .env file}

services:
  node1:
    <<: *node
    command: ["--cluster-enabled", "yes", "--requirepass", "${HOTKV_PASSWORD}", "--masterauth", "${HOTKV_PASSWORD}",
              "--appendonly", "yes", "--shutdown-on-sigterm", "failover",
              "--cluster-announce-ip", "node1",
              "--cluster-announce-client-ipv4", "127.0.0.1", "--cluster-announce-client-port", "7000"]
    ports: ["127.0.0.1:7000:6379"]
    volumes: ["node1-data:/data"]
  node2:
    <<: *node
    command: ["--cluster-enabled", "yes", "--requirepass", "${HOTKV_PASSWORD}", "--masterauth", "${HOTKV_PASSWORD}",
              "--appendonly", "yes", "--shutdown-on-sigterm", "failover",
              "--cluster-announce-ip", "node2",
              "--cluster-announce-client-ipv4", "127.0.0.1", "--cluster-announce-client-port", "7001"]
    ports: ["127.0.0.1:7001:6379"]
    volumes: ["node2-data:/data"]
  node3:
    <<: *node
    command: ["--cluster-enabled", "yes", "--requirepass", "${HOTKV_PASSWORD}", "--masterauth", "${HOTKV_PASSWORD}",
              "--appendonly", "yes", "--shutdown-on-sigterm", "failover",
              "--cluster-announce-ip", "node3",
              "--cluster-announce-client-ipv4", "127.0.0.1", "--cluster-announce-client-port", "7002"]
    ports: ["127.0.0.1:7002:6379"]
    volumes: ["node3-data:/data"]
  node4:
    <<: *node
    command: ["--cluster-enabled", "yes", "--requirepass", "${HOTKV_PASSWORD}", "--masterauth", "${HOTKV_PASSWORD}",
              "--appendonly", "yes", "--shutdown-on-sigterm", "failover",
              "--cluster-announce-ip", "node4",
              "--cluster-announce-client-ipv4", "127.0.0.1", "--cluster-announce-client-port", "7003"]
    ports: ["127.0.0.1:7003:6379"]
    volumes: ["node4-data:/data"]
  node5:
    <<: *node
    command: ["--cluster-enabled", "yes", "--requirepass", "${HOTKV_PASSWORD}", "--masterauth", "${HOTKV_PASSWORD}",
              "--appendonly", "yes", "--shutdown-on-sigterm", "failover",
              "--cluster-announce-ip", "node5",
              "--cluster-announce-client-ipv4", "127.0.0.1", "--cluster-announce-client-port", "7004"]
    ports: ["127.0.0.1:7004:6379"]
    volumes: ["node5-data:/data"]
  node6:
    <<: *node
    command: ["--cluster-enabled", "yes", "--requirepass", "${HOTKV_PASSWORD}", "--masterauth", "${HOTKV_PASSWORD}",
              "--appendonly", "yes", "--shutdown-on-sigterm", "failover",
              "--cluster-announce-ip", "node6",
              "--cluster-announce-client-ipv4", "127.0.0.1", "--cluster-announce-client-port", "7005"]
    ports: ["127.0.0.1:7005:6379"]
    volumes: ["node6-data:/data"]

  init:
    <<: *node
    restart: "no"
    entrypoint: ["/bin/bash", "/init.sh"]
    command: []
    depends_on: [node1, node2, node3, node4, node5, node6]
    volumes:
      - ./init.sh:/init.sh:ro

volumes:
  node1-data:
  node2-data:
  node3-data:
  node4-data:
  node5-data:
  node6-data:

init.sh waits for the nodes, introduces them to each other, gives the three primaries a third of the 16384 slots each, attaches a replica to each primary and waits for the cluster to report cluster_state:ok. It is plain CLUSTER MEET, CLUSTER ADDSLOTSRANGE and CLUSTER REPLICATE, so no other tool or image is needed. Run again against a cluster that exists, it only says so:

set -eu
cli() { hotkv-cli -h "$1" "${@:2}"; }

for n in node1 node2 node3 node4 node5 node6; do
  until cli "$n" ping >/dev/null 2>&1; do sleep 1; done
done

if cli node1 cluster info | grep -q 'cluster_state:ok'; then
  echo "The cluster already exists."
  exit 0
fi

# Introduce every node to node1, which tells the others.
for n in node2 node3 node4 node5 node6; do
  cli node1 cluster meet "$(getent hosts "$n" | cut -d' ' -f1)" 6379
done
until [ "$(cli node1 cluster nodes | grep -c .)" = 6 ]; do sleep 1; done

# Three primaries share the 16384 slots.
cli node1 cluster addslotsrange 0 5460
cli node2 cluster addslotsrange 5461 10922
cli node3 cluster addslotsrange 10923 16383

# The other three nodes become their replicas.
for pair in node4:node1 node5:node2 node6:node3; do
  replica=${pair%%:*}; primary=${pair##*:}
  until cli "$replica" cluster nodes | grep -q "$(cli "$primary" cluster myid)"; do sleep 1; done
  cli "$replica" cluster replicate "$(cli "$primary" cluster myid)"
done

for n in node1 node2 node3 node4 node5 node6; do
  until cli "$n" cluster info | grep -q 'cluster_state:ok'; do sleep 1; done
done
echo "The cluster is ready."

Start the cluster

docker compose up -d

 Network hotkv-cluster_default  Creating
 Network hotkv-cluster_default  Created
 ...
 Container hotkv-cluster-init-1  Starting
 Container hotkv-cluster-init-1  Started

The init container exits when the cluster is ready, a few seconds later. Its log says so, and docker compose ps lists the six nodes (the status changes from health: starting to healthy within half a minute, as the image’s own health check pings each node):

docker compose logs init | tail -3
docker compose ps --format "table {{.Service}}\t{{.Status}}\t{{.Ports}}"

init-1  | OK
init-1  | OK
init-1  | The cluster is ready.
SERVICE   STATUS                            PORTS
node1     Up 6 seconds (health: starting)   127.0.0.1:7000->6379/tcp
node2     Up 6 seconds (health: starting)   127.0.0.1:7001->6379/tcp
node3     Up 6 seconds (health: starting)   127.0.0.1:7002->6379/tcp
node4     Up 6 seconds (health: starting)   127.0.0.1:7003->6379/tcp
node5     Up 6 seconds (health: starting)   127.0.0.1:7004->6379/tcp
node6     Up 6 seconds (health: starting)   127.0.0.1:7005->6379/tcp

Check it from the host

Give redis-cli the password through its environment, then connect with -c so that it follows redirects. The nodes tell clients the address they can reach from the host, 127.0.0.1 and the published port:

export REDISCLI_AUTH=$(grep ^HOTKV_PASSWORD .env | cut -d= -f2)
redis-cli -c -p 7000

127.0.0.1:7000> SET user:1000 ada
OK
127.0.0.1:7000> SET city Lisbon
-> Redirected to slot [11479] located at 127.0.0.1:7002
OK
127.0.0.1:7002> GET city
"Lisbon"
127.0.0.1:7002> CLUSTER KEYSLOT city
(integer) 11479

redis-cli -p 7000 CLUSTER NODES | awk '{print $2, $3, $9}' | sort
redis-cli --cluster check 127.0.0.1:7000 | tail -4

127.0.0.1:7000@16379 myself,master 0-5460
127.0.0.1:7001@16379 master 5461-10922
127.0.0.1:7002@16379 master 10923-16383
127.0.0.1:7003@16379 slave
127.0.0.1:7004@16379 slave
127.0.0.1:7005@16379 slave
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

(The number after @ is the cluster bus port inside the container.)

What the settings do

Setting	Why
`--requirepass`, `--masterauth`	One password for clients and for the nodes among themselves: replicas log in with it to follow their primary, and an atomic slot migration logs in with `masterauth` to the node it moves slots to. Without `--masterauth` that migration fails with `NOAUTH`.
`--appendonly yes` and the `/data` volume	Each node keeps its data and its `nodes.conf` in its own volume, so a restart finds the cluster as it was.
`--cluster-announce-ip nodeN`	The nodes reach each other by service name, which Docker resolves to the container’s address when they connect. A container that comes back with another address is found again.
`--cluster-announce-client-ipv4`, `--cluster-announce-client-port`	What clients are told: `127.0.0.1` and the port you published. Inside a container `127.0.0.1` is the container itself, so only clients on the host can use these addresses.
`--shutdown-on-sigterm failover`	`docker compose stop` sends SIGTERM. With this setting a primary hands its slots to a replica before it exits, so a planned stop causes no outage.
`restart: unless-stopped`, `stop_grace_period: 2m`	A node that dies comes back, and a stop gives a large data set time to be saved.
`ports: ["127.0.0.1:7000:6379"]`	Publishes the client port on the loopback address only. The bus port (16379) stays inside the network.

The two cluster-announce-client settings are what make the cluster usable from the host. They are also why the host cannot use every tool: see Resharding below.

Applications inside the Compose project

Clients that run as services in the same project reach the nodes by name, so they need the names and not 127.0.0.1. Remove the two --cluster-announce-client-* settings from every node and the nodes hand out what they announce, node3:6379 and so on. With the same file minus those two settings, hotkv-cli run inside a container shows the redirect, and a cluster-aware Python client run in another container on the same network connects and finds all six nodes:

docker compose exec node1 hotkv-cli get foo
docker compose exec node1 hotkv-cli cluster nodes | awk '{print $2, $3, $9}' | sort

MOVED 12182 node3:6379
node1:6379@16379 myself,master 0-5460
node2:6379@16379 master 5461-10922
node3:6379@16379 master 10923-16383
node4:6379@16379 slave
node5:6379@16379 slave
node6:6379@16379 slave

import os
from redis.cluster import RedisCluster, ClusterNode

rc = RedisCluster(startup_nodes=[ClusterNode("node1", 6379)], password=os.environ["HOTKV_PASSWORD"], decode_responses=True)
rc.set("foo", "bar")
print(rc.get("foo"))
print(sorted(f"{n.host}:{n.port}" for n in rc.get_nodes()))

docker run --rm --network hotkv-cluster_default --env-file .env -v "$PWD/client.py:/client.py:ro" python:3-slim sh -c "pip install -q --root-user-action=ignore redis 2>&1 | tail -1; python /client.py"

bar
['node1:6379', 'node2:6379', 'node3:6379', 'node4:6379', 'node5:6379', 'node6:6379']

A cluster gives each node one endpoint. If you need both kinds of client, run your tools inside the network, or announce addresses that both can reach, as described in Deploy on VMs and cloud.

Resharding

Moving slots needs the nodes to reach each other, and in this layout that works with the atomic slot migration only. valkey-cli --cluster reshard with --cluster-use-atomic-slot-migration moved slot 0, which holds a key, to the other node, and CLUSTER MIGRATESLOTS, sent to the node that owns the slot and naming the node that is to get it, moved it back. The redis-cli --cluster reshard that moves keys with MIGRATE does not work here: it names the nodes by the addresses clients are given, and the source node, inside its container, cannot connect to 127.0.0.1:7001. It stops with IOERR and leaves the slot open on both nodes; CLUSTER SETSLOT 0 STABLE on both closes it. (redis-cli 8.4 and later, which asks the target to import, does not work in this layout either.)

SRC=$(redis-cli -p 7000 CLUSTER MYID); DST=$(redis-cli -p 7001 CLUSTER MYID)
redis-cli -c -p 7000 SET 06S x
redis-cli --cluster reshard 127.0.0.1:7000 --cluster-from $SRC --cluster-to $DST --cluster-slots 1 --cluster-yes 2>&1 | tail -4
redis-cli -p 7000 CLUSTER SETSLOT 0 STABLE; redis-cli -p 7001 CLUSTER SETSLOT 0 STABLE
export VALKEYCLI_AUTH=$REDISCLI_AUTH; valkey-cli --cluster reshard 127.0.0.1:7000 --cluster-from $SRC --cluster-to $DST --cluster-slots 1 --cluster-yes --cluster-use-atomic-slot-migration 2>&1 | tail -3
redis-cli -c -p 7000 GET 06S
redis-cli -p 7001 CLUSTER MIGRATESLOTS SLOTSRANGE 0 0 NODE $SRC
redis-cli -c -p 7000 GET 06S

OK
Node 127.0.0.1:7000 replied with error:
IOERR error or timeout connecting to the specified instance

clusterManagerMoveSlot failed: IOERR error or timeout connecting to the specified instance
OK
OK
  Resharding plan:
    Moving slot range 0-0 from f15f91672565e9d0a623c91a55a9923c6326ef92
Moving slot range 0-0 from 127.0.0.1:7000 to 127.0.0.1:7001 via atomic slot migration.#
-> Redirected to slot [0] located at 127.0.0.1:7001
"x"
OK
"x"

(06S is a key name in slot 0. The first GET is redirected to 7001 because the slot had moved there; after MIGRATESLOTS it is back on 7000. The OK lines are the replies of the commands that print one.)

Fail a primary

A planned stop is the common case: docker compose stop sends SIGTERM, and because of --shutdown-on-sigterm failover the replica of the stopped primary takes over at once. Stop node1, which serves slots 0 to 5460, and read and write through the cluster:

docker compose stop node1
redis-cli -p 7001 CLUSTER NODES | awk '{print $2, $3, $9}' | sort
redis-cli -c -p 7001

 Container hotkv-cluster-node1-1  Stopping
 Container hotkv-cluster-node1-1  Stopped
127.0.0.1:7000@16379 master
127.0.0.1:7001@16379 myself,master 5461-10922
127.0.0.1:7002@16379 master 10923-16383
127.0.0.1:7003@16379 master 0-5460
127.0.0.1:7004@16379 slave
127.0.0.1:7005@16379 slave
127.0.0.1:7001> GET user:1000
-> Redirected to slot [1649] located at 127.0.0.1:7003
"ada"
127.0.0.1:7003> SET user:1000 grace
OK

node4 (port 7003) serves slots 0 to 5460 at once after the stop. Start node1 again and it rejoins as a replica of node4:

docker compose start node1
sleep 5
redis-cli -p 7001 CLUSTER NODES | awk '{print $2, $3, $9}' | sort

 Container hotkv-cluster-node1-1  Starting
 Container hotkv-cluster-node1-1  Started
127.0.0.1:7000@16379 slave
127.0.0.1:7001@16379 myself,master 5461-10922
127.0.0.1:7002@16379 master 10923-16383
127.0.0.1:7003@16379 master 0-5460
127.0.0.1:7004@16379 slave
127.0.0.1:7005@16379 slave

A crash is different: nobody tells the cluster. docker compose kill sends SIGKILL, and the replica waits for the other nodes to agree that the primary is gone, which takes the node timeout (15 seconds by default) plus the election. Kill node2, serving slots 5461 to 10922, and poll CLUSTER NODES through a node that is still up, here node1, until its replica node5 is the primary:

docker compose kill node2
until redis-cli -p 7000 CLUSTER NODES | grep -q '127.0.0.1:7004@16379 master'; do sleep 0.5; done
echo "node5 is the primary"

 Container hotkv-cluster-node2-1  Killing
 Container hotkv-cluster-node2-1  Killed
node5 is the primary

Writes to those slots fail until then. docker compose start node2 brings the old primary back as a replica.

Restart and keep the data

docker compose down removes the containers and the network but keeps the volumes, and with them each node’s data and nodes.conf. Starting again brings the cluster back with the same node IDs and data, even when the containers get other addresses, because the nodes find each other by name. To prove it, start the nodes in reverse order, which changes the addresses:

for n in node1 node2 node3; do docker inspect -f "$n {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}" hotkv-cluster-$n-1; done
docker compose down
for n in node6 node5 node4 node3 node2 node1; do docker compose up -d $n 2>/dev/null; done
for n in node1 node2 node3; do docker inspect -f "$n {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}" hotkv-cluster-$n-1; done
redis-cli -p 7000 CLUSTER INFO | head -3
redis-cli -c -p 7000

node1 172.18.0.4
node2 172.18.0.5
node3 172.18.0.7
node1 172.18.0.7
node2 172.18.0.6
node3 172.18.0.5
cluster_enabled:1
cluster_state:ok
cluster_slots_assigned:16384
127.0.0.1:7000> GET user:1000
-> Redirected to slot [1649] located at 127.0.0.1:7003
"grace"
127.0.0.1:7003> GET city
-> Redirected to slot [11479] located at 127.0.0.1:7002
"Lisbon"

The init service did not run again, because up -d <node> starts only the nodes you name. If you run docker compose up -d for the whole project instead, init runs, finds cluster_state:ok and prints The cluster already exists..

Tear down

docker compose down -v

-v also deletes the volumes, and with them the data. Leave it out to keep the cluster for next time.

Before you rely on it

All six nodes share one machine, one disk and one Docker daemon, so they fail together. For availability, put the nodes on different hosts.
The password is in the .env file and in each container’s command line. Keep the file private and the published ports on 127.0.0.1.

Next steps

Run a cluster on Kubernetes for the same layout as a StatefulSet.
Cluster clients: connect your application.
Deploy on VMs and cloud: the production layout.

Ask the HotKV docs