Skip to content

Ask the HotKV docs

Answers come only from these docs and can be wrong; check the linked pages before you rely on them.

Cluster clients

Last updated
Not on HotKV CloudOn-Premise: no license needed

A cluster-aware client library learns which node serves which slot by asking the cluster, sends each command to the right node, and follows the redirects when the layout changes. HotKV speaks the Redis Cluster protocol, so the cluster libraries you already use work against it, and the HotKV SDKs add typed commands for the AI engines on top.

The snippets on this page assume a cluster of three primaries and three replicas on ports 7000 to 7005, protected by a password. Start the nodes as in the quick start, adding --requirepass cluster-password --masterauth cluster-password to each node and the password to the redis-cli --cluster create command. Replace the addresses and the password with yours.

  • Seed nodes. You give the client a few node addresses. It asks one of them for the slot map (CLUSTER SLOTS), then connects to the nodes it learns about. Give two or three seeds in case one is down at start. Seeds are used only to find the cluster: the client talks to the addresses the cluster announces.
  • The announced address must be reachable from the client. MOVED replies and CLUSTER SLOTS carry the address each node announces, and the client connects to exactly that. A cluster whose nodes announce a container or private address works from inside that network only. See Deploy on VMs and cloud for the announce settings, and the Compose and Kubernetes pages for containers.
  • Routing by key. The client hashes the key to a slot, so a command with several keys needs them in one slot: use a hash tag such as {user:1000}.name. A multi-key command across slots fails with CROSSSLOT.
  • Password and TLS are connection settings applied to every node. TLS options are covered in Cluster TLS.
  • Client-side caching is per node. A client that keeps a local cache turns CLIENT TRACKING on for each node connection it reads from, and is told of the changes that node makes to the keys it read. A node that gives up a slot tells such clients of each key as it removes them, so a cache can drop what it held from that node. See Client-side caching.

What the redirects and errors mean to a client

Section titled “What the redirects and errors mean to a client”
ReplyMeaningWhat a good client does
MOVED slot addressThe slot is served by another node.Refresh the slot map and resend to that address.
ASK slot addressThe slot is being moved and this key is already on the target.Send ASKING and the command to that address, once. Do not update the slot map.
CLUSTERDOWNThe slot is not served, or the cluster is down (for example, during a failover that has not finished).Wait briefly, refresh the map and retry.
TRYAGAINA multi-key command found its keys split across a slot that is moving. An AI-engine entry of a slot that this node is handing off gets it too.Retry after a short pause.
LOADINGThe node is loading or syncing its data.Retry, or use another node.
CROSSSLOTThe keys of a command are in different slots.Not retryable: fix the keys (hash tags).

Each snippet connects with two seed nodes and the password, writes and reads a key, runs a multi-key command on keys that share a hash tag, and reads from replicas with a second client. The output of each one is shown under it.

redis-py (works with 8.1.0 and later): pip install redis, in a virtual environment (python3 -m venv venv && . venv/bin/activate) on systems whose Python is managed by the operating system.

from redis.cluster import RedisCluster, ClusterNode
rc = RedisCluster(
startup_nodes=[ClusterNode("127.0.0.1", 7000), ClusterNode("127.0.0.1", 7001)],
password="cluster-password",
decode_responses=True,
)
rc.set("user:1000", "ada")
print(rc.get("user:1000"))
# Keys with a common hash tag live in one slot, so a multi-key command works.
rc.mset({"{user:1000}.name": "Ada", "{user:1000}.city": "London"})
print(rc.mget("{user:1000}.name", "{user:1000}.city"))
# Keys in different slots need one command each.
rc.mset_nonatomic({"user:1": "a", "user:2": "b", "user:3": "c"})
print(rc.mget_nonatomic("user:1", "user:2", "user:3"))
# Read from replicas: a second client that sends READONLY to replica nodes.
rr = RedisCluster(
startup_nodes=[ClusterNode("127.0.0.1", 7000)],
password="cluster-password",
decode_responses=True,
read_from_replicas=True,
)
print(rr.get("user:1000"))
rc.close()
rr.close()
ada
['Ada', 'London']
['a', 'b', 'c']
ada

read_from_replicas=True sends reads to the replicas of the slot’s primary (and the library sends READONLY to them). Without it every command goes to the primary.

A client in a cluster should ride out a failover: the slot moves to another node, the old connection closes, and for a moment commands for those slots get CLUSTERDOWN or a refused connection. A cluster-aware client refreshes its slot map and retries those commands, so a coordinated failover (CLUSTER FAILOVER, docker compose stop with --shutdown-on-sigterm failover, a rolling update) pauses writes for a moment and hands the slot over without losing a write.

A crash is different: the slot has no primary until the other nodes agree that it is gone, which takes the node timeout (15 seconds by default) plus the election, and a client that gives up sooner reports errors in that time. See Replicas and failover for the timeline. How long a client library retries before it raises an error is a setting of that library. Set the retry count and timeout of your library so that they cover the node timeout plus the election.

The Python and Node.js SDKs have a cluster client that routes every command, including the typed commands of the AI engines, by the key or name it carries, follows MOVED and ASK, and rides out a failover, a reshard in progress, CLUSTERDOWN, TRYAGAIN and LOADING by refreshing the slot map and trying again for up to 10 seconds (retry_timeout in Python, retryTimeoutMs in Node.js) before it raises ClusterError. A command that may have reached a node whose connection then broke is sent again, so a command that is not idempotent (incr, rpush, a budget consume) can be applied twice; pass retry_writes=False (retryWrites: false) to get the error instead. Install them as shown on Connecting and SDKs.

ClusterHotKV, in the hotkv.cluster module:

from hotkv.cluster import ClusterHotKV
cluster = ClusterHotKV(
[("127.0.0.1", 7000), ("127.0.0.1", 7001)],
password="cluster-password",
decode_responses=True,
)
cluster.set("user:1000", "ada")
print(cluster.get("user:1000"))
# Keys with a common hash tag live in one slot, so a multi-key command works.
print(cluster.execute("MSET", "{user:1000}.name", "Ada", "{user:1000}.city", "London"))
print(cluster.execute("MGET", "{user:1000}.name", "{user:1000}.city"))
# Commands that list one node's data are sent to every primary and merged.
print(cluster.dbsize())
print(sorted(cluster.keys("{user:1000}*")))
# A command for one node's own state goes where you say.
print(cluster.execute_on_primaries("CONFIG", "GET", "maxmemory-policy"))
# Read from replicas: replica connections send READONLY.
replicas = ClusterHotKV(
[("127.0.0.1", 7000)],
password="cluster-password",
decode_responses=True,
read_from_replicas=True,
)
print(replicas.get("user:1000"))
replicas.close()
cluster.close()
ada
True
['Ada', 'London']
103
['{user:1000}.city', '{user:1000}.name']
{('127.0.0.1', 7000): {'maxmemory-policy': 'noeviction'}, ('127.0.0.1', 7001): {'maxmemory-policy': 'noeviction'}, ('127.0.0.1', 7002): {'maxmemory-policy': 'noeviction'}}
ada

The asyncio client has the same methods as coroutines:

import asyncio
from hotkv.asyncio import ClusterHotKV
async def main():
cluster = ClusterHotKV(
[("127.0.0.1", 7000), ("127.0.0.1", 7001)],
password="cluster-password",
decode_responses=True,
)
await cluster.set("user:1000", "ada")
print(await cluster.get("user:1000"))
print(await cluster.execute("MGET", "{user:1000}.name", "{user:1000}.city"))
await cluster.close()
asyncio.run(main())
ada
['Ada', 'London']

Some commands report on one node’s own data: the LIST commands of the AI engines, KEYS, DBSIZE, FLUSHALL, TAG.MEMBERS, and the script and function commands (see What cannot be clustered). The HotKV cluster clients send these to every primary and merge the answers, so they cover the whole cluster: dbsize() above counted the keys of all three primaries (the number is however many keys your cluster holds), and pcache_list() lists the prompt-cache namespaces of the whole cluster:

from hotkv.cluster import ClusterHotKV
cluster = ClusterHotKV([("127.0.0.1", 7000), ("127.0.0.1", 7001)], decode_responses=True)
for ns in ("notes", "faqs", "plans", "agents"):
cluster.pcache_create(ns)
cluster.pcache_set(ns, "question", "answer")
print(cluster.pcache_get("faqs", "question"))
# pcache_list() asks every primary and merges, so it lists the whole cluster.
print(sorted(cluster.pcache_list()))
cluster.close()
answer
['agents', 'faqs', 'notes', 'plans']

(The AI engines need a license on every node: see AI engines in a cluster.)

State that belongs to one node, such as an ACL user, is not merged: the client sends ACL, CONFIG, INFO and similar commands to one primary, and you send them to a node you choose with execute_on_node (Python) or executeOnNode (Node.js), or to every primary with execute_on_primaries or executeOnPrimaries, which return the answer of each node, as the CONFIG GET calls in the snippets above did. An ACL user created on one node does not exist on the others, or on its replicas.

Each of these SDKs has a cluster client that builds its slot map from the cluster and follows MOVED and ASK. Install them as shown on Connecting and SDKs.

Language Cluster class Replica reads
Java com.hotkv.HotKVCluster commandReadOnly
C# HotKV.Cluster.HotKVCluster ReadFromReplicas = true
C++ hotkv::ClusterClient set_read_from_replicas(true)
PHP HotKV\Cluster\ClusterClient the readFromReplicas option
Ruby HotKV::Cluster -
Rust hotkv::ClusterClient ClusterOptions::read_from_replicas with execute_readonly
Elixir HotKV.Cluster -
Perl HotKV::Cluster the read_from_replicas option

The Go and C SDKs have a function that computes the slot of a key (KeySlot, hotkv_key_slot) and recognise redirect errors. In Go, use go-redis’s ClusterClient as shown above.