# Cluster clients > Connect an application to a HotKV cluster with redis-py, node-redis, ioredis, go-redis, Jedis, Lettuce or a HotKV SDK, with replica reads and retries through failover. URL: https://docs.hotkv.com/clustering/clients/ **Availability:** Not on HotKV Cloud. On-Premise: no license needed. A cluster-aware client library learns which node serves which slot by asking the cluster, sends each command to the right node, and follows the redirects when the layout changes. HotKV speaks the Redis Cluster protocol, so the cluster libraries you already use work against it, and the HotKV SDKs add typed commands for the AI engines on top. The snippets on this page assume a cluster of three primaries and three replicas on ports 7000 to 7005, protected by a password. Start the nodes as in the [quick start](https://docs.hotkv.com/clustering/quick-start/index.md), adding `--requirepass cluster-password --masterauth cluster-password` to each node and the password to the `redis-cli --cluster create` command. Replace the addresses and the password with yours. ## How a cluster client works - **Seed nodes.** You give the client a few node addresses. It asks one of them for the slot map (`CLUSTER SLOTS`), then connects to the nodes it learns about. Give two or three seeds in case one is down at start. Seeds are used only to find the cluster: the client talks to the addresses the cluster announces. - **The announced address must be reachable from the client.** `MOVED` replies and `CLUSTER SLOTS` carry the address each node announces, and the client connects to exactly that. A cluster whose nodes announce a container or private address works from inside that network only. See [Deploy on VMs and cloud](https://docs.hotkv.com/clustering/production/index.md) for the announce settings, and the [Compose](https://docs.hotkv.com/clustering/docker-compose/index.md) and [Kubernetes](https://docs.hotkv.com/clustering/kubernetes/index.md) pages for containers. - **Routing by key.** The client hashes the key to a slot, so a command with several keys needs them in one slot: use a [hash tag](https://docs.hotkv.com/clustering/keys-and-commands/index.md#hash-tags) such as `{user:1000}.name`. A multi-key command across slots fails with `CROSSSLOT`. - **Password and TLS** are connection settings applied to every node. TLS options are covered in [Cluster TLS](https://docs.hotkv.com/clustering/tls/index.md). - **Client-side caching** is per node. A client that keeps a local cache turns `CLIENT TRACKING` on for each node connection it reads from, and is told of the changes that node makes to the keys it read. A node that gives up a slot tells such clients of each key as it removes them, so a cache can drop what it held from that node. See [Client-side caching](https://docs.hotkv.com/develop/client-side-caching/index.md#in-a-cluster-and-on-replicas). ### What the redirects and errors mean to a client
ReplyMeaningWhat a good client does
MOVED slot addressThe slot is served by another node.Refresh the slot map and resend to that address.
ASK slot addressThe slot is being moved and this key is already on the target.Send ASKING and the command to that address, once. Do not update the slot map.
CLUSTERDOWNThe slot is not served, or the cluster is down (for example, during a failover that has not finished).Wait briefly, refresh the map and retry.
TRYAGAINA multi-key command found its keys split across a slot that is moving. An AI-engine entry of a slot that this node is handing off gets it too.Retry after a short pause.
LOADINGThe node is loading or syncing its data.Retry, or use another node.
CROSSSLOTThe keys of a command are in different slots.Not retryable: fix the keys (hash tags).
## Connect and use Each snippet connects with two seed nodes and the password, writes and reads a key, runs a multi-key command on keys that share a hash tag, and reads from replicas with a second client. The output of each one is shown under it. **Python** `redis-py` (works with 8.1.0 and later): `pip install redis`, in a virtual environment (`python3 -m venv venv && . venv/bin/activate`) on systems whose Python is managed by the operating system. ```python from redis.cluster import RedisCluster, ClusterNode rc = RedisCluster( startup_nodes=[ClusterNode("127.0.0.1", 7000), ClusterNode("127.0.0.1", 7001)], password="cluster-password", decode_responses=True, ) rc.set("user:1000", "ada") print(rc.get("user:1000")) # Keys with a common hash tag live in one slot, so a multi-key command works. rc.mset({"{user:1000}.name": "Ada", "{user:1000}.city": "London"}) print(rc.mget("{user:1000}.name", "{user:1000}.city")) # Keys in different slots need one command each. rc.mset_nonatomic({"user:1": "a", "user:2": "b", "user:3": "c"}) print(rc.mget_nonatomic("user:1", "user:2", "user:3")) # Read from replicas: a second client that sends READONLY to replica nodes. rr = RedisCluster( startup_nodes=[ClusterNode("127.0.0.1", 7000)], password="cluster-password", decode_responses=True, read_from_replicas=True, ) print(rr.get("user:1000")) rc.close() rr.close() ``` ```text ada ['Ada', 'London'] ['a', 'b', 'c'] ada ``` `read_from_replicas=True` sends reads to the replicas of the slot's primary (and the library sends `READONLY` to them). Without it every command goes to the primary. **Node.js** `node-redis` (works with 6.3.0 and later): `npm install redis`. ```javascript import { createCluster } from 'redis'; const options = { rootNodes: [{ url: 'redis://127.0.0.1:7000' }, { url: 'redis://127.0.0.1:7001' }], defaults: { password: 'cluster-password' }, }; const cluster = createCluster(options); cluster.on('error', (err) => console.error('cluster error:', err.message)); await cluster.connect(); await cluster.set('user:1000', 'ada'); console.log(await cluster.get('user:1000')); // Keys with a common hash tag live in one slot, so a multi-key command works. await cluster.mSet({ '{user:1000}.name': 'Ada', '{user:1000}.city': 'London' }); console.log(await cluster.mGet(['{user:1000}.name', '{user:1000}.city'])); // Read from replicas: a second client with useReplicas. const replicas = createCluster({ ...options, useReplicas: true }); replicas.on('error', (err) => console.error('cluster error:', err.message)); await replicas.connect(); console.log(await replicas.get('user:1000')); await replicas.close(); await cluster.close(); ``` ```text ada [ 'Ada', 'London' ] ada ``` `ioredis` (works with 6.0.0 and later): `npm install ioredis`. ```javascript import { Cluster } from 'ioredis'; const nodes = [{ host: '127.0.0.1', port: 7000 }, { host: '127.0.0.1', port: 7001 }]; const redisOptions = { password: 'cluster-password' }; const cluster = new Cluster(nodes, { redisOptions }); await cluster.set('user:1000', 'ada'); console.log(await cluster.get('user:1000')); // Keys with a common hash tag live in one slot, so a multi-key command works. await cluster.mset('{user:1000}.name', 'Ada', '{user:1000}.city', 'London'); console.log(await cluster.mget('{user:1000}.name', '{user:1000}.city')); // Read from replicas: a second client with scaleReads. const replicas = new Cluster(nodes, { redisOptions, scaleReads: 'slave' }); console.log(await replicas.get('user:1000')); replicas.disconnect(); cluster.disconnect(); ``` ```text ada [ 'Ada', 'London' ] ada ``` In node-redis the client options for the nodes (password, TLS) go in `defaults`, because the nodes it discovers do not inherit the options of the root nodes. `useReplicas: true` in node-redis and `scaleReads: 'slave'` in ioredis send reads to replicas. **Go** `go-redis` (works with v9.22.0 and later): `go get github.com/redis/go-redis/v9`. ```go package main import ( "context" "fmt" "github.com/redis/go-redis/v9" ) func main() { ctx := context.Background() options := &redis.ClusterOptions{ Addrs: []string{"127.0.0.1:7000", "127.0.0.1:7001"}, Password: "cluster-password", } rdb := redis.NewClusterClient(options) defer rdb.Close() rdb.Set(ctx, "user:1000", "ada", 0) fmt.Println(rdb.Get(ctx, "user:1000").Val()) // Keys with a common hash tag live in one slot, so a multi-key command works. rdb.MSet(ctx, "{user:1000}.name", "Ada", "{user:1000}.city", "London") fmt.Println(rdb.MGet(ctx, "{user:1000}.name", "{user:1000}.city").Val()) // Read from replicas: a second client with ReadOnly (reads go to the primary or its replicas). replicaOptions := *options replicaOptions.ReadOnly = true replicaOptions.RouteRandomly = true rr := redis.NewClusterClient(&replicaOptions) defer rr.Close() fmt.Println(rr.Get(ctx, "user:1000").Val()) } ``` ```text ada [Ada London] ada ``` `ReadOnly: true` lets the client read from replicas, and `RouteRandomly: true` spreads those reads over the primary and its replicas. **Java** Jedis (works with 5.2.0 and later): Maven `redis.clients:jedis:5.2.0`. ```java import java.util.List; import java.util.Set; import redis.clients.jedis.DefaultJedisClientConfig; import redis.clients.jedis.HostAndPort; import redis.clients.jedis.JedisCluster; public class JedisBasic { public static void main(String[] args) { Set seeds = Set.of(new HostAndPort("127.0.0.1", 7000), new HostAndPort("127.0.0.1", 7001)); DefaultJedisClientConfig config = DefaultJedisClientConfig.builder().password("cluster-password").build(); try (JedisCluster jc = new JedisCluster(seeds, config)) { jc.set("user:1000", "ada"); System.out.println(jc.get("user:1000")); // Keys with a common hash tag live in one slot, so a multi-key command works. jc.mset("{user:1000}.name", "Ada", "{user:1000}.city", "London"); List values = jc.mget("{user:1000}.name", "{user:1000}.city"); System.out.println(values); } } } ``` ```text ada [Ada, London] ``` Lettuce (works with 6.5.0 and later): Maven `io.lettuce:lettuce-core:6.5.0.RELEASE`. ```java import io.lettuce.core.ReadFrom; import io.lettuce.core.RedisURI; import io.lettuce.core.cluster.RedisClusterClient; import io.lettuce.core.cluster.api.StatefulRedisClusterConnection; import io.lettuce.core.cluster.api.sync.RedisAdvancedClusterCommands; import java.util.List; public class LettuceBasic { public static void main(String[] args) { char[] password = "cluster-password".toCharArray(); RedisClusterClient client = RedisClusterClient.create(List.of( RedisURI.Builder.redis("127.0.0.1", 7000).withPassword(password).build(), RedisURI.Builder.redis("127.0.0.1", 7001).withPassword(password).build())); try (StatefulRedisClusterConnection connection = client.connect()) { RedisAdvancedClusterCommands commands = connection.sync(); commands.set("user:1000", "ada"); System.out.println(commands.get("user:1000")); // Keys with a common hash tag live in one slot, so a multi-key command works. commands.mset(java.util.Map.of("{user:1000}.name", "Ada", "{user:1000}.city", "London")); System.out.println(commands.mget("{user:1000}.name", "{user:1000}.city")); // Read from replicas. connection.setReadFrom(ReadFrom.REPLICA_PREFERRED); System.out.println(commands.get("user:1000")); } finally { client.shutdown(); } } } ``` ```text ada [KeyValue[{user:1000}.name, Ada], KeyValue[{user:1000}.city, London]] ada ``` Lettuce reads from replicas with `connection.setReadFrom(...)`. The Jedis snippet has no replica reads: it uses the primaries only. ## Survive a failover A client in a cluster should ride out a failover: the slot moves to another node, the old connection closes, and for a moment commands for those slots get `CLUSTERDOWN` or a refused connection. A cluster-aware client refreshes its slot map and retries those commands, so a coordinated failover (`CLUSTER FAILOVER`, `docker compose stop` with `--shutdown-on-sigterm failover`, a rolling update) pauses writes for a moment and hands the slot over without losing a write. A crash is different: the slot has no primary until the other nodes agree that it is gone, which takes the node timeout (15 seconds by default) plus the election, and a client that gives up sooner reports errors in that time. See [Replicas and failover](https://docs.hotkv.com/clustering/replicas-and-failover/index.md) for the timeline. How long a client library retries before it raises an error is a setting of that library. Set the retry count and timeout of your library so that they cover the node timeout plus the election. ## HotKV SDK cluster clients The Python and Node.js SDKs have a cluster client that routes every command, including the typed commands of the AI engines, by the key or name it carries, follows `MOVED` and `ASK`, and rides out a failover, a reshard in progress, `CLUSTERDOWN`, `TRYAGAIN` and `LOADING` by refreshing the slot map and trying again for up to 10 seconds (`retry_timeout` in Python, `retryTimeoutMs` in Node.js) before it raises `ClusterError`. A command that may have reached a node whose connection then broke is sent again, so a command that is not idempotent (`incr`, `rpush`, a budget consume) can be applied twice; pass `retry_writes=False` (`retryWrites: false`) to get the error instead. Install them as shown on [Connecting and SDKs](https://docs.hotkv.com/sdks/index.md). **Python** `ClusterHotKV`, in the `hotkv.cluster` module: ```python from hotkv.cluster import ClusterHotKV cluster = ClusterHotKV( [("127.0.0.1", 7000), ("127.0.0.1", 7001)], password="cluster-password", decode_responses=True, ) cluster.set("user:1000", "ada") print(cluster.get("user:1000")) # Keys with a common hash tag live in one slot, so a multi-key command works. print(cluster.execute("MSET", "{user:1000}.name", "Ada", "{user:1000}.city", "London")) print(cluster.execute("MGET", "{user:1000}.name", "{user:1000}.city")) # Commands that list one node's data are sent to every primary and merged. print(cluster.dbsize()) print(sorted(cluster.keys("{user:1000}*"))) # A command for one node's own state goes where you say. print(cluster.execute_on_primaries("CONFIG", "GET", "maxmemory-policy")) # Read from replicas: replica connections send READONLY. replicas = ClusterHotKV( [("127.0.0.1", 7000)], password="cluster-password", decode_responses=True, read_from_replicas=True, ) print(replicas.get("user:1000")) replicas.close() cluster.close() ``` ```text ada True ['Ada', 'London'] 103 ['{user:1000}.city', '{user:1000}.name'] {('127.0.0.1', 7000): {'maxmemory-policy': 'noeviction'}, ('127.0.0.1', 7001): {'maxmemory-policy': 'noeviction'}, ('127.0.0.1', 7002): {'maxmemory-policy': 'noeviction'}} ada ``` The asyncio client has the same methods as coroutines: ```python import asyncio from hotkv.asyncio import ClusterHotKV async def main(): cluster = ClusterHotKV( [("127.0.0.1", 7000), ("127.0.0.1", 7001)], password="cluster-password", decode_responses=True, ) await cluster.set("user:1000", "ada") print(await cluster.get("user:1000")) print(await cluster.execute("MGET", "{user:1000}.name", "{user:1000}.city")) await cluster.close() asyncio.run(main()) ``` ```text ada ['Ada', 'London'] ``` **Node.js** `HotKVCluster`, exported by the `hotkv` package: ```javascript import { HotKVCluster } from '@hotkv/client'; const cluster = new HotKVCluster(['127.0.0.1:7000', '127.0.0.1:7001'], { password: 'cluster-password', }); await cluster.set('user:1000', 'ada'); console.log(await cluster.get('user:1000')); // Keys with a common hash tag live in one slot, so a multi-key command works. console.log(await cluster.execute(['MSET', '{user:1000}.name', 'Ada', '{user:1000}.city', 'London'])); const values = await cluster.execute(['MGET', '{user:1000}.name', '{user:1000}.city']); console.log(values.map((v) => v.toString())); // Commands that list one node's data are sent to every primary and merged. console.log(await cluster.dbSize()); // A command for one node's own state goes to every primary when you ask. const perNode = await cluster.executeOnPrimaries(['CONFIG', 'GET', 'maxmemory-policy']); console.log([...perNode.keys()]); // Read from replicas: replica connections send READONLY. const replicas = new HotKVCluster(['127.0.0.1:7000'], { password: 'cluster-password', readFromReplicas: true, }); console.log(await replicas.get('user:1000')); await replicas.close(); await cluster.close(); ``` ```text ada OK [ 'Ada', 'London' ] 103 [ '127.0.0.1:7000', '127.0.0.1:7001', '127.0.0.1:7002' ] ada ``` ### Commands that answer for one node Some commands report on one node's own data: the `LIST` commands of the AI engines, `KEYS`, `DBSIZE`, `FLUSHALL`, `TAG.MEMBERS`, and the script and function commands (see [What cannot be clustered](https://docs.hotkv.com/clustering/keys-and-commands/index.md#what-cannot-be-clustered)). The HotKV cluster clients send these to every primary and merge the answers, so they cover the whole cluster: `dbsize()` above counted the keys of all three primaries (the number is however many keys your cluster holds), and `pcache_list()` lists the prompt-cache namespaces of the whole cluster: ```python from hotkv.cluster import ClusterHotKV cluster = ClusterHotKV([("127.0.0.1", 7000), ("127.0.0.1", 7001)], decode_responses=True) for ns in ("notes", "faqs", "plans", "agents"): cluster.pcache_create(ns) cluster.pcache_set(ns, "question", "answer") print(cluster.pcache_get("faqs", "question")) # pcache_list() asks every primary and merges, so it lists the whole cluster. print(sorted(cluster.pcache_list())) cluster.close() ``` ```text answer ['agents', 'faqs', 'notes', 'plans'] ``` (The AI engines need a license on every node: see [AI engines in a cluster](https://docs.hotkv.com/clustering/ai-engines/index.md).) State that belongs to one node, such as an ACL user, is not merged: the client sends `ACL`, `CONFIG`, `INFO` and similar commands to one primary, and you send them to a node you choose with `execute_on_node` (Python) or `executeOnNode` (Node.js), or to every primary with `execute_on_primaries` or `executeOnPrimaries`, which return the answer of each node, as the `CONFIG GET` calls in the snippets above did. An ACL user created on one node does not exist on the others, or on its replicas. ## Other HotKV SDKs Each of these SDKs has a cluster client that builds its slot map from the cluster and follows `MOVED` and `ASK`. Install them as shown on [Connecting and SDKs](https://docs.hotkv.com/sdks/index.md). | Language | Cluster class | Replica reads | | --- | --- | --- | | Java | `com.hotkv.HotKVCluster` | `commandReadOnly` | | C# | `HotKV.Cluster.HotKVCluster` | `ReadFromReplicas = true` | | C++ | `hotkv::ClusterClient` | `set_read_from_replicas(true)` | | PHP | `HotKV\Cluster\ClusterClient` | the `readFromReplicas` option | | Ruby | `HotKV::Cluster` | - | | Rust | `hotkv::ClusterClient` | `ClusterOptions::read_from_replicas` with `execute_readonly` | | Elixir | `HotKV.Cluster` | - | | Perl | `HotKV::Cluster` | the `read_from_replicas` option | The Go and C SDKs have a function that computes the slot of a key (`KeySlot`, `hotkv_key_slot`) and recognise redirect errors. In Go, use go-redis's `ClusterClient` as shown above. ## Next steps - [Keys, slots and commands](https://docs.hotkv.com/clustering/keys-and-commands/index.md): hash tags and what answers per node. - [Replicas and failover](https://docs.hotkv.com/clustering/replicas-and-failover/index.md): what a client sees during a failover. - [Cluster TLS](https://docs.hotkv.com/clustering/tls/index.md): the connection options for TLS.