AI engines in a cluster

Last updated 12 Oct 2026

Not on HotKV CloudOn-Premise: no license needed

HotKV’s AI engines (the prompt, semantic, tool, negative, guard and rerank caches, vector search, RAG pipelines, agent memory, LLM budgets, statistics and rate limits, the prompt registry, embedding de-duplication, the feature store and time series) are clustered like keys: every entry has a name, the name decides the hash slot, and the entry lives on the node that serves that slot. You use the same commands as on a single server, and a cluster-aware client sends each one to the right node. This page shows where each kind of entry goes, what a replica and a failover keep, how to put entries that belong together on one node, which state is shared by all nodes and which is not, and what a reshard does with entries.

The examples use six nodes (three primaries, three replicas) on ports 7000 to 7005. Cluster mode needs no license, but the engines do: give every node the license file with the license-path setting, as described in AI features. The license belongs to the node, so LICENSE.CHECK answers for the node you ask:

$ redis-cli -p 7000 LICENSE.CHECK promptcache; redis-cli -p 7003 LICENSE.CHECK promptcache
1
1

Where an entry lives

An entry is found by its name: the cache namespace, the index name, the pipeline name, the agent or session id, the budget, bucket or limiter name, the template name, the series name. CLUSTER KEYSLOT of that name is the slot, and a node that does not serve the slot answers MOVED for the command, just as it does for a key.

Commands	Routed by
`PCACHE.`, `TOOL.CACHE.`, `NCACHE.`, `GUARD.`, `RERANK.*`	the namespace
`SCACHE.*`	the namespace
`VSIM.*` (indexes), except `VSIM.SAVE`, `VSIM.LOAD`, `VSIM.DROP` and `VSIM.LIST`	the index name
`RAG.*`	the pipeline name
`AMEM.CONV.`, `AMEM.WORK.`	the session id
`AMEM.SEM.`, `AMEM.EPIS.`, `AMEM.SEARCH`, `AMEM.SUMMARY.*`	the agent id
`LLMBUDGET.`, `RATELIMIT.`	the budget or limiter name
`LLMSTATS.*` (buckets and records)	the bucket name
`PROMPT.`, `EMBED.DEDUP.`, `FEATURE.`, `TS.`	the template, namespace, feature group or series name

On a node that does not serve the slot of a name, the command is refused with MOVED and the slot you get from CLUSTER KEYSLOT. redis-cli -c follows the redirect:

127.0.0.1:7000> CLUSTER KEYSLOT faq
(integer) 16230
127.0.0.1:7000> SCACHE.CREATE faq 3
(error) MOVED 16230 127.0.0.1:7002
127.0.0.1:7000> CLUSTER KEYSLOT products
(integer) 10285
127.0.0.1:7000> VSIM.CREATEINDEX products 3 METRIC COSINE
(error) MOVED 10285 127.0.0.1:7001
127.0.0.1:7000> CLUSTER KEYSLOT api-limit
(integer) 11870
127.0.0.1:7000> RATELIMIT.CREATE api-limit 100 60000
(error) MOVED 11870 127.0.0.1:7002

$ redis-cli -c -p 7000
127.0.0.1:7000> PCACHE.CREATE prompts
OK
127.0.0.1:7000> SCACHE.CREATE faq 3
-> Redirected to slot [16230] located at 127.0.0.1:7002
OK
127.0.0.1:7002> VSIM.CREATEINDEX products 3 METRIC COSINE
-> Redirected to slot [10285] located at 127.0.0.1:7001
OK
127.0.0.1:7001> RATELIMIT.CREATE api-limit 100 60000
-> Redirected to slot [11870] located at 127.0.0.1:7002
OK
127.0.0.1:7002> LLMBUDGET.SET team-budget 100000 5.0 3600
-> Redirected to slot [521] located at 127.0.0.1:7000
OK
127.0.0.1:7000> RAG.CREATE support-docs 3
-> Redirected to slot [10518] located at 127.0.0.1:7001
OK
127.0.0.1:7001> AMEM.CONV.CREATE sess-1
-> Redirected to slot [16334] located at 127.0.0.1:7002
OK
127.0.0.1:7002> AMEM.SEM.CREATE agent-7 3
OK
127.0.0.1:7002> TS.CREATE sensor-1
OK
127.0.0.1:7002> PROMPT.PUT greeting "Hello {{name}}"
(integer) 1

The remaining families route the same way: each command is answered by the primary that serves the slot of the name.

Command	Name	Slot
`TOOL.CACHE.CREATE`	`tools`	11861
`NCACHE.CREATE`	`misses`	545
`GUARD.CREATE`	`safety`	7426
`RERANK.CREATE`	`ranks`	12077
`LLMSTATS.RECORD`	`api-bucket`	5653
`AMEM.WORK.SET`	`sess-9`	16070
`AMEM.EPIS.LOG`, `AMEM.SUMMARY.SET`	`agent-9`	3625
`AMEM.SEARCH`	`agent-7`	12263

A cluster-aware client does this routing for you. The Python and Node.js HotKV client libraries route every command in the table by its name: see Cluster clients.

An entry is never split between nodes. A namespace, an index, a limiter, a budget, a time series or an agent’s memory is held by exactly one node, so its size and its request rate are bounded by that node. To spread the load, use several names, for example one cache namespace per tenant. An entry larger than 512 MB cannot be moved by a reshard.

Some entries belong together but have different names, and so land on different nodes: an agent’s conversation sessions (AMEM.CONV, by session id) and its long-term memory (AMEM.SEM, by agent id), or a cache namespace and the vector index you keep beside it. Give them a common hash tag and they share a slot, and so a node, and move together:

$ redis-cli -c -p 7000
127.0.0.1:7000> CLUSTER KEYSLOT sess-2
(integer) 4013
127.0.0.1:7000> CLUSTER KEYSLOT agent-8
(integer) 7688
127.0.0.1:7000> CLUSTER KEYSLOT {agent-8}:sess-2
(integer) 7688
127.0.0.1:7000> CLUSTER KEYSLOT {agent-8}
(integer) 7688
127.0.0.1:7000> AMEM.CONV.CREATE {agent-8}:sess-2
-> Redirected to slot [7688] located at 127.0.0.1:7001
OK
127.0.0.1:7001> AMEM.SEM.CREATE {agent-8} 3
OK
127.0.0.1:7001> CLUSTER KEYSLOT {docs}:cache
(integer) 13556
127.0.0.1:7001> CLUSTER KEYSLOT {docs}:index
(integer) 13556
127.0.0.1:7001> SCACHE.CREATE {docs}:cache 3
-> Redirected to slot [13556] located at 127.0.0.1:7002
OK
127.0.0.1:7002> VSIM.CREATEINDEX {docs}:index 3
OK

A RAG pipeline needs no tag for its own vectors: they are kept with the pipeline, under its name.

TS.DOWNSAMPLE is the one AI command that names two entries, the source series and the destination series, so they must be in the same slot, like the keys of a multi-key command. Give the pair a common tag, and the rule and both series stay on one node:

127.0.0.1:7000> TS.CREATE sensor-2:raw
-> Redirected to slot [5485] located at 127.0.0.1:7001
OK
127.0.0.1:7001> TS.CREATE sensor-2:1m
-> Redirected to slot [12175] located at 127.0.0.1:7002
OK
127.0.0.1:7002> TS.DOWNSAMPLE sensor-2:raw sensor-2:1m avg 60000
(error) CROSSSLOT Keys in request don't hash to the same slot
127.0.0.1:7002> TS.CREATE {sensor-3}:raw
-> Redirected to slot [5109] located at 127.0.0.1:7000
OK
127.0.0.1:7000> TS.CREATE {sensor-3}:1m
OK
127.0.0.1:7000> TS.DOWNSAMPLE {sensor-3}:raw {sensor-3}:1m avg 60000
OK

Tables every node holds

A few tables are not sharded. Every node holds the whole table: the LLM price table (LLMSTATS.PRICE), the tenants (TENANT.*, except TENANT.STATS), and the IAM roles and who has them (IAM.ROLE.*, IAM.USER.ASSIGN, IAM.USER.UNASSIGN). You change them on any primary, and the change reaches the other nodes. A replica refuses the change, as it refuses any write.

Here the price table and a tenant are written on one primary, and read from another:

127.0.0.1:7000> LLMSTATS.PRICE SET gpt-4o INPUT 2.5 OUTPUT 10
OK
127.0.0.1:7000> TENANT.CREATE acme "Acme Corp" MAX_KEYS 1000000
OK
127.0.0.1:7002> LLMSTATS.PRICE GET gpt-4o
 1) "model"
 2) "gpt-4o"
 3) "source"
 4) "custom"
 5) "priced"
 6) (integer) 1
 7) "retired"
 8) (integer) 0
 9) "input_usd_per_mtok"
10) "2.500000"
11) "output_usd_per_mtok"
12) "10.000000"
13) "cache_read_usd_per_mtok"
14) "0.250000"
15) "cache_write_5m_usd_per_mtok"
16) "3.125000"
17) "cache_write_1h_usd_per_mtok"
18) "5.000000"
127.0.0.1:7002> TENANT.LIST
1) "acme"

State that belongs to one node

Some state is per node and the cluster does not copy it:

The license. Each node has its own, as shown above.
Encryption at rest, the audit log, OBSERVE and ALERTS. Each node keeps its own, and they are not copied to the other nodes.
Listings. The LIST commands of the engines (PCACHE.LIST and the others), TS.LIST and VSIM.INFO without an index name answer for the node you ask. To list the whole cluster, ask every primary and merge.
Sessions, hot keys and usage counters. IAM.SESSION.* (a login belongs to the node it is on), HOTKEYS, TENANT.STATS, and INFO, CONFIG, CLIENT, SLOWLOG and LATENCY answer for the node you ask.
Vector snapshots. VSIM.SAVE, VSIM.LOAD, VSIM.DROP and VSIM.LIST are a manual store of named snapshots on the node you send them to. The live indexes are the ones that shard.
REPLICATION.*. These commands act on one node and are for a standalone primary and replica. A cluster promotes with CLUSTER FAILOVER and fails over on its own.
Limits. The count limits an engine applies to its own commands (the number of cache namespaces, limiters, buckets and so on) are checked by each node for the entries it creates. An entry a reshard brings in is accepted past the limit, so a node can hold more than its limit until some are dropped.
Scripts, functions and ACL users, as on any cluster: see What cannot be clustered.

$ for p in 7000 7001 7002; do echo "port $p: $(redis-cli -p $p PCACHE.LIST | tr "\n" " ")"; done
port 7000: prompts summaries
port 7001: answers
port 7002: tools
$ for addr in $(redis-cli -p 7000 CLUSTER NODES | awk "/master/ {sub(/@.*/, \"\", \$2); print \$2}"); do redis-cli -h ${addr%:*} -p ${addr#*:} PCACHE.LIST; done | sort
answers
prompts
summaries
tools
$ for p in 7000 7001 7002; do echo "port $p: $(redis-cli -p $p VSIM.INFO | tr "\n" " ")"; done
port 7000:
port 7001: products
port 7002: {docs}:index
$ for p in 7000 7001 7002; do echo "port $p: $(redis-cli -p $p TS.LIST | tr "\n" " ")"; done
port 7000: {sensor-3}:raw {sensor-3}:1m
port 7001: sensor-2:raw
port 7002: sensor-2:1m sensor-1

The second command lists the primaries from CLUSTER NODES and merges what each one lists.

The engines are not in databases. An entry is found by its name whatever database the connection selected, and FLUSHDB leaves the entries alone: see Databases.

Replication and failover

The writes of the engines reach the replicas and the append-only file like the writes of keys, so a promoted replica has the entries. Write one entry of each of these engines on its primary (these commands, sent through redis-cli -c, add to what the earlier examples created), fail every primary over with CLUSTER FAILOVER on its replica, one after the other, and read the data back through the new primaries:

redis-cli -c -p 7000 <<'EOF'
PCACHE.CREATE answers
PCACHE.CREATE summaries
PCACHE.CREATE tools
PCACHE.SET prompts "capital of France" Paris
SCACHE.SET faq "How do I reset my password?" "Use the account page" 0.1 0.9 0.2
VSIM.SET products sku-1 0.1 0.9 0.2
RATELIMIT.CHECK api-limit user-1
LLMBUDGET.CONSUME team-budget 1200 0.05
TS.ADD sensor-1 1700000000000 21.5
AMEM.SEM.STORE agent-7 fact-1 "Ada likes tea" 0.1 0.2 0.3
AMEM.CONV.APPEND sess-1 user Hello
RAG.INGEST support-docs doc:1 "Reset your password on the login page." VECTORS 0.9,0.1,0.0
EMBED.DEDUP.PUT ns1 h1 model abc
FEATURE.CREATE users "Users" SCHEMA age int64 country string
FEATURE.SET users user:1 age 31 country DE
EOF

Then the failover of every primary:

$ for p in 7003 7004 7005; do redis-cli -p $p CLUSTER FAILOVER; until [ "$(redis-cli -p $p ROLE | head -1)" = master ]; do sleep 0.5; done; done
OK
OK
OK
$ redis-cli -p 7000 CLUSTER NODES | awk "{print \$2, \$3, \$9}" | sort
127.0.0.1:7000@17000 myself,slave
127.0.0.1:7001@17001 slave
127.0.0.1:7002@17002 slave
127.0.0.1:7003@17003 master 10923-16383
127.0.0.1:7004@17004 master 0-5460
127.0.0.1:7005@17005 master 5461-10922

127.0.0.1:7000> PCACHE.GET prompts "capital of France"
-> Redirected to slot [1197] located at 127.0.0.1:7004
"Paris"
127.0.0.1:7004> SCACHE.GET faq 0.1 0.9 0.2
-> Redirected to slot [16230] located at 127.0.0.1:7003
1) "Use the account page"
2) "1"
127.0.0.1:7003> VSIM.SEARCH products 1 0.1 0.9 0.2
-> Redirected to slot [10285] located at 127.0.0.1:7005
1) "sku-1"
2) "0"
127.0.0.1:7005> RATELIMIT.CHECK api-limit user-1
-> Redirected to slot [11870] located at 127.0.0.1:7003
1) (integer) 1
2) (integer) 98
3) (integer) 0
4) (integer) 100
5) (integer) 1
127.0.0.1:7003> LLMBUDGET.STATUS team-budget
-> Redirected to slot [521] located at 127.0.0.1:7004
 1) "tokens_used"
 2) (integer) 1200
 3) "token_limit"
 4) (integer) 100000
 5) "cost_used_usd"
 6) "0.050000"
 7) "cost_limit_usd"
 8) "5.000000"
 9) "window_seconds"
10) (integer) 3600
11) "window_remaining"
12) (integer) 3591
13) "requests_allowed"
14) (integer) 1
15) "requests_rejected"
16) (integer) 0
17) "unpriced_requests"
18) (integer) 0
127.0.0.1:7004> PROMPT.GET greeting
-> Redirected to slot [12714] located at 127.0.0.1:7003
1) (integer) 1
2) "Hello {{name}}"
3) ""
4) (integer) 1791065805
127.0.0.1:7003> TS.RANGE sensor-1 - +
1) 1) (integer) 1700000000000
   2) "21.500000"
127.0.0.1:7003> AMEM.SEM.RECALL agent-7 1 0.1 0.2 0.3
1) 1) "key"
   2) "fact-1"
   3) "content"
   4) "Ada likes tea"
   5) "similarity"
   6) "0.9999999"
127.0.0.1:7003> AMEM.CONV.GET sess-1
1) 1) "role"
   2) "user"
   3) "content"
   4) "Hello"
127.0.0.1:7003> RAG.SEARCH support-docs "reset password" K 1 VECTOR 0.9,0.1,0.0
-> Redirected to slot [10518] located at 127.0.0.1:7005
1) 1) "doc:1:0"
   2) "doc:1"
   3) "Reset your password on the login page."
   4) "1.000000"
   5) "1.000000"
   6) "0.575364"
127.0.0.1:7005> EMBED.DEDUP.GET ns1 h1
-> Redirected to slot [15122] located at 127.0.0.1:7003
"abc"
127.0.0.1:7003> FEATURE.GET users user:1
1) 1) "age"
   2) (integer) 31
   3) "country"
   4) "DE"
2) (integer) 1791065810600
3) (integer) 4259

The prompt cache, semantic cache, vector index, rate limiter, budget, prompt registry, time series, semantic and conversation memory, RAG pipeline, de-duplication cache and feature store all answered from the promoted replica. The rate limiter and the budget kept their counters (98 requests left after the second check, 1200 tokens used). The price table, written before the failover, was there too:

127.0.0.1:7004> LLMSTATS.PRICE GET gpt-4o
 1) "model"
 2) "gpt-4o"
 3) "source"
 4) "custom"
 5) "priced"
 6) (integer) 1
 7) "retired"
 8) (integer) 0
 9) "input_usd_per_mtok"
10) "2.500000"
11) "output_usd_per_mtok"
12) "10.000000"
13) "cache_read_usd_per_mtok"
14) "0.250000"
15) "cache_write_5m_usd_per_mtok"
16) "3.125000"
17) "cache_write_1h_usd_per_mtok"
18) "5.000000"

Time to live and clocks

An entry written with a time to live expires by the clock of each node that holds it. Keep the clocks of all nodes of a cluster in step with NTP: a replica whose clock runs ahead of its primary’s drops entries earlier than the primary does, and one that runs behind keeps them longer.

What a reshard does with entries

Engine entries have no key, so MIGRATE never sees them. A reshard moves them along with the slot anyway, whichever tool you use. Each of the four ways below moves a slot that holds an entry, and the entry is read from the new owner.

CLUSTER MIGRATESLOTS moves the keys and the entries of the slot in one atomic step. Run it on the node that serves the slot, with the target’s node ID:

$ redis-cli -p 7000 CLUSTER NODES | awk "/master/ {print \$2, \$9}" | sort
127.0.0.1:7003@17003 10923-16383
127.0.0.1:7004@17004 0-5460
127.0.0.1:7005@17005 5461-10922
$ SRC=$(redis-cli -p 7004 CLUSTER MYID); DST=$(redis-cli -p 7005 CLUSTER MYID)
$ redis-cli -p 7004 CLUSTER MIGRATESLOTS SLOTSRANGE 1197 1197 NODE $DST
OK
$ redis-cli -p 7000 CLUSTER NODES | awk "/master/ {print \$2, \$9, \$10, \$11}" | sort
127.0.0.1:7003@17003 10923-16383
127.0.0.1:7004@17004 0-1196 1198-5460
127.0.0.1:7005@17005 1197 5461-10922
127.0.0.1:7000> PCACHE.GET prompts "capital of France"
-> Redirected to slot [1197] located at 127.0.0.1:7005
"Paris"

valkey-cli --cluster reshard --cluster-use-atomic-slot-migration asks the nodes for the same atomic migration. The slot it moves first is the lowest slot of the source; the name 06S is in slot 0, which is why the example uses it:

127.0.0.1:7000> CLUSTER KEYSLOT 06S
(integer) 0
127.0.0.1:7000> PCACHE.CREATE 06S
-> Redirected to slot [0] located at 127.0.0.1:7004
OK
127.0.0.1:7004> PCACHE.SET 06S question answer
OK
$ SRC=$(redis-cli -p 7004 CLUSTER MYID); DST=$(redis-cli -p 7005 CLUSTER MYID)
$ valkey-cli --cluster reshard 127.0.0.1:7000 --cluster-from $SRC --cluster-to $DST --cluster-slots 1 --cluster-yes --cluster-use-atomic-slot-migration | tail -5
       slots:[1197],[5461-10922] (5463 slots) master
       1 additional replica(s)
  Resharding plan:
    Moving slot range 0-0 from f1e432f739a537dc459403c56b61eeaf10e78aac
Moving slot range 0-0 from 127.0.0.1:7004 to 127.0.0.1:7005 via atomic slot migration.#
127.0.0.1:7000> PCACHE.GET 06S question
-> Redirected to slot [0] located at 127.0.0.1:7005
"answer"

redis-cli --cluster reshard moves keys with MIGRATE, one at a time. The entries move when the source is told the slot is migrating (the first step of the manual procedure below), and the tool needs no change:

$ redis-cli -p 7000 CLUSTER NODES | awk "/master/ {print \$2, \$9, \$10, \$11, \$12}" | sort
127.0.0.1:7003@17003 10923-16229 16231-16383
127.0.0.1:7004@17004 1-1196 1198-5460
127.0.0.1:7005@17005 0 1197 5461-10922 16230
$ SRC=$(redis-cli -p 7005 CLUSTER MYID); DST=$(redis-cli -p 7004 CLUSTER MYID)
$ redis-cli --cluster reshard 127.0.0.1:7000 --cluster-from $SRC --cluster-to $DST --cluster-slots 1 --cluster-yes | tail -3
  Resharding plan:
    Moving slot 0 from 51ee6626e7f80308c24fd9ceb9b556b6c3a9dcd7
Moving slot 0 from 127.0.0.1:7005 to 127.0.0.1:7004:
127.0.0.1:7000> PCACHE.GET 06S question
-> Redirected to slot [0] located at 127.0.0.1:7004
"answer"

Step by step with CLUSTER SETSLOT. SETSLOT MIGRATING on the source moves the entries of the slot to the target before it returns. While the slot is open, a command for an entry that has moved gets ASK, and the target serves it after ASKING:

$ SRC=$(redis-cli -p 7003 CLUSTER MYID); DST=$(redis-cli -p 7005 CLUSTER MYID)
$ redis-cli -p 7005 CLUSTER SETSLOT 16230 IMPORTING $SRC
OK
$ redis-cli -p 7003 CLUSTER SETSLOT 16230 MIGRATING $DST
OK
127.0.0.1:7003> SCACHE.GET faq 0.1 0.9 0.2
(error) ASK 16230 127.0.0.1:7005
127.0.0.1:7005> SCACHE.GET faq 0.1 0.9 0.2
(error) MOVED 16230 127.0.0.1:7003
127.0.0.1:7005> ASKING
OK
127.0.0.1:7005> SCACHE.GET faq 0.1 0.9 0.2
1) "Use the account page"
2) "1"
$ redis-cli -p 7005 CLUSTER SETSLOT 16230 NODE $DST; redis-cli -p 7003 CLUSTER SETSLOT 16230 NODE $DST
OK
OK
127.0.0.1:7000> SCACHE.GET faq 0.1 0.9 0.2
-> Redirected to slot [16230] located at 127.0.0.1:7005
1) "Use the account page"
2) "1"

If the target cannot take every entry of a slot, SETSLOT MIGRATING fails and the slot stays where it was, so you can retry. The nodes use the cluster password to talk to each other during a reshard: give every node the same requirepass, and the same masterauth for the atomic migration, as described in Deploy on VMs and cloud.

The entries arrive complete: a semantic cache entry can still be found by its embedding on the new owner.

Next steps

Keys, slots and commands: hash tags and what answers per node.
Cluster clients: the HotKV client libraries route the AI commands for you.
Scaling a cluster: reshard methods in full.

Ask the HotKV docs

AI engines in a cluster

Where an entry lives

Tables every node holds

State that belongs to one node

Replication and failover

Time to live and clocks

What a reshard does with entries

Next steps

Was this page helpful?

AI engines in a cluster

Where an entry lives

Keep related entries together

Tables every node holds

State that belongs to one node

Replication and failover

Time to live and clocks

What a reshard does with entries

Next steps

Was this page helpful?