# Client-side caching

> Keep copies of values in your application's memory and let HotKV tell you when a key changes, with CLIENT TRACKING.

URL: https://docs.hotkv.com/develop/client-side-caching/


A read answered from your application's own memory takes no network round
trip. The hard part is knowing when a copy has gone stale. With server-assisted
client-side caching, a connection asks HotKV to remember which keys it reads,
and HotKV sends it an invalidation when one of them changes. The application
keeps its copy until that message arrives.

You turn it on for one connection with
[CLIENT TRACKING](https://docs.hotkv.com/commands/client-tracking/index.md). A connection that does not use
it is not affected.

## How it works

1. The client reads a key with an ordinary read command such as
   [GET](https://docs.hotkv.com/commands/get/index.md). HotKV notes that this connection read the key, and a
   change made at the same moment is never missed.
2. The client keeps the value in its local cache.
3. When the key next changes, HotKV sends the client an invalidation and
   forgets the note. The client drops the key from its cache.
4. The next time the client reads the key it is noted again, so the client is
   told of the next change, and only then.

A key is told once for each time it was read. A client that wants to hear about
a key it still uses must read it again after each invalidation.

## Try it with redis-cli

A RESP2 connection cannot receive messages it did not ask for, so it sends its
invalidations to a second connection that is subscribed to a channel. Open three
`redis-cli` sessions.

The listener subscribes to the channel HotKV publishes invalidations on and
tells you its connection id:

```
127.0.0.1:6379> CLIENT ID
(integer) 52
127.0.0.1:6379> SUBSCRIBE __redis__:invalidate
1) "subscribe"
2) "__redis__:invalidate"
3) (integer) 1
```

The reader turns tracking on, redirecting its invalidations to the listener, and
reads a key:

```
127.0.0.1:6379> CLIENT TRACKING ON REDIRECT 52
OK
127.0.0.1:6379> GET user:1
(nil)
```

A third session changes the key:

```
127.0.0.1:6379> SET user:1 bob
OK
```

The listener is sent the invalidation:

```
1) "message"
2) "__redis__:invalidate"
3) 1) "user:1"
```

The reader read `user:1` before it existed. A key that is missing is noted too,
so the client is told when it appears.

## RESP3: one connection

A RESP3 connection (after `HELLO 3`) can be pushed to, so it needs no second
connection. In the transcripts below, a line that starts with `push>` is a
message the server sent on its own, after the reply of the command that caused
it. Client libraries receive it as a push frame, `>2`, then `invalidate`, then
an array of keys.

```
# Connection A (the reply of HELLO 3 is left out)
127.0.0.1:6379> HELLO 3
127.0.0.1:6379> CLIENT TRACKING ON
OK
127.0.0.1:6379> SET user:1 alice
OK
127.0.0.1:6379> MGET user:1 user:2
1) "alice"
2) (nil)

# Connection B
127.0.0.1:6379> SET user:1 bob
OK

# Connection A is sent, without asking:
push> 1) "invalidate"
      2) 1) "user:1"
```

Connection A read both keys, so it is told of either. It is not told again when
`user:1` changes a second time, because it has not read the key since. When
`user:2` is created it is told:

```
# Connection B
127.0.0.1:6379> SET user:1 carol
OK
127.0.0.1:6379> SET user:2 dave
OK

# Connection A
push> 1) "invalidate"
      2) 1) "user:2"
```

A change that empties a whole database is told with a null in place of the keys,
which means "forget everything you cached":

```
# Connection B
127.0.0.1:6379> FLUSHDB
OK

# Connection A
push> 1) "invalidate"
      2) (nil)
```

## Modes

`CLIENT TRACKING ON` takes options that decide which keys a client is told of.

| Mode | The client is told of |
| --- | --- |
| Default | The keys it read. |
| `OPTIN` | The keys read by a command that comes right after `CLIENT CACHING YES`. |
| `OPTOUT` | The keys it read, except those of a command that comes right after `CLIENT CACHING NO`. |
| `BCAST` | Every key that starts with one of its `PREFIX` values, whether it read the key or not. HotKV remembers nothing per key for this client. |

`NOLOOP`, in any mode, leaves out the changes made by the client's own commands:
it updates its cache itself when it writes. `REDIRECT client-id` sends the
invalidations to another connection (see
[Where invalidations go](#where-invalidations-go)).

`OPTIN` and `OPTOUT` cannot be used together, or with `BCAST`. A `PREFIX` needs
`BCAST`. Prefixes of one client must not overlap: one being the start of
another, or giving the same one twice, is refused with an error that names both.
Once tracking is on, `BCAST` cannot be switched on or off, and neither can
`OPTIN` for `OPTOUT`: turn tracking off first. Asking for `CLIENT TRACKING ON`
again replaces the redirect and the other options, and adds any new prefixes.

### OPTIN and OPTOUT

`CLIENT CACHING` chooses for the next command only. Sent before `MULTI`, it
covers the commands queued up to `EXEC`.

```
# Connection A, tracking in OPTIN mode
127.0.0.1:6379> CLIENT TRACKING ON OPTIN
OK
127.0.0.1:6379> GET p:1
"1"
127.0.0.1:6379> CLIENT CACHING YES
OK
127.0.0.1:6379> GET p:2
"2"

# Connection B changes both keys
127.0.0.1:6379> SET p:1 changed
OK
127.0.0.1:6379> SET p:2 changed
OK

# Connection A is told only of the key it opted in for
push> 1) "invalidate"
      2) 1) "p:2"
```

In `OPTOUT` mode the choice is the other way round: `CLIENT CACHING NO` before a
command leaves its keys out.

### BCAST

A client in `BCAST` mode does not need to read a key to hear about it, which suits
a cache that holds everything under a prefix. The client is told of the keys one
command changed in one message for each prefix:

```
# Connection A
127.0.0.1:6379> CLIENT TRACKING ON BCAST PREFIX user: PREFIX order:
OK

# Connection B
127.0.0.1:6379> MSET user:1 x user:2 y order:9 z other:1 w
OK

# Connection A
push> 1) "invalidate"
      2) 1) "user:1"
         2) "user:2"
push> 1) "invalidate"
      2) 1) "order:9"
```

`CLIENT TRACKING ON BCAST` with no prefix covers every key. A client is told
only of the keys its ACL user may read.

### NOLOOP

```
# Connection A
127.0.0.1:6379> CLIENT TRACKING ON BCAST PREFIX n: NOLOOP
OK
127.0.0.1:6379> SET n:1 mine
OK

# Connection B
127.0.0.1:6379> SET n:1 theirs
OK

# Connection A is told of the change by B only
push> 1) "invalidate"
      2) 1) "n:1"
```

## What counts as a read, and what as a change

**Reads.** A key is noted when it is named by a command that only reads:
[GET](https://docs.hotkv.com/commands/get/index.md), [MGET](https://docs.hotkv.com/commands/mget/index.md), [HGETALL](https://docs.hotkv.com/commands/hgetall/index.md),
[EXISTS](https://docs.hotkv.com/commands/exists/index.md), [TTL](https://docs.hotkv.com/commands/ttl/index.md) and the like. A key that
does not exist is noted too. Notes are made as each command runs, so inside
[MULTI](https://docs.hotkv.com/commands/multi/index.md) and [EXEC](https://docs.hotkv.com/commands/exec/index.md) every read command notes
its keys, and a script notes the keys of the read commands it calls, for the
client that ran the script. `EVAL`, `EVALSHA` and `FCALL` note nothing
themselves. Commands that only write note nothing. The note is made by key name
whatever database the client had selected, so an invalidation can arrive for a
key of another database; the client may be told more than it needs, and never
less.

**Changes.** A key is told as changed when:

- a command that writes it has run, whether or not the command altered the value
  (inside `MULTI` and scripts too). This includes changing its time to live;
- the key expires, or the fields of a hash expire (a client that reads a key just
  as its time runs out can be told of the expiry right after the reply);
- it is evicted to stay under `maxmemory`;
- it is moved by [MIGRATE](https://docs.hotkv.com/commands/migrate/index.md) without `COPY`, or created by
  [RESTORE](https://docs.hotkv.com/commands/restore/index.md);
- its slot is cleared or taken away (`CLUSTER FLUSHSLOT`, `SFLUSH`, the end of a
  slot migration);
- it is written on a replica by the stream from its primary, for the clients
  connected to that replica.

[FLUSHDB](https://docs.hotkv.com/commands/flushdb/index.md), [FLUSHALL](https://docs.hotkv.com/commands/flushall/index.md),
[SWAPDB](https://docs.hotkv.com/commands/swapdb/index.md), [CLUSTER RESET](https://docs.hotkv.com/commands/cluster-reset/index.md) of a
replica, `CLUSTER REPLICATE NO ONE` and a replica's data being replaced by a
full sync tell every client that tracks with a null instead of keys.

The data the AI commands keep outside the keyspace (for example the entries of a
prompt cache) is not noted and not told.

## Where invalidations go

On a **RESP3** connection the invalidation is a push on the same connection. It
comes after the reply of the command that caused it, and whatever
[CLIENT REPLY](https://docs.hotkv.com/commands/client-reply/index.md) says. A client's own change is told once
its reply is out, and the changes of a transaction come after the `EXEC` reply.
Evictions that make room for a command are told before that command's reply.

A **RESP2** connection cannot receive a push. With `REDIRECT` its invalidations go
to the other connection: as a push if that one speaks RESP3, otherwise as a
message on the channel `__redis__:invalidate`, provided that connection is
subscribed to something. A RESP2 client with no redirect, or one that redirects
to a connection that is not subscribed, is told nothing, though `CLIENT TRACKING
ON` is accepted. The target of a `REDIRECT` must exist when the command runs.

```
127.0.0.1:6379> CLIENT TRACKING ON REDIRECT 99999
(error) ERR The client ID you want redirect to does not exist
```

When the connection redirected to is gone, a RESP3 client is sent
`tracking-redir-broken` with that connection's id, each time an invalidation
would have gone to it:

```
push> 1) "tracking-redir-broken"
      2) (integer) 48
```

[CLIENT TRACKINGINFO](https://docs.hotkv.com/commands/client-trackinginfo/index.md) then lists
`broken_redirect` among the flags until you turn tracking on again.

A RESP3 connection that is subscribed to a channel can still run any command, so
a single connection can hold tracking, subscriptions and ordinary reads. See
[Pub/Sub](https://docs.hotkv.com/develop/pubsub/index.md).

## What your cache must do

- **Drop the key on an invalidation, and read it again if you still want it.**
  That read is what makes the server tell you about the next change.
- **Treat a null invalidation as "forget everything".**
- **Empty the cache when the connection is lost.** Changes made while you were
  disconnected were not delivered. Turn tracking on again after you reconnect.
- **Watch for `tracking-redir-broken`** if you redirect. Your cache is no longer
  being kept current: empty it and set up the redirect again.
- **Keep reads and the cache consistent.** Put a value in the cache only for a
  read made on the connection that has tracking on, and do not let a value in
  flight overwrite an invalidation that arrived meanwhile.

## How many keys the server remembers

The server remembers which keys tracking clients have read. Four `INFO`
fields describe it:

| Field | Section | Meaning |
| --- | --- | --- |
| `tracking_clients` | clients | The connections with tracking on. |
| `tracking_total_keys` | stats | The keys that have a note. |
| `tracking_total_items` | stats | The notes: a key read by two clients counts twice. |
| `tracking_total_prefixes` | stats | The prefixes `BCAST` clients registered. |

[`tracking-table-max-keys`](https://docs.hotkv.com/reference/configuration/index.md#tracking-table-max-keys)
limits the keys in the table (1,000,000 by default; `0` means no limit). Past it,
HotKV forgets keys, in no particular order, and tells each client that had read
one as if the key had changed, so a cache is never left holding a value the
server stopped watching. `CONFIG SET` applies a new limit at once. With a limit
of 3 and a client that read five keys:

```
127.0.0.1:6379> CONFIG SET tracking-table-max-keys 3
OK
127.0.0.1:6379> MGET k1 k2 k3 k4 k5
1) (nil)
2) (nil)
3) (nil)
4) (nil)
5) (nil)
push> 1) "invalidate"
      2) 1) "k2"
push> 1) "invalidate"
      2) 1) "k4"
```

A client that is told of a key it just read has to read it again; if you see this
often, raise the limit or use `BCAST` with prefixes, which keeps nothing per key.
A connection that closes leaves its notes in the table until the key changes or
is forgotten.

## Memory and limits

Invalidations count in the output of the connection they are sent to, like any
other message. A client that does not read them is closed by
[`client-output-buffer-limit`](https://docs.hotkv.com/reference/configuration/index.md#client-output-buffer-limit)
or evicted by [`maxmemory-clients`](https://docs.hotkv.com/reference/configuration/index.md#maxmemory-clients),
as any client that does not read its replies. A client with no hard output limit
(the `normal` class, by default) is also closed when it leaves more than 32 MiB
of pushes unread. Prefixes a `BCAST` client registers count towards
that client's memory (`tot-mem` in [CLIENT LIST](https://docs.hotkv.com/commands/client-list/index.md)). See
[Client memory](https://docs.hotkv.com/operate/memory-and-eviction/index.md#client-memory).

## In a cluster and on replicas

Every node keeps its own table, and a client is told of the changes the node it
reads from makes. For a key in a slot that node serves, that is every change. A
client that reads from a replica (after `READONLY`) is told of the writes the
replica applies from its primary. When Redis 8's atomic slot migration leaves
slots on the node they came from, each removed key is told to the clients that
track it. A
client that follows a slot to another node should treat the move as a reason to
drop what it cached from the old one. See [Cluster clients](https://docs.hotkv.com/clustering/clients/index.md).

## Seeing what is tracked

[CLIENT TRACKINGINFO](https://docs.hotkv.com/commands/client-trackinginfo/index.md) shows the state of the
current connection, and [CLIENT LIST](https://docs.hotkv.com/commands/client-list/index.md) shows `t` (tracking
is on), `B` (`BCAST`) and `R` (the redirect is broken) among a connection's
flags and `redir=` for the connection invalidations go to:

```
127.0.0.1:6379> CLIENT TRACKINGINFO
1) "flags"
2) 1) "on"
   2) "bcast"
3) "redirect"
4) (integer) 0
5) "prefixes"
6) 1) "order:"
   2) "user:"
```

## From a client library

Redis client libraries that have a local cache turn tracking on for you and
handle the invalidations. The examples below use the `redis` packages, which
work against HotKV like any Redis-compatible client (see
[Connecting and SDKs](https://docs.hotkv.com/sdks/index.md)).

**Python**

As of redis-py 8.1.0 the built-in cache (`cache_config`) connects only to a
server whose `HELLO` reply says it is Redis, so it refuses a HotKV connection.
This cache manages tracking itself, with a RESP2 listener connection and
`REDIRECT`, which works with any client library:

```python
import threading
import time

import redis

HOST, PORT = "localhost", 6379

class TrackedCache:
    """A read-through cache that the server keeps current."""

    def __init__(self):
        self.local = {}
        self.lock = threading.Lock()
        self.hits = 0

        # This connection only listens. Its invalidations arrive as
        # messages on a channel because the connection speaks RESP2.
        self.pubsub = redis.Redis(
            host=HOST, port=PORT, decode_responses=True, protocol=2
        ).pubsub()
        self.pubsub.execute_command("CLIENT", "ID")
        listener_id = self.pubsub.parse_response()
        self.pubsub.subscribe("__redis__:invalidate")
        self.pubsub.get_message(timeout=5)  # the subscribe confirmation

        # This connection reads. The pool holds one connection so that
        # every read goes through the one that has tracking turned on.
        self.reader = redis.Redis(
            host=HOST, port=PORT, decode_responses=True, protocol=2,
            max_connections=1,
        )
        self.reader.execute_command(
            "CLIENT", "TRACKING", "ON", "REDIRECT", listener_id
        )

        threading.Thread(target=self._listen, daemon=True).start()

    def _listen(self):
        for message in self.pubsub.listen():
            if message["type"] != "message":
                continue
            with self.lock:
                if message["data"] is None:  # a flush: forget everything
                    self.local.clear()
                else:
                    for key in message["data"]:
                        self.local.pop(key, None)

    def get(self, key):
        with self.lock:
            if key in self.local:
                self.hits += 1
                return self.local[key]
        value = self.reader.get(key)
        with self.lock:
            self.local[key] = value
        return value

cache = TrackedCache()
writer = redis.Redis(host=HOST, port=PORT, decode_responses=True)
writer.set("user:1", "alice")

print(cache.get("user:1"), "local hits:", cache.hits)
print(cache.get("user:1"), "local hits:", cache.hits)
writer.set("user:1", "bob")
time.sleep(0.2)  # let the invalidation arrive
print(cache.get("user:1"), "local hits:", cache.hits)
print(cache.get("user:1"), "local hits:", cache.hits)
```

Run against a server, this prints:

```
alice local hits: 0
alice local hits: 1
bob local hits: 1
bob local hits: 2
```

The second read is answered locally. After the write, the invalidation removes
the entry, so the next read goes to the server and the one after it is local
again.

**Node.js**

node-redis turns on tracking and keeps the cache when you give `createClient`
RESP3 and a `clientSideCache`:

```js
import { createClient } from 'redis';

const url = 'redis://localhost:6379';

const writer = createClient({ url });
await writer.connect();

const cached = createClient({
  url,
  RESP: 3,
  clientSideCache: { ttl: 0, maxEntries: 1000, evictPolicy: 'LRU' },
});
await cached.connect();

await writer.set('user:1', 'alice');
console.log(await cached.get('user:1')); // asks the server
console.log(await cached.get('user:1')); // answered from the local cache

await writer.set('user:1', 'bob');
await new Promise((resolve) => setTimeout(resolve, 200)); // let the invalidation arrive
console.log(await cached.get('user:1')); // asks the server again
console.log(await cached.get('user:1')); // local again

console.log(cached.clientSideCache.stats());
await cached.quit();
await writer.quit();
```

Run against a server, this prints:

```
alice
alice
bob
bob
CacheStats {
  hitCount: 2,
  missCount: 2,
  loadSuccessCount: 2,
  loadFailureCount: 0,
  totalLoadTime: 60.73290000000043,
  evictionCount: 0
}
```

`totalLoadTime` is milliseconds and differs from run to run. The server saw two
`GET` commands for the four reads.

## Compatibility

`CLIENT TRACKING`, `CLIENT CACHING`, `CLIENT GETREDIR` and
`CLIENT TRACKINGINFO` take the options, give the replies and refuse with the
error texts that Redis 8 and Valkey 9 do, on RESP2 and RESP3, so a client library
written for their client-side caching works, apart from the check some libraries
make of the server's name (see the Python example above). The HotKV SDKs do not
include a local cache; use a Redis client library that does, as above.

Things to keep in mind:

- Expiry is told like any other change, for a key and for the fields of a hash.
- `tracking-table-max-keys` forgets keys in no particular order.
- Tracking is per node: invalidations come from the node the client reads from.

## See also

[CLIENT TRACKING](https://docs.hotkv.com/commands/client-tracking/index.md),
[CLIENT CACHING](https://docs.hotkv.com/commands/client-caching/index.md),
[CLIENT GETREDIR](https://docs.hotkv.com/commands/client-getredir/index.md),
[CLIENT TRACKINGINFO](https://docs.hotkv.com/commands/client-trackinginfo/index.md),
[RESP3](https://docs.hotkv.com/develop/resp3/index.md), [Pub/Sub](https://docs.hotkv.com/develop/pubsub/index.md),
[Keyspace notifications](https://docs.hotkv.com/develop/keyspace-notifications/index.md)