# Hot keys

> Find the most frequently accessed keys with the HOTKEYS command to diagnose traffic skew, overloaded cluster nodes and cache stampedes.

URL: https://docs.hotkv.com/develop/hot-keys/


**Availability:** All HotKV Cloud plans. On-Premise: no license needed. Also in the community build.

A hot key is a key that receives a large share of all traffic: a popular
product page, a global counter, a feature flag every request reads. Hot keys
are a common cause of uneven load. In a cluster, all requests for one key go
to the same node, so a single hot key can saturate that node while the others
sit idle. [HOTKEYS](https://docs.hotkv.com/commands/hotkeys/index.md) shows you which keys those are,
without sampling tools or a `MONITOR` session.

## Turning tracking on

Hot key tracking is off by default. Turn it on at runtime:

```
127.0.0.1:6379> CONFIG SET hotkey-tracking yes
OK
```

or in the configuration file:

```
hotkey-tracking yes
```

or in `hotkv.toml`:

```toml
[hotkeys]
tracking = true
```

While tracking is on, HotKV counts accesses to the first key of every command
that names a key (reads and writes alike). Turn it off again with
`CONFIG SET hotkey-tracking no` when you are done diagnosing.

## Reading the top keys

`HOTKEYS TOP [count]` returns the most accessed keys, most frequent first, as
pairs of key and access count. `count` defaults to 10.

```
127.0.0.1:6379> HOTKEYS TOP 3
1) 1) "product:1187"
   2) (integer) 48211
2) 1) "config:flags"
   2) (integer) 30518
3) 1) "session:9f2c"
   2) (integer) 912
```

`HOTKEYS RESET` clears the counts, so you can start a fresh measurement:

```
127.0.0.1:6379> HOTKEYS RESET
OK
```

The counts are approximate. Use the numbers to compare keys with each other
rather than as exact request totals. Turning tracking off stops the counting but keeps the counts
until `HOTKEYS RESET`; if tracking has never been on, `HOTKEYS TOP` returns
an empty list.

> **Note**
> `HOTKEYS` is an administrative command. A user needs permission for it in
> their ACL rules (it is in the `@admin` and `@dangerous` categories), so grant it to operators
> rather than to application users.

**Python**

```python
import hotkv

client = hotkv.HotKV(host="localhost", port=6379)

for key, count in client.hotkeys_top(10):
    print(key, count)
client.hotkeys_reset()
```
**Node.js**

```ts
import { HotKV } from "@hotkv/client";

const client = new HotKV({ url: "hotkv://127.0.0.1:6379" });

for (const { key, count } of await client.hotKeysTop(10)) {
  console.log(key, count);
}
await client.hotKeysReset();
```
**Go**

```go
entries, err := client.HotKeysTop(ctx, 10).Result()
if err != nil {
    log.Fatal(err)
}
for _, e := range entries {
    fmt.Println(e.Key, e.Count)
}
```
**Java**

```java
for (var entry : client.hotKeysTop(10)) {
    System.out.println(entry.key() + " " + entry.count());
}
```

## Diagnosing traffic skew

1. **Confirm the imbalance.** Compare CPU and command rates across nodes, for
   example with [INFO](https://docs.hotkv.com/commands/info/index.md) `commandstats` and `cpu` on each one.
   In a cluster, [CLUSTER SLOT-STATS](https://docs.hotkv.com/commands/cluster-slot-stats/index.md) shows
   which slots receive the most traffic.
2. **Measure.** On the busy node, run `HOTKEYS RESET`, let traffic run for a
   minute or two, then run `HOTKEYS TOP 20`. Each node counts only the keys it
   serves, so in a cluster ask the node that is under pressure.
3. **Read the result.** One or two keys far ahead of the rest means a hot
   key. Counts that fall off gradually across many keys mean the load is
   simply spread over a busy keyspace, which more nodes or shards will help
   with.

## Fixing a hot key

What helps depends on whether the key is mostly read or mostly written.

**Hot reads.**
- Cache the value in the application, so most requests never reach HotKV.
  With [client-side caching](https://docs.hotkv.com/develop/client-side-caching/index.md) HotKV tells the
  application when the key changes, so the copy can be kept until then.
- Keep several copies of the value under different keys (`config:flags:0`
  to `config:flags:7`, with different hash tags so they land on different
  nodes) and have each client read a random one. Update every copy when the
  value changes.
- Send reads to replicas with `READONLY` if your application tolerates
  slightly stale data.

**Hot writes.**
- Split a counter into several keys (`views:1187:0` to `views:1187:15`),
  increment a random one, and sum them when you read.
- Batch updates in the application and write them less often, for example
  once a second.

**Cache stampedes.** When a hot cached key expires, many clients may miss at
once and all rebuild it. Let one client rebuild it (for example behind a lock
taken with `SET lock:key token NX PX 5000`) while the others keep serving the
old value, or refresh the key shortly before its TTL runs out.

## Compatibility

`HOTKEYS` is a HotKV command; Redis and Valkey have no equivalent built-in
command (`redis-cli --hotkeys` samples keys from the client side instead,
and needs an LFU eviction policy).