Skip to content
HotKVDocs

Ask the HotKV docs

Answers come only from these docs and can be wrong; check the linked pages before you rely on them.

AI features guide

Last updated

HotKV’s AI features are native commands in the same engine as your key-value data, not a bolt-on service. This guide covers the prompt cache, semantic cache, the smaller keyed caches, LLM cost tracking and budgets, vector search, agent memory, the RAG pipeline, and embedding de-duplication, with realistic examples for each.

Licensing and plans

Section titled “Licensing and plans”

Vector search (VSIM.*) is a core HotKV command family included on every plan, with no license required. Every other feature on this page is an enterprise AI feature that needs its license feature. On HotKV Cloud, they are included from the Performance plan up; on On-Premise, your license lists which of them are enabled. Once licensed, a feature’s commands are available straight away; to keep clients away from a feature you do not use, deny its commands with an ACL rule.

LICENSE.INFO lists the features your server has, by the names in the table below. LICENSE.CHECK <feature> checks one and replies 1 or 0; it takes the name without hyphens, for example LICENSE.CHECK promptcache.

FeatureCommand familyLicense featureConfig section
Prompt CachePCACHE.*prompt-cache[pcache]
Semantic CacheSCACHE.*semantic-cache[scache]
Tool CacheTOOL.CACHE.*tool-cache[tool]
Negative CacheNCACHE.*negative-cache[ncache]
Guard CacheGUARD.*guard-cache[guard]
Rerank CacheRERANK.*rerank-cache[rerank]
LLM StatsLLMSTATS.*llm-stats[llmstats]
LLM BudgetLLMBUDGET.*llm-budget[llmbudget]
Agent MemoryAMEM.*agent-memory[agent_memory]
Agent SummaryAMEM.SUMMARY.*amem-summary[amem_summary]
RAG PipelineRAG.*rag-pipeline[rag]
Prompt RegistryPROMPT.*prompt-registry[prompt]
Embedding DedupEMBED.DEDUP.*embed-dedup[embed]
VSIM PersistenceVSIM.SAVE/LOAD/LIST/DROPvsim-persist[vsim_persist]

When a feature is not licensed, its commands reply -NOLICENSE <feature> requires a HotKV license. Contact sales@hotkv.com, where <feature> is the name from the table. The official SDKs raise a distinct, catchable error for it (for example hotkv.NotLicensedError in Python), so your application can degrade gracefully instead of crashing.

Vector search (VSIM)

Section titled “Vector search (VSIM)”

VSIM.* is HotKV’s built-in HNSW-indexed approximate nearest neighbor search, with cosine, L2, and inner product distance. The semantic cache, agent memory’s semantic store, and the RAG pipeline below build on it, and you can also use it directly for your own embeddings.

VSIM.CREATEINDEX products 384 METRIC COSINE
VSIM.SET products sku-1 0.12 0.98 0.44 ...
VSIM.SEARCH products 5 0.10 0.95 0.40 ...
VSIM.INFO products

COSINE indexes store the L2-normalized vector, so VSIM.GET on one returns the normalized vector, not necessarily byte-identical to what you set; ranking is unaffected, since cosine similarity is scale invariant. VSIM indexes are saved with your snapshots and AOF rewrites like the rest of your data (see Persistence and backups).

With the vsim-persist license feature, VSIM.SAVE <name> <blob> stores a blob of vector data you supply as a named file (under the dir of [vsim_persist] when that section sets enabled = true, otherwise the default vsim_snapshots), VSIM.LOAD <name> returns it (nil if there is none), VSIM.LIST names the saved ones, and VSIM.DROP <name> deletes one.

Prompt cache (PCACHE)

Section titled “Prompt cache (PCACHE)”

An exact-match cache for LLM prompt/response pairs, so it has no false positives. Use it for deterministic prompts, system prompts, and tool specs, where the same input should always return the same cached output.

CommandPurpose
PCACHE.CREATE <ns> [MAX_ENTRIES n] [TTL s] [COST_PER_HIT usd]Create a namespace
PCACHE.SET <ns> <prompt> <response> [TTL s] [META json] [usage]Upsert an entry, optionally with token usage
PCACHE.GET <ns> <prompt>Response only (nil on miss)
PCACHE.GETEX <ns> <prompt>Response, metadata and hit count (nil on miss)
PCACHE.DEL <ns> <prompt>Delete one entry (0/1)
PCACHE.STATS <ns>Entries, hits, misses, evictions, memory, hit rate, p50/p95/p99 lookup latency, cost saved
PCACHE.FLUSH <ns>Empty but keep namespace
PCACHE.DROP <ns>Delete namespace
PCACHE.LISTAll namespaces
PCACHE.CONFIG <ns> SET <MAX_ENTRIES|TTL|COST_PER_HIT> <v>Reconfigure at runtime

The optional usage group on PCACHE.SET is MODEL <model> INPUT <n> OUTPUT <n> [CACHEREAD <n>] [CACHEWRITE5M <n>] [CACHEWRITE1H <n>]. Token counts without MODEL are rejected with an error. Pass the group and HotKV attributes the original call’s tokens, and their exact cost when the model is priced (see LLM cost tracking below), to every later hit. Without it, savings are tracked only through the namespace’s static COST_PER_HIT.

PCACHE.CREATE llm_prompts MAX_ENTRIES 100000 TTL 3600 COST_PER_HIT 0.012
PCACHE.SET llm_prompts "Translate: Hello" "Bonjour" MODEL gpt-4o INPUT 20 OUTPUT 3
PCACHE.GET llm_prompts "Translate: Hello"
"Bonjour"
PCACHE.STATS llm_prompts
 1) "entries"                   -> (integer) 1
 3) "hits"                      -> (integer) 1
19) "cost_saved_usd"            -> "0.012"          # static COST_PER_HIT metric
21) "usage_saved_requests"      -> (integer) 1
31) "usage_saved_cost_usd"      -> "0.000080"        # exact: (20 in + 3 out) @ gpt-4o rates

TOOL.CACHE.SET, NCACHE.SET, GUARD.SET and RERANK.SET accept the same usage group.

Semantic cache (SCACHE)

Section titled “Semantic cache (SCACHE)”

A similarity-based cache: your application supplies the query embedding, and HotKV returns the closest cached response if its similarity clears the namespace’s threshold. Use it for paraphrased requests where an exact prompt cache would miss.

CommandPurpose
SCACHE.CREATE <ns> <dims> [METRIC cosine|l2|ip] [THRESHOLD 0-1]Create a namespace (threshold default 0.85)
SCACHE.SET <ns> <query_text> <response> <val1> … <valN> [TTL s] [usage]Store a cached response with its embedding (vector values follow response)
SCACHE.GET <ns> <val1> … <valN> [THRESHOLD 0-1]On a hit, the response and its similarity; nil on a miss. THRESHOLD overrides the namespace threshold for this lookup
SCACHE.STATS <ns>Entries, hits, misses, hit rate, p50 search latency, savings
SCACHE.FLUSH <ns>Remove all entries, keep the namespace
SCACHE.DROP <ns>Delete the namespace
SCACHE.LISTAll namespaces
SCACHE.THRESHOLD <ns> <threshold>Update the similarity threshold

Like PCACHE.SET, SCACHE.SET accepts an optional trailing usage group (MODEL <model> INPUT <n> OUTPUT <n> [CACHEREAD <n>] [CACHEWRITE5M <n>] [CACHEWRITE1H <n>], in any order after the vector) so a later hit attributes tokens and cost saved. Every cache family’s STATS reports the savings as usage_saved_requests, usage_saved_input_tokens, usage_saved_output_tokens, usage_saved_cache_read_tokens, usage_saved_cache_write_tokens and usage_saved_cost_usd:

SCACHE.CREATE llm_cache 1536 METRIC cosine THRESHOLD 0.92
SCACHE.SET llm_cache "What is the capital of France?" "The capital of France is Paris." 0.1 0.2 0.3 ... TTL 3600 MODEL gpt-4o INPUT 15 OUTPUT 8
SCACHE.GET llm_cache 0.1 0.2 0.29 ...
1) "The capital of France is Paris."
2) "0.9993"

Tiered caching

Section titled “Tiered caching”

Query the prompt cache first for a zero-risk exact hit, then fall back to the semantic cache for a paraphrase-tolerant match, and only call the model on a double miss:

def complete(prompt: str, embedding: list[float]) -> str:
    if hit := client.execute("PCACHE.GET", "llm_prompts", prompt):
        return hit
    if hit := client.execute("SCACHE.GET", "llm_cache", *embedding):
        return hit[0]  # [response, similarity]
    response = call_llm(prompt)
    client.execute("PCACHE.SET", "llm_prompts", prompt, response)
    client.execute("SCACHE.SET", "llm_cache", prompt, response, *embedding)
    return response

Keyed caches

Section titled “Keyed caches”

Four more caches work like the prompt cache, each dedicated to a specific shape of LLM-adjacent data. All of them support the same optional usage group on SET for savings attribution.

CacheCommand prefixUse it for
Tool CacheTOOL.CACHE.*Memoizing deterministic tool/function-call results, keyed by a deterministic input string
Negative CacheNCACHE.*Short-TTL caching of negative results (“user not found”, “refused by moderation”) to absorb repeated lookups
Guard CacheGUARD.*Caching only responses that already passed schema validation or a guardrail, so a hit is known-safe to replay
Rerank CacheRERANK.*Caching cross-encoder rerank scores for a (query, document) pair in a RAG pipeline

Each has CREATE <ns> [MAX_ENTRIES n] [TTL s], SET, GET, DEL, STATS, FLUSH, DROP and LIST under its own prefix, with the same arguments as the PCACHE command of that name:

TOOL.CACHE.CREATE sqltool TTL 300
TOOL.CACHE.SET sqltool "SELECT 1" "[[1]]"
TOOL.CACHE.GET sqltool "SELECT 1"
"[[1]]"

LLM cost tracking and budgets

Section titled “LLM cost tracking and budgets”

LLMSTATS.* records per-tenant or per-model usage with a one-minute rolling window, and owns a runtime-overridable price table that both LLMSTATS.RECORD and LLMBUDGET.CONSUME share, so a model’s price is defined once and used consistently everywhere, including the cache savings numbers above. LLMBUDGET.* enforces a hard cap on tokens or cost over a fixed window that starts again every window_seconds, rejecting a request server-side before you pay the upstream provider.

LLMSTATS

Section titled “LLMSTATS”
CommandPurpose
LLMSTATS.BUCKET <name>Create a bucket (RECORD also creates it)
LLMSTATS.RECORD <name> <in> <out> <cached> <cost_usd> <lat_ms> [ERROR 0|1] [MODEL <model>] [CACHEWRITE5M <n>] [CACHEWRITE1H <n>]Record one request
LLMSTATS.GET <name>Totals + 1-minute rolling window
LLMSTATS.RESET <name>Zero counters
LLMSTATS.DROP <name>Delete bucket
LLMSTATS.LISTAll buckets
LLMSTATS.PRICE SET <model> INPUT <usd/Mtok> OUTPUT <usd/Mtok> [CACHEREAD <usd>] [CACHEWRITE5M <usd>] [CACHEWRITE1H <usd>]Set or override a model’s price
LLMSTATS.PRICE GET <model>Show a model’s price and source (default/custom/unpriced)
LLMSTATS.PRICE LISTAll known models
LLMSTATS.PRICE DEL <model>Remove a custom override

<cached> is the count of cache-read tokens. ERROR 1 marks the request as failed. Passing MODEL computes cost from the price table using the token counts given, and <cost_usd> is ignored (pass 0); an unpriced model still records its tokens but contributes $0 to cost, and increments unpriced_requests so that is never mistaken for a real zero-cost request. Without MODEL, <cost_usd> is used exactly as given. You can also seed custom prices at startup with price_overrides under [llmstats], one "model:input:output" entry per model (USD per million tokens, optionally followed by :cacheread:cachewrite5m:cachewrite1h).

HotKV ships current list prices for Anthropic and OpenAI models in general availability. Anthropic reports uncached input tokens, cache-write tokens split by 5-minute/1-hour TTL, and cache-read tokens as separate fields; OpenAI reports a single prompt-token count that includes cached tokens, so subtract the cached count yourself before passing it as <in> when recording an OpenAI call with MODEL.

LLMSTATS.BUCKET model_sonnet5
# 100k uncached input, 2k output, 50k served from the provider's own
# cache, 10k written to a 5-minute-TTL cache. cost_usd (0) is ignored
# because MODEL is given.
LLMSTATS.RECORD model_sonnet5 100000 2000 50000 0 420 MODEL claude-sonnet-5 CACHEWRITE5M 10000
LLMSTATS.GET model_sonnet5
11) "cost_usd"  "0.255"


# A self-hosted model with no built-in price:
LLMSTATS.PRICE SET my-finetuned-model INPUT 0.80 OUTPUT 2.40
LLMSTATS.RECORD model_custom 1000 200 0 0 15 MODEL my-finetuned-model

LLMBUDGET

Section titled “LLMBUDGET”
CommandPurpose
LLMBUDGET.SET <name> <token_limit> <cost_limit_usd> <window_seconds>Define a budget (0 = no cap)
LLMBUDGET.CONSUME <name> <tokens> <cost_usd> [MODEL <model>] [INPUT <n>] [OUTPUT <n>] [CACHEREAD <n>] [CACHEWRITE5M <n>] [CACHEWRITE1H <n>]1 = allowed, 0 = rejected
LLMBUDGET.STATUS <name>Tokens and cost used against their limits, the window and the seconds left in it, allowed/rejected counts, unpriced_requests
LLMBUDGET.RESET <name>Zero consumption
LLMBUDGET.DROP <name>Delete budget
LLMBUDGET.LISTAll budgets

LLMBUDGET.SET is purely positional; there is no TOKENS/COST/WINDOW keyword form. As with LLMSTATS.RECORD, passing MODEL to CONSUME prices INPUT+OUTPUT+CACHEREAD+CACHEWRITE5M+CACHEWRITE1H from the shared price table and ignores the given <tokens>/<cost_usd>.

LLMBUDGET.SET tenant_acme 0 5.00 3600
LLMBUDGET.CONSUME tenant_acme 0 0 MODEL claude-sonnet-5 INPUT 100000 OUTPUT 2000
(integer) 1
LLMBUDGET.STATUS tenant_acme
 5) "cost_used_usd"  "0.220000"
if client.execute("LLMBUDGET.CONSUME", tenant, 0, 0, "MODEL", model, "INPUT", in_tok, "OUTPUT", out_tok) == 0:
    raise RateLimitError("budget exhausted")
response = call_llm(...)

Agent memory (AMEM)

Section titled “Agent memory (AMEM)”

Four memory types for AI agents, each with its own command family and lifecycle:

TypeWhat it isKey commands
ConversationSession-scoped chat history, a bounded sliding windowAMEM.CONV.CREATE/APPEND/GET/WINDOW/CLEAR/DROP
SemanticLong-term facts, retrieved by embedding similarityAMEM.SEM.CREATE/STORE/RECALL/FORGET/LIST
WorkingScratchpad key/value state for an in-progress taskAMEM.WORK.SET/GET/DEL/CLEAR/LIST
EpisodicTimestamped, append-only event log with time-range queriesAMEM.EPIS.LOG/QUERY/CLEAR/STATS

AMEM.SEARCH <agent_id> <k> <val1> ... <valN> returns the k closest entries of that agent’s semantic memory, each with its source, key, content and similarity (k precedes the vector). With the amem-summary license feature, AMEM.SUMMARY.SET/APPEND/GET/DROP/LIST keep a versioned rolling summary per agent, so a long-running agent’s prompt does not have to grow unbounded.

AMEM.CONV.CREATE agent-42 WINDOW 500
AMEM.CONV.APPEND agent-42 user "What is the weather today?"
AMEM.CONV.APPEND agent-42 assistant "I'll check the weather for you."
AMEM.CONV.WINDOW agent-42 5


AMEM.SEM.CREATE knowledge-base 1536
AMEM.SEM.STORE knowledge-base fact-1 "Paris is the capital of France." 0.1 0.2 0.3 ...
AMEM.SEM.RECALL knowledge-base 3 0.1 0.2 0.3 ...


AMEM.WORK.SET task-123 current_step "analyzing data"
AMEM.EPIS.LOG agent-42 action "Called weather API for New York"

RAG pipeline (RAG)

Section titled “RAG pipeline (RAG)”

RAG.* runs chunking plus retrieval inside the same engine as the caches above, so a RAG pipeline does not need a separate vector database. The command family is RAG.CREATE, RAG.INGEST, RAG.SEARCH, RAG.STATS, RAG.DROP, and RAG.LIST: create a pipeline with its vector dimension (chunking defaults to STRATEGY fixed, CHUNK_SIZE 512, CHUNK_OVERLAP 64), ingest documents with your own precomputed chunk embeddings (the reply is the number of chunks), and search by query text and embedding together, with ALPHA weighting the two. The official SDKs expose a typed RAG helper; for example, in the PHP SDK:

$client->rag()->create('kb', 384);
$client->rag()->ingest('kb', 'doc:1', $text, vectors: $chunkVectors);
$client->rag()->search('kb', 'how do I reset my password?', vector: $queryVector);

The raw commands behind those calls:

RAG.CREATE kb 384 STRATEGY sentence CHUNK_SIZE 512 CHUNK_OVERLAP 64
RAG.INGEST kb doc:1 "How do I reset my password? Click Forgot password on the login screen..." VECTORS 0.02,0.11,... 0.05,0.09,...
RAG.SEARCH kb "how do I reset my password?" K 5 ALPHA 0.5 VECTOR 0.03,0.10,...

Pair RAG.SEARCH results with the Rerank Cache above to cache cross-encoder rerank scores for repeated (query, document) pairs.

Embedding de-duplication (EMBED.DEDUP)

Section titled “Embedding de-duplication (EMBED.DEDUP)”

Content-hash de-duplication for embedding vectors, with reference counting. If the same content is embedded more than once, for example the same PDF uploaded by many users, EMBED.DEDUP.PUT stores the vector once and increments a refcount on every duplicate, instead of paying for a fresh embedding call and storing the same vector again.

EMBED.DEDUP.PUT org abc123 ada-002 "vector-bytes"    -> 1   # stored
EMBED.DEDUP.PUT org abc123 ada-002 "vector-bytes"    -> 0   # duplicate, refcount 2
EMBED.DEDUP.GET org abc123                            -> "vector-bytes"
EMBED.DEDUP.RELEASE org abc123                        -> 1   # refcount 1

An entry is removed when EMBED.DEDUP.RELEASE takes its refcount to zero. EMBED.DEDUP.STATS <ns> reports hits, misses, and entries; EMBED.DEDUP.PURGE <ns> removes every entry but keeps the namespace, EMBED.DEDUP.DROP <ns> deletes the namespace, and EMBED.DEDUP.LIST names them all.

Also included: prompt registry

Section titled “Also included: prompt registry”

With the prompt-registry license feature, PROMPT.* is a versioned registry for prompt templates with an active-version pointer, so you can roll back a bad prompt with one PROMPT.SETACTIVE call instead of a code deploy: PROMPT.PUT, PROMPT.GET, PROMPT.LATEST, PROMPT.SETACTIVE, PROMPT.VERSIONS, PROMPT.DROP, PROMPT.LIST.

Next steps

Section titled “Next steps”