Skip to content
HotKVDocs

Ask the HotKV docs

Answers come only from these docs and can be wrong; check the linked pages before you rely on them.

SCACHE.SET

Last updated 
SCACHE.SET namespace query_text response value [value ...] [TTL seconds] [MODEL model [INPUT n] [OUTPUT n] [CACHEREAD n] [CACHEWRITE5M n] [CACHEWRITE1H n]]
Group
Semantic cache
Complexity
O(log N) where N is the number of entries
Arity
-5 (at least 5 words, counting the command name)
Flags
write, slow
Keys
first 1, last 1, step 1
HotKV Cloud
Performance and Scale
On-Premise
requires the semantic-cache license feature

The semantic cache returns a cached LLM response when a new query embedding is similar enough to one you stored, so paraphrased requests can hit. Your application computes the embeddings. SCACHE.SET stores response with the query’s embedding, given as one argument per vector value after response. The number of values must match the namespace’s dimensions. query_text is kept with the entry for your reference; matching uses only the embedding.

Every call adds a new entry, even for a query you stored before.

Options

Section titled “Options”

Options follow the last vector value, in name and value pairs, in any order.

Option Effect
TTL seconds Time to live for this entry. 0 or omitted means it does not expire.
MODEL model The model that produced the response. Required when you pass any token count below.
INPUT n Uncached input tokens the original call used.
OUTPUT n Output tokens the original call used.
CACHEREAD n Provider cache-read tokens the original call used.
CACHEWRITE5M n Tokens the original call wrote to a 5-minute provider cache.
CACHEWRITE1H n Tokens the original call wrote to a 1-hour provider cache.

The usage options record what the original, uncached model call consumed. Every later hit on the entry adds those tokens to the namespace’s usage_saved_* counters, and when the model has a price (built in, or set with LLMSTATS.PRICE) it also adds the cost those tokens would have had to usage_saved_cost_usd. Token counts are non-negative integers. Passing a token count without MODEL is rejected.

Return value

Section titled “Return value”
Case RESP2 RESP3
Entry stored Simple string OK Same

Errors:

  • ERR namespace not found or dimension mismatch if the namespace does not exist, the vector has the wrong number of values, or the namespace is at its entry limit (max_entries_per_namespace under [scache]) with no expired entries to remove.
  • ERR invalid vector component if a vector value is not a finite number.
  • ERR empty embedding vector if no vector values are given.
  • ERR syntax error if an option has no value.
  • ERR unknown option for any other option name.
  • ERR invalid TTL if TTL is not a non-negative integer.
  • ERR usage token counts (INPUT, OUTPUT, CACHEREAD, CACHEWRITE5M, CACHEWRITE1H) need MODEL if you pass a token count without MODEL.
  • ERR invalid INPUT, ERR invalid OUTPUT, ERR invalid CACHEREAD, ERR invalid CACHEWRITE5M or ERR invalid CACHEWRITE1H if a token count is not a non-negative integer.
  • ERR wrong number of arguments for 'scache.set' command if the argument count is wrong.
  • NOLICENSE semantic-cache requires a HotKV license. Contact sales@hotkv.com if your license does not include the semantic-cache feature.

Examples

Section titled “Examples”
127.0.0.1:6379> SCACHE.CREATE small 3
OK
127.0.0.1:6379> SCACHE.SET small "What is the capital of France?" "The capital of France is Paris." 0.1 0.2 0.3
OK
127.0.0.1:6379> SCACHE.SET small "Capital of France?" "Paris." 0.1 0.2 0.3 TTL 3600 MODEL gpt-4o INPUT 15 OUTPUT 8
OK
127.0.0.1:6379> SCACHE.SET small "Too short" "x" 0.1 0.2
(error) ERR namespace not found or dimension mismatch

Compatibility

Section titled “Compatibility”

HotKV-only command. Redis and Valkey have no equivalent.

See also

Section titled “See also”

SCACHE.CREATE, SCACHE.GET, SCACHE.STATS, SCACHE.FLUSH, SCACHE.DROP, SCACHE.LIST, SCACHE.THRESHOLD, AI features guide