0
Cache hits
0
LLM calls
0%
Hit rate
0
Tokens saved
·
Cache speedup
$0.00
Estimated savings
How this demo works
- Every message searches LangCache first. The embedding model runs inside LangCache, on every call.
- Hit: the cached response returns verbatim, in milliseconds, zero LLM tokens. Miss: the LLM answers once and the response is stored. A miss also shows how close the nearest entry was vs the similarity threshold.
- Entries only match inside the scope they were stored in (the 🔒 chip on each reply): person, business unit or company. Attributes are a deterministic filter combined with the vector search in one query.
- "My role is..." steps are app intents writing a profile entry, not the LLM. LangCache is a semantic response cache; fact extraction and episodic memory are a different tool (Redis Agent Memory Server).
Cache scope
who shares the cached answers?G
D