Caching

merido can avoid paying for the same answer twice. There are two layers — an exact-match dedup cache and an opt-in semantic cache — plus a high-level note on prompt-cache economics.

Exact request dedup

The simplest layer: when two identical non-streaming requests arrive within a TTL window, the second returns the first's cached response instead of calling upstream.

• Enable it by setting MERIDO_DEDUP_TTL_MS (the cache window in milliseconds). Unset disables dedup.
• It's exact-match — a single differing byte is a miss — so it's always safe: a hit is the response you would have gotten anyway.

Use this for retried or fan-out workloads that issue identical calls in a short burst.

Semantic cache (opt-in)

The fuzzy sibling of dedup: a request whose prompt embedding is close enough to a recent one reuses that response, even if the wording differs.

• Enable with MERIDO_SEMANTIC_CACHE_ENABLED=true.
• It needs an external embedding model to embed prompts — configure MERIDO_SEMANTIC_CACHE_EMBED_MODEL (and a TTL via MERIDO_SEMANTIC_CACHE_TTL_MS).
• The similarity bar is MERIDO_SEMANTIC_CACHE_THRESHOLD (how close two prompts must be to count as a hit).

A fuzzy hit risks returning a plausible but wrong answer, so semantic caching is guarded:

• Precision floor (MERIDO_SEMANTIC_CACHE_PRECISION_FLOOR) — the cache will not serve fuzzy hits whose measured precision falls below this bar.
• Revalidation sampling (MERIDO_SEMANTIC_CACHE_REVALIDATE_RATE) — a fraction of would-be hits are re-run against the model and compared, continuously measuring real precision so the guard stays honest.
• Per-entry adaptive thresholds — the similarity bar above is the floor. Each cached answer also keeps its own threshold that automatically tightens when a near-identical prompt is later cached with a different response: that region is proven ambiguous, so neither answer will be served unless a query is closer than the known-divergent neighbour. This is free (no extra request) and lets you safely run a lower MERIDO_SEMANTIC_CACHE_THRESHOLD for more hits — ambiguous clusters self-protect.

Enable the semantic cache when you have repetitive, paraphrased prompts (FAQs, templated agent steps) and can tolerate occasional revalidation cost. Keep it off if every answer must be freshly generated.

In a multi-instance deployment, the semantic cache is per-instance; only completed dedup responses are shared cluster-wide.

Prompt-cache economics (high level)

Several providers offer prompt caching: a stable prefix (system prompt, tool definitions, long context) is cached on their side, so repeat requests that share that prefix are billed at a steep discount on the cached portion.

On the live request path merido helps with this in two concrete ways:

• Cache-control injection — for providers whose API takes explicit cache breakpoints (e.g. Anthropic), merido injects cache_control markers (by default at the end of the system prompt) so the stable prefix is actually flagged as cacheable. This is a per-route setting (cache_injection).
• Cache affinity — routing prefers an account/target that has likely already cached your prefix, so repeat requests hit a warm cache rather than re-paying the write cost.

merido does not rewrite or restructure your prompt to manufacture a stable prefix, and it does not pre-warm caches on its own. The deeper prefix-hashing / TTL / break-even machinery is used by the Advisor to flag where you're leaving prompt-cache savings on the table — it is analysis and advice, not an engine that fires automatically on every request.

Related

• Token saving — shrink what you do send.
• Usage & the Advisor — measure and act on cache opportunities.