Fast, but never wrong
A four-tier response cache — exact, canonical, near-dup and semantic — with safety gates that reject a match when numbers, negation or names differ. Catch the noise; don't serve an answer to a different question.
"Catch noise (spaces, commas, a trailing question mark) without serving wrong answers when the meaning changes."
L1 · Exact
A SHA-256 over the raw request — model, messages, parameters and tools. Identical calls return instantly, with no fuzzy matching involved.
L1b · Canonical
Collapses runs of whitespace and trims the ends, nothing else. No lower-casing or punctuation stripping, because those can change meaning.
L2 · Near-dup
A 64-bit SimHash catches requests that differ only cosmetically — a trailing question mark, an extra comma — and every candidate still has to pass the safety gates.
L3 · Semantic
Cosine similarity on a built-in bag-of-words embedding with no external call, or on an embedder you choose: an OpenAI-compatible endpoint or a model on the gateway.
Anti-false-hit gates
"Capital of France" and "capital of Spain" are a miss. "Refund order 12345" and "12346" are a miss. Numbers, negation, and entities break a hit.
Real latency & cost saved
A hit skips the upstream call entirely — the response carries x-waeg-cache so you can see exactly what was served from cache.
One principal per answer
A cached answer belongs to one user, application, team or key, never the whole organization. Two users never share an answer, not even on the fuzzy tiers.
Provider prompt caching
Session affinity keeps a conversation on one deployment, and optional hints mark Anthropic cache breakpoints or send OpenAI a stable prompt_cache_key.
Conservative eligibility
Streams, temperature above zero and requests with tools skip the response cache by default. Images, audio, embeddings and OCR only ever get exact hits.
Step by step
- 01
Normalize
The request is hashed for an exact-match probe, then canonicalized — whitespace collapsed and trimmed, nothing that could change meaning.
- 02
Widen carefully
If exact misses, near-dup (SimHash) and semantic (cosine) tiers widen the net — each one looser than the last, each one still gated.
- 03
Gate the meaning
Before any near-dup or semantic hit is served, safety gates check that numbers, negation, and named entities match. If meaning shifted, it is a miss.
- 04
Serve or fall through
A gated hit returns immediately and is marked in the response headers. A miss falls through to routing and the upstream provider.
# Cosmetic noise → HIT (near-dup, gates pass)
"capital of France" ≈ "capital of France?"
→ near_dup HIT
# Entity changed → gate rejects → MISS
"capital of France" vs "capital of Spain"
→ MISS (entity)
# Number changed → gate rejects → MISS
"Refund order 12345" vs "Refund order 12346"
→ MISS (number)Semantic Caching is one of six control surfaces enforced in the same request path, applying to every application behind the endpoint — including the ones that were never instrumented.
See how they combineOne endpoint. Every model. Complete control.
Run it in your own infrastructure
Everything you need to run a control plane in production is Apache-2.0 and stays that way — routing, HA, caching, failover, budgets, policy, MCP, jobs and observability. Self-hosted, no account, no phone-home.
$ docker run -p 4000:4000 <registry>/waeg-gateway:<version>Not a working pull: the images are not public yet. Design partners receive the registry details privately.
Deploy in 5 minutesShape it as a design partner
Wäg is not publicly released yet. We are working with a small cohort of teams running multi-provider AI in production, and serious about governance and sovereignty, to harden the control plane on real workloads.
- Every Enterprise capability included, at no licence cost
- Hands-on migration — we do it with you, including from LiteLLM
- A direct line to the engineers building it, and real influence on what ships next