From direct API calls to one governed endpoint

Not sure you need a gateway yet? This explains what one does, and when calling the provider directly is still the better choice: What is an AI gateway?

Five steps, one application at a time

1 · Run Wäg and add your providers

Give each provider your existing account and name the aliases after the model names your code already sends. Then no model reference in any application has to change.

# waeg.yaml — aliases named after the models your code already asks for
providers:
  openai:
    provider: openai
    api_key_env: OPENAI_API_KEY
  anthropic:
    provider: anthropic
    api_key_env: ANTHROPIC_API_KEY

models:
  - name: gpt-4o
    deployments:
      - provider: openai
        model_id: gpt-4o

  - name: claude-haiku-4-5
    deployments:
      - provider: anthropic
        model_id: claude-haiku-4-5

Keys are resolved from the environment or your vault, never written into the file.

2 · Issue one gateway key per application

Each application gets its own virtual key instead of the provider key. That is what makes cost per application, limits per team and revoking a single service possible — none of which a shared provider key can do.

3 · Change the base URL and the key

Wäg serves the OpenAI-compatible surface — chat completions, the Responses API (stateless), embeddings, images, audio, video, OCR, moderations, rerank, files, batches, realtime and models — with true SSE streaming, plus /v1/mcp and /v1/a2a for agents. In the application, two lines change:

  client = OpenAI(
-     api_key=os.environ["OPENAI_API_KEY"],
+     base_url=os.environ["WAEG_BASE_URL"],   # e.g. https://waeg.internal/v1
+     api_key=os.environ["WAEG_KEY"],         # this application's gateway key
  )

Applications built on Anthropic's or Gemini's own SDKs are supported too: /v1/messages and Gemini's /v1beta are first-class request shapes, not only translations of the OpenAI shape.

4 · Check, then move the next application

Send one normal, one streaming and one tool-calling request, and read the response headers: x-waeg-routed-model says which model served it, x-waeg-route-reason why (with intelligent routing), x-waeg-cache whether a cache answered. The request is already in the console with its tokens and cost.

A reasonable order: internal tools first, then batch and offline workloads, then one production service, then the rest.

5 · Take the provider keys out of the applications

When every application goes through Wäg, rotate the provider keys and delete them from application environments, CI secrets and config files. From then on, only Wäg holds them — which is most of the security gain, and the step teams most often forget.

What you gained

• Cost per application, team and user, as it happens — not reconstructed from an invoice.
• Budgets that stop spend before the provider is called, not dashboards that turn red afterwards.
• Failover to another provider or model when one is down.
• Data rules — mask, redact, block — applied to every prompt, including in applications nobody touched.
• A record of every request: who, which model, why, what it cost, which rules applied.
• Models added or swapped in configuration, without a deploy of every application.

What it costs you

• One more service to run. Embedded mode needs nothing external but is single-node and in-memory, for trials; production runs on Postgres and OpenFGA, plus Redis once there are several replicas.
• One more network hop. A few milliseconds against model calls of hundreds of milliseconds to seconds — and cache hits come back faster than any provider.
• Someone has to own it. Usually the platform team, the same people who own your other shared infrastructure.

Questions that come up