Module ai 

The AI provider-adapter pack (DW-075): one client dialect in, three provider dialects out.

Clients send OpenAI chat-completions shaped requests to a route whose action is ai; the gateway resolves the request’s model through the AiRuntime alias table, translates the request to the serving provider’s wire format via that provider’s ProviderAdapter, places the call through the provider’s named UPSTREAM (the same pooling, TLS, timeout, breaker, and health machinery every other upstream gets), and translates the response back to the OpenAI shape. The provider model identifier never reaches the client; the client’s alias never reaches the provider.

Layout

• types — the canonical chat vocabulary every adapter shares
• adapter — the ProviderAdapter trait, the pure-translation seam DW-076 (routing/failover) composes on, and the domain error type
• adapters — the OpenAI, Anthropic, and Gemini dialect impls
• openai_compat — the client-facing facade (parse/serialize)
• sse — hand-rolled SSE framing (in-house by the locked M4 dependency decision)
• guardrails — the DW-082 guardrail engine (prompt-injection, PII, banned-content, output schema enforcement)
• semantic_cache — the DW-083 semantic cache (embedding-similarity cache for AI prompts; feature-gated behind the semantic_cache cargo feature)
• policy — the DW-085 routing-policy engine (within-request escalation via an external classifier, and latency-vs-cost static selection)
• a2a — the DW-114 A2A (agent-to-agent) protocol scaffold (adapter, Agent Card parser, stubbed task lifecycle; feature- gated behind the a2a cargo feature)

Dependency direction

ai depends on config only (the ai: block’s schema types). The transport lives in dataplane (dataplane::ai_proxy), which may depend on everything; DW-076’s failover wraps the call, DW-077 wires the streaming path, DW-078/079 meter it — none of them need to touch adapter internals, which is the point of the trait.

Re-exports

pub use adapter::adapter_for;

pub use adapter::AiError;

pub use adapter::ProviderAdapter;

pub use adapter::ProviderErrorBody;

pub use adapter::ProviderRequest;

Modules

a2a

A2A (agent-to-agent) protocol support (DW-114, AI-12 #252).

adapter

The ProviderAdapter trait (DW-075): the seam every provider dialect implements and every higher layer composes on.

adapters

Provider dialect implementations (DW-075). Each module is a stateless ProviderAdapter singleton; see the module docs of each for its translation notes.

batch

AI-04 (#194): Batch API support and half-price metering.

budget

AI token budgets (DW-078): tokens/min and cost/day per consumer or per policy-shared team.

cost

AI cost attribution (DW-079): pricing tables and per-call cost computation.

credentials

Provider credential pools (DW-080, Ent).

experiments

Prompt experimentation (DW-086): A/B model comparison, prompt versioning, regression evals, and feedback ingestion.

governance

AI model governance (DW-084): per-team model allowlists and the shadow-audit switch.

guardrails

AI guardrails (DW-082): prompt-injection heuristics, PII detection, banned-content filters, and output schema enforcement as a middleware chain on the AI proxy action.

image_fetch

AI-13 (#201): Gateway-side image URL fetcher.

ingress

AI-05 (#195): Multi-dialect ingress parsers.

logging

AI prompt/response logging engine (DW-081): the per-generation compiled logging config plus redactor.

mcp

MCP gateway (DW-087): a hand-rolled Model Context Protocol server/router over JSON-RPC 2.0.

openai_compat

The OpenAI-compatibility facade (DW-075).

policy

AI routing policies (DW-085): within-request escalation and latency-vs-cost selection. Composed over DW-076 routing.

redaction

PII redaction (DW-081): scrubs PII/secrets from prompt and response text before it reaches the log store.

routing

AI routing decisions (DW-076): which provider/model serves one request, and in what order to fail over.

semantic_cache

AI semantic caching (DW-083): embedding-similarity cache for AI prompts. A paraphrased prompt within the similarity threshold returns the cached response (no provider call, no token spend).

sse

Hand-rolled SSE framing (DW-075).

stream

Streaming translation (DW-077): provider SSE in, OpenAI SSE out, frame by frame.

token_estimator

Local prompt-token estimation (PERF-01): a lightweight, dependency- free heuristic that estimates the prompt token count of a canonical ChatRequest BEFORE the upstream provider is contacted. The estimate feeds a budget pre-check that rejects requests exceeding the remaining token-budget window with 429 ai_budget_exceeded BEFORE the provider call, so a holder near the limit cannot overrun by a large prompt.

types

Canonical chat-completion types (DW-075).

Structs

AiRuntime

The per-generation compiled AI table (DW-075): the provider pool with resolved credentials plus the model alias map. Built at dataplane refresh from the published config; immutable once built.

CompiledProvider

One compiled provider: everything a request needs after the model alias resolves, with the auth value already RESOLVED (DW-045 compile-time contract — validation resolved it once at publish; re-resolution here keeps the runtime table self-sufficient and a reload picks up rotated env/file values).

RouteTarget

One routable provider/model pair (DW-076): a failover-chain member or a canary version. version is the canary attribution label (None for plain chain targets).

Enums

CompiledModel

The compiled alias entry (DW-076): either a failover chain (primary first), a canary split, a routing policy (DW-085), or an A/B test experiment (DW-086). A policy alias composes over OTHER aliases’ routing plans, so it has no provider/model pair of its own — validation enforces that routing_policy is mutually exclusive with failover and canary. An experiment alias composes over OTHER aliases’ routing plans too — validation enforces that ab_test is mutually exclusive with failover, canary, and routing_policy.