Module stream 

Streaming translation (DW-077): provider SSE in, OpenAI SSE out, frame by frame.

The StreamTranslator is the pure half of the streaming path: feed it the provider’s response BYTES as they arrive and it returns the client-facing SSE TEXT to forward for exactly those bytes — nothing is held: a complete provider frame becomes complete client frames in the same poll, and a partial frame stays in the SseDecoder buffer (bounded by the provider’s own frame size) until its terminator arrives.

The dataplane’s AiStreamBody (see dataplane/ai_proxy.rs) drives this translator over the upstream body and owns the metrics hooks.

Client-facing stream shape

The client is OpenAI-shaped regardless of provider dialect:

• each provider delta becomes one data: {chat.completion.chunk} frame (via openai_compat::stream_event_to_openai_chunk),
• provider Usage events are ACCUMULATED (not forwarded inline) — they merge across the stream (Anthropic reports input tokens at message_start and output tokens at message_delta) —
• at stream end the accumulated usage (if any) is emitted as ONE terminal choices: [] usage chunk (the stream_options. include_usage shape), and the gateway ALWAYS writes its own data: [DONE] terminator (the provider’s own terminator is swallowed, even OpenAI’s, so the terminal ordering is uniform: deltas, usage, DONE).

Token counting (the locked decision)

Token counts are PROVIDER-REPORTED ONLY. The translator accumulates the usage events the provider emits mid-stream; there is no local estimation, and chunk counts / text lengths are never presented as token estimates.

Mid-stream aborts

StreamTranslator::abort_tail produces the terminal frames for a provider stream that died after frames were already forwarded: an OpenAI-shaped error chunk then data: [DONE] — the stream ends cleanly for the client instead of a connection reset, and the already-forwarded content stands.

Structs

StreamTranslator

Pure streaming translator state (DW-077).

Type Aliases

ClientFrame

One translated client frame (raw SSE text, terminator included).