Expand description
AI routing decisions (DW-076): which provider/model serves one request, and in what order to fail over.
Pure functions over the compiled AiRuntime
structures — no I/O, no clocks, no shared state. The dataplane’s AI
action calls route once per request
and walks the returned candidates; this module owns the two
decision kinds:
- Failover chain (an alias with
failover:): candidates are[primary, alternates...]in config order. The action tries each in turn on 429/5xx/transport errors and STOPS at the first non-retryable outcome. Same-provider retries are not the chain’s job — the provider’s upstream breaker owns those. - Canary split (an alias with
canary:): ONE candidate is picked deterministically by a weighted hash of the pick key (the request id), using the same slot semantics as the dataplane’s traffic splitting: cumulative bounds over weights,hash % totalselects the slot, and the entry whose bound first exceeds the slot wins. Deterministic means re-sends with the same request id land on the same version, and ratios hold per request (converging statistically over distinct ids).
The hash is an explicit 64-bit FNV-1a rather than
std::collections::hash_map::DefaultHasher: the pick must be stable
across toolchain versions (it is arithmetic on the bytes, not a
seeded SipHash) because canary attribution compares series across
process restarts. This mirrors why dataplane::balance::key_hash
exists — but duplicated here in four lines rather than importing
upward from dataplane, which the dependency direction forbids.
Functions§
- pick_
hash - Toolchain-stable 64-bit FNV-1a over UTF-8 bytes. Deterministic for a given input forever; good enough distribution for slot picks.
- weighted_
pick - The deterministic weighted pick over
(weight, T)entries: the entry whose cumulative weight bound first exceedspick_hash(key) % total. Mirrorsdataplane::split’s slot walk over at most a handful of entries (validation bounds canary lists), so a linear scan is cheaper than any index.totalmust be > 0 (validation guarantees it: weights are >= 1 and the list non-empty). - weighted_
pick_ with_ index - DW-091: like
weighted_pickbut also returns the picked entry’s INDEX (0 = baseline, 1 = canary for a 2-version canary split). The auto-canary controller needs the index to record the outcome on the correct side. Same hash and slot arithmetic.