Module routing

Module routing 

Source
Expand description

AI routing decisions (DW-076): which provider/model serves one request, and in what order to fail over.

Pure functions over the compiled AiRuntime structures — no I/O, no clocks, no shared state. The dataplane’s AI action calls route once per request and walks the returned candidates; this module owns the two decision kinds:

  • Failover chain (an alias with failover:): candidates are [primary, alternates...] in config order. The action tries each in turn on 429/5xx/transport errors and STOPS at the first non-retryable outcome. Same-provider retries are not the chain’s job — the provider’s upstream breaker owns those.
  • Canary split (an alias with canary:): ONE candidate is picked deterministically by a weighted hash of the pick key (the request id), using the same slot semantics as the dataplane’s traffic splitting: cumulative bounds over weights, hash % total selects the slot, and the entry whose bound first exceeds the slot wins. Deterministic means re-sends with the same request id land on the same version, and ratios hold per request (converging statistically over distinct ids).

The hash is an explicit 64-bit FNV-1a rather than std::collections::hash_map::DefaultHasher: the pick must be stable across toolchain versions (it is arithmetic on the bytes, not a seeded SipHash) because canary attribution compares series across process restarts. This mirrors why dataplane::balance::key_hash exists — but duplicated here in four lines rather than importing upward from dataplane, which the dependency direction forbids.

Functions§

pick_hash
Toolchain-stable 64-bit FNV-1a over UTF-8 bytes. Deterministic for a given input forever; good enough distribution for slot picks.
weighted_pick
The deterministic weighted pick over (weight, T) entries: the entry whose cumulative weight bound first exceeds pick_hash(key) % total. Mirrors dataplane::split’s slot walk over at most a handful of entries (validation bounds canary lists), so a linear scan is cheaper than any index. total must be > 0 (validation guarantees it: weights are >= 1 and the list non-empty).
weighted_pick_with_index
DW-091: like weighted_pick but also returns the picked entry’s INDEX (0 = baseline, 1 = canary for a 2-version canary split). The auto-canary controller needs the index to record the outcome on the correct side. Same hash and slot arithmetic.