Module balance 

Per-upstream load balancing (DW-011, feature analysis 4.3).

One UpstreamLb per upstream holds the endpoint set behind an [ArcSwap] (LbState), so UpstreamLb::pick is lock-free: it loads the current state snapshot and runs the configured algorithm with no mutex on the hot path. Endpoint mutation happens only on config reload via UpstreamLb::rebuild, which swaps in a new state atomically — in-flight requests holding an older state keep working against it.

Algorithms (config load_balancer):

• round_robin — smooth weighted round-robin (the nginx algorithm): each pick adds every endpoint’s effective weight to its running current_weight, selects the maximum, and subtracts the total from the winner. The result interleaves endpoints deterministically in proportion to their weights (weights {5,1,2} produce the classic a a b a c a a pattern) with period = sum of weights; over any full period each endpoint is picked exactly its weight many times.
• least_requests — the endpoint with the fewest in-flight requests (see the inflight counters below) wins; ties break to the lowest index. Slow start is NOT applied here (documented choice: least-conn balances on observed load, not weights; a ramping endpoint naturally receives traffic only when others are busier, which is the same conservative effect without inventing a weight model).
• random — “power of two choices”: two distinct endpoints are drawn uniformly at random and the one with the lower in-flight count wins (ties to the lower index). With large fan-out this converges to least-conn at a fraction of the coordination cost.
• ip_hash — consistent hashing (ketama): each endpoint is hashed onto a ring with vnode count per unit of weight (KETAMA_VNODES = 160 per weight unit, so a weight-w endpoint gets 160*w vnodes); a pick hashes its key (the client IP, plumbed by the proxy) and takes the first ring entry at or after that hash. Because vnodes are additive (no max-normalization), adding or removing an endpoint remaps only ~1/(n+1) of keys for equal weights and keys between unchanged endpoints stay put. Ring and key hashing use an inline, fully specified FNV-1a hash with a murmur3-style finalizer — stable across Rust versions, unlike DefaultHasher (SipHash), which makes no cross-version stability guarantee and would silently remap every sticky session after a toolchain bump. Slow start is NOT applied to the ring (documented choice: rebuilding a ring mid-ramp would remap keys; consistency is the point of the algorithm). With no key (e.g. the TLS-passthrough path) ip_hash falls back to smooth weighted round-robin.

In-flight counting: every endpoint carries an AtomicU64 in-flight counter. The upstream handle increments the picked endpoint’s counter at dispatch and decrements when the response (headers) resolves — NOT when the streaming body completes, which the pool cannot observe without wrapping bodies (documented approximation; it biases least-conn/random-2 slightly optimistic during long streams).

Slow start (slow_start_ms, default 0 = off): an endpoint entering the set ramps its effective weight from a floor of 1 to its configured weight over the window, measured from the moment it entered the set. The ramp applies to smooth WRR and (through weights) ip_hash vnode counts are exempt as above. Endpoints already in the set when slow_start is newly enabled do NOT ramp — their carried entry instant predates the window; only genuinely new addresses ramp. Recovery-ramp after a health failure is future work (DW-015 owns health state) — the effective-weight function is the same one that integration will call.

Hot endpoint sets / weight changes without restart: config reload rebuilds the registry and calls UpstreamLb::rebuild with the new endpoint list. Endpoints whose address:port is unchanged KEEP their in-flight counter, current_weight (WRR phase), and slow-start entry instant; new addresses start fresh (in-flight 0, full slow-start ramp if configured). Documented reset semantics: removed endpoints drop their state outright; re-adding one later is a fresh entry.

Randomness for random (random-2) is a process-local xorshift64* seed under a CAS loop — no rand dependency, adequate uniformity for two choices.

Structs

Dispatch

One dispatch’s pick: the endpoint chosen, its address:port, the passive-health report handle (DW-012), and the in-flight guard for the same snapshot the pick came from. Dropping the Dispatch (after the response headers resolve) releases the guard.

InflightGuard

Drops => decrement of the endpoint’s in-flight counter. Holds an Arc to the state it counted in, so a concurrent rebuild never strays the decrement onto a different endpoint set.

PeakEwmaTracker

Peak-EWMA latency tracker (DW-090): one per endpoint when the upstream’s algorithm is peak_ewma. Holds the current EWMA cost (cost_ns, in nanoseconds) and the timestamp of the last update (stamp, in nanoseconds since the Unix epoch) as atomics so UpstreamLb::record_latency is lock-free on the hot path. Carried across rebuilds by address:port (exactly like inflight and health): the live tracker survives a config reload so a latency history accumulated under one generation is not thrown away by the next.

UpstreamLb

A load balancer for one upstream: lock-free picks over an atomically swappable endpoint set. Share via Arc; cheap to read from any task.