Module retries 

Upstream retries: resolved parameters, retry budget, backoff with full jitter (DW-014, feature analysis 4.11).

What retries are

All knobs live on the UPSTREAM (upstreams[].retries); there is no per-route retry configuration in v1. RetryParams is the resolved, validated form carried by each UpstreamHandle; RetryConfig is the serde schema form (defaults applied there).

Retry budget

Each upstream owns one RetryBudget: a rolling 10-second window of (timestamp, is_retry) events. A request may be retried only while the invariant (retries + 1) * 100 <= percent * totals holds inside the window, where totals counts proxied requests to that upstream (one record per request, not per attempt) and retries counts retry attempts. The check-and-reserve is a single lock-protected step, so the invariant is never transiently violated by concurrent requests; the conservative form (charging the retry BEFORE it happens) means a fresh window with little volume grants few or no retries rather than allowing a burst — a deliberately safe bias for a gateway protecting an already struggling upstream. Budget state is carried across config reloads (keyed by upstream name, exactly like balancer state).

Backoff

Nominal delay before retry n (n = 1 for the first retry) is min(base * 2^(n-1), cap) (saturating). The actual sleep applies FULL JITTER, AWS style: a uniform random duration in [0, nominal] (backoff_with_full_jitter). Full jitter avoids the thundering-herd synchronization of decorrelated jitter while still bounding the worst case by the nominal delay. Randomness is a process-local xorshift64* seed under a CAS loop — the same pattern as the load balancer’s random-2 draws, no rand dependency.

Total latency

The cross-attempt total deadline (total_deadline, REL-01) caps the wall-clock time from the first attempt to the last, INCLUDING backoff delays between attempts. When set, a retry whose backoff would cross the deadline is aborted and the last response/error is returned to the client; the per-attempt read_ms timeout still bounds each individual attempt. Absent (the default) leaves the cross-attempt budget unbounded — worst case the retry loop adds up to attempts * (read_ms + backoff_cap_ms) of latency to a single request (each attempt pays its own read timeout plus at most the backoff cap before it). Standard in Envoy and NGINX.

Structs

HedgeParams

Resolved (validated) hedge parameters for one upstream (DW-063). hedge_after is None when hedging is disabled (no hedge block or hedge_after_ms == 0). When enabled, a speculative duplicate is sent after hedge_after without a response; at most hedge_max copies per request.

RetryBudget

Per-upstream rolling-window retry budget. Share via Arc from the upstream handle; the lock is taken once per proxied request (record) and once per retry decision (reserve) — never per attempt frame.

RetryParams

Resolved (validated) retry parameters for one upstream. attempts == 0 disables retries entirely — the single-attempt path then performs no body buffering and no extra per-request work (the budget denominator is still recorded: the window counts all proxied traffic).

Constants

DEFAULT_RETRY_STATUSES

Default retry statuses (502, 503, 504).

RETRY_BUDGET_WINDOW_MS

Rolling window the retry budget accounts over (milliseconds).

Functions

backoff_with_full_jitter

Full-jitter sleep duration for retry number retry (1 = the first retry): a uniform draw in [0, nominal_backoff_ms(base, cap, retry)] (AWS “Exponential Backoff and Jitter” full jitter). Pure in rand so tests can pin bounds; the production draw is jitter_delay.

jitter_delay

backoff_with_full_jitter drawing from the process-local rng.

nominal_backoff_ms

Nominal (pre-jitter) backoff for retry number retry (1 = the first retry): min(base * 2^(retry-1), cap), saturating on both the shift and the multiply.

retry_sleep_under_total_deadline

Returns the sleep duration to use for a retry under an optional cross-attempt total deadline (REL-01), or None when the deadline has been exhausted and the retry must be aborted. When the deadline is set and the full delay would cross it, the sleep is clamped to the remaining slice so the loop never sleeps past the deadline. Pure in elapsed so tests can pin the clock; the production wrapper is retry_sleep_within_total.

retry_sleep_within_total

retry_sleep_under_total_deadline reading the wall clock from started (the instant captured at the first attempt of the retry loop).