Module rate_limiter 

Rate limiting extension point.

Contract: RateLimiter

Purpose: decide, per caller-supplied key (consumer id, IP, route, or a composite), whether a request costing cost units may proceed.

Semantics: check is the hot path — it MUST be non-blocking from the caller’s perspective (in-flight for at most the backend round-trip) and MUST be atomic: concurrent check calls for the same key are linearized by the implementation. Ordering across distinct keys is unspecified. check both decides AND reserves: if it returns allowed: true the cost has already been deducted from the key’s budget; callers must not call again to “commit”. There is no separate refund in M1.

Failure model: returns ExtensionsError; a limiter that cannot reach its backend should report ExtensionsError::Backend. Callers are expected to apply their own fail-open/fail-closed policy — the trait does not prescribe one. No retries are built in.

Editions: OSS ships InMemoryRateLimiter (fixed window, kept for reference) and GcraRateLimiter (DW-017: the real sharded GCRA limiter behind this same trait). Additional distributed limiter backends may be provided separately in future editions.

DW-017: GCRA limiter and policy engine

GcraRateLimiter implements the RateLimiter trait with the governor crate’s GCRA cells over a sharded keyed state store (GcraShardStore: fixed-count shard-locked maps per window, no global mutex; contrast InMemoryRateLimiter’s single Mutex<HashMap>, which is why it stayed a skeleton). A limiter may STACK several windows (e.g. 10 r/s AND 100 r/hour): each window is an independent GCRA cell per key and the decision is the AND of all windows — denied if ANY window denies, retry_after from the denying (binding) window.

Stacking consumption semantics (documented trade-off): windows are evaluated shortest-first and evaluation STOPS at the first denial, so windows before the binding one have already consumed their cell. A request denied by the hourly window still spends one second-window token. This is fail-fast and slightly STRICTER than a fully-atomic all-windows decision (governor’s public API has no non-consuming peek), never more permissive, and the waste is bounded to the short-window bucket (which also replenishes fastest).

Multi-rule denial semantics: when several RULES apply (route- and service-attached policies), a denial in one rule does not stop evaluation of the others — every applicable rule’s state advances and the reported Retry-After is the MAXIMUM wait across all denying rules, while the Limit/Remaining headers come from the first (binding) denying rule. Headers thus show the tightest constraint in resolution order; Retry-After is the longest wait, so a compliant client never retries into a second 429 early. This max-wait rule is across RULES; the stricter-not-looser stop-at-first-denial semantics above stay, per limiter, for WINDOWS stacked within one rule.

Dry-run bundles (DW-041): a policy bundle may set dry_run — its rules still evaluate (buckets advance exactly as if enforcing, so denial rates preview enforcement) but its denials are reported to the caller (RateLimitEngine::evaluate’s dry_denied]) instead of enforced, and its allowed outcomes contribute no X-RateLimit-* headers. LIVE bundles attached to the same request enforce unaffected: monitor mode observes without ever making enforcement more permissive. Extensions may not import observability, so the log/metric side of the report lives on the dataplane caller.

Per-key eviction (#122): every window’s keyed GCRA state lives in a GcraShardStore — GCRA_STORE_SHARDS independent lock-guarded maps replacing governor’s unbounded DashMapStateStore (whose key set a reload can only reset wholesale). Each shard holds at most crate::config::limits::MAX_RATE_LIMITER_KEYS_PER_SHARD keys, so a window’s worst case is GCRA_STORE_SHARDS times that cap — an [ip]-selector limiter under key spray is memory-bounded for the process lifetime. The sweep runs INLINE, on the shard lock a reservation already holds (cascade-on-insert; no background task, no O(cap) work per request): when a shard reaches its cap it first drops IDLE cells — keys untouched for at least the window’s full-refill time, whose GCRA state is indistinguishable from a fresh cell, so dropping them cannot change any decision — and if the shard is still crowded (every key fresh — sustained spray) it evicts the idlest half, ordered by (last_touch_ms, key), down to half the cap so the O(cap) sweep amortizes to O(1) per insertion. Idle-first ordering means a key actively under enforcement (denied, hence freshly touched) is evicted only when the shard holds more than a cap-full of keys ALL touched within one full-refill window; losing a cell then merely resets that key’s bucket (a fresh bucket can only be MORE permissive for that key, never stricter for anyone) — the documented fail-open trade under spray. GcraRateLimiter::evictions exposes the dropped-cell count as a monotonic counter in the balancer’s fail-open-picks style; extensions may not import observability, so no metric family is wired here (a future gauge can scrape it from the dataplane).

Legacy field mapping: a policy’s old rate_limit {requests, window_seconds} compiles to one rule with selector [route] and a single window of requests per window_seconds (burst = requests). Both fields may be set; both apply.

Burst vs sustained: a window of requests_per.s = 10 with burst: 20 is a GCRA quota replenishing 1 token per 1/10 s with a 20-token bucket: 20 rapid requests pass (burst), sustained traffic above 10 r/s starts drawing 429s once the bucket empties. burst defaults to the window’s request count; under GCRA the first window can admit up to burst + replenished cells (documented, standard GCRA shape).

Structs

GcraOutcome

Result of one GcraRateLimiter::check across its stacked windows.

GcraRateLimiter

Sharded GCRA limiter over one or more stacked windows (DW-017).

GcraWindowSpec

One window specification for GcraRateLimiter::new.

InMemoryRateLimiter

In-memory fixed-window limiter (OSS skeleton).

RateDecision

Outcome of a RateLimiter::check call.

RateLimitEngine

Policy-resolution and key-building engine for request rate limiting (DW-017). Compiled once per config generation.

RateLimitEvaluation

The full rate-limit decision for one request (DW-041): the LIVE enforcement outcome (exactly the pre-dry-run check semantics over live bundles) plus the first dry bundle’s would-be denial, if any evaluated to one.

RateLimitKeyContext

The per-request attributes a rule key can be built from (DW-017).

Enums

RateLimitOutcome

What the engine decided for one request (DW-017).

Constants

GCRA_STORE_SHARDS

Number of independent shard locks in a GcraShardStore. Fixed (not CPU-count-derived) so the worst-case key bound per window (GCRA_STORE_SHARDS * MAX_RATE_LIMITER_KEYS_PER_SHARD) is the same number on every machine. Public for the eviction tests’ bound assertions.

Traits

RateLimiter

Swappable rate-limit backend.

Functions

denied_outcome