Distributed Redis rate limiter

The default rate limiting (capping how many requests a client may make in a window) limiter is local and in-memory: each gateway instance keeps its own per-key GCRA buckets. This is correct for a single instance, but two or more instances behind a load balancer each get their own independent budget, so the effective limit is multiplied by the instance count.

The distributed Redis (an in-memory data store often used for shared state) rate limiter (an enterprise feature) moves the bucket state to Redis so every instance shares one limit. The same GCRA (Generic Cell Rate Algorithm — a rate-limiting algorithm that models a virtual queue) algorithm runs, but the theoretical arrival time (TAT) (the time the next request is allowed — the state GCRA tracks per key) for each key lives in Redis and is updated atomically via a Lua script (a script run atomically inside Redis) in a single round-trip (one request and its response).

When to use this

The distributed limiter is for a fleet of two or more gateway instances behind a load balancer that must share one rate-limit budget (so the effective limit is not multiplied by the instance count). For a single instance, the local in-memory limiter is correct and needs no Redis.

Requirements

All three conditions must hold for the Redis limiter to activate:

1. The ent cargo feature is compiled in (cargo build --features ent).
2. The config carries a redis_rate_limiter block.
3. The loaded license grants the redis_rate_limiter feature claim.

When any condition is missing, the block is accepted but inert and the local in-memory limiter is used. A one-line notice is logged at startup.

Configuration

Add a redis_rate_limiter block to your gateway config:

gateway:
  redis_rate_limiter:
    url: redis://127.0.0.1:6379
    fail_open: true           # optional, default true
    key_prefix: "dwara:rl:"   # optional, default "dwara:rl:"
    connection_timeout_ms: 1000  # optional, default 1000, range 100..=30000
    key_ttl_s: 3600           # optional, default 3600, range 60..=86400

Field	Default	Range	Description
url	required	n/a	Redis connection URL (e.g. redis://host:6379).
fail_open	true	bool	When Redis is unreachable: true lets requests through (no rate limiting); false rejects with 429.
key_prefix	dwara:rl:	non-empty string	Prefix for rate-limit keys in Redis.
connection_timeout_ms	1000	100..=30000	Timeout for the initial connection at startup.
key_ttl_s	3600	60..=86400	Minimum TTL for rate-limit keys in Redis (stale keys auto-expire).

How it works

The limiter uses the same GCRA (Generic Cell Rate Algorithm) as the local limiter. Each check is one atomic Redis round-trip, which is what keeps the decision consistent across every gateway instance:

For each rate-limit check:

1. The key is built from the policy's selectors (e.g. ip, ip+route, consumer+route) exactly as the local limiter does.
2. A Lua script runs atomically in Redis: it reads the key's TAT (theoretical arrival time — the time the next request is allowed), computes the new TAT, and writes it back in a single round-trip.
3. The script returns whether the request is allowed, the remaining budget, and the retry-after duration.

The Lua script is atomic (Redis executes it as a single command), so two gateway instances checking the same key at the same time will serialize correctly through Redis.

Fail-open vs fail-closed

When Redis is unreachable (network error, timeout, Redis down):

• fail_open: true (default) -- the request is allowed with no rate limiting. This is the safer default for availability: a Redis outage should not take down the gateway. The limiter logs a warning and falls back to allowing all traffic.
• fail_open: false -- the request is rejected with 429. Use this when hard limits matter more than availability (e.g. compliance requirements).

At startup, if the Redis connection cannot be established:

• fail_open: true -- the gateway starts with the local rate limiter and logs a warning.
• fail_open: false -- the gateway refuses to start (exit 1).

Key expiry

Each rate-limit key in Redis carries a TTL (how long a record lives before expiring) so stale keys auto-expire. The Lua script sets an EXPIRE based on the burst tolerance (the time it takes a fully-spent bucket to refill); the key_ttl_s config value is a floor that ensures cleanup even for long-burst windows.

Connection pooling

The limiter uses redis::aio::ConnectionManager -- a multiplexed connection (one TCP connection carrying many logical requests) that clones cheaply (Arc-based) and reconnects automatically on failure. The connection is established once at startup and cloned per-rule at engine compile time. Reloads recompile the rate-limit engine with the same connection.

High availability (HA)

The ha block controls the Redis deployment topology. It applies to all Redis-backed features (rate limiter, quotas, shared cache).

Single (default)

The default topology: one Redis instance, no HA. If the instance is down, the fail_open policy applies.

gateway:
  redis_rate_limiter:
    url: redis://127.0.0.1:6379
    # ha block omitted — defaults to single

Sentinel

Redis Sentinel provides automatic failover: a set of Sentinel nodes monitor the master and promote a replica if the master fails. The gateway resolves the master via SENTINEL GET-MASTER-ADDR-BY-NAME at startup, then connects to the resolved master with a ConnectionManager (auto-reconnecting on failover is handled by re-resolving on disconnect).

gateway:
  redis_rate_limiter:
    url: redis://127.0.0.1:6379  # used if ha.nodes is empty
    ha:
      topology: sentinel
      master_name: mymaster
      nodes:
        - redis://sentinel-1:26379
        - redis://sentinel-2:26379
        - redis://sentinel-3:26379
      request_timeout_ms: 500  # optional, default 500, range 50..=30000

Field	Default	Range	Description
topology	single	single, sentinel, cluster	Deployment topology.
nodes	[]	list of URLs	Sentinel/cluster node URLs. For sentinel, these are the Sentinel nodes. For cluster, any subset of the cluster's nodes.
master_name	none	string	Required for sentinel: the master_name in Sentinel config.
request_timeout_ms	500	0 or 50..=30000	Per-request timeout. 0 means no timeout (use the connection's default). A timed-out command is treated as a backend error and the fail_open policy applies.

Cluster

Redis Cluster shards data across multiple nodes. The gateway connects to one cluster node and uses hash tags ({key}) to ensure all rate-limit windows for one logical key land on the same slot. If the connected node is not the owner of that slot, Redis returns a MOVED error which is treated as a backend error (fail-open or fail-closed per config).

Full cluster support with automatic MOVED/ASK handling is a future enhancement (requires a Redis client library with a Send-compatible async cluster connection).

gateway:
  redis_rate_limiter:
    url: redis://127.0.0.1:6379  # used if ha.nodes is empty
    ha:
      topology: cluster
      nodes:
        - redis://cluster-node-1:6379
        - redis://cluster-node-2:6379
        - redis://cluster-node-3:6379
      request_timeout_ms: 500

Multi-region guidance

For multi-region deployments, run one Redis (or Sentinel/Cluster) per region and point each region's gateway instances at their local Redis. Rate limits are shared within a region but not across regions. This is the recommended pattern: cross-region Redis adds latency and complexity that is rarely worth the benefit of a single global limit.

If a single global limit is required, use Redis Sentinel or Cluster in one region and accept the cross-region latency. The request_timeout_ms field bounds the impact: a timed-out Redis command is treated as a backend error and the fail_open policy applies, so a slow cross-region Redis link degrades gracefully (no rate limiting) rather than blocking the request path.

Runnable demo

The demos/11-enterprise/ directory in the repository includes the redis_rate_limiter block (inert in the OSS build, which uses the local GCRA limiter) and verifies the proxy path (test script: test-04-redis-limiters.sh). The category README covers prerequisites and teardown.