Traffic splitting and sticky sessions

A service can send its traffic to more than one upstream: a weighted split for canary releases (rolling a new version to a small slice of traffic first) and blue-green (running two environments and switching traffic between them) switches, plus an optional sticky cookie that pins a session to its branch. This is the layer above per-upstream load balancing — each split target is a full upstream with its own endpoints, health, and balancer.

When to use this

Traffic splitting is for canary releases (send 5% to a new version and watch for errors before rolling forward), blue-green switches (flip all traffic at once with an instant rollback), and sticky sessions (pin a user to one branch for stateful backends). It sits above per-upstream load balancing — each split target is a full upstream with its own endpoints and balancer.

Weighted splits

Instead of upstream, give the service a split block listing two to eight target upstreams with weights:

services:
  - name: api
    split:
      targets:
        - { upstream: api-stable, weight: 95 }
        - { upstream: api-canary, weight: 5 }
    base_path: /v1

Each request lands on one target by a deterministic weighted hash. With no sticky cookie, the key is the request id, so the realized ratios converge on the configured weights statistically — over enough requests, 95/5 serves roughly 95% stable and 5% canary.

Weights are relative shares (default 1 each). A weight of 0 parks a target: it is compiled and validated but serves no traffic — the parked side of a blue-green pair.

Ramping a canary: keep the total constant

The pick is hash % total_weight. When a weight change KEEPS the total constant, only the changed share of traffic moves — 95/5 to 90/10 moves exactly the 5% that became canary, and existing sessions on the stable side stay put. When a change ALTERS the total, the modulus changes and every key reshuffles. So ramp a canary by RE-BALANCING the pair, never by growing one side alone:

Change	Total	Effect
95/5 -> 90/10	100 (unchanged)	only the canary's 5% delta moves
95/5 -> 95/10	100 -> 105	every session reshuffles
100/0 -> 0/100	100 (unchanged)	the stable side moves wholesale (the blue-green flip)

Blue-green switches

A blue-green switch is a split with one side parked at weight 0. Flip the weights and re-publish — the next request dispatches by the new generation, with no restart and no drain:

# before: all green
split:
  targets:
    - { upstream: green, weight: 100 }
    - { upstream: blue,  weight: 0 }

# after: all blue (republish)
split:
  targets:
    - { upstream: green, weight: 0 }
    - { upstream: blue,  weight: 100 }

Because the total stays 100, the displacement is exactly the stable side. Reloads are live (file change or SIGHUP), so the switch takes effect on the next request after the republish.

Sticky sessions

Add a sticky block to pin a session to its branch. The gateway sets a cookie on the first response; the cookie's value consistently selects the same upstream for every later request from that session:

services:
  - name: api
    split:
      targets:
        - { upstream: api-stable, weight: 95 }
        - { upstream: api-canary, weight: 5 }
    sticky:
      cookie: dwara_affinity     # the cookie name the gateway reads and sets
      ttl_s: 3600                # default 3600; 1..=2592000 (30 days)
    base_path: /v1

How it behaves:

• The cookie is set on the FIRST response of a session (with Max-Age) and never re-set when the client already presents it.
• The cookie guarantees BRANCH affinity (which upstream). Endpoint affinity within the branch comes from the branch upstream's own balancer: when it runs ip_hash (a balancing strategy that pins a client IP to one endpoint), the cookie value becomes the ring key (the input to a consistent-hash ring) and the session pins one endpoint; with other balancers the endpoint is free to float.
• The cookie value is an opaque handle generated by the gateway — not a secret, carrying no identity. A client reusing or "forging" one can only pick a branch it could have landed on anyway.
• A session whose first request is a response cache HIT is served without dispatching, so no cookie is minted until the first cache MISS — while hits last there is nothing to pin.

sticky.cookie must be a valid cookie name (RFC 6265 token: letters, digits, and !#$%&'*+-.^_|~— no spaces or separators).ttl_sis the cookie'sMax-Age` in seconds, 1 to 2592000 (30 days); the default is one hour.

Metrics

• dwara_split_picks_total{service,upstream} — one increment per request dispatched through a weighted split. Both labels are the config-declared names, so the canary share is the upstream's share of the service's total.
• dwara_sticky_sessions_total — affinity cookies set on a first response (a plain counter).

Both are visible at /metrics.

Auto-canary analysis

A canary_analysis block on a service split (exactly 2 targets: baseline + canary) or an AI model alias arms a background controller that automatically adjusts the canary weight based on error rate or latency. When the canary is healthy, the controller promotes it (increases its weight by step); when it regresses, the controller rolls it back (decreases by step, or to 0 on severe regression). The total weight stays constant — the baseline absorbs the delta — so existing sessions are not reshuffled beyond the changed share. Weight changes are transient and revert on config reload.

services:
  - name: api
    split:
      targets:
        - { upstream: api-stable, weight: 90 }
        - { upstream: api-canary, weight: 10 }
      canary_analysis:
        enabled: true
        window_seconds: 60
        step: 5
        min_requests: 10
        cooldown_seconds: 30
        promote:
          metric: error_rate       # or latency_p99
          threshold: 0.01          # canary error < 1% -> promote
        rollback:
          metric: error_rate
          threshold: 0.05          # canary error > 5% -> rollback

The controller uses per-version sliding windows (1000-sample cap) and waits for min_requests before acting. cooldown_seconds prevents rapid oscillation. Severe regression (canary metric > 2x the rollback threshold) triggers an immediate rollback to 0.

New metrics: dwara_canary_promotions_total, dwara_canary_rollbacks_total, dwara_canary_weight{group}. New events: canary_promoted, canary_rolled_back.

Runnable demo

Run the balancers and splits against a live gateway: demos/02-load-balancing/ in the repository. Test scripts cover each per-upstream strategy (test-01-round-robin.sh through test-04-peak-ewma.sh), cookie-affinity sticky sessions (test-05-sticky-sessions.sh), and a 90/10 weighted split between two pools (test-06-traffic-split.sh). The category README covers prerequisites and teardown.