Expand description
Webhook delivery (DW-044, feature analysis section 5 “Platform”): drains the event bus and POSTs each event to the configured targets.
§Envelope (stable shape)
One JSON object per delivery, serialized once per event and shared by every matching target:
{
"id": "evt-18f3c2a1b9d0-00000a",
"kind": "breaker_opened",
"timestamp": "2026-08-27T09:00:00.123Z",
"gateway": "dwara-8213-18f3c2910b07",
"payload": { "upstream": "billing", "detail": "error_ratio" }
}id is process-unique and monotonic (event correlation, not
ordering across processes), kind is the closed
EventKind spelling, timestamp is RFC 3339 UTC
with millisecond precision, gateway identifies the emitting
process, and payload carries only the bounded labels and numbers
of EventPayload — no request-derived data,
no secrets. The envelope is byte-capped at MAX_ENVELOPE_BYTES
(an over-cap envelope — only possible via absurd config label
lengths — is dropped and counted, never truncated: a receiver should
never see a malformed event).
§Delivery contract
One delivery = one target + one event. Every attempt of a delivery
(the first POST plus its retries) shares ONE total budget of
timeout_ms, the OTLP exporter’s model (#133): a flapping or hung
target cannot stretch a delivery past the budget any more than a
slow one could, which is the whole failure-isolation guarantee. The
retry shape is equally borrowed from that client:
- retryable outcomes: transport failures and the transient status
set 429/502/503/504. A seconds-form
Retry-Afterreplaces the computed backoff for that wait (the HTTP-date form is deliberately uninterpreted);Retry-After: 0falls back to the computed backoff so a hostile zero cannot busy-loop. - every other non-2xx answer (4xx, 500, 3xx — redirects are NOT followed) fails the delivery immediately: retrying cannot fix an answer that is this delivery’s fault.
- backoff is exponential from
backoff_base_ms, doubling up tobackoff_cap_ms; a wait that would exhaust the remaining budget gives up instead of sleeping past it.
Outcomes are counted in dwara_webhook_events_total{kind,outcome}:
delivered (2xx on some attempt), failed (retries exhausted,
non-transient answer, or budget spent — the target was tried and did
not take it), and dropped (never tried: envelope over the byte
cap, or the delivery-concurrency semaphore saturated). Queue drops
at EMIT time are the separate dwara_events_dropped_total gauge
(see the parent module). Cardinality is bounded: kind is the
closed event set, outcome is these three spellings.
§Isolation from the dataplane
run_deliverer is one background task; each delivery is a further
task, and at most MAX_CONCURRENT_DELIVERIES run at once
(a saturated semaphore drops-and-counts rather than queueing — an
unbounded delivery queue would just move the unbounded buffer from
the bus to the deliverer). The emit path (try_send + atomics) is
unaffected by everything here: a dead target costs its semaphore
slot for at most timeout_ms, a hung one exactly the same.
§Egress posture
Webhook URLs are OPERATOR configuration, exactly like upstream
endpoints and JWKS URLs: the gateway dials what its config names, so
there is no SSRF boundary to enforce against the config author.
There is deliberately no private-address egress filter in v1 (an
internal alerting listener on 127.0.0.1 or 10/8 is a normal shape);
https:// targets verify against the public webpki root set with no
trusted_ca_file override (v1 scope — the alerting fan-out is
public SaaS), and HTTP/1.1 only. Header values may carry secrets
via ${...} references (DW-045): they are resolved at compile time,
held only by the compiled target, and NEVER logged (delivery logs
name the URL and the outcome, never the headers).
§Target lifecycle
Targets are compiled per config generation (secret references
re-resolved on every publish) and pushed to the deliverer over a
tokio::sync::watch channel by the dataplane’s refresh — a config
change applies to the NEXT event, with no deliverer restart.
Structs§
- Webhook
Target - One compiled
gateway.webhooks[]entry (DW-044): the URL decomposed for dialing, the event-kind filter, and the RESOLVED header set. Built per config generation byWebhookTarget::compile; held by the deliverer via the watch channel. Header values may be secret material (a${...}reference): they are never logged and never appear inDebug(the impl prints names only).
Constants§
- MAX_
CONCURRENT_ DELIVERIES - At most this many deliveries in flight across every target (per gateway process). Over it: drop and count — a stuck fleet of targets must not accumulate an unbounded task pile.
- MAX_
ENVELOPE_ BYTES - Hard cap on one serialized envelope (bytes). Payload fields are bounded config labels, so this only binds absurd label lengths; an over-cap envelope is dropped and counted, never sent truncated.
Functions§
- deliver
- Deliver one alert-event envelope to one target, with the documented
retry shape, counting exactly one outcome (
deliveredorfailed) indwara_webhook_events_total{kind,outcome}. Public for the delivery contract’s unit tests; the deliverer calls it per delivery. - envelope_
json - Serialize one event into its envelope JSON.
- rfc3339_
ms - RFC 3339 UTC with millisecond precision from Unix milliseconds
(
civil_from_days, the standard fully-specified conversion — no datetime dependency, stable across toolchains like the FNV ring hash). UTC by definition of theZsuffix. - run_
deliverer - The deliverer loop (DW-044): drain the bus, filter by kind, dispatch one bounded-concurrency delivery task per (event, target). Returns on shutdown (pending queue abandoned — documented: the gateway is not a durable queue) or when the bus closes.