Module events

Module events 

Source
Expand description

In-process event bus (DW-044, feature analysis section 5 “Platform”): the notice plumbing behind alert & event webhooks.

§What it is

A bounded tokio::sync::mpsc channel plus cheap clone-able Emitter handles. Subsystems that own a state machine worth alerting on (the circuit breaker, passive/active health, the config publish pipeline) emit a small Event at every transition; the webhook deliverer (webhook) drains the channel in a background task and POSTs each event to the configured targets. std + tokio only — no dependency beyond config (the schema types and their bounds) and observability (delivery-outcome metrics).

§Emission contract: never backpressure the dataplane

Emitter::emit is a try_send onto the bounded channel. When the queue is full (or nobody is draining it) the event is DROPPED and the drop is counted (EventBus::dropped_total, surfaced as the dwara_events_dropped_total gauge at scrape time). There is no blocking wait, no unbounded buffer growth, and no failure mode that can stall the caller — emission sites sit on request paths (the breaker check/report wire) and on the reload path, and a misbehaving webhook target must never show up as gateway latency. The drop policy is drop-NEWEST at emit time (the oldest queued events keep their place in line): a slow drain sheds the burst, not the history.

§Event kinds and payloads

Kinds are a closed set (EventKind::ALL) — a webhook’s events list is validated against it, so a config cannot subscribe to an event the gateway will never emit. The payload is a flat struct of bounded config labels and numbers (EventPayload); there is no free-form field, so the serialized envelope is small by construction (the deliverer still enforces a hard byte cap). No request-derived data (paths, headers) enters a payload — identifiers are CONFIG-DECLARED labels only (upstream, and consumer for quota budgets, whose consumers are config records by construction in this edition) — an envelope is safe to POST to a third party by construction.

Emission sites:

  • breaker state transitions (DW-015): opened (with the rule that tripped), half-open, closed (resilience::breaker);
  • outlier ejection and recovery (DW-012 passive + DW-013 active — both report through the same tracker, so both emit): endpoint_ejected, endpoint_recovered (resilience::health);
  • config published (generation + content hash + route count) and config rejected (validation issue count) (snapshot);
  • a consumer’s request budget crossing the near-limit threshold (DW-033): quota_near_limit, edge-triggered once per (consumer, budget, window) from the dataplane’s quota phase (dataplane::proxy — the emit lives there because the state domain must not import events).

Deliberately NOT emitted (documented hook points, not gaps):

  • rate-limiter eviction (#132): already observable as dwara_rate_limiter_evictions_total; an event per eviction would be high-frequency noise, not an alert.

§Failure isolation

The deliverer (see webhook) runs as its own task, dispatches each delivery as a further task, bounds concurrent deliveries with a semaphore (over-cap deliveries are dropped and counted, never queued), and gives every delivery ONE total timeout shared by all its retry attempts. A slow, hung, or dead target can therefore consume at most timeout_ms of one semaphore slot — never the queue, never the emit path, never the dataplane.

§Bus identity and lifecycle

One bus per gateway process, owned by the DataPlane and attached to the ConfigState it was built from (so config publish events and dataplane events share one queue and one deliverer). The gateway field of every envelope identifies the instance: dwara-<pid>-<boot unix ms> — unique per process start, no hostname dependency, stable for the process lifetime.

Modules§

stream
Real-time access-record stream (DW-121, feature analysis section 5 “Platform”): the record plumbing that mirrors the notice plumbing of DW-044. Where the event bus carries a handful of discrete operational events per minute (breaker transitions, ejections, config publishes), this pipeline carries ONE record per completed request — the raw firehose an operator points at their own warehouse or SIEM when the embedded analytics store (DW-043) is not the system of record. It complements, never replaces: the two are configured independently (a deployment can run either, both, or neither) and share nothing at runtime — different channels, different tasks, different failure domains.
webhook
Webhook delivery (DW-044, feature analysis section 5 “Platform”): drains the event bus and POSTs each event to the configured targets.

Structs§

Emitter
A clone-able emission handle (an Arc<EventBus>). Option<Emitter> at every emission site: None (tests, unwired constructions) is a documented no-op, so the subsystems never need a bus to exist.
EndpointEvents
UpstreamEmitter further bound to one endpoint label (DW-044): held by the per-endpoint passive health tracker (resilience::health::EndpointHealth), which knows its state machine but not its own address:port (the balancer does).
Event
One emitted event. id and timestamp_ms are assigned by the bus at emit time (single place), gateway identifies the process.
EventBus
The bounded event queue plus its identity and counters. Share via Arc; emitters hold an Emitter (a cheap clone of that Arc).
EventPayload
The payload of one event: a flat set of bounded labels and numbers. Every field is optional (each event kind sets the fields it owns); unset fields are omitted from the serialized envelope, and there is deliberately NO free-form/detail-string-from-input field — an envelope must stay small and safe to hand to a third party.
UpstreamEmitter
Emitter pre-bound to an upstream label (DW-044): held by the per-upstream circuit breaker and balancer.

Enums§

EventKind
Every event kind the gateway emits (DW-044). Closed set: webhook events lists are validated against EventKind::from_config.

Constants§

EVENT_CHANNEL_CAPACITY
Capacity of the event channel (DW-044). Bound chosen for alert-rate shapes: transition events are rare per upstream (a breaker flapping once per open_ms, ejections once per eject_ms), so 256 queued events is already a gateway in serious trouble — beyond it, drops (counted, gauge-exported) are the right answer, not buffering.

Traits§

EventDurability
REL-14 (#249): the durability seam for the event bus. The events domain owns the trait; the dataplane (which already holds both the StateStore and the EventBus) provides a concrete implementation that wraps StateStore methods. This keeps the dependency direction correct: events does not import state.

Functions§

generate_instance_id
Process-unique instance label: dwara-<pid>-<boot unix ms hex>. Two gateway processes on one host never share it; it carries no hostname (a third-party envelope reader needs process identity, not topology).
refresh_event_gauges
Refresh the event-bus observation gauge at scrape time (the dwara_rate_limiter_evictions_total model: the emit path bumps a plain atomic, and only the scrape couples it to the registry — a gauge rather than a counter so the hot path stays registry-free).