Module analytics

Module analytics 

Source
Expand description

Analytics bounded context (DW-043): the embedded analytics store.

Observability answers “is it healthy right now”; analytics answers “what happened over time, to whom, and why”. This domain owns the DURABLE side of that question: a local SQLite database (separate file from the state store, DW-018 — different lifecycle, different write pattern, independent bounded-disk story) holding raw access records for a short retention window and additive rollup tables at 1m/5m/1h/1d granularities with per-granularity retention, plus the query surface over them (dashboard series, Top-N, structured query) served by the admin API, and the scheduled usage-report exports (DW-120, exports) that turn the same rollups into durable per-consumer statements.

§Write path (never blocks the dataplane)

The dataplane’s DataPlane (see dataplane::proxy) holds an EmbeddedAnalytics (set once at startup when the config carries an analytics block) and calls EmbeddedAnalytics::record at request completion — a try_send onto a bounded channel. When the channel is full the record is DROPPED (counted, logged throttled): the analytics pipeline must never slow or block a request, the same fire-and-forget posture DW-121’s record stream (events::stream, a sibling pipeline that ships each record to an external sink instead of the local store) holds. A background writer drains the channel in batched transactions.

§The sink seam

The M1 extension contract extensions::analytics::AnalyticsSink is the OSS/Ent seam (feature analysis 11.4): the embedded store implements it (over the richer per-request fields DW-043 added to extensions::analytics::Event). DW-121’s raw-record firehose is NOT an implementation of this contract — it streams the completion-time access record (which carries request_id and the redacted path the extension event deliberately omits) through its own sink seam in events::stream. The federated analytics pipeline (DW-095, SCALE-07 #186) IS an implementation of this contract: cp_dp::analytics::FederatedAnalyticsSink batches events on the edge and ships them to the controller over gRPC; the controller’s EmbeddedCollector forwards them to the aggregate EmbeddedAnalytics store, tagging each event with the originating edge_id for fleet-wide query filtering.

§Custom dimensions

Config-declared dimensions extracted from request HEADERS at completion-time capture (e.g. x-plan -> dimension plan): they ride the AccessRecord into raw rows (a JSON object column) and aggregate into their own narrow rollup table. They are analytics-only — deliberately NOT added to the access log, whose field list is redacted by construction and stays that way.

Re-exports§

pub use crate::config::ANALYTICS_DEFAULT_RETENTION_MS as DEFAULT_RETENTION_MS;

Modules§

exports
Scheduled usage-report exports (DW-120).
insights
ML traffic insights (DW-092): EWMA-based capacity forecasting and seasonal-baseline anomaly detection over the live sketch window rotations.
partition
Raw table daily partitioning and retention automation (SCALE-09, #188).
query
Read side of the analytics store (DW-043): dashboard series, Top-N reports, and the structured query endpoint’s translator.
rollup
Rollup cascade and retention (DW-043).
schema
Analytics store schema (DW-043).
sketch
Percentile sketches (PERF-09, #242): a DDSketch-based bounded-memory mergeable quantile sketch with relative-error guarantees, replacing the previous ring-buffer + nearest-rank approach in the live sketches.

Structs§

AiEvalResultRecord
One AI eval result record (DW-086): the per-case record written to the ai_eval_results table when an eval is run via the admin API. A PLAIN DTO — the admin endpoint constructs it from the eval runner’s output.
AiExperimentAssignment
One AI experiment assignment record (DW-086): the per-request record written to the ai_experiment_assignments table when a request is served by an A/B test alias. A PLAIN DTO — deliberately not importing ai::experiments (the analytics domain may not import the ai domain; see scripts/check_deps.py).
AiFeedbackRecord
One AI feedback record (DW-086): the per-feedback record written to the ai_feedback table when feedback is ingested via the admin API.
AiGovernanceEvent
One AI governance audit event (DW-084): the per-check record written to the ai_governance_events table. A PLAIN DTO — deliberately not importing ai::governance (the analytics domain may not import the ai domain; see scripts/check_deps.py). The dataplane converts from ai::governance::GovernanceVerdict at the call site.
AiPromptLogRecord
One AI prompt/response log record (DW-081): the per-request capture written to the ai_prompt_logs table. A PLAIN DTO — deliberately not importing ai::types (the analytics domain may not import the ai domain; see scripts/check_deps.py). The dataplane serializes and redacts the ChatRequest/ChatResponse at the call site, passing the redacted JSON strings here. Capture is opt-in (privacy-first); the redaction pass runs BEFORE the record is offered.
AiSpendRecord
One AI spend record (DW-079): the per-request spend dimensions written to the ai_spend table. A PLAIN DTO — deliberately not importing ai::types::Usage (the analytics domain may not import the ai domain; see scripts/check_deps.py). The dataplane converts from ai::types::Usage at the call site.
EmbeddedAnalytics
The embedded analytics store (DW-043): SQLite file + bounded channel + background writer. Rollup/retention workers are spawned by EmbeddedAnalytics::spawn_workers.
LiveRouteSnapshot
One route’s snapshot from the current live window (DW-092).
LiveSketches
Live in-process sketches (DW-092): sub-second-freshness per-route rolling window with counts, errors, and capped latency samples. Maintained inside EmbeddedAnalytics and updated synchronously on every record() call — never blocks the dataplane (atomics for counts, a short mutex over a capped vector for samples). When the window expires, the completed window’s aggregates are handed to the insights engine (if attached) for forecasting and anomaly detection.
LiveSnapshot
A snapshot of the current live window across all routes (DW-092). Returned by EmbeddedAnalytics::live_snapshot.
McpToolCallRecord
One MCP tool call record (DW-087): the per-call record written to the mcp_tool_calls table when a tool call is proxied through the MCP gateway. A PLAIN DTO — deliberately not importing ai::mcp (the analytics domain may not import the ai domain; see scripts/check_deps.py). The dataplane constructs it at the call site.
SampledDegradation
REL-10 (#221): sampled degradation parameters, resolved from AnalyticsConfig::sampled_degradation at store build time. Stored inline in EmbeddedAnalytics so the offer path reads them without indirection.

Constants§

ROLLUP_GRACE_MS
Completion GRACE: a fine window is rolled only once its end is at least this far in the past — headroom for the writer’s flush tick and batch latency, so a straggler record still lands in the window it belongs to. Records later than the grace are documented as rollup-lost (they remain in raw until raw retention expires).