Module exports 

Scheduled usage-report exports (DW-120).

DW-043 made the analytics store QUERYABLE live; this module makes it REPORTABLE durably: a background worker closes each UTC calendar window of the configured kind (hourly/daily/monthly) and writes the per-consumer usage statement — period totals of requests, errors, error rate, rate-limited and shed counts, plus the quota budget figures a billing pipeline consumes — as one deterministic file per configured format (CSV and JSON in v1; Parquet is deferred to the DW-156 backlog) into a configured directory. Distinct from the query API on purpose: this is durable, scheduled OUTPUT, not an ad hoc query.

Reconciliation with the query API (the acceptance contract)

The statement’s analytics numbers ARE the structured query endpoint’s numbers: run_export builds them by calling query::structured itself (grouped by consumer, plus one ungrouped totals row) over the same rollup_fixed tables through the same aggregation helpers. Equality with POST /analytics/query for the same period holds BY CONSTRUCTION — and is pinned by a test that parses the written JSON file and compares it against an independent structured() call. The per-statement consumer cap is 10_000 (the grouped query’s limit).

Scheduling

The worker (spawned by the dataplane via DataPlane::spawn_export_worker in dataplane::proxy) ticks on the same interval-plus-shutdown-watch machinery as the rollup cascade, reads the CURRENT config generation each tick (a reload can add/remove exports without a restart), and exports every closed window that has no successful run record yet — oldest first, so a restart backfills missed windows. A window is due once its close is at least EXPORT_DELAY_MS in the past (writer flush + rollup grace + cascade headroom); the delay keeps the statement as complete as the store can make it. run_export also forces one maintain() pass first, so even a manual trigger reads settled rollups.

Quota columns and window alignment

The quota figures come from the STATE store’s quota_counters (DW-033) — a domain this module cannot import (see scripts/check_deps.py), so callers ABOVE both domains (the dataplane worker, the admin trigger endpoint) read the counters and hand them in as plain QuotaFigures per consumer name. The alignment rule: a budget’s figures appear only when its quota window FULLY CONTAINS the export window — a daily export carries the same-UTC-day daily counter (exact period consumption) and the month-to-date monthly counter; a monthly export carries only the monthly counter (the daily budget’s per-day rows do not sum to the month by construction and are omitted, never fabricated). The monthly column reads the store’s LIVE month-to-date counter, so a backfilled or re-exported window can carry a different quota_monthly_used than the original run: analytics columns are deterministic, quota columns are generation-time snapshots (QuotaBudget’s window_start_epoch_s/reset_epoch_s bound what used covers).

Retention and partial windows

Rollup retention deletes old windows (DW-043); a window older than the chosen granularity’s retention may be undercounted without the store being able to know. Such runs are written with partial: true in the file header and run record — real data, flagged — and the scheduler never AUTO-exports windows past the horizon (only a manual trigger can, still flagged). Quiet windows (no traffic) are not loss: the statement’s counts are exact whenever partial is false.

Output shape

Files are named dwara-usage-{window}-{utc-stamp}.{ext} (stamp YYYY-MM-DDThHZ hourly, YYYY-MM-DD daily, YYYY-MM- trimmed to YYYY-MM monthly) and written atomically and DURABLY (temp file, then fsync, rename, and a directory fsync — all inside the destination directory): a re-export of the same window simply overwrites — output is idempotent, the rollup recompute philosophy. CSV is RFC 4180 (CRLF rows, quoted fields containing commas/quotes/CR/LF, doubled quotes); absent quota cells are EMPTY strings (zero means configured-and-zero-used). JSON is pretty and self-describing (window bounds, generation time, partial flag, totals, per-consumer rows).

Structs

ExportRun

One export attempt’s ledger row (the export_runs table and the admin list endpoint’s shape).

ManualRunBody

The manual trigger’s request body (DW-120 admin endpoint): which window kind to export and optionally which aligned window start.

QuotaBudget

One budget’s figures as carried in a statement (plain DTO: the state-store types stay in the state domain, which this module cannot import).

QuotaFigures

A consumer’s quota figures for one export window (see the module docs’ alignment rule: only budgets whose window fully contains the export window appear).

SpendByModel

One per-model spend breakdown row (DW-079): the JSON statement’s spend_by_model array member. Aggregated from the ai_spend table grouped by model over the export window.

StatementRow

One per-consumer statement row (the JSON member shape; the CSV writer flattens the same fields in a fixed column order).

Enums

ExportFormat

One output format (DW-120). Parquet is deliberately absent (lean dependency rule; see the DW-156 backlog issue).

WindowKind

The export window kinds (DW-120): fixed UTC calendar windows, the closed set behind analytics.exports.window.

Constants

EXPORT_DELAY_MS

How long after a window’s close before the scheduler will export it: the writer’s flush tick plus the rollup grace plus cascade headroom, with margin. Not a correctness knob — a manual trigger bypasses it.

EXPORT_TICK_MS

The scheduler worker’s tick (same cadence class as the rollup cascade’s 30 s; the per-tick work is one cheap SQL probe).

FAILURE_RETRY_MS

How long a FAILED window stays skipped before the scheduler retries it: without this, a persistent failure (unwritable directory, unusable store) would re-burn a full export attempt on every 30 s tick, forever. Time-based, not attempt-count-based — no schema change.

MAX_CATCHUP_WINDOWS

Backfill bound per tick: after a long downtime the oldest-first catch-up drains at most this many windows per tick (one query + file writes each), keeping any single tick’s work bounded.

Functions

effective_formats

The effective formats: the configured list, or both when the list is omitted or empty (the config default and the validation rule agree — an empty list MEANS both, it is never an error).

effective_window

The effective exports schedule from the config block: window kind (default daily) and formats (default both).

last_closed_window

The most recent CLOSED window of a kind (the manual trigger’s default): the window containing now is still open by construction, so the answer is always the previous one — including at the exact boundary instant, where the previous window has just closed.

list_runs

List run records, newest first (the admin GET endpoint’s read).

run_due

The scheduler’s per-tick entry (DW-120): export every closed, settled window of the configured kind that has no successful run record yet — oldest first, bounded by MAX_CATCHUP_WINDOWS per tick and by the query granularity’s retention horizon (windows past it would be partial; a manual trigger can still force them). Failed runs are retried on later ticks, at most once per FAILURE_RETRY_MS window (the backoff).

run_export

Export one window: settle rollups, read the statement through the query API’s own aggregation, write one file per format, and upsert the run record. quota_at supplies the quota figures for the window (period bounds in epoch SECONDS). Never panics on I/O: a failed run is recorded failed with the error string.