Struct SemanticCacheEngine 

pub struct SemanticCacheEngine { /* private fields */ }

The semantic cache engine (DW-083). Constructed once at startup and stored on the dataplane (persists across reloads — the HNSW index and cached entries survive config refreshes). Config is updated in place via update_config.

Implementations

impl SemanticCacheEngine

pub fn new(config: SemanticCacheConfig) -> Self

Build the engine from a config. Creates a fresh HNSW index sized for max_entries.

pub fn config_of(cfg: Option<&AiConfig>) -> Option<SemanticCacheConfig>

Extract the semantic-cache config from the ai: block. Returns None when the block or the semantic_cache field is absent. Does NOT check enabled — a disabled engine still compiles (so a reload can flip it on); the runtime checks enabled on every lookup/store.

pub fn compile(cfg: Option<&AiConfig>) -> Option<Self>

Compile from the ai: config block. Returns None when the semantic_cache field is absent. The returned engine is live only when enabled is true (checked at runtime).

pub fn update_config(&self, config: SemanticCacheConfig)

Update the config in place (reload path). The HNSW index and cached entries PERSIST — only the config (threshold, TTL, timeout, etc.) changes. A max_entries change does NOT rebuild the index immediately; the next reset (when the cache fills) sizes the new index to the new value.

pub fn is_enabled(&self) -> bool

Whether the cache is enabled (the runtime gate for every lookup/store).

pub fn entry_count(&self) -> usize

Number of cached entries (introspection / tests).

pub async fn lookup( &self, prompt_text: &str, model: &str, ) -> Option<CachedResponse>

Look up a cached response for prompt_text + model. Returns the cached response (JSON or SSE frames) when a match is found within the similarity threshold, within TTL, and for the same model alias. None otherwise (or when disabled, or on any embedding/search error — the cache fails open: a miss never blocks the request).

PERF-02: the exact-match fast tier checks a hash map BEFORE calling the embedding service. An exact prompt-text match skips the embedding call entirely (the common case for repeated identical prompts).

pub async fn store(&self, prompt_text: &str, response_json: &Value, model: &str)

Store a non-streaming response in the cache (fire-and- forget from the request path). When the cache is full (entry_count >= max_entries), the least recently used entry is evicted (PERF-02 LRU, not a wholesale reset). Errors are logged and swallowed.

pub async fn store_streaming( &self, prompt_text: &str, frames: Vec<String>, model: &str, )

Store a streaming response (SSE frames) in the cache (PERF-02). The frames are the collected SSE data: ...\n\n strings as tee’d by the stream body. Same LRU eviction as the non-streaming store.

pub fn reset(&self)

Reset the cache: a fresh HNSW index replaces the old, and all entries are evicted. Called on explicit reset only (PERF-02: LRU eviction handles the bounded-memory policy at max_entries; a wholesale reset is no longer the default eviction strategy).

Trait Implementations

impl Debug for SemanticCacheEngine

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.