Enum EvalScorer 

pub enum EvalScorer {
    ExactMatch,
    Contains,
    Regex,
    LlmJudge,
    SemanticSimilarity,
}

The scorer for an eval case (DW-086, AI-10/#199).

Variants

ExactMatch

Exact string match between the output and expected.

Contains

The output must CONTAIN the expected string.

Regex

The output must MATCH the expected regex pattern.

LlmJudge

AI-10 (#199): LLM-judge scorer. An LLM evaluates whether the output is a good response to the input, comparing against the expected value as a reference. The judge prompt is a simple yes/no comparison. This scorer requires a provider URL and auth to be configured at eval-run time.

SemanticSimilarity

AI-10 (#199): Semantic-similarity scorer. Uses a simple token-overlap heuristic (Jaccard similarity) as a proxy for semantic similarity. Passes when the Jaccard index between the output and expected tokens exceeds 0.5. This avoids adding a new embedding dependency; a future enhancement can plug in a real embedding model.

Implementations

impl EvalScorer

pub fn parse(s: Option<&str>) -> Self

Parse a scorer name string. Defaults to ExactMatch for an unrecognized or None value.

pub fn as_str(&self) -> &'static str

The stable name string (for analytics storage).

pub fn score(&self, output: &str, expected: &str) -> bool

Score an output against an expected value. For LlmJudge, this is a fallback heuristic (the real LLM judge runs asynchronously in the eval runner); the fallback uses Contains semantics. For SemanticSimilarity, uses Jaccard token overlap with a 0.5 threshold.

Trait Implementations

impl Clone for EvalScorer

fn clone(&self) -> EvalScorer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for EvalScorer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for EvalScorer

fn eq(&self, other: &EvalScorer) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for EvalScorer

impl Eq for EvalScorer

impl StructuralPartialEq for EvalScorer