pub struct AiLatencyCostPolicy {
pub candidates: Vec<AiLatencyCostCandidate>,
pub preference: AiLatencyPreference,
pub live: bool,
pub live_window: Option<usize>,
}Expand description
A latency-vs-cost routing policy (DW-085 kind: latency_cost):
static config-based selection. The operator declares cost/latency
scores per candidate and a preference; the policy sorts candidates
at compile time and picks the best one deterministically (no
runtime metrics needed).
Fields§
§candidates: Vec<AiLatencyCostCandidate>The candidate models with their cost/latency scores. Each
model must reference an existing alias in ai.models.
Validation rejects an empty list.
preference: AiLatencyPreferenceThe selection preference: cost (cheapest), latency
(fastest), or balanced (best cost/latency sum).
live: boolAI-15 (#203): when true, the policy uses live latency
observations to dynamically adjust candidate ordering. The
static latency scores are used as priors; observed latencies
from recent requests refine the selection. Default false
(static selection, the DW-085 behavior).
live_window: Option<usize>AI-15 (#203): the window size for live latency tracking
(number of recent observations to average). Default 10. Only
used when live is true.
Trait Implementations§
Source§impl Clone for AiLatencyCostPolicy
impl Clone for AiLatencyCostPolicy
Source§fn clone(&self) -> AiLatencyCostPolicy
fn clone(&self) -> AiLatencyCostPolicy
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for AiLatencyCostPolicy
impl Debug for AiLatencyCostPolicy
Source§impl<'de> Deserialize<'de> for AiLatencyCostPolicy
impl<'de> Deserialize<'de> for AiLatencyCostPolicy
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Source§impl JsonSchema for AiLatencyCostPolicy
impl JsonSchema for AiLatencyCostPolicy
Source§fn schema_id() -> Cow<'static, str>
fn schema_id() -> Cow<'static, str>
Source§fn json_schema(generator: &mut SchemaGenerator) -> Schema
fn json_schema(generator: &mut SchemaGenerator) -> Schema
Source§fn inline_schema() -> bool
fn inline_schema() -> bool
$ref keyword. Read moreSource§impl PartialEq for AiLatencyCostPolicy
impl PartialEq for AiLatencyCostPolicy
Source§impl Serialize for AiLatencyCostPolicy
impl Serialize for AiLatencyCostPolicy
impl StructuralPartialEq for AiLatencyCostPolicy
Auto Trait Implementations§
impl Freeze for AiLatencyCostPolicy
impl RefUnwindSafe for AiLatencyCostPolicy
impl Send for AiLatencyCostPolicy
impl Sync for AiLatencyCostPolicy
impl Unpin for AiLatencyCostPolicy
impl UnwindSafe for AiLatencyCostPolicy
Blanket Implementations§
§impl<T> AnyExt for T
impl<T> AnyExt for T
§fn downcast_ref<T>(this: &Self) -> Option<&T>where
T: Any,
fn downcast_ref<T>(this: &Self) -> Option<&T>where
T: Any,
T behind reference§fn downcast_mut<T>(this: &mut Self) -> Option<&mut T>where
T: Any,
fn downcast_mut<T>(this: &mut Self) -> Option<&mut T>where
T: Any,
T behind mutable reference§fn downcast_rc<T>(this: Rc<Self>) -> Result<Rc<T>, Rc<Self>>where
T: Any,
fn downcast_rc<T>(this: Rc<Self>) -> Result<Rc<T>, Rc<Self>>where
T: Any,
T behind Rc pointer§fn downcast_arc<T>(this: Arc<Self>) -> Result<Arc<T>, Arc<Self>>where
T: Any,
fn downcast_arc<T>(this: Arc<Self>) -> Result<Arc<T>, Arc<Self>>where
T: Any,
T behind Arc pointer§fn downcast_box<T>(this: Box<Self>) -> Result<Box<T>, Box<Self>>where
T: Any,
fn downcast_box<T>(this: Box<Self>) -> Result<Box<T>, Box<Self>>where
T: Any,
T behind Box pointer§fn downcast_move<T>(this: Self) -> Option<T>
fn downcast_move<T>(this: Self) -> Option<T>
Self to T,
useful only in generic context as a workaround for specializationSource§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<T, X> CoerceTo<T> for Xwhere
T: CoerceFrom<X> + ?Sized,
impl<T, X> CoerceTo<T> for Xwhere
T: CoerceFrom<X> + ?Sized,
fn coerce_rc_to(self: Rc<X>) -> Rc<T>
fn coerce_box_to(self: Box<X>) -> Box<T>
fn coerce_ref_to(&self) -> &T
fn coerce_mut_to(&mut self) -> &mut T
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more