async-bulkhead-llm - v3.17.0
    Preparing search index...

    Type Alias LLMRequest

    Minimal request shape.

    model is optional at the request level. When present, the estimator uses it for ratio lookup instead of the bulkhead-level default. This supports A/B testing, canary deployments, and mixed-model routing through a single bulkhead.

    type LLMRequest = {
        extraInputTokens?: number;
        max_tokens?: number;
        messages: LLMMessage[];
        model?: string;
        system?: string | ContentBlock[];
    }
    Index
    extraInputTokens?: number

    Additional input tokens the character-based estimators cannot see (v3.7) — e.g. provider-side cost of tool schemas kept outside messages, or media priced by out-of-band rules. Built-in estimators add this verbatim to their input estimate; custom estimators may honor or ignore it.

    Must be a non-negative integer when present. Participates in the default deduplication key like any other request field, so requests differing only here are never conflated.

    max_tokens?: number
    messages: LLMMessage[]
    model?: string
    system?: string | ContentBlock[]

    Optional system prompt (v3.7). Counted by the built-in estimators exactly like message content: a plain string, or an array of content blocks (text blocks counted by characters, non-text blocks subject to the model-aware estimator's opaqueBlockTokens surcharge).

    Before v3.7, callers had to fold the system prompt into a synthetic message for it to participate in estimation — which also distorted events, logs, and deduplication keys. Carry it here instead.