async-bulkhead-llm - v3.17.0
    Preparing search index...

    Type Alias ModelAwareEstimatorOptions

    type ModelAwareEstimatorOptions = {
        defaultModel?: string;
        onUnknownModel?: (model: string) => void;
        opaqueBlockTokens?: OpaqueBlockTokens;
        outputCap?: number;
    }
    Index
    defaultModel? onUnknownModel? opaqueBlockTokens? outputCap?
    defaultModel?: string

    Model string used for ratio lookup when the request does not carry its own model field. Should match the model field on the owning LLMBulkhead.

    onUnknownModel?: (model: string) => void

    Called when the model string matches no built-in prefix and no override. The estimator falls back to the naive 4.0 ratio when this fires.

    opaqueBlockTokens?: OpaqueBlockTokens

    Input-token surcharge for opaque (non-text) content blocks (v3.7).

    Without this, opaque blocks contribute 0 input tokens — an image-heavy request estimates as nearly free, which is the wrong direction for admission control. A flat conservative number (e.g. 2048 per block) or a per-type map converts each opaque block into a fixed reservation. Validated at estimator creation. Omitted: surcharge disabled (previous behavior).

    outputCap?: number

    Fallback output reservation when request.max_tokens is absent. Default: 2048.

    Settings
    Member Visibility

    On This Page
    Properties