async-bulkhead-llm - v3.17.0
    Preparing search index...

    Type Alias LLMStats

    async-bulkhead-llm — public API surface.

    This entry point re-exports everything the package supports; the implementation lives in focused modules:

    • types.ts — request/result/options/stats/event types
    • errors.ts — LLMBulkheadRejectedError
    • profiles.ts — PROFILES presets
    • estimators.ts — naive + model-aware estimators, extractTextLength
    • adaptive.ts — createAdaptiveTokenEstimator (v3.8)
    • dedup.ts — deduplication internals (keying, share safety)
    • validation.ts — internal numeric/estimate/usage guards
    • bulkhead.ts — createLLMBulkhead (admission, budget, events)

    Deep-importing the internal modules is not supported; the package exports map exposes only this entry point.

    type LLMStats = {
        admissionClasses?: {
            classes: Record<string, LLMAdmissionClassStats>;
            defaultClass: string;
            shared: LLMAdmissionClassSharedStats;
        };
        bulkhead: Stats;
        deduplication?: { active: number; hits: number };
        limits: LLMAdmissionLimits;
        llm: LLMRequestStats;
        observe?: LLMObserveStats;
        tokenBudget?: {
            available: number;
            budget: number;
            highPriorityReserve: number;
            inFlightTokens: number;
            totalConsumed: number;
            totalOverrun: number;
            totalRefunded: number;
            totalReserved: number;
        };
    }
    Index
    admissionClasses? bulkhead deduplication? limits llm observe? tokenBudget?
    admissionClasses?: {
        classes: Record<string, LLMAdmissionClassStats>;
        defaultClass: string;
        shared: LLMAdmissionClassSharedStats;
    }

    Present only when bounded admission classes are configured.

    bulkhead: Stats

    Underlying async-bulkhead-ts-compatible stats.

    deduplication?: { active: number; hits: number }

    Present only when deduplication is enabled.

    Type Declaration

    • active: number

      Number of distinct in-flight deduplication keys.

    • hits: number

      Cumulative requests that shared an existing in-flight call.

    limits: LLMAdmissionLimits

    Currently applied atomic admission-limit snapshot.

    llm: LLMRequestStats

    LLM-layer request stats.

    observe?: LLMObserveStats

    Present after at least one observe-mode bypass has occurred.

    tokenBudget?: {
        available: number;
        budget: number;
        highPriorityReserve: number;
        inFlightTokens: number;
        totalConsumed: number;
        totalOverrun: number;
        totalRefunded: number;
        totalReserved: number;
    }

    Present only when tokenBudget is configured.

    Type Declaration

    • available: number
    • budget: number
    • highPriorityReserve: number

      Budget headroom reserved for priority: "high" requests.

    • inFlightTokens: number
    • totalConsumed: number

      Cumulative actual tokens consumed (usage.input + usage.output), summed across releases that reported TokenUsage. Releases without usage contribute 0 — totalConsumed is meaningful only when getUsage is wired up consistently. Not clamped: over-consumption (actual > reserved) is reported as-is.

    • totalOverrun: number

      Cumulative tokens held beyond original reservations because reportUsage() reported consumption exceeding the reserved hold. Overrun expands inFlightTokens (possibly above budget), which correctly blocks new admissions until the overrunning work releases.

    • totalRefunded: number

      Cumulative tokens returned to the budget via the refund mechanism — both early refunds from reportUsage() and refunds at release.

    • totalReserved: number

      Cumulative tokens reserved at admission across all successful admissions. Monotonically increasing.

    Settings
    Member Visibility

    On This Page
    Properties