async-bulkhead-llm - v3.17.0
    Preparing search index...

    Type Alias LLMEventMap

    async-bulkhead-llm — public API surface.

    This entry point re-exports everything the package supports; the implementation lives in focused modules:

    • types.ts — request/result/options/stats/event types
    • errors.ts — LLMBulkheadRejectedError
    • profiles.ts — PROFILES presets
    • estimators.ts — naive + model-aware estimators, extractTextLength
    • adaptive.ts — createAdaptiveTokenEstimator (v3.8)
    • dedup.ts — deduplication internals (keying, share safety)
    • validation.ts — internal numeric/estimate/usage guards
    • bulkhead.ts — createLLMBulkhead (admission, budget, events)

    Deep-importing the internal modules is not supported; the package exports map exposes only this entry point.

    type LLMEventMap = {
        admit: {
            admissionClass?: string;
            admissionId: string;
            decisionDurationNs?: number;
            limitRevision: number;
            priority: LLMPriority;
            queueWaitNs?: number;
            request: LLMRequest;
            reservedTokens: number;
            resources: LLMAdmissionResources;
        };
        borrowedConcurrencyAbandon: {
            admissionClass?: string;
            admissionId: string;
            cause: LLMBorrowedConcurrencyAbandonCause;
            heldTokens: number;
            limitRevision: number;
            priority: LLMPriority;
            request: LLMRequest;
            resources: LLMAdmissionResources;
        };
        bypass: {
            admissionClass?: string;
            admissionId: string;
            detail?: LLMRejectDetail;
            limitRevision: number;
            priority: LLMPriority;
            raced: boolean;
            reason: LLMShadowableRejectReason;
            request: LLMRequest;
            reservation: LLMReservationEstimate
            | null;
        };
        bypassRelease: {
            admissionClass?: string;
            admissionId: string;
            limitRevision: number;
            priority: LLMPriority;
            reason: LLMShadowableRejectReason;
            request: LLMRequest;
            reservation: LLMReservationEstimate
            | null;
            usage?: TokenUsage;
            usageSequence: number;
        };
        bypassUsage: {
            admissionClass?: string;
            admissionId: string;
            limitRevision: number;
            outputCap: number
            | null;
            outputRemaining: number | null;
            overReservation: boolean;
            priority: LLMPriority;
            progressive?: true;
            reason: LLMShadowableRejectReason;
            remainingOutputTokens?: number;
            request: LLMRequest;
            reservation: LLMReservationEstimate | null;
            safetyMarginTokens?: number;
            sequence: number;
            usage: TokenUsage;
        };
        dedup: { admissionClass?: string; request: LLMRequest };
        reconfigure: { current: LLMAdmissionLimits; previous: LLMAdmissionLimits };
        reject: {
            admissionClass?: string;
            decisionDurationNs?: number;
            detail?: LLMRejectDetail;
            limitRevision: number;
            queueWaitNs?: number;
            reason: LLMRejectReason;
            request: LLMRequest;
        };
        release: {
            admissionClass?: string;
            admissionId: string;
            heldTokens: number;
            limitRevision: number;
            priority: LLMPriority;
            refundedTokens: number;
            request: LLMRequest;
            reservedTokens: number;
            resources: LLMAdmissionResources;
            usage?: TokenUsage;
            usageSequence: number;
        };
        usage: {
            admissionClass?: string;
            admissionId: string;
            deltaTokens: number;
            heldTokens: number;
            limitRevision: number;
            outputCap: number
            | null;
            outputRemaining: number | null;
            overReservation: boolean;
            previousHeldTokens: number;
            priority: LLMPriority;
            progressive?: true;
            remainingOutputTokens?: number;
            request: LLMRequest;
            reservedTokens: number;
            safetyMarginTokens?: number;
            sequence: number;
            usage: TokenUsage;
        };
    }
    Index
    admit borrowedConcurrencyAbandon bypass bypassRelease bypassUsage dedup reconfigure reject release usage
    admit: {
        admissionClass?: string;
        admissionId: string;
        decisionDurationNs?: number;
        limitRevision: number;
        priority: LLMPriority;
        queueWaitNs?: number;
        request: LLMRequest;
        reservedTokens: number;
        resources: LLMAdmissionResources;
    }

    Fired after a request is admitted (slot + budget acquired).

    Admission timing fields are present for normal admission decisions and absent for lifecycle events that are not admission decisions. Both values are integer nanoseconds measured with Node's monotonic process.hrtime.bigint() clock.

    queueWaitNs is the complete span spent awaiting the underlying concurrency acquire. decisionDurationNs excludes that await and measures synchronous admission work, including acquire-option normalization, precheck, postcheck, revision capture, admission id creation, and token wrapping.

    Type Declaration

    • OptionaladmissionClass?: string
    • admissionId: string
    • OptionaldecisionDurationNs?: number

      Synchronous admission-decision time, excluding queueWaitNs.

    • limitRevision: number
    • priority: LLMPriority
    • OptionalqueueWaitNs?: number

      Time spent awaiting the underlying concurrency acquire.

    • request: LLMRequest
    • reservedTokens: number
    • resources: LLMAdmissionResources

      Exact protected/shared resource attribution at admission time.

    borrowedConcurrencyAbandon: {
        admissionClass?: string;
        admissionId: string;
        cause: LLMBorrowedConcurrencyAbandonCause;
        heldTokens: number;
        limitRevision: number;
        priority: LLMPriority;
        request: LLMRequest;
        resources: LLMAdmissionResources;
    }

    Fired when a borrowed local concurrency slot is deliberately returned before the admitted work reaches final settlement.

    This event does not mean the provider stopped work. heldTokens remain charged until the eventual release event so upstream capacity is never presented as reclaimed based only on client-side abandonment.

    Type Declaration

    bypass: {
        admissionClass?: string;
        admissionId: string;
        detail?: LLMRejectDetail;
        limitRevision: number;
        priority: LLMPriority;
        raced: boolean;
        reason: LLMShadowableRejectReason;
        request: LLMRequest;
        reservation: LLMReservationEstimate | null;
    }

    Fired when observe mode executes a callback without holding capacity. This is separate from admit: bypassed work does not affect concurrency or token-budget accounting.

    Type Declaration

    bypassRelease: {
        admissionClass?: string;
        admissionId: string;
        limitRevision: number;
        priority: LLMPriority;
        reason: LLMShadowableRejectReason;
        request: LLMRequest;
        reservation: LLMReservationEstimate | null;
        usage?: TokenUsage;
        usageSequence: number;
    }

    Fired when bypassed work settles, whether successfully or by throwing.

    bypassUsage: {
        admissionClass?: string;
        admissionId: string;
        limitRevision: number;
        outputCap: number | null;
        outputRemaining: number | null;
        overReservation: boolean;
        priority: LLMPriority;
        progressive?: true;
        reason: LLMShadowableRejectReason;
        remainingOutputTokens?: number;
        request: LLMRequest;
        reservation: LLMReservationEstimate | null;
        safetyMarginTokens?: number;
        sequence: number;
        usage: TokenUsage;
    }

    Fired after an effective cumulative usage update for bypassed work.

    dedup: { admissionClass?: string; request: LLMRequest }

    Fired when a request joins an existing in-flight call via dedup.

    Type Declaration

    • OptionaladmissionClass?: string

      Deduplication is automatically partitioned by class when configured.

    • request: LLMRequest
    reconfigure: { current: LLMAdmissionLimits; previous: LLMAdmissionLimits }

    Fired after a higher-revision admission-limit snapshot is applied.

    reject: {
        admissionClass?: string;
        decisionDurationNs?: number;
        detail?: LLMRejectDetail;
        limitRevision: number;
        queueWaitNs?: number;
        reason: LLMRejectReason;
        request: LLMRequest;
    }

    Fired when a request is rejected at any stage.

    Timing is attached only when the event represents an admission decision. Deduplication-follower failures and observe-mode bypasses are deliberately left uninstrumented because they are not admission decisions.

    Boundaries for instrumented rejection paths:

    • precheck reject: queueWaitNs === 0; decision time is the precheck span
    • acquire reject / timeout: full acquire await + synchronous precheck, acquire-option normalization, and reject-detail work
    • postcheck reject: full acquire await + synchronous precheck, acquire-option normalization, postcheck, acquired-slot cleanup, and reject-detail work

    Type Declaration

    • OptionaladmissionClass?: string
    • OptionaldecisionDurationNs?: number

      Synchronous admission-decision time, excluding queueWaitNs.

    • Optionaldetail?: LLMRejectDetail

      Capacity snapshot; absent for dedup-wait rejections.

    • limitRevision: number

      Limit revision captured with the rejection decision.

    • OptionalqueueWaitNs?: number

      Time spent awaiting the underlying concurrency acquire.

    • reason: LLMRejectReason
    • request: LLMRequest
    release: {
        admissionClass?: string;
        admissionId: string;
        heldTokens: number;
        limitRevision: number;
        priority: LLMPriority;
        refundedTokens: number;
        request: LLMRequest;
        reservedTokens: number;
        resources: LLMAdmissionResources;
        usage?: TokenUsage;
        usageSequence: number;
    }

    Fired when a slot is released.

    reservedTokens is the pre-admission reservation (input estimate + max_tokens reservation). refundedTokens is what was returned to the budget — non-zero only when usage was reported and usage.input + usage.output < reservedTokens.

    Per-request actual consumption: usage ? usage.input + usage.output : null. The library does not pre-derive this onto the event payload — null (no usage reported) and 0 (genuinely zero usage) are different states that observers should distinguish.

    For cumulative consumption across all releases, prefer stats().tokenBudget.totalConsumed over aggregating these events.

    Type Declaration

    • OptionaladmissionClass?: string
    • admissionId: string
    • heldTokens: number

      Tokens held immediately before release returned them.

    • limitRevision: number
    • priority: LLMPriority
    • refundedTokens: number
    • request: LLMRequest
    • reservedTokens: number
    • resources: LLMAdmissionResources

      Exact protected/shared resource attribution captured at admission.

    • Optionalusage?: TokenUsage
    • usageSequence: number

      Last emitted usage-event sequence for this admission.

    usage: {
        admissionClass?: string;
        admissionId: string;
        deltaTokens: number;
        heldTokens: number;
        limitRevision: number;
        outputCap: number | null;
        outputRemaining: number | null;
        overReservation: boolean;
        previousHeldTokens: number;
        priority: LLMPriority;
        progressive?: true;
        remainingOutputTokens?: number;
        request: LLMRequest;
        reservedTokens: number;
        safetyMarginTokens?: number;
        sequence: number;
        usage: TokenUsage;
    }

    Fired after an effective cumulative reportUsage() update.

    Stale or duplicate reports that do not increase either cumulative usage field are ignored and do not emit an event. sequence starts at 1 per admission and increases monotonically, allowing external coordinators to reject duplicate or out-of-order updates.

    Settings
    Member Visibility

    On This Page
    Properties