async-bulkhead-llm - v3.17.0
    Preparing search index...

    Type Alias LLMToken

    Admission token returned by acquire().

    Call release() exactly once when the request completes. Pass TokenUsage to enable the refund mechanism — the bulkhead returns the difference between the pre-admission reservation and actual consumption to the budget immediately.

    For streaming workloads, call reportUsage() with cumulative usage as stream events arrive:

    • If the reported input is lower than the pre-admission estimate, the surplus is refunded to the budget immediately (the full output reservation is retained).
    • If reported consumption exceeds the hold, the hold expands (overrun), which blocks new admissions until this request releases.
    • Reports are clamped to be monotonically non-decreasing per field.
    • If release() is later called without usage, the last reported usage is used for the final refund.
    type LLMToken = {
        admissionClass?: string;
        admissionId: string;
        limitRevision: number;
        reservation: LLMReservationEstimate | null;
        resources: LLMAdmissionResources;
        abandonBorrowedConcurrency(): boolean;
        release(usage?: TokenUsage): void;
        reportUsage(
            usage: TokenUsage,
            reconciliation?: ProgressiveReconciliationOptions,
        ): UsageReport;
    }
    Index
    admissionClass? admissionId limitRevision reservation resources
    abandonBorrowedConcurrency release reportUsage
    admissionClass?: string

    Bounded policy class used for this admission, when configured.

    admissionId: string

    Stable identifier for this successful admission.

    limitRevision: number

    Immutable limit revision captured at successful admission.

    reservation: LLMReservationEstimate | null

    Exact reservation used at admission, or null without a token budget.

    resources: LLMAdmissionResources

    Exact protected/shared resource attribution captured at admission.

    • Return a borrowed local concurrency slot before final settlement while retaining any token hold. Returns true only for the first effective call.

      Returns boolean

    Settings
    Member Visibility

    On This Page
    Properties
    Methods