async-bulkhead-llm - v3.17.0
    Preparing search index...

    Type Alias LLMRejectDetail

    Capacity snapshot attached to rejections so callers (e.g. gateways) can build informative 429/503 responses.

    Note: no retry-after estimate is provided — a fail-fast bulkhead has no honest ETA for capacity. Expose these numbers instead.

    type LLMRejectDetail = {
        admissionClass?: {
            availableConcurrent: number | null;
            availableProtectedConcurrent: number;
            borrowedConcurrent: number;
            id: string;
            inFlight: number;
            maxConcurrent: number | null;
            protectedConcurrent: number;
            protectedConcurrentInUse: number;
            tokenBudget?: {
                available: number | null;
                availableProtectedTokens: number;
                borrowedInFlightTokens: number;
                inFlightTokens: number;
                maxInFlightTokens: number | null;
                protectedInFlightTokens: number;
                protectedTokensInUse: number;
                requested: number;
            };
        };
        constraint?: LLMCapacityConstraint;
        inFlight: number;
        limitRevision: number;
        maxConcurrent: number;
        maxQueue: number;
        pending: number;
        sharedCapacity?: {
            concurrency: {
                available: number;
                capacity: number;
                inUse: number;
                requestedBorrowed: number;
            };
            tokenBudget?: {
                available: number;
                capacity: number;
                inUse: number;
                requestedBorrowed: number;
            };
        };
        tokenBudget?: {
            available: number;
            budget: number;
            effectiveBudget: number;
            inFlightTokens: number;
            requested: number;
        };
    }
    Index
    admissionClass? constraint? inFlight limitRevision maxConcurrent maxQueue pending sharedCapacity? tokenBudget?
    admissionClass?: {
        availableConcurrent: number | null;
        availableProtectedConcurrent: number;
        borrowedConcurrent: number;
        id: string;
        inFlight: number;
        maxConcurrent: number | null;
        protectedConcurrent: number;
        protectedConcurrentInUse: number;
        tokenBudget?: {
            available: number | null;
            availableProtectedTokens: number;
            borrowedInFlightTokens: number;
            inFlightTokens: number;
            maxInFlightTokens: number | null;
            protectedInFlightTokens: number;
            protectedTokensInUse: number;
            requested: number;
        };
    }

    Bounded policy-class snapshot used for this decision.

    Type Declaration

    • availableConcurrent: number | null

      null means availability is governed only by the global ceiling.

    • availableProtectedConcurrent: number
    • borrowedConcurrent: number
    • id: string
    • inFlight: number
    • maxConcurrent: number | null

      null means no class-specific concurrency ceiling.

    • protectedConcurrent: number
    • protectedConcurrentInUse: number
    • OptionaltokenBudget?: {
          available: number | null;
          availableProtectedTokens: number;
          borrowedInFlightTokens: number;
          inFlightTokens: number;
          maxInFlightTokens: number | null;
          protectedInFlightTokens: number;
          protectedTokensInUse: number;
          requested: number;
      }
      • available: number | null

        null means availability is governed only by the global budget.

      • availableProtectedTokens: number
      • borrowedInFlightTokens: number
      • inFlightTokens: number
      • maxInFlightTokens: number | null

        null means no class-specific token ceiling.

      • protectedInFlightTokens: number
      • protectedTokensInUse: number
      • requested: number
    constraint?: LLMCapacityConstraint

    Present for capacity decisions when admission classes are configured.

    inFlight: number
    limitRevision: number

    Limit revision captured with this capacity decision.

    maxConcurrent: number
    maxQueue: number
    pending: number
    sharedCapacity?: {
        concurrency: {
            available: number;
            capacity: number;
            inUse: number;
            requestedBorrowed: number;
        };
        tokenBudget?: {
            available: number;
            capacity: number;
            inUse: number;
            requestedBorrowed: number;
        };
    }

    Shared capacity remaining after every configured class floor.

    tokenBudget?: {
        available: number;
        budget: number;
        effectiveBudget: number;
        inFlightTokens: number;
        requested: number;
    }

    Type Declaration

    • available: number

      Tokens available at this priority (clamped at 0).

    • budget: number

      Configured budget ceiling.

    • effectiveBudget: number

      Budget this request was admitted against (priority-adjusted).

    • inFlightTokens: number

      Tokens currently held by in-flight reservations.

    • requested: number

      Tokens this request needed.

    Settings
    Member Visibility

    On This Page
    Properties