async-bulkhead-llm - v3.17.0
    Preparing search index...

    Type Alias LLMAdmissionClassLimits

    Protected floors and hard ceilings for one statically configured admission class.

    Admission classes are intentionally bounded at construction time. A gateway should map trusted tenant/application identity to one of these policy-class IDs rather than using arbitrary tenant IDs directly.

    Protected capacity is reserved before shared capacity is lent to requests above their class floor. Existing work is never revoked when a floor grows; new borrowing pauses until attrition restores the protected capacity. Omitted ceilings inherit the enclosing bulkhead's global ceiling. Class limits are fail-fast and do not create a separate per-class queue.

    type LLMAdmissionClassLimits = {
        maxConcurrent?: number;
        maxInFlightTokens?: number;
        protectedConcurrent?: number;
        protectedInFlightTokens?: number;
    }
    Index
    maxConcurrent?: number

    Maximum requests from this class that may be in flight.

    maxInFlightTokens?: number

    Maximum tokens from this class that may be held in flight.

    protectedConcurrent?: number

    Requests reserved for this class before shared concurrency is lent.

    protectedInFlightTokens?: number

    Tokens reserved for this class before shared token capacity is lent.