Maximum tokens allowed in-flight simultaneously across all active requests.
Admission is always fail-fast when this ceiling is reached,
independent of concurrency headroom and independent of profile.
Must be a non-negative integer. 0 is valid and intentional — it
represents a pool with no budget to grant this cycle (e.g. a lease
ledger reporting exhaustion) and results in every budget-gated
admission being rejected with "budget_limit" until the ceiling is
raised (via applyLimits() or setBudget()) or tokenBudget admits a zero-token
request.
OptionalestimatorEstimator used to calculate token reservation pre-admission.
Defaults to createModelAwareTokenEstimator seeded with the bulkhead's model.
OptionalhighTokens of budget headroom reserved for priority: "high" requests.
Normal-priority admission is checked against
budget - highPriorityReserve; high-priority admission is checked
against the full budget. This lets interactive traffic keep
admitting while batch traffic has saturated the shared pool.
Must satisfy 0 <= highPriorityReserve <= budget. Default: 0
(priority has no effect).
OptionaloutputFallback output reservation when request.max_tokens is absent.
Default: 2048.
async-bulkhead-llm — public API surface.
This entry point re-exports everything the package supports; the implementation lives in focused modules:
types.ts— request/result/options/stats/event typeserrors.ts—LLMBulkheadRejectedErrorprofiles.ts—PROFILESpresetsestimators.ts— naive + model-aware estimators,extractTextLengthadaptive.ts—createAdaptiveTokenEstimator(v3.8)dedup.ts— deduplication internals (keying, share safety)validation.ts— internal numeric/estimate/usage guardsbulkhead.ts—createLLMBulkhead(admission, budget, events)Deep-importing the internal modules is not supported; the package
exportsmap exposes only this entry point.