async-bulkhead-llm - v3.17.0
    Preparing search index...

    Function createAdaptiveTokenEstimator

    • A self-calibrating wrapper around createModelAwareTokenEstimator (v3.8).

      Character-ratio estimation is ±15% at best and drifts with content mix (code vs prose vs CJK) and provider tokenizer changes. This estimator closes the loop: feed it actual usage from completed calls via observe(), and it maintains a per-model EWMA of actual input / estimated input, multiplying future input estimates by that factor (clamped, and only after minSamples observations). Output reservations are never corrected — max_tokens / outputCap is a ceiling, not an estimate.

      The correction applies to the whole input estimate, including any opaqueBlockTokens surcharge and extraInputTokens — the observed ratio necessarily includes those components too, so scaling the total is the self-consistent choice (the fixed point is "corrected estimate ≈ actual"). If your extraInputTokens values are exact, expect the factor to settle slightly differently than a pure-prose deployment; the clamps bound the damage either way.

      observe() always measures against the uncorrected base estimate, so feedback does not compound: an already-corrected estimator does not drag its own factor back toward 1.

      Not distributed and not persisted — calibration lives in this instance. Share one instance per bulkhead (create it, pass .estimator into tokenBudget, and subscribe .observe to the same bulkhead's release events).

      Parameters

      Returns AdaptiveTokenEstimator