dsh-llm-repetition-guard

English | 中文

Function plugin that fails a model request whose streamed output degenerates into repetition — a loop over one word, phrase, or n-gram — so the agent loop's agent/request-error recovery can retry it under the provider's retry policy. It wraps the llm/stream waterfall on ctx.llm: each model call gets a fresh per-stream detector, and the wrapper yields a terminal DEGENERATE_OUTPUT error finish (then truncates the stream) as soon as the trailing text shows one unit repeated maxRepeats times. Retry happens inside the same turn/step — the loop re-issues the request against the same durable history — and the truncated chunks never enter derived messages. This plugin only classifies; it never re-dispatches the adapter.

Detection is per content block, keyed by stream index, under a bounded sliding window. text deltas are guarded by default; reasoning deltas are ignored unless checkReasoning is set, because reasoning text is pattern-heavy by nature and would trip conservative thresholds. Tool-call argument streams are never guarded — JSON fragments legitimately repeat keys and separators. A block must accumulate minOutputLength characters before detection starts, so short replies (even fully repetitive ones) are left alone.

The guard trips on the longest trailing run of one unit, trying every alignment: "data data data …" is caught whether the last unit lands on a window boundary or not. The repeated unit names the failure message, and the DEGENERATE_OUTPUT code routes through the same recovery path as EMPTY_RESPONSE.

Usage

Install the package:

npm install dsh-llm-repetition-guard

Mount the plugin beside @deepseek-ai/dsh-llm-retry and add DEGENERATE_OUTPUT to the provider's retryableCodes:

- name: '@deepseek-ai/dsh-llm-deepseek'
  config:
    retryPolicy:
      mode: normal
      maxRetries: 2
      retryableCodes: ['EMPTY_RESPONSE', 'DEGENERATE_OUTPUT', 'RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT']

- name: 'dsh-llm-repetition-guard'
  config:
    minOutputLength: 64
    windowChars: 1024
    unitLength: 8
    maxRepeats: 6
    checkReasoning: false

- name: '@deepseek-ai/dsh-llm-retry'

Why the explicit retryableCodes? The published @deepseek-ai/dsh-llm (through 0.1.0-rc.7) does not include DEGENERATE_OUTPUT in its default retryable set. Without it, the guard still truncates the degenerate stream and fails the request, but the loop treats the failure as terminal and the turn ends with an error. Add the code to make retry effective. A harness checkout that already carries the code in DEFAULT_RETRYABLE_CODES may omit the list.

The plugin has no other configuration. Thresholds are optional and default conservatively:

Verifying it works

Send a prompt that makes the model loop (or lower maxRepeats to 2 and use a degenerate output). The session log records the recovery:

• an llm/retry event whose failure.code is DEGENERATE_OUTPUT and whose failure.message names the repeated unit;
• the retried request completes normally and the turn ends completed.

Model Experience

Degenerate-output recovery

What the model sees

The failure itself is not model-visible. The retried request reconstructs the same explicit provider/model request from durable surface history; chunks that were truncated never enter derived messages. The llm/retry scheduling event is non-surface.

Token effect

Each retry is a new provider request and may repeat input-token billing; the truncated degenerate attempt still bills its own output tokens at the provider. Normal retry policy has a finite budget.

KV Cache effect

The reconstructed request preserves the prior prefix and is eligible for provider cache reuse under that provider's rules.

Known Limitations and Deferred Work

• Truncation discards the partial block — already-streamed chunks stay in the session log (matching the existing error-path convention), but no assistant/message is assembled for the failed attempt.
• Non-consecutive repetition is not detected — only a trailing run of one unit trips the guard; a stream that alternates two phrases or scatters a short unit sparsely stays below the threshold.
• Direct ctx.llm.stream() consumers receive the DEGENERATE_OUTPUT error finish but must implement their own recovery; retry is owned by the agent loop.
• The guard is opt-in — compose the plugin (or add it to a bundle) to activate it; it is not part of dsh-base.