Compatibility

What you get

dsh-auto-review puts a second model on the approval/request answerer chain:

1. Official seam — an answerer that claims only the requests it owns (ai policy) and delegates everything else via next(); the human approval flow is never short-circuited.
2. Read-only reviewer subagent — a one-shot fork with a read/glob/grep tool allow-list returns a structured verdict { decision, reason, riskLevel }. Reviewer asks are recognized by identity and delegated; maxDepth + the allow-list keep the reviewer non-delegating.
3. Fail closed — reviewer crash, timeout, or schema mismatch resolves through fallbackPolicy (default rejected); a deny verdict feeds its reason back to the calling model.
4. Config-driven routing — per-tool policies (ai/human/never) plus regex risk rules, all changeable from cordis.yml.
5. Deny reasons reach the model — the reviewer's reason is injected into the denied tool result (callId-linked); fallback and never-policy rejections inject auditable markers too ([auto-review] / [auto-review-fallback] / [auto-review-never]).
6. Full audit trail — log-only autoReview/verdict + autoReview/rejection session events (envelope ignorable: true) plus an optional invariant companion enforcing marker ⟺ event.
7. Safety knobs — a rejection circuit breaker (3 consecutive denials, or 6 of the last 10 verdicts, per turn), a risk-level policy, a one-shot /auto-review approve override, and a never-policy hard disable that explains itself to the model.
8. Optional reviewer context — a bounded compact transcript (contextBudget) plus a Codex-style Markdown ruling policy (reviewerPolicyText).

Every decision reconstructs from the session log: approval/asked → autoReview/verdict (or autoReview/rejection) → approval/decided.

Why a second model instead of rules?

Pattern-based auto-approvers decide before dispatch, with no evidence. dsh-auto-review gives the decision to a reviewer subagent that reads the actual workspace (through its read-only tool face), the already-streamed tool-call arguments (sensitive values redacted), the request reason, and your risk rules — then returns a structured verdict. A deny verdict feeds its reason back to the calling model, so the agent learns why instead of retrying blindly.

Quick start

# 1. install the bundle into your profile
dsh plugin --profile web add "github:PerryLink/dsh-auto-review#main"

# or from npm (published releases)
dsh plugin --profile web add dsh-auto-review

# 2. restart and verify the row
dsh --profile web --dump-config | grep -A4 'id: auto-review'

Out of the box the shipped patch AI-reviews bash and write; every other tool (including edit — in-place modification) delegates to the human chain. Add edit: ai explicitly if you accept in-place edits without a human in the loop.

Install & uninstall

• git channel (latest main): dsh plugin --profile web add "github:PerryLink/dsh-auto-review#main" — the isolated prepare build needs the single allowBuilds: { esbuild: true } key the dsh CLI prints for dsh-auto-review.
• npm channel (published releases): dsh plugin --profile web add dsh-auto-review.
• tarball channel: pnpm pack in this repo, then dsh plugin --profile web add ./dsh-auto-review-<version>.tgz.
• uninstall: dsh plugin --profile web remove dsh-auto-review (or remove the row from the profile patch).

Configuration

All tunables are Schemastery Config fields (changeable from cordis.yml). An id-targeted override replaces the whole row — restate every key you need.

Example (annotated full form: fixtures/config/config-full.yaml):

- insert:
    - id: auto-review
      name: dsh-auto-review
      config:
        toolsPolicy:
          overrides: { bash: ai, write: ai }
        riskRules:
          - pattern: '(?i)(rm\s+(-[a-z]+\s+)*/|git\s+push\s+--force)'
            policy: never
          - pattern: 'write'
            policy: never
            field: toolName
        reviewerTimeoutMs: 30000
        fallbackPolicy: delegate
        riskPolicy: { maxAutoAllow: medium, onHighRisk: delegate }
        circuitBreaker: { consecutiveDenies: 3, windowDenies: 6, windowSize: 10, action: delegate }

Tools & surfaces

Session command

/auto-review on|off|status|approve [n]

on/off append the durable autoReview/state override (the fold survives restart/resume — replay IS the state) and inject a switch notice the model sees (logged as a user/message event). status reports the effective state, both per-turn budgets (AI verdicts and reviewer failures), a tripped circuit breaker when one is active, and the session's cumulative statistics (allows/denies/fallbacks/never rejects, mean duration, recent verdicts). approve [n] records a single-use autoReview/override for the n-th most recent denial (1 = most recent): the next same-tool review within overrideTtlMs carries the authorization as reviewer context — the reviewer still decides, and the override is consumed by that review regardless of its outcome.

Web review panel

In the Web GUI (web profile), the package contributes a session-header action (AI Review) that opens a panel with the session's auto-review state: the switch with on/off buttons (they execute /auto-review on|off), both per-turn budgets, cumulative statistics (including hard-disable rejections), the circuit trip, the recent verdicts, and one-shot approve buttons for recent denials (they execute /auto-review approve [n]).

How it is wired:

• The host registers an autoReview session projection (folded from the log-only autoReview/* events) and serves it through the session-projection channel.
• The browser half is a client module (auto-discovered from the dsh.client declaration) registered on the conversation.session.header.actions seat.
• No extra patch rows are needed: the panel loads whenever the plugin is installed in a profile whose web build provides the session-projection capability (the web profile does). Without that capability the panel reports itself unavailable; the answerer is unaffected.

The panel reads only whole projection values — it never receives the raw session event stream.

How it works

                       approval/request waterfall (answerer chain)
                        │
┌───────────────────────┴──────────────────────┐
│ dsh-auto-review answerer                     │
│  · session enabled?  · policy = ai?         │   no ── next() ──▶ human answerer (UI)
│  · risk rules → toolsPolicy → default       │
└───────────────────────┬──────────────────────┘
                        │ yes
                        ▼
        ┌───────────────────────────────────┐
        │ reviewer subagent (fork, one-shot)│
        │  · toolFilter: read/glob/grep     │
        │  · outputSchema: {decision,       │
        │    reason, riskLevel}             │
        │  · timeout + req.signal abort     │
        └───────────────┬───────────────────┘
                        │ verdict / failure (fail-closed fallback)
                        ▼
 allow → allowed-once        deny → rejected + reason injected into the
                                       denied tool result (callId-linked)
                        │   never → rejected + [auto-review-never] feedback
                        │            (hard disable, no reviewer runs)
                        ▼
 audit: approval/asked → autoReview/verdict | autoReview/rejection
        → approval/decided (session events, log-only, invariant-checked)

Composition order. The answerer runs at its registration position in the waterfall: if a human UI answerer is composed BEFORE the auto-review row, humans answer first and the reviewer only sees what is delegated downstream. Verify with dsh --profile <name> --dump-config and place the auto-review row before your human answerer rows when you want ai-policy tools routed to the reviewer first.

dsh-eval — agent evaluation engine

Beyond the approval reviewer, dsh-auto-review ships dsh-eval: a YAML-driven agent evaluation platform that runs real headless DSH sessions (one isolated agent + scratch workspace per case, the official Minimal persona as the baseline system prompt), collects the tool-call trace from the session event log, and evaluates structured assertions plus an optional second-model review — the same reviewer seam as the approval answerer.

# eval/cases/demo.yaml (abridged)
suite:
  name: my-suite
  cases:
    - id: math-output
      input: Solve 17 × 24 and reply with only the final number, nothing else.
      expect:
        output: { contains: "408" }
    - id: glob-trace
      seedFrom: '.'
      input: Use the glob tool with pattern "src/**" to list the source files…
      expect:
        toolCalls: [{ tool: glob, arguments: { contains: { pattern: "src" } } }]
        results: [{ tool: glob, contains: "index.ts" }]

Run it (a DeepSeek API key must be in the environment):

dsh-eval eval/cases --model deepseek-v4-flash --timeout-ms 240000 --out .eval-reports

CI gate: the process exits 0 only when every case of every suite passed — drop it into a GitHub Action step and failing evaluations fail the build. Each case leaves a replayable session JSONL and a trace JSON beside report.md/report.json; assertion results, token usage, and the review verdict are all written into the report files.

Permissions & data

• Permissions: the workshop manifest declares session:append, approval:answer, subagent:spawn, command:register, and tools:observe.
• Data: nothing is stored on disk; the report ring buffer is in-memory and bounded. No network requests of its own.
• Session log: autoReview/* events carry reviewer identity, verdict, reason, risk, and duration — appended with the envelope's ignorable: true marker so any build loads the log.

Security boundaries

• The reviewer is a model. Its verdicts are advisory policy, not a security kernel; prefer human/never rules for irreversible operations.
• Fail closed. Every abnormal path (provider missing, capability gaps, start rejection, timeout, non-completed stop reason, missing/malformed verdict, audit-correlation failure) resolves through fallbackPolicy, default rejected — and the rejection feeds an auditable reason back to the model. allow-once grants unconditionally; it exists only for unattended deployments whose admin accepts that risk.
• Read-only reviewer. The reviewer's toolFilter allow-list (read/glob/grep) cannot write, edit, run bash, fetch the network, or delegate (maxDepth = its own depth). Its session log is persisted and auditable.
• Sensitive arguments are redacted (key-name matching: token, password, api_key, Authorization, credentials, private keys …) before entering the reviewer prompt; the plugin never executes the reviewed arguments. Redaction is key-based, not content-based — do not AI-review tools whose argument values you cannot afford to show a model.
• Hard disables explain themselves. A never tool or risk rule rejects deterministically AND records a log-only autoReview/rejection event, then injects a [auto-review-never] marker into the denied tool result — the model learns the action is hard-disabled instead of retrying it (invariant-checked: marker ⟺ event).
• Rejection circuit breaker. A run of denials in one turn trips the breaker (consecutiveDenies / windowDenies inside windowSize), recorded as a log-only autoReview/circuit event; later requests follow its action (delegate / reject / abort-turn).
• Reviewer context is presented transcript. contextBudget feeds already-presented session content to the reviewer. With the default same-route reviewer model that content stays inside one provider; configure reviewerModel to a different provider only if you accept presenting that transcript to it.
• never is one-way at this layer. A never tool or risk rule rejects before the human chain sees the request — a lockdown knob, not a default.

Known limitations

• The reviewer needs a working LLM route (inherited by default); without one every review falls back per fallbackPolicy — never a silent grant.
• reviewerTools names must exist as global tools in the profile; an unknown name fails the reviewer child loudly at the earliest point and falls back.
• Risk rules match the request reason, the toolName, or the redacted call arguments per their field; other conditions belong in toolsPolicy.overrides.
• The /auto-review approve override authorizes the next same-tool review, not the exact historical call; a different action on the same tool consumes it.
• The verdict events are log-only; the Web review panel reads the folded autoReview projection (the raw event stream never reaches browser plugins).
• autoReview/state and autoReview/verdict are appended with the envelope's ignorable: true marker, so any harness build loads the log — readers that do not know the out-of-repo types simply skip those records. (rc.6 hosts accept and ignore the marker; sessions written by pre-0.1.1 versions can be repaired with scripts/repair-session-logs.mjs from dsh-permission-rules.)
• The git channel needs the single allowBuilds key the dsh CLI prints for dsh-auto-review itself. The repo ships its own pnpm-workspace.yaml with allowBuilds: { esbuild: true }; typescript + tsdown are regular dependencies.
• The optional invariant companion needs the invariants service (agent-spine compositions such as headless/ACP); the plain web profile does not provide it, so the row ships commented out in the bundle patch.

Related work

• Andy8647/dsh-auto-approval — two-state allow/deny classifier on the tools/pre-execute waterfall with file-log audit. dsh-auto-review deliberately differs: official answerer chain, always delegates what it does not own, read-only second model with a structured verdict, deny reasons fed back to the model, session-log audit.
• ACP automation bridge — one-shot machine decisions for its own ACP-owned agents. dsh-auto-review is session- and tool-policy-scoped for the interactive harness; it never infers durable grants.

Development

pnpm install                # node ^22.19 || >=24
pnpm run typecheck          # tsc: src + tests against the local harness checkout
pnpm test                   # vitest: 190 tests, 14 files
pnpm run build              # tsc declarations + tsdown bundles (lib/, incl. the client bundle)
pnpm run verify:self-contained
pnpm pack                   # the published tarball

Repository layout: src/index.ts (plugin contract) · src/config.ts (Schemastery schema + resolution) · src/runtime.ts (answerer, command, deny-reason injection) · src/review.ts (reviewer orchestration, prompt, sanitization) · src/events.ts (session-event vocabulary + folds) · src/projection.ts + src/projection-types.ts (the autoReview session projection) · src/invariant.ts (invariant companion) · src/eval/ (the dsh-eval engine) · eval/ (shipped evaluation composition) · bin/dsh-eval.mjs (CLI launcher) · src/client/ (browser half) · test/ · fixtures/.

Topics

deepseek-harness, dsh, dsh-plugin, cordis, approval, auto-review, second-model, ai-safety, sandbox, subagent

Contributors

• @PerryLink — creator and maintainer: the approval answerer, the reviewer subagent, risk policy and circuit breaker, the session-projection review panel, the invariant companion, dsh-eval, and the five-language docs.

PerryLink DSH Plugin Family

This project is one of the DeepSeek Harness plugins maintained by PerryLink. If this one helps you, the others likely will too:

License

Apache License 2.0 © 2026 dsh-auto-review contributors