dsh-outcome-loop

Task outcome ledger & acceptance plugin for DeepSeek Harness (DSH) · 任务结果账本与验收插件

Know whether a task was actually completed — with near-zero extra tokens and mechanically re-checkable evidence — and keep the result as a portable, user-owned record.

dsh-outcome-loop is a local-first, user-owned, vendor-neutral task outcome ledger and acceptance plugin for DSH. It organizes one DSH session's goal, constraints, acceptance criteria, execution evidence, user feedback, cost, and final result into a re-auditable task record.

• Stops "the model said done" from being mistaken for "the task is done": acceptance is based on mechanical evidence — tests, build, lint, exit codes, file state, diagnostics, git scope;
• Zero extra model cost by default: no extra LLM calls, no model-visible tools, no system-prompt injection;
• Local by default, offline by default: all result data lives in your own DSH storage backend;
• User owns and controls the data: inspect, delete, or export — with a preview + redaction gate before any export;
• Separates success, failure, unknown, stale evidence, user acceptance and user abandonment — an unknown is never silently converted into a success.

Interview oral (3 minutes)

One-page whiteboard script (no ADR binder): docs/ORAL-3MIN.md. Covers problem → data/control plane → non-negotiables → proof we are not cosplaying Datadog.

Quick start

Install

# From source
pnpm install
pnpm build
pnpm pack   # produces dsh-outcome-loop-<version>.tgz

# Install into a DSH profile
dsh plugin --profile <name> add ./dsh-outcome-loop-0.1.0-beta.8.tgz

Prerequisite — storageDomain (real-host verified): the plugin requires the storageDomain service, which the official dsh-base bundle does not provide (only upper bundles such as @deepseek-ai/dsh-web-app do; the web profile already has it). On a bare/headless profile, add it once:

dsh plugin --profile <name> add @deepseek-ai/dsh-storage-domain@0.1.0-rc.8
# then append to ~/.dsh/profiles/<name>/cordis.patch.yml:
# - insert:
#     - id: storage
#       name: "@deepseek-ai/dsh-storage"
#     - id: storage-json
#       name: "@deepseek-ai/dsh-storage-json"
#       config:
#         root: !!js dshHomePath("storages")
#     - id: storage-domain
#       name: "@deepseek-ai/dsh-storage-domain"
#       config:
#         backend: json

Without it the profile boot fails loud with waiting for service: storageDomain (fail-loud by design; see COMPATIBILITY.md §4 for the verified lifecycle: add → dump-config → boot → real /outcome commands → restart-read → uninstall).

The bundle mounts four plugin rows (see cordis.patch.yml):

Use (via /outcome)

/outcome new 修复登录页按钮在移动端溢出问题        # create a task contract
/outcome criterion add 移动端 375px 宽度下无横向滚动  # add an acceptance criterion (manual)
/outcome criterion add-command "pnpm test"          # command criterion (exit code 0)
/outcome criterion add-test                          # test-report criterion
/outcome criterion add-file dist/bundle.js          # artifact file criterion
/outcome criterion add-test --min-passed 2 --max-failed 1
                                                     # structured test counts (TAP auto-parsed)
/outcome verify                                     # run verification (passive observation only)
/outcome status                                     # mechanical verification × user disposition
/outcome accept | reject | revise | abandon         # user disposition (independent axis)
/outcome export [<contract>]                             # two-phase export: preview → digest
/outcome export <contract> --approve <digest> --out <path> [--overwrite]
                                                     # approve and atomically write JSONL
/outcome exports [<contract>]                        # list export manifests
/outcome import <path>                              # import a Task Contract file (outcome-loop.contract.v1)
/outcome export-contract <id> --out <path>          # export a contract file
/outcome cost [<contract>] [--summary]              # token usage (+ optional price table → cost)
/outcome calibration [<contract>]                 # dsh-code-reference decision calibration
/outcome skills [--out <path>]                    # skill candidates (read-only aggregation)
/outcome delete <contract-id> --yes                 # delete sidecar data (session log untouched)

Contribution mode (off by default, ADR-0005)

Contribution mode is a separate, not-installed-by-default consumer. Add it manually to the profile patch:

- insert:
    - id: outcome-loop-contribute
      name: dsh-outcome-loop/lib/consumers/contribute.js
      config:
        enabled: true

/contribute preview <contract>                 # batch preview (fields/sensitivity/digest)
/contribute approve <digest> <contract> --out <dir> [--summary-only]
                                               # writes consent manifest + records.jsonl (or summary.json)
/contribute revoke <contract> --out <dir> --yes  # withdrawal = delete the dataset directory

Datasets contain only the export-v1 minimal fields (no message bodies / code / credentials / absolute paths); the deterministic redaction gate blocks a whole batch on any sensitive hit; the plugin never uploads anything — delivery is entirely the user's decision.

Via the Host API

import type { Context } from '@deepseek-ai/cordis'

// Create a contract
const created = await ctx.outcomeLoop.createContract({
  sessionId: session.id,
  goalText: 'fix login bug',
  workspaceRoot: session.header.cwd,
  criteria: [
    { description: 'pnpm test passes', kind: 'command-exit',
      specification: { kind: 'command-exit', command: 'pnpm test', expectExitCode: 0 } },
  ],
})
// created: OutcomeResult<TaskContract>

// Run verification (passive: observes existing events, never executes commands on its own)
const run = await ctx.outcomeLoop.verify({ contractId: created.value.id })

// User disposition (an independent axis from mechanical verification)
await ctx.outcomeLoop.setDisposition({ contractId, status: 'accepted' })

// Two-phase export
const preview = await ctx.outcomeLoop.previewExport({ contractId })
const receipt = await ctx.outcomeLoop.exportJsonl({ contractId, previewDigest: preview.value.previewDigest })

// Record a prior decision from dsh-code-reference (or any integration, §15 — calibration only)
await ctx.outcomeLoop.recordDecisionEvidence({
  contractId,
  source: 'dsh-code-reference',
  decisionId: 'decision-42',
  strategy: 'reuse',
  predictedMatch: 0.87,
})

The full API surface is OutcomeLoopApi in src/service.ts.

Runtime flow

flowchart LR
    A[DSH session events] -->|observer| B[fact log<br/>session-sidecar]
    C[/outcome new + criteria/] --> D[TaskContract<br/>outcome_loop domain]
    E[verify] --> F{policy allows<br/>active checks?}
    F -- no --> G[passive: replay facts<br/>+ prior evidence]
    F -- yes --> H[active: sandboxed commands<br/>file / git / diagnostics / TAP]
    G --> I[VerificationRun]
    H --> I
    I --> J[disposition: accept/reject...]
    J --> K[outcome view<br/>mechanical × user axes]
    K --> L[two-phase export<br/>preview → digest → JSONL]

Active verification is never run by default: the policy layer gates every invocation (autoRun, allowedVerifierIds, timeout, output cap, allowlisted env). Commands are spawned argv-first — never through a shell string.

Active-verifier safety (beta.8)

• Infrastructure failures are always unknown: a command that timed out, failed to start, or had its output truncated is never parsed into pass/fail evidence; git-scope requires both git commands to exit 0 (non-repo directories are unknown, never pass, and git stderr is never parsed as changed paths); diagnostic-count treats a non-zero exit with zero parsed diagnostics as unknown (tool crash) while keeping tsc/eslint semantics (non-zero + findings → fail);
• Workspace confinement is realpath-based: every user- or contract-supplied path is checked against realpath of the workspace root — reads (file-exists, file-digest, json-schema, JUnit reportPath, contract import) and writes (export --out, contribute approve/revoke) reject symlinks escaping the workspace (unknown/error, never pass); in-workspace symlinks keep working.

Conceptual model

A task result is not a boolean. The ledger keeps at least five mutually independent axes (full rules in ARCHITECTURE.md and src/domain/reducer.ts):

A user may accept a result even when mechanical verification failed — and user acceptance never erases the mechanical failure. Both axes are kept.

Verification aggregation (summary)

1. Any required + blocking criterion fail → overall failed;
2. No failure but at least one required criterion unknown → inconclusive;
3. All required pass/not-applicable → passed;
4. Nothing verified → not-run;
5. Warning criteria do not change passed/failed but are always shown;
6. Conflicting current evidence → inconclusive by default — never pick the success-favoring row;
7. Contract revision change, workspace change, or stale age → old evidence is stale; stale rows never imply pass;
8. User acceptance only changes disposition, not mechanical verification;
9. An LLM judge (future, separate plugin) can at most produce a weak label.

Privacy & security (summary)

• Zero model calls, zero network, zero proactive command execution by default;
• Only structured facts are stored: command digests, exit codes, counts, digests, seq references — never full prompts, tool arguments, tool output, source code, or message bodies;
• Outcome data lives in a separate sidecar domain (outcome_loop), never in the session log, never in telemetry;
• Export is an explicit two-phase operation: preview (with digest) → approval (digest-bound; content changes invalidate it);
• Full threat model: SECURITY.md; default hard privacy gates: PRIVACY.md.

Repository layout

src/
├── domain/        # pure domain: ids / types / errors / reducer / aggregate / freshness (no DSH deps)
├── dsh/           # DSH adapters: events / observer / replay / registry / token-bridge / feedback-bridge / compatibility
├── persistence/   # storage-domain sidecar: schema / repository / queue / repair
├── verification/  # engine: registry / policy / engine / adapters(passive, active) / paths (realpath confinement)
├── export/        # redact / schema / preview / jsonl
├── consumers/     # /outcome commands + optional projection (service-only, no domain truth)
├── service.ts     # ctx.outcomeLoop (OutcomeLoopApi)
├── config.ts      # Schemastery config (defaults locked to the safe side)
└── index.ts       # plugin entry

Development

pnpm install
pnpm typecheck     # tsc --noEmit
pnpm lint          # eslint
pnpm test          # vitest (151 tests)
pnpm test:coverage # coverage with thresholds (security-critical code targets 100% branch)
pnpm build         # tsc → lib/
pnpm pack          # npm tarball (dsh plugin add install)
pnpm smoke         # plain-Node import smoke of the built bundle

Test matrix: Node 22.19+ / Node 24 (engines), CI in .github/workflows/ci.yml. Coverage thresholds (statements 80 / branches 68 / functions 80 / lines 80) are enforced in CI; security-critical modules (path confinement, consumers) are never excluded from measurement.

Relationship to dsh-code-reference

dsh-code-reference handles pre-development candidate discovery and reuse decisions; this plugin handles post-development factual verification. They install independently with a one-way optional integration: outcome-loop never imports code-reference internals.

Docs

• ARCHITECTURE.md — architecture decisions, layering, event flow, replay & idempotency
• PRIVACY.md — default privacy hard gates and data minimization
• SECURITY.md — threat model and controls
• DATA_FORMAT.md — sidecar schema and open export format
• COMPATIBILITY.md — DSH compatibility matrix and release baseline
• CHANGELOG.md — change log

License

MIT — see LICENSE.

DSH compatibility statement: developed and verified against DeepSeek Harness 0.1.0-rc.7 (see COMPATIBILITY.md). DSH is in developer preview and its APIs may change incompatibly; check the compatibility matrix before upgrading.