dsh-jev

The Jev decision layer for DeepSeek Harness (DSH).

DSH runs the agent. TypeSafe Jev makes the small, fast decisions. Your code decides what the answers mean.

   user task
       │
       ▼
 ┌───────────────────────────────────────────────────────────┐
 │  DeepSeek Harness agent loop                              │
 │  planning · tools · execution · sessions                  │
 │                                                           │
 │   pre-step ──▶ Jev: which tools are relevant?   → restrict│
 │   tool call ──▶ Jev: is this call safe to run?  → ask/hold│
 │   result ─────▶ deterministic loop guard (no model)       │
 │   request ────▶ Jev: which model route fits?    → route   │
 └───────────────────────────────────────────────────────────┘
       │
       ▼
  TypeSafe Jev (System One)  ← choice · score · noul

No Jev call is planned by the LLM, no model output becomes an explanation, and no model answer can widen a permission. Jev only ever narrows or gates.

At a glance

What this is — and is not

Is: a plugin that binds Jev to real DSH extension points, plus a reusable decision core you can embed in any application (games, search, MCP routers).

Is not: a Jev training or hosting project, a DSH fork, a dashboard, a database, or an MCP platform. It never auto-applies a model suggestion, never caches approvals, and never sends repositories, logs, or transcripts by default.

Try it in 60 seconds

Everything below runs offline with synthetic answers. No key, no network.

git clone https://github.com/buberlo/dsh-jev
cd dsh-jev
pnpm install
pnpm build

pnpm example:coding    # tool selection + call assessment
pnpm example:dsh       # real DSH services + real plugin (still synthetic)
pnpm example:ops       # read-only incident router
pnpm example:game      # standalone game, imports only jev-core

What pnpm example:coding shows (excerpt):

=== 1. Dynamic tool selection ===
input task : Fix the failing billing test: read src/billing.test.ts and run the test suite
categories (independent relevance questions):
  - files    relevance=0.97 relevant=true, pick=read_file p=0.88 conf=0.88
  - tests    relevance=0.93 relevant=true, pick=run_tests p=0.91 conf=0.91
selected : read_file, run_tests

=== 2. Pre-execution assessment (proposed model call) ===
policy     : allow (applied=true)
values     : matches=0.96 missing=0.06 violates=0.04

pnpm example:dsh goes further: it mounts the real DSH tool pipeline and the actual plugin, then shows a call being held with the policy rule that caused it. Every model value is visibly synthetic (mock/jev-synthetic).

What Jev decides here

Jev answers three question types; the core keeps their meanings distinct:

Independent questions are sent in one request (they cannot see each other's answers), so the code sends every question it might need and ignores the rest.

Modes and providers

Two independent switches — where answers come from, and whether they may act:

Start with the default mock + shadow, watch the logs, then move to enforce, then to live if you want real Jev answers. live + shadow still transmits state to TypeSafe — it only skips applying the decisions.

Use cases

• Coding assistant — keep only the tools a task needs, gate risky calls before they run, stop identical retry loops. examples/coding
• Read-only ops router — route an incident to diagnostics while a hard policy keeps remediation out of reach. examples/ops-readonly
• Interactive apps and games — map free text onto a bounded action set with deterministic consequences, without a chat model. examples/standalone-game
• Any agent harness — the core is harness-independent; the LangChain team describes the same pattern with TypeSafeClassifier, model routing, and tool-risk gating middleware (see docs/architecture.md).

Details and design notes: docs/use-cases.md.

Install into a DSH profile

Both packages are on npm:

dsh plugin --profile <name> add @buberlo/dsh-jev
dsh --profile <name> --dump-config   # shows the "# == @buberlo/dsh-jev" layer

For an unreleased checkout (workspace 0.1.2), pack both tarballs. The assessment wording fix lives in @buberlo/jev-core; a plugin-only tarball still resolves jev-core@0.1.0 from npm:

pnpm --filter @buberlo/jev-core pack --pack-destination ./packs
pnpm --filter @buberlo/dsh-jev pack --pack-destination ./packs
dsh plugin --profile <name> add ./packs/buberlo-dsh-jev-0.1.2.tgz
# overlay packs/buberlo-jev-core-0.1.2.tgz on the profile
# (see BENCH_LOCAL_PACKS in docs/benchmark.md)

The bundle inserts one row; configure it by overriding that row's config:

- id: jev
  name: '@buberlo/dsh-jev'
  config:
    provider: mock            # mock | live
    mode: shadow              # off | shadow | enforce

    selection:                # dynamic tool preselection
      enabled: true
      alwaysAllow: [read_file]
      categories:
        files: 'Reading or writing workspace files'
        tests: 'Running or inspecting tests'
      toolCategories:
        read_file: [files]
        write_file: [files]
        run_tests: [tests]

    assessment:               # per-call semantic check
      enabled: true
      onFailure: ask          # ask | hold   (never an allow)

    skills:                   # route the vendored TypeSafe skill
      enabled: true
      routingHints:
        typesafe-ai: 'especially for TypeSafe/Jev integration, System One models, classifiers'

    modelRouting:             # map route classes to real models
      enabled: false
      routes:
        fast: { provider: deepseek-official, model: deepseek-v4-flash }
        reasoning: { provider: deepseek-official, model: deepseek-v4-pro }

Live API — two explicit settings, never implicit:

    provider: live
    apiKey: !!js process.env.TYPESAFE_API_KEY

Configure it from the web client

In a web/desktop profile the bundle ships a configuration page: open the Plugins page and this bundle's own page to find the Jev card. It explains what Jev does in the loop and edits the safe subset live:

• mode (off / shadow / enforce) — applied immediately, no restart;
• the five feature toggles (selection, assessment, loop guard, skills, model routing).

Provider, model, and API key stay in cordis.yml (the key is a secret and is never displayed). Headless profiles have no web client and simply ignore this half; the plugin runs identically from its composed configuration.

How the integration stays safe

• No implicit live access. provider: live without an explicit key fails at plugin load — verified through the real dsh loader.
• No model-derived permissions. Failures, timeouts, aborts, stale snapshots, and validation errors produce ask/hold; configuration rejects anything else.
• No premature allow. The assessment listener always calls next() first and composes monotonically (deny > hold > ask > allow); later policies are never skipped.
• No widening. A Jev selection intersects with existing restrictions and can only hide tools, never add them.
• No stale decisions. Async results are bound to a turn/catalog snapshot; changed snapshots discard the result and apply the failure rule.
• No persistent cache. Approvals are per call; changed arguments are assessed again. The loop guard is a bounded, deterministic counter.
• Bounded, redacted data. Only task text, tool metadata, and bounded arguments leave the process; redaction is an extra measure, not anonymization.

Measured value

Explainer video with narration: English · German Raw terminal replays: GIF · EN · DE

Jev stopped every attempt: on a weaker model the agent tried to delete the protected audit trail in 10/10 runs — 31 denials, zero executions — while without Jev it got through. Footnote, honestly: a rule inside the prompt also held in our runs and is cheaper; Jev is the guarantee when that rule cannot live in the model context, when the model cannot be trusted, or when a denial must be auditable. Full method, raw numbers, videos and limits: docs/benchmark.md.

Status

Repository layout

packages/jev-core/     harness-independent decision core
packages/dsh-jev/      real Cordis/DSH plugin (bundle)
examples/              coding · ops-readonly · standalone-game · DSH runtime
evals/fixtures/        versioned de/en evaluation dataset
.agents/skills/        vendored TypeSafe agent skill (pinned upstream commit)
docs/                  architecture · upstream compatibility · policy · evaluation · roadmap · skills · use cases
scripts/               verify.sh · packaging-test.mjs · run-evals.ts

Documentation

Development

pnpm install          # workspace install
pnpm build            # tsc for both packages
pnpm test             # 120 tests (82 core + 38 DSH)
pnpm calibrate        # threshold sweep over the fixtures (mock; --live with a key)
pnpm bench:compare    # with/without Jev: real loop, scripted model, no key needed
pnpm bench:cli        # CLI A/B run harness (needs an OpenAI-compatible gateway)
pnpm evals            # 25 mock evaluation fixtures
pnpm verify           # install → build → typecheck → tests → evals → examples → packaging

AGENTS.md holds the permanent project rules (pinned deps, no implicit keys, monotonic decisions, no release automation).

License

MIT