dsh-llmasking

🌐 English · 简体中文

Transport-layer data masking for deepseek-harness (dsh): sensitive values never leave the process on their way to the model — while your session log, UI, and tool executions keep seeing real values, restored live in the stream.

session log (real values)
        │ deriveMessages()
        ▼
┌─ dsh-llmasking (llm/stream) ─────────────────────────┐
│  mask request copy: 13800138000 → [PHONE_1]          │
│  re-dispatch masked copy through the waterfall       │
│  restore every response chunk on the way back,       │
│  including placeholders split across SSE boundaries  │
└──────────────────┬───────────────────────────────────┘
                   ▼
        provider / model sees only placeholders

The threat model is logs keep truth, the wire carries masks: your dsh session log, terminal UI, and every tool execution see real values; only what crosses the network to the LLM provider is masked. Session titles and compaction summaries are covered too — they ride the same llm/stream seam.

Powered by the llmasking engine: universal detectors (email, bank card with Luhn check, IP, URL, international phone, secret family: cloud keys / PEM / JWT / git tokens / high-entropy passwords), CN rules (mobile, ID card with ISO 7064 check, landline), US rules (SSN, phone), plus your own keywords. Same value → same placeholder within a session; secrets are redacted one-way ([SECRET_1] never maps back).

Quick start

1. Install dsh (skip if you already run it — needs Node ≥ 22, dsh ≥ 0.1.0-rc.6):

npm install -g @deepseek-ai/dsh
dsh --version

2. Install the plugin into a profile. Pick any profile name — dsh initializes it with dsh-base on first use:

dsh plugin --profile my add dsh-llmasking

3. Boot it:

dsh --profile my

4. Configure a model. In the Web UI: Settings → Models — set your Base URL and API key (dsh stores credentials under $DSH_HOME, never in the repo). Or edit ~/.dsh/settings.yaml:

llm-deepseek:
  baseURL: https://api.deepseek.com

…with your key in $DSH_HOME/.credentials.yaml or the DEEPSEEK_API_KEY environment variable.

5. Verify the plugin is active (two ways):

dsh --profile my --dump-config | grep -A1 "id: llmasking"

or in the Web UI: Settings → Plugins → search llmasking → status should be active.

6. See it work — run the secret-echo test below. That's the whole setup.

Installing from GitHub instead

Installs source rather than the npm build; pnpm ≥ 10 will ask you to allow the build script — only do this for sources you trust:

dsh plugin --profile my add github:yolorouter/dsh-llmasking
# then follow pnpm's hint: add "dsh-llmasking: true" under allowBuilds
# in the profile's pnpm-workspace.yaml and re-run

Upgrade, disable, uninstall

dsh plugin --profile my update dsh-llmasking   # upgrade to the latest npm release
dsh plugin --profile my remove dsh-llmasking   # uninstall (removes the dependency AND the layer)

To temporarily disable without uninstalling, add this to the profile's cordis.patch.yml and remove it to re-enable:

- replace:
    - id: llmasking
      disabled: true

Compatibility

dsh moves fast; if a newer dsh breaks the plugin, pin your profile's dsh or open an issue — the public surface this plugin touches is the documented llm/stream waterfall, systemPrompt.section, and ctx.commands.

Configuration

Defaults are deliberate; most users need none of this. Override per profile in cordis.patch.yml (row config replaces wholesale, no deep merge):

- replace:
    - id: llmasking
      config:
        keywords: ["acme-corp-token"]
        regions: ["CN", "US"]
        maskSystem: true
        teachModel: true

How it works

• Intercepts every model call in the llm/stream waterfall (the seam dsh documents for exactly this: "yield your own chunks to short-circuit"). Requests are immutable there, so it builds a frozen masked copy — system prompt, every text/reasoning block (user input, assistant history, tool results), and tool-call arguments (parsed, masked per decoded string value, re-serialized so JSON-escaped values can't hide) — then re-dispatches it. A process-local marker stops the second pass from recursing.
• The response stream is wrapped: text/reasoning deltas flow through per-block restorers that withhold and stitch placeholders split across chunk boundaries (flushed at block close, so a withheld tail is never silently dropped), and each assembled block-end block is restored authoritatively. That last part is also the write-back path: when the model writes [PHONE_1] into a tool call, the assembled arguments are restored before the tool executes — the file/command operates on the real value.
• Placeholder mapping lives in memory, one mapping per dsh session, shared by main-loop, title, and compaction calls. No custom session events are written: dsh currently refuses to load logs containing event types unknown to the harness, and persistence isn't needed anyway — the log stores real values, so the next request re-masks deterministically.
• Sensitive-free requests take a zero-overhead passthrough (next(), original request, no stream wrapping).
• Masking fails closed (if a string exceeds the engine's input cap the request is refused rather than sent unmasked); restore fails open (on a restore error the masked text passes through with a warning — masking is the security boundary and it already happened).

Observing it: the log line and the /llmasking command

Every masked turn writes one receipt line to the dsh log (counts and entity types only — never values):

llmasking: 3 value(s) masked on the wire this turn (PHONE, EMAIL, SECRET)

The /llmasking slash command (works in the TUI and the Web UI) is the receipt and the self-test:

• /llmasking — status: mode, detector config, this session's masking stats, totals since load
• /llmasking verify — runs a sentinel value through the real masking pipeline locally (zero network) and shows the before/after: My phone number is 13800138000… → …[PHONE_1]…. PASS means the pipeline is live
• /llmasking status — same as the bare command

Permissions & data

• Files: none of yours. The plugin writes nothing and reads no user files — the only disk read is its own package manifest (for the version string); the placeholder mapping lives in memory and dies with the process.
• Network: none of its own. It has no endpoints, telemetry, or third-party calls — it only transforms the requests dsh was already sending.
• Credentials: none. API keys travel in adapter headers the plugin never sees (it sits above the adapter, and GenerateOptions carries no key material).
• Session data: masking derives from conversation content already in the session log. Statistics (/llmasking, log receipts) record counts and entity TYPES only — never values.
• What leaves the process: the masked request (placeholders instead of values). Nothing else is added.

How do I know it is working?

The plugin is invisible by design — your logs, UI, and tool executions all show real values. Two ways to see the masking with your own eyes:

The secret-echo test (30 seconds, no tools). Send one message containing a phone number and a labeled API key, asking the model to repeat both back:

我的手机号是 13800138000，API key 是 OPENAI_API_KEY=sk-proj-xxxx，请原样复述这两项。

In the reply, the phone number appears as the real value (restored), while the key position shows [SECRET_1] — secrets are masked one-way and never restored. That [SECRET_1] is the proof the model never saw the real key: if it had, the restored echo would show it. For a control experiment, disable the plugin (set disabled: true on the llmasking row in your profile's cordis.patch.yml) and ask again — this time the model recites your real key.

Wire inspection (for the unconvinced). Point llm-deepseek.baseURL at any logging proxy and inspect what actually leaves the process: your real values never appear; [PHONE_1]-style placeholders do. What dsh logs locally is original data BY DESIGN ("logs keep truth, the wire carries masks") — so the trace view is never the place to look.

What it does NOT do (honest boundaries)

• Not a vault. It is not an exec-time credential broker and never writes mappings to disk. If you need the model to use a credential without seeing it, that's a different product category.
• Secrets never come back. The secret family (API keys, PEM blocks, JWTs, git tokens, high-entropy passwords) is redacted one-way. When the model echoes [SECRET_1], it stays [SECRET_1].
• Detectors are patterns, not oracles. Novel formats, unusual spellings, or values split across separate JSON string fragments (e.g. a tool output chunked into array elements mid-value) can pass through. Masking narrows the leak surface dramatically; it does not promise zero leakage.
• The provider still learns metadata — that a conversation happened, its shape, and the placeholders themselves.
• Chunk-log fragments of tool arguments keep placeholders. Only the assembled block (what tools execute and what the durable message stores) is guaranteed restored; dsh's in-tree adapters always emit it, but a hypothetical delta-only adapter would leave tool arguments masked. Tool arguments that DO get masked are re-serialized, which may normalize JSON number formatting (1e2 → 100) and collapse duplicate keys.
• Mapping is per-process. After a restart or fork, placeholders re-number deterministically from the real-value log (the first masked phone is [PHONE_1] again), but cross-fork numbering is not inherited.

Troubleshooting

• Is it even loaded? Web UI → Settings → Plugins → search llmasking → status should be active. Or: dsh --profile my --dump-config | grep -A1 "id: llmasking".
• Quick self-test: /llmasking verify runs a sentinel value through the real pipeline locally (zero network) — PASS means masking is live.
• Error: unknown tool "" after a tool call: that is a gateway/upstream bug, not this plugin — some OpenAI-compatible gateways emit empty id/name on tool-call continuation chunks, which dsh assembles into a nameless call. Verify by pointing the same dsh at the official provider endpoint; if it works there, report it to your gateway (we hit exactly this with one gateway on 2026-08-16 and documented the fix: the gateway must not forward empty-string id/name on continuation chunks).
• Nothing gets masked? Check mode is not monitor, check regions (e.g. regions: ["CN"] disables US rules such as SSN), and remember secrets need label context (OPENAI_API_KEY=sk-... masks; a bare sk-... string does not).
• Restore problems: restore failures degrade open — masked text passes through and a warning containing restore failed is logged; see log location below.
• Where are the logs? dsh writes to the standard output of the process that started it (the terminal running dsh --profile my, or the service console for a Web deployment); dsh does not write a log file by default. The plugin's receipt lines all start with llmasking:.
• Rollback: to stop the behavior immediately, disable the plugin (see the patch snippet above under Quick start) — it takes effect on reload. To pin/roll back the version: dsh plugin --profile my add dsh-llmasking@0.1.0 (the profile's pnpm then holds that exact version).

Development

npm install        # also builds dist/ (prepare script)
npm test           # vitest: transform units + a waterfall simulation
npm run build

The test suite includes a zero-leak assertion: the fake provider-side adapter asserts it never received a real phone, email, or API key.

License & security

MIT — same as the llmasking engine it builds on.

Found a security issue (e.g. a value that reaches the provider unmasked)? Please report it privately via GitHub Security Advisories instead of a public issue.