dsh-skillnotary

A DSH plugin that pins what your agent's skills are allowed to do — and tells you when that changes.

An agent skill is a folder with a SKILL.md your agent loads and follows. It usually contains runnable shell, scripts and MCP config: unreviewed code with tool access, pinned to nothing, changing silently.

Most skill-security tools answer "is this skill dangerous right now?" skillnotary answers "is this still the skill I approved — and is it still only allowed to do what it did?" That is a different question, and it is the one that catches the change that happens after your review.

dsh plugin --profile web add github:liyixuan201211/dsh-skillnotary

中文：安装、更新、审查 agent skill 或 DSH 插件之前，先锁定它、审计它的能力，并在它被上游改动时发现。多数工具回答"这个 skill 现在危不危险"，本插件回答的是"它还是我批准的那个吗，还只被允许做当初那件事吗"。

What it changes

Once installed, the agent follows skills/skillnotary/SKILL.md: before adding or updating a skill it runs skillnotary audit (what can it do?), verify (is it what was approved?) and policy (is it allowed here?) — and reports a critical/high finding instead of quietly installing.

It ships two skills:

• skills/skillnotary/ — the review loop above, for the moment you adopt or update something.
• skills/skillnotary-drift-watch/ — for after that moment: it sets up a recurring verify (via DSH's scheduling tools) so a skill that changes once you have stopped looking is noticed, instead of being discovered later.

The moment it exists for:

$ skillnotary verify
✗ 2 drift(s) detected:
  ✗ release-notes [integrity-changed]    content digest changed: e4qHXFFicS -> Cj7IuwNp3Gb
  ✗ release-notes [capabilities-changed] capabilities changed: [exec] -> [agent.spawn,exec]

A hash tells you "something changed". That second line tells you the skill gained the ability to spawn further agents. Reproduce it yourself offline:

git clone https://github.com/liyixuan201211/skillnotary
cd skillnotary && bash demo/run.sh

Install

# this plugin
dsh plugin --profile web add github:liyixuan201211/dsh-skillnotary

# the tool it drives (not on npm yet; run it from git today)
npx --yes github:liyixuan201211/skillnotary --help

Pin the commit if you prefer:

dsh plugin --profile web add github:liyixuan201211/dsh-skillnotary#<commit>

This plugin ships no boot-time code

cordis.patch.yml is an empty patch, deliberately:

• a boot-time insert would run code inside the DSH process, for every profile, with your full privileges — a security plugin that widens the boot graph is itself attack surface;
• a bad insert can take down the whole boot layer (~/.dsh/cordis.patch.yml carries an explicit warning about this), and shipping something that could do it would be careless;
• the skill needs no privileges — it asks the agent to run npx skillnotary … through the normal shell tool, where you can see and approve each command.

So if you are reviewing this plugin before installing it: cordis.patch.yml is the whole story, and nothing there executes.

It audits itself

Both skills are checked by skillnotary like any other skill, and both declare allowed-tools: Bash because they do ask the agent to run commands. Two findings survive, and we left them in rather than gaming them:

skillnotary              MEDIUM  R007 Runtime package installation (SKILL.md:45)
                                > npx --yes github:liyixuan201211/skillnotary --help
skillnotary-drift-watch  MEDIUM  R007 Runtime package installation (SKILL.md:23)
                                > sn() { npx --yes github:liyixuan201211/skillnotary "$@"; }

· 4 finding(s) suppressed (config.ignore matched R001:reference/*; config.ignore matched R007:reference/*)
✓ no findings at or above "high"

npx is runtime package installation. Suppressing it would have been the dishonest move, and medium is below the default maxSeverity: high, so the CI gate still passes.

The suppressed findings are the same shape of false positive, in the one place it is unavoidable: reference/capabilities.md and reference/workflow.md quote commands in order to document them, and a capability table has to name the strings the detector looks for. So skillnotary.config.json exempts reference/* — by rule and by path, never an entire file:

{ "version": 1,
  "ignore": ["R001:reference/*", "R007:reference/*"],
  "allowInlineSuppressions": false }

The scoping is the point: R001 is silenced only inside reference/, so a capability the entry-point SKILL.md genuinely exercises is still reported.

What skillnotary does, in one table

Zero runtime dependencies; TypeScript with no build step (Node ≥ 23.6).

Honest limits

• Detection is context-scoped regex, not a sandbox. Targeted obfuscation evades it; a clean report means "no known signals", not "safe".
• Only the head of each file is scanned (1 MB per file, 64 MB per skill); rule R025 says so when that truncates instead of hiding it.
• Attestations are DSSE + in-toto, but this is not Sigstore — no keyless/OIDC, no transparency log.
• The lockfile pins a skill; it does not sandbox what the skill does when your agent runs it.

Links

• skillnotary — the tool: https://github.com/liyixuan201211/skillnotary
• Security audit of the tool itself (8 findings, all fixed): https://github.com/liyixuan201211/skillnotary/blob/main/SECURITY-AUDIT.md
• Plugin marketplace: https://awesome-dsh-plugin.com

Licence

MIT