DSH-WM

A playable world-model toolkit for DeepSeek Harness — look at a strip, name the route, score the run, and iterate the research loop.

Point the agent at a rollout (or just fixtures/sunset) and ask: did the second half melt, is Sora even a world simulator, and which memory recipe is allowed to win.

🚀 One command to install | Play sunset with no GPU | Built-in WM map | RSI on skills and evals

🌐 English | 中文

World-model work inside DeepSeek Harness is more fun when the agent can see the strip, name the lineage, and measure the claim. DSH-WM is the profile bundle for that: contact-sheet inspect, three-route knowledge (3D display / pixel video-gen / latent prediction), run scoring, and an RSI loop on skills and wm.yaml.

dsh plugin --profile wm add github:WayneJin0918/dsh-wm
dsh --profile wm

Then try: Triage fixtures/sunset. Look at first, mid, last. Is this late-horizon?

DeepSeek’s product mainline can skip world models. Harness is still the research OS — this plugin is the WM lab on top of it.

Runtime: deepseek-ai/deepseek-harness

Play it in 30 seconds

fixtures/sunset is an 8-frame toy strip. Early pred frames stay warm and close to GT; the second half is wiped to cool blue so late-horizon collapse is obvious. No checkpoint, cluster, or GPU.

node cli.js inspect fixtures/sunset --indices first,mid,last
node cli.js diff --pred fixtures/sunset/pred --gt fixtures/sunset/gt
node cli.js diagnose "is Sora a world simulator"
node cli.js knowledge --id wm-routes

wm_inspect prints a luma sketch you can read in a terminal:

pred #0  luma=148.7  low contrast, warm / orange
    ****************
    **##************
pred #7  luma=62    near-uniform, cool / blue
    ::::::::::::::::
gt   #7  luma=160.4  low contrast, warm / orange
    ***#############

wm_rollout_diff on the same strip reports a second-half SSIM drop and names frames 4–6 as the worst window. That is the whole game: look, score, then open a card.

Highlights

• Install and play. Official DSH bundle, pure JavaScript, no prepare / allowBuilds. Sunset works from node cli.js before you even open Harness.
• Look at the frames in-repo. wm_inspect samples first / mid / last (or named indices), writes a contact sheet, and returns a luma sketch plus a color/contrast look.
• Name the route first. 3D display, pixel / video-gen WM, and latent prediction are three exams. wm-routes then display-3d / pixel-wm / latent-wm.
• A run is a directory. Optional wm.yaml declares pred / gt / log / metrics. No manifest → heuristics. Cannot tell → candidates and warnings, never invented paths.
• Measure when you have a run. wm_discover → wm_summarize → wm_rollout_diff → wm_inspect for layout, logs, numbers, and a look.
• Built-in WM knowledge. Technique cards for chunk-AR, memory, KV, exposure bias, revisit, ablation, action following, cache eviction, and RSI-in-Harness. wm_knowledge / wm_diagnose before a new architecture.
• RSI on the harness layer. Skill wm-rsi uses DSH trajectory, fork, Creator, and sunset to evolve skills, wm.yaml, and eval notes.
• Skills that keep the game honest. Triage, knowledge, RSI, fair ablation, revisit.
• No GPU required to start. Rollout scores are luminance SSIM + MSE in pure JS. Videos need ffmpeg; PNG/PPM folders do not.

Three routes

The word “world model” is three research games. Cards follow the map in Awesome World Models. Open wm-routes before you design a backbone.

A forgotten room is revisit-eval on a pixel strip, a pose / occupancy check on a 3D scene, and a latent-state mismatch on JEPA / Dreamer. Name the route, then measure.

node cli.js knowledge --id display-3d
node cli.js knowledge --id pixel-wm
node cli.js knowledge --id latent-wm
node cli.js diagnose "Gaussian explorable 3D"
node cli.js diagnose "JEPA latent Dreamer"

Who it is for

Quick start: three steps

1. Install

Into a dedicated research profile:

dsh plugin --profile wm add github:WayneJin0918/dsh-wm

Web or Headless also work:

dsh plugin --profile web add github:WayneJin0918/dsh-wm
dsh plugin --profile headless add github:WayneJin0918/dsh-wm

From a local checkout (path install does not need GitHub access):

dsh plugin --profile wm add /path/to/dsh-wm

The package is pure JS. Git installs do not need pnpm allowBuilds. Pin a commit if you want a frozen default: github:WayneJin0918/dsh-wm#<sha>.

2. Restart and check it

dsh --profile wm --dump-config    # look for "# == dsh-wm"
dsh --profile wm

Restart a running Web profile after adding the bundle, then start a new session so the skill catalog reloads.

3. Ask something you would actually say

Triage fixtures/sunset. What failed, and is it late-horizon?
Look at first, mid, last — what do the pixels do in the second half?
Is Sora a world simulator, or a pixel WM that still has to pass the stick?
The return trip forgot the room — which memory recipe is even allowed?
These two runs claim a memory win — are they paired on scene/protocol/seed?
Use Harness RSI to tighten the revisit skill; keep sunset as the gate.

Common workflows

Toolbox

Three families you can compose in one session:

Rollout scores are luminance SSIM + MSE. wm_inspect is the built-in way to look at the strip.

Knowledge cards

Routes: wm-routes · display-3d · pixel-wm · latent-wm

Technique: chunk-ar · memory-types · kv-memory · exposure-bias · revisit-eval · ablation-protocol · action-following · cache-eviction · rsi-harness · diagnosis-map

node cli.js knowledge
node cli.js knowledge --id wm-routes
node cli.js knowledge kv memory
node cli.js knowledge --id rsi-harness
node cli.js diagnose "is Sora a world simulator"
node cli.js diagnose "late collapse after the first chunk"

Skills

• wm-run-triage — walk a run: discover → summarize → diff → inspect, then name the failure
• wm-knowledge — open a route or technique card before designing
• wm-rsi — one claim, one card, one measurement, one skill / wm.yaml change, sunset gate
• wm-ablation — paired scene / protocol / seed before any mean
• wm-revisit — geometric loop vs frame-similarity proxy

RSI with Harness

DeepSeek Harness already gives you append-only trajectories, fork/replay, and Creator mode (inspect the live plugin tree). DSH-WM points that at world-model process:

1. Write a falsifiable claim.
2. Open wm_knowledge (rsi-harness + the technique, after wm-routes if the lineage is unclear).
3. Measure (wm_summarize / wm_rollout_diff) and look (wm_inspect).
4. Change one skill, wm.yaml field, or eval note.
5. Gate on fixtures/sunset (must still report late-horizon drop) and a paired user scene.
6. Solidify or roll back; keep the session.

The repeatable core is numbers plus cards.

wm.yaml

name: sunset-revisit
pred: outputs/pred          # frame directory or mp4
gt: outputs/gt
log: logs/train.log
metrics: metrics.json       # any JSON; keys are summarized, not schema-validated

Without the file, the plugin looks for pred|preds|recon, gt|target|ref, train.log / logs/*.log, and metrics.json / *eval*.json.

How it works

flowchart LR
  play[Ask or point at a run] --> know[wm_knowledge / wm_diagnose]
  know --> measure[wm_discover / summarize / diff / inspect]
  measure --> rsi[wm-rsi on skills and wm.yaml]
  rsi --> gate[sunset fixture plus paired scene]

Three layers, one session:

1. Knowledge — name the route, then open a technique card.
2. Measure — filesystem tools plus wm_inspect.
3. RSI — evolve the research loop and pass the sunset gate.

Offline fixture

fixtures/sunset is the built-in playground. Pred frames 0–3 stay close to GT; 4–7 are wiped so second-half SSIM drops.

npm test
npm run check
node scripts/generate-fixtures.js    # regenerate after changing the painter

Configuration and limits

Requirements

• DeepSeek Harness 0.1.0-rc.6 or compatible, with pnpm on PATH for dsh plugin.
• Node.js 18+.
• Optional ffmpeg for JPEG or video inputs. PNG/PPM frame directories work offline.

Install, upgrade, disable, and uninstall

dsh plugin --profile wm update github:WayneJin0918/dsh-wm
dsh plugin --profile wm remove dsh-wm

To disable the bundle temporarily, set this in the profile patch:

- id: dsh-wm
  disabled: true

Restart the profile after enabling or upgrading.

Troubleshooting

Development

npm test
npm run check

• See CHANGELOG.md for releases.
• Use GitHub Issues on this repository for bugs and focused requests.

Acknowledgements

DSH-WM stands on these upstream projects. Thank you to their authors and the maps they made reusable.

• DeepSeek Harness (dsh) — the official runtime this bundle installs into. Docs: deepseek.com/harness.
• DSH Vision Toolkit by Anionex, with agent-vision-toolkit — thanks for the open plugin and the homepage this README learned from.
• Awesome World Models — the map of 3D / pixel / latent lineages the built-in route cards follow.

License

MIT