dsh-multi-model-orchestrator

A DeepSeek Harness (dsh) plugin that turns your primary AI into a multi-model "main brain". When a task is complex, it decomposes the task into subtasks and dispatches each to the sub-agent model best suited for it (GLM / Kimi / Qwen / any OpenAI-compatible route you configure), while tracking per-model token usage.

Features

1. Multi-model orchestration guidance — a system-prompt section coaches the main brain to decompose complex tasks, inspect available routes with list_subagent_models, and dispatch each subtask via subagent with an explicit provider / model.
2. Model-aware dispatch — you describe each route's strengths/weaknesses once (see config); the main brain assigns work accordingly (strong models for hard reasoning, fast/cheap ones for high-volume work).
3. Per-model token usage — a model_token_usage tool reports input / output / cache read / cache write tokens and request counts per provider/model, accumulated since the process started.

The plugin does not touch any harness internals. All model routes and the sub-agent allow-list are ordinary harness settings — see examples/settings.yaml.

Install

Install into a dsh profile (usually web):

dsh plugin --profile web add dsh-multi-model-orchestrator

or, from a git checkout:

dsh plugin --profile web add github:YOU/dsh-multi-model-orchestrator

Restart the harness (or reload the profile) afterwards. The plugin loads itself as a profile layer via its cordis.patch.yml.

Quick start

1. Add your third-party model routes to $DSH_HOME/settings.yaml (default ~/.dsh/settings.yaml). GLM / Kimi / Qwen presets ready to paste: see examples/settings.yaml. Each route needs only an API key behind apiKeyEnv (env var, $DSH_HOME/.credentials.yaml, or the web Models page).

2. Enable sub-agent model selection so the main brain can pick the model for each child:

   subagent-model-selection:
     enabled: true
     allowedModels:
       - { provider: glm, model: glm-4.6 }
       # ... your other routes

3. (Optional) Add model notes the main brain reads when assigning work:

   multi-model-orchestrator:
     modelNotes:
       glm/glm-4.6:
         description: Zhipu flagship, strong reasoning.
         strengths: complex reasoning, coding, agentic tool use
         weaknesses: slower and pricier

4. Restart the harness and ask, e.g.:

   Break this into parallel subtasks and dispatch each to the best model.

Credentials

Keys are resolved per request through each route's apiKeyEnv. Provide them any of these ways:

• Environment variables: GLM_API_KEY / KIMI_API_KEY / DASHSCOPE_API_KEY (per your route names).
• Credential store $DSH_HOME/.credentials.yaml:

  GLM_API_KEY: sk-xxxx
  KIMI_API_KEY: sk-xxxx
  DASHSCOPE_API_KEY: sk-xxxx

• Web "Models" page: paste the key directly (stored in the managed credential document).

Routes without a key fail at request time with MISSING_CREDENTIAL and do not affect configured ones.

How the main brain works

Ask for a complex task in the main conversation. The main brain will:

1. use todo_write to record the decomposed subtasks;
2. call list_subagent_models to see available routes;
3. dispatch each subtask through subagent (with provider / model) to the best-fit model — launching independent delegations in one message, running them in the background by default;
4. gather results and synthesize the final deliverable.

Route names are whatever your profile actually registers — the GLM / Kimi / Qwen names in the examples are illustrative. If you registered routes under other providers (e.g. a zai gateway), reference those exact provider/model ids in subagent-model-selection and modelNotes. Ask list_subagent_models for the live list.

To see usage, ask the main brain to call model_token_usage (or just ask "how many tokens has each model used?").

Configuration reference

Everything is configured in $DSH_HOME/settings.yaml:

To add your own vendor, extend llm-pi-ai.providers with { api, baseURL, apiKeyEnv, models } (any OpenAI-compatible api: openai-completions gateway works), then add matching entries to subagent-model-selection.allowedModels and multi-model-orchestrator.modelNotes.

Notes & limitations

• Token usage is per-process (cleared when the harness restarts); per-session usage still shows in the built-in token meter.
• model_token_usage needs at least one model call that returned a usage chunk before it reports anything.
• The orchestration guidance is injected globally, so sub-agents read it too; its wording keeps sub-agents from recursively re-decomposing their single focused subtask.
• Third-party routes are registered as non-reasoning models by default; reasoning flags (reasoningEfforts / compat.thinkingFormat) can be added per model on the web Models page.
• Some reasoning models reject a call without an explicit reasoning tier. Zhipu's GLM-5.3 line, for example, fails with default parameters and only responds when the sub-agent is dispatched with an explicit reasoning_effort (e.g. low). If a dispatched child errors on a model you expect to work, add an explicit reasoning_effort to the dispatch. A route that returns no tokens despite requests succeeding at the API level is usually a provider-side issue, not an orchestrator bug.

Troubleshooting

• ERR_MODULE_NOT_FOUND: Cannot find package '@deepseek-ai/...' — the plugin was linked manually (raw link: + a hand-made node_modules junction) instead of installed through dsh plugin ... add. Install it as a real dependency of the profile and remove the junction.
• model_token_usage returns "No records yet" — no model call has completed with usage since load; make a request and ask again.
• Children cannot be dispatched to a route / list_subagent_models is empty — check that subagent-model-selection is enabled: true with a non-empty allowedModels, and that the routes exist under llm-pi-ai.providers.

License

MIT