简体中文

Model Catalog — Model Catalog Auto-Discovery (dsh plugin)

This plugin is designed for the dsh ecosystem (a plugin-based DeepSeek harness where everything is a plugin): once you configure an OpenAI-compatible API host in dsh (official API, relay gateway, local inference service, etc.), the plugin automatically pulls model information from that host's endpoints — model listings, pricing (input/output/cache prices per million tokens), inference parameters (context length, max output, capability flags: tool calling / structured output / vision / parallel tools, etc.) — normalizes it, and generates ready-to-use model configs, saving you the manual work.

Core value:

• Host-type auto-detection, no protocol details to configure;
• End-to-end unit normalization (per-token USD / per-million USD / multipliers / per-call);
• Traceable pricing (captured_at + source endpoint), structured dynamic pricing;
• Optional lightweight live probing for missing capabilities;
• Three outputs: full catalog, dsh config fragment, human-readable report.

Features

• Host type auto-detection: standard compatible (minimal fields) / rich metadata (/models with pricing and parameter lists) / multiplier-priced gateway (quota system) / capability-flag proxy (/model/info capability booleans) / Ollama / vLLM; fails with an error when all endpoint probes fail, and supports manual --kind.
• Normalized model records: id / context window / max output / pricing family (input, output, cache_read, cache_write, internal_reasoning, in USD/1M) / capability flags / source provenance.
• Pricing source chain: host endpoint → user override config → external price mirror → built-in default table → marked unknown (with warning).
• Built-in default table: model facts and pricing fallback for the official API (DeepSeek); dynamic pricing (peak/off-peak billing from 2026-08-16) expressed as structured tiers.
• Capability probe verification (switchable): sends minimal requests to live-test capabilities missing from metadata (tool calling / structured output / streaming), results cached per (baseUrl, model).
• Cache and concurrency safety: classification/probe/mirror results cached with TTL; file locking + atomic writes prevent concurrent processes from corrupting each other; corrupted caches auto-recover.
• Interactive config generation: the pick command lists models and prices; select one to generate a dsh config fragment.
• Self-contained: manifest + entry factory + tool/event interfaces, directly loadable by the harness (see integration docs).

How It Works

baseUrl + apiKey
   │
   ▼
① Host detection (probe endpoints, result cached 1h)
   ├─ /models rich metadata     → augmented
   ├─ /v1/models                → ├─ /api/pricing → quota
   │                              ├─ /version     → vllm
   │                              ├─ /model/info  → flag
   │                              └─ otherwise    → bare
   ├─ /api/tags                 → ollama
   └─ all failed                → unknown (error; override with --kind)
   │
   ▼
② Fetch (parse model listings/pricing/capabilities by type)
   │
   ▼
③ Normalize (unit conversion + source-chain completion + alias resolution)
   │
   ▼
④ Capability probing (optional; only fills gaps missing from metadata)
   │
   ▼
⑤ Output
   ├─ out/catalog.json     full catalog (schema: model-catalog/v1)
   ├─ out/dsh-models.json  dsh config fragment (schema: dsh/models/v1)
   └─ out/report.md        human-readable report

Installing in DSH

dsh plugin --profile demo add github:JohnXu22786/model-catalog

Remove with:

dsh plugin --profile demo remove model-catalog

Quick Start

Requirements: Node.js ≥ 21 (no runtime dependencies; only TypeScript needed to build).

npm install          # install dev dependencies
npm run build        # compile to dist/

# Discover the DeepSeek official API model catalog
export DEEPSEEK_API_KEY=sk-xxx
node dist/src/main.js discover --base-url https://api.deepseek.com

# Discover a local Ollama
node dist/src/main.js discover --base-url http://127.0.0.1:11434 --probe always

# Relay gateway (multiplier pricing)
node dist/src/main.js discover --base-url https://gateway.example.com --api-key-env GATEWAY_KEY

# Interactively pick models and generate a dsh config fragment (answers can be piped in one shot:
#   line 1 = baseUrl, line 2 = key env var name (blank = auto-detect), line 3 = model number)
node dist/src/main.js pick --base-url https://api.deepseek.com

After running, three artifacts land in the out/ directory. Hand dsh-models.json to the dsh harness to complete model configuration (field docs in integration docs).

Host Types

Normalization and Unit Conversion

All per-token prices in the catalog are normalized to USD per million tokens (USD/1M); per-call prices are marked separately:

augmented pricing field mapping: prompt → input, completion → output, input_cache_read → cacheRead, input_cache_write → cacheWrite, internal_reasoning → internalReasoning.

Pricing Source Priority

1. Host endpoint (full trust: if the host provides some prices, no other sources are consulted)
2. User override config data/overrides.json (replacement semantics: override only the fields configured)
3. External price mirror --external-url (only fills fully-missing pricing; result cached 1h)
4. Built-in default table data/builtin-table.json (only fills gaps; includes DeepSeek official model facts)
5. None of the above → pricing: null, marked "unknown" in the report with a warning

Capability fields, context and max output follow the same "host > override > mirror > built-in table" priority (override is replacement; the rest only fill gaps).

Dynamic Pricing

The DeepSeek official API has billed by peak/off-peak time-of-day since 2026-08-16 (peak: 01:00–04:00 and 06:00–10:00 UTC; half price at other times). In the catalog:

• pricing.dynamic: true;
• pricing.amounts holds the baseline tier (first off-peak tier) prices;
• pricing.tiers carries all tiers (label, UTC window, per-tier prices);
• the report (report.md) lists dynamic-priced models separately with a note that billing is per-tier;
• the built-in default table records the official prices of the day; if they differ from the official docs, follow the docs and update data/builtin-table.json.

Capability Probing

For capabilities missing from metadata (tool calling / structured output / streaming), sends minimal probe requests (max_tokens tiny, message very short) to live-test host support:

• Tool calling: sends tools + forced tool_choice;
• Structured output: response_format: {type: "json_object"} (message contains the word "json" to dodge the classic JSON-mode rejection trap);
• Streaming: stream: true; the response must actually be SSE (has data: events and a [DONE] terminator); a plain JSON body is judged unsupported.

Interpretation: 2xx = supported; 400/404/405/422 = unsupported (error-body summary kept as evidence); 401/403 = abort all probing with a warning (avoid wasting quota); 5xx/network/timeout = stays unknown with a short-lived error cache.

• Modes: auto (default; probes only with a key or a local host) / never / always;
• Results cached per (baseUrl, model, capability) for 24h (errors 30m);
• Capabilities that cannot be cheaply live-tested (vision, parallel tools) rely on metadata only; no probing.

Cache and Concurrency Safety

• Cache directory var/ (vault.json): host classification (1h), probe results (24h/30m), external mirror (1h);
• All writes go through "temp file + atomic rename"; corrupted cache files are reset automatically without affecting operation;
• Cross-process file lock (var/.lock): stale locks (>5 minutes) are taken over automatically; wait timeout (10s) raises an error;
• cache --clear wipes everything in one command.

Configuration

Config file catalog.config.json (in the plugin root; overridable with --config FILE; all fields optional):

{
  "baseUrl": "https://api.deepseek.com",
  "apiKeyEnv": "DEEPSEEK_API_KEY",
  "probe": "auto",
  "externalUrl": "https://example.com/mirror.json",
  "outputDir": "out",
  "cacheDir": "var",
  "httpTimeoutMs": 10000
}

Keys are only passed via environment variables (--api-key is for transient CLI use only); no artifact file ever contains a plaintext key.

Data Files

Outputs

CLI Overview

model-catalog discover [args]          discover and output the model catalog (default command)
model-catalog pick [args]              interactively pick models and generate a dsh config fragment
model-catalog probe --model ID [args]  run capability probing for a single model
model-catalog cache --clear            clear the cache
model-catalog config                   show effective config

--base-url URL      --api-key-env NAME   --api-key KEY
--kind KIND         --probe MODE         --out DIR
--cache DIR         --external-url URL   --config FILE
--model ID          --help

dsh Integration

The plugin is self-contained: manifest.json declares the entry and interfaces, and dist/src/plugin.js exports a createPlugin() factory registering 5 tools (catalog.discover/list/refresh/select/probe) and 2 events (catalog.updated/catalog.failed). It is also installable via the dsh bundle (package.json → dsh.bundle → cordis.patch.yml): the Cordis entry dist/src/dsh.js exports name/inject/apply and registers the same 5 tools on the harness. Harness loading, tool parameters and returns, event payloads, and how config fragments are consumed are all covered in docs/integration.md.

Limitations and Notes

• Capabilities like vision and parallel tools cannot be cheaply live-tested; they come from metadata only (unknown when missing);
• If a multiplier gateway's group_ratio is a map without a default group, it counts as 1 — actual group multipliers can be corrected in the override config;
• Probe requests consume a small number of tokens (at most 3 requests per model, 1–16 tokens each); keyless standard hosts are not probed in auto mode;
• Built-in default table prices are fallbacks; for dynamic pricing, follow the official docs and the tiers.

Development

npm run build    # compile TypeScript
npm test         # compile + run all tests (node:test, no external test deps)

Test coverage: unit conversion, host detection, per-type fetching, normalization priority chain, probe interpretation and caching, cache store (TTL/locking/corruption recovery), output artifacts, and an end-to-end pipeline (local mock host).

License

MIT — see LICENSE. Copyright (c) 2026 JohnXu22786.