dsh-mineru

Repository: https://github.com/Lee-Hilex/dsh-mineru · npm: dsh-mineru · Changelog

English | 简体中文

dsh-mineru is a MinerU-powered multimodal document parsing plugin for DeepSeek Harness (dsh): turn PDF, Word, PPT, Excel, HTML, and images into structured Markdown. Configure a MinerU token to use the 🎯 Precision API, or leave it blank and use the ⚡ Agent API (tokenless, IP rate-limited).

⚠️ Data leaves this machine: every parse uploads your file to MinerU's remote service — there is no fully offline mode. Do not use it for confidential / case material that must stay on-premises. See Security model.

Contents

• Overview · Quick start · Two API modes · Tools · Results & artifacts · Configuration · Usage tips · Web UI · Rate limits & reliability · Error code reference · Security model · FAQ · Development & publishing · Contributing · License

Overview

What is this?

dsh-mineru plugs MinerU (OpenDataLab's high-precision document parsing engine) into the DeepSeek Harness agent ecosystem, letting text-only models "read" documents: the model receives structured Markdown (tables, formulas, and images preserved) instead of binary files it cannot open.

Typical use cases:

• Read papers / contracts / reports: parse a PDF, then ask the agent to summarize or extract data;
• Batch-processing documents: parse dozens of files in one call, each producing Markdown;
• OCR for images / screenshots: scanned pages and web screenshots become searchable text;
• Formulas & tables: math formulas in papers and Excel tables survive with full structure.

Supported file formats

Note: the Agent API does not support .doc / .ppt / .xls (legacy Office formats) or HTML — use the Precision API (token required) for those.

Highlights

• Dual API modes, auto-selected — chooses Precision or Agent parsing based on whether a token is configured; can be overridden per call;
• Progressive tool exposure — only the tiny mineru_activate bootstrap is visible until the agent activates the full toolset, saving context;
• Drag-and-drop uploads — drop a file into the chat; its path lands in the composer draft so text-only models can process documents;
• Auto-chunked batch parsing — submit hundreds of files/URLs at once; batching respects official limits automatically;
• Rate-limit aware — built-in token buckets, daily caps, 429 backoff, and actionable messages for every error code;
• Artifact-backed results — parsed output lands in the workspace as files with signed preview links in the Web UI.

Quick start

Step 1: Install

Note: Requires an existing DeepSeek Harness installation (0.1.0-rc.6 or a compatible 0.1.x) with the dsh CLI available.

From npm (recommended)

dsh plugin --profile web add dsh-mineru        # Web profile
dsh plugin --profile headless add dsh-mineru   # Headless profile

Build from source (development / preview latest changes)

git clone git@github.com:Lee-Hilex/dsh-mineru.git
cd dsh-mineru
npm install        # installs peer deps (no build step for this package)
npm pack           # produces dsh-mineru-<version>.tgz
dsh plugin --profile web add ./dsh-mineru-<version>.tgz

Run npm pack again and re-install after changing the source. Local installs must use the tarball (pnpm link: installs resolve @deepseek-ai peers outside the profile).

Step 2: Verify the install

dsh --profile web --dump-config | grep -A 2 "id: mineru"

Expected output (a # == comment line plus your plugin row):

# == dsh-mineru
  - id: mineru
    name: dsh-mineru

Then restart a running Web profile (dsh web) and open Settings → Plugins → MinerU 解析.

Step 3: Configure (optional)

No token needed to get started — the plugin automatically falls back to the tokenless Agent API. Add a token to unlock the full feature set (Precision API, batch parsing, HTML, more formats):

• Web UI: Settings → Plugins → MinerU 解析 → paste the token into the write-only field (blank keeps the existing value);
• Headless / CLI: put MINERU_API_TOKEN: <token> in $DSH_HOME/.credentials.yaml, or export the MINERU_API_TOKEN environment variable.

The token is resolved once per operation — rotations take effect on the very next call without a restart. Get a token by registering at https://mineru.net (API management page).

Step 4: Use it

Web UI (drag-and-drop friendly):

1. Drag a PDF/Word/image into the chat — the file is saved to the session workspace and its path is filled into the composer draft (never auto-sent);
2. Add your request, e.g. "Parse this PDF to Markdown and summarize the key points";
3. Send. The agent activates the toolset and calls mineru_parse; results come back as files you can preview.

URL parsing: just tell the agent "parse this document: https://example.com/paper.pdf".

Headless smoke test: one-off tasks run on the headless profile:

dsh --profile headless "Parse C:/docs/sample.pdf to Markdown and summarize the first 3 paragraphs"

Headless and Web share the same tool semantics; you can also type /mineru-tools in a conversation to load the usage skill.

Two API modes

Mode selection (mode parameter or config):

• auto (default) — Precision when a token is configured, Agent otherwise;
• precision — force Precision; errors loudly when no token is configured;
• agent — force Agent.

To see which API the current session is actually using: call mineru_activate once — the api field in its result tells you (precision or agent).

Tools

By default the toolset is progressively exposed: only the mineru_activate bootstrap is registered at first. One call unlocks the other three tools, and mineru_activate then hides itself for that session.

mineru_activate — activation bootstrap

mineru_parse — parse a single document

Parses a workspace file path or http(s) URL into structured Markdown.

Result highlights: ok, the api/modelVersion used, taskId, duration, runDir, a truncated Markdown preview, the artifact list (with signed preview links on Web), and — with anchor: true — an anchor summary naming the document.md that was written.

Page-anchored Markdown (anchor)

full.md is prose: nothing in it says which page a sentence came from, so a claim drawn from a scan cannot be cited back to a page. anchor: true adds a document.md that carries that information as an HTML comment per block:

<!-- p12 b3 -->
君子务本，本立而道生。

<!-- p12 b4 -->
![image](images/1efeb151….jpg)

<!-- p13 b1 -->
**表一 孔门弟子年表**
<table><tr><td>姓名</td><td>字</td><td>生年</td></tr>…</table>

• pN is the physical page position in this file (1-based, cover and front matter included) — not the page number printed on the page. The cloud renumbers the pages you asked for, so with pageRanges: "2,4-6" the anchors are p2, p4, p5, p6 — the mapping is requestedPages[page_idx], not a fixed offset;
• bK is the K-th block of that page in content_list.json order. Page headers, footers and page numbers stay out of the body but still occupy their number, so bK can have gaps (e.g. b1, b2, b4) — that is deliberate: bK always names the same record in content_list.json;
• text blocks render their text, a cloud heading level becomes the matching Markdown heading, tables render as their caption plus table_body, images as ![](<img_path>);
• a block the cloud marked [Unreadable] keeps its anchor and shows [Unreadable]; a block with empty text keeps its anchor too;
• the precision API is the only mode that returns content_list.json, so anchor: true under the Agent API returns a warning and no document.md instead of failing.

The number printed on a page is a printed page number, and it need not match the physical order: front matter numbered with Roman numerals, or a document that keeps the page numbers of the journal it came from, shifts every later page. When the result carries readable page_number blocks the renderer compares them with the physical order on a best-effort basis, records each page's printed − physical difference in the pageNumbers field of the anchor summary ({ detected, total, offsets }, offsets sorted and de-duplicated), and reports a single constant non-zero difference as one warning line (messages are Chinese, like the rest of the renderer's warnings):

anchor.pageNumbers = { "detected": 4, "total": 4, "offsets": [3] }
anchor.warnings    = ["印刷页码与物理页序整体相差 +3（前言用罗马数字，或用析出文献保留原刊页码，都会造成整体偏移）；锚点仍用物理页序，可用页码映射声明两者的对应关系"]

Several different differences are reported as such and need checking page by page. Nothing is reported when the two agree (every offset 0) or when no printed number could be read. The check only reads the blocks: the anchors are never changed by it.

Without pageRanges the pages are numbered from 1 in request order. With a negative range ("2--2") the cloud does not tell us the document length, so the anchors fall back to the first page you asked for and the result carries a warning telling you to check them.

Long PDFs — automatic chunking: when one precision request would ask for more than 200 pages (a workspace PDF larger than that, or a pageRanges selection over 200 pages), the plugin splits it into sequential page_ranges chunks itself, submits and polls them one by one, and merges the outcome into the layout a single request produces. No extra parameter, and no local PDF rewrite: the file is uploaded once, only the requested range changes per request.

• full.md — concatenated in page order, with one <!-- chunk i/n: pages … --> line before each chunk;
• content_list.json — merged, with page_idx mapped back to the original document (the API renumbers it from 0 for every requested subset);
• images/ — the union of all chunks (names are content hashes, so they cannot collide);
• layout.json / <uuid>_model.json — kept per chunk under chunks/chunk-<n>/, because they are page-indexed arrays that a merge would break;
• run.json — per-chunk page range, batchId, state and duration; the tool result carries the same list as chunkCount / chunks.

Boundaries: splitting needs the page count locally, so it covers workspace PDFs in precision mode only. URL sources, non-PDF formats (the host cannot read docx/pptx page counts) and the tokenless Agent API keep their previous behaviour, and their page-limit error names the reason no split happened. The Agent API is deliberately never split: it is rate-limited per caller IP, so fanning one call out into a dozen requests would exhaust the caller's quota — use Precision, or segment it yourself. timeoutMs applies to each chunk. Only a merged multi-chunk result renumbers pages; a single request (including one with an explicit pageRanges) keeps the cloud's own page_idx, exactly as before.

mineru_batch_parse — batch parsing (Precision only)

Parses many documents in one call; local paths and URLs can be mixed, submissions are auto-chunked within official limits, and per-item success/failure is reported.

Chunking rules: local files ≤ 50 per batch, URLs ≤ 200 per batch, ≤ 1000 total per call; exceeding these errors with a hint to split the call.

Result highlights: success/failure counts, batchIds, per-item results (each success includes its full.md directory), and error messages for failures.

mineru_task — task query & result collection

A timed-out parse is never lost: use the taskId + api returned by mineru_parse to query progress and collect results later — no resubmission needed.

Typical scenario: mineru_parse timed out on a large PDF → tell the agent "collect the result for taskId=xxx with mineru_task".

mineru-tools skill

The plugin ships a skill of the same name. Type /mineru-tools in a conversation or ask the agent to load it via the skill tool: it contains the full agent-facing manual — tool list, dual-API comparison, usage notes, and official limits.

Results & artifacts

All results land in <workspace>/.dsh-mineru/artifacts/<run>/ (<workspace> is the session workspace, <run> a fresh directory per parse):

.dsh-mineru/artifacts/<run>/
├── full.md                  # Structured Markdown (both modes)
├── run.json                 # Run metadata (source, API, model, duration, …)
├── document.md              # Page-anchored Markdown (Precision + anchor: true only)
├── *_content_list.json      # Structured content list (Precision only)
├── layout.json              # Layout data (Precision only)
├── *_model.json             # Raw model output (Precision only)
├── *_origin.pdf             # Server-side original file (Precision only)
├── images/                  # Embedded images (Precision only)
└── result.zip               # Raw server result package (Precision only)

• Tool results carry a bounded Markdown preview (first 12 KB by default); read the full text from full.md with the read tool;
• document.md is opt-in (anchor: true) and purely additional: full.md, content_list.json and layout.json keep exactly the shape they have today, so nothing changes for users who do not pass anchor;
• Web UI: artifacts get HMAC-signed preview URLs (24 h lifetime by default) — click to view/download; tool-result cards open files directly;
• Headless: use the absolute paths returned by the tool.

Batch parses keep each document in its own subdirectory: <run>/<file-name>/full.md.

Configuration

Configuration resolves in three layers, later wins: schema defaults ← composition layer (the mineru row config in a profile patch) ← user settings (written through the Web Settings card).

# Composition-layer example (cordis.patch.yml or a profile patch)
- id: mineru
  config:
    mode: auto               # auto | precision | agent
    modelVersion: vlm        # pipeline | vlm | MinerU-HTML
    language: ch             # document language pack
    enableTable: true        # table recognition
    enableFormula: true      # formula recognition
    isOcr: false             # force OCR
    extraFormats: []         # docx | html | latex (Precision only)
    exposeMode: progressive  # progressive: bootstrap activation; always: register everything globally
    timeoutMs: 600000        # whole-operation timeout (ms)
    pollIntervalMs: 3000     # result poll interval (ms)
    inlineMarkdownBytes: 12000  # inline preview bytes in tool results
    submitRatePerMin: 40     # submission rate cap (per minute)
    pollRatePerMin: 900      # query rate cap (per minute)
    dailySubmitLimit: 5000   # daily submission cap
    artifactRootName: .dsh-mineru  # artifact root directory name

Full option reference

Token (DSH Credentials)

The configuration stores only the credential reference name (default MINERU_API_TOKEN); the token value lives in the credentials layer. Any of three ways:

1. Web Settings card: the token field is write-only — blank keeps the existing value, 清除 removes it;
2. Headless / CLI: put MINERU_API_TOKEN: <token> in $DSH_HOME/.credentials.yaml;
3. Environment variable: export MINERU_API_TOKEN.

The token is resolved once per operation — a rotation applies to the very next call without a restart. Logs, errors, and Settings responses never contain the secret.

language values

Usage tips

• Drag-and-drop: drop a file into the chat — it is saved to the session workspace and its path is filled into the composer draft (never auto-sent); add your request and send. This bypasses the native image-attachment channel that text-only models reject;
• URL caveat: MinerU fetches the URL server-side — it cannot reach blocked sites (github.com, AWS, etc.). Download such documents locally first and pass a path;
• Page ranges: Precision uses pageRanges ("2,4-6"; "2--2" = 2nd from last); Agent uses pageRange ("1-10"). A workspace PDF that would exceed 200 pages per request is split automatically;
• Long documents: in Precision mode a workspace PDF over 200 pages is split automatically — no manual pageRanges juggling; you can still pass pageRanges to parse only a part. The vlm model handles formulas and complex layouts best;
• Timeout recovery: a timeout is not a failure — the task keeps running server-side; collect it later with mineru_task + taskId;
• HTML documents: Precision only, and the MinerU-HTML model is forced automatically;
• HTML daily quota: HTML submissions have a separate official cap (max 100/day).

Web UI

• Settings → Plugins → MinerU 解析: write-only token, mode, model, language, table/formula/OCR switches, extra formats, plus a collapsible Advanced section (timeouts/polling/rate limits, API base URL, credential reference, artifact root name). Every action shows immediate feedback (busy states and result messages); saves are revision-fenced (409 conflict → reload and retry);
• Test buttons: 测试 Token submits the official one-page demo PDF to verify the key (≈ 1 page of quota); 测试 Agent API verifies the tokenless channel (IP rate-limited);
• Tool cards: results of the three parsing tools render as dedicated cards — mode badge, duration, task id, artifact chips (open / preview), bounded preview, per-item batch outcomes.

Rate limits & reliability

Official MinerU limits (the plugin enforces local equivalents and backoff; requests beyond the limits are rejected with HTTP 429):

Plugin behavior:

• Built-in token buckets (separate for submit/query) — local requests queue instead of hammering the API;
• HTTP 429 honors Retry-After with automatic backoff and retry;
• Polling carries random jitter (pollJitterMs) so requests do not line up;
• On whole-operation timeout (timeoutMs) the task is not lost — collect it later with mineru_task + taskId.

Error code reference

Every error is mapped to an actionable message instead of a raw code.

Security model

Data flow (where your content goes)

This plugin is not offline. Both API modes send the full source file to a remote MinerU service, then download the parsed result — nothing is parsed on this machine.

There is no "local-only" mode: MinerU runs server-side. For material that cannot leave the machine, use a fully local toolchain (local PDF conversion + offline OCR + local template rendering).

• Path isolation: input paths resolve against the session workspace; files outside it cannot be read;
• ZIP unpacking guards: rejects absolute paths, .. traversal, encrypted entries, and ZIP64, with entry-count and byte caps, and verifies CRC32 on every extracted entry;
• Artifact boundary: written only under <workspace>/.dsh-mineru; preview URLs are HMAC-signed with expiry and re-validated on every read;
• Web routes: same-origin only; HTML/SVG artifacts are served as attachments with nosniff;
• Cancellable: all network work honors the tool-call abort signal — cancel/timeout stops uploads, polling, and downloads;
• Secrets never leave the vault: the token lives only in the credentials layer; logs/errors/Settings responses never carry it.

FAQ

Q: Can I use it without a token? Yes. It falls back to the tokenless Agent API (≤ 10 MB, ≤ 20 pages, Markdown only) — fine for quick reads.

Q: Why is only mineru_activate in my tool list? That is progressive exposure by design. Call mineru_activate once and the other three tools unlock; or set exposeMode: always to register everything globally.

Q: My parse timed out — is the file lost? No. The task keeps running on MinerU's servers; use mineru_task (taskId + api) anytime to query progress and collect the result without resubmitting.

Q: Why didn't the parse start after I dropped a file? Drag-and-drop only fills the path into the composer draft (to avoid accidental sends). Add your request and press send; the agent then starts parsing.

Q: A0202 / A0211 errors? The token is invalid or expired. Re-enter it in Settings → Plugins → MinerU 解析 (a blank save keeps the old value).

Q: Can I parse HTML pages? Yes, but Precision API only (token required), and the MinerU-HTML model is forced. Note the server cannot reach blocked URLs.

Q: How do I batch-parse? Use mineru_batch_parse with a list of paths/URLs — auto-chunked (local ≤ 50/batch, URLs ≤ 200/batch, ≤ 1000 per call).

Q: What's the difference between the Agent and Precision results? Agent produces Markdown only. Precision additionally yields content_list.json, layout.json, model.json, embedded images, optional docx/html/latex exports, and table/formula recognition controls.

Development & publishing

No build step: lib/ ships loadable ESM and lib/client.js is the packed browser bundle.

node --check lib/*.js      # syntax check
npm pack                   # pack a local tarball

For a new release, follow the checklist in RELEASE.md (bump version → update CHANGELOG and both READMEs → pack & verify → tag & push → npm publish); see CONTRIBUTING.md for the full development loop.

Contributing

Issues and pull requests are welcome — see CONTRIBUTING.md. Please never paste a MinerU token into an issue.

License

MIT. MinerU is provided by OpenDataLab; its official docs (API, rate limits) remain authoritative.