kestrel

_{dsh-deep-research · a DeepSeek Harness plugin and standalone library · zero runtime dependencies}

Quick start · Run the demo · How it works · Tools · Architecture

The problem

Every AI research tool optimizes for plausible prose with URLs attached. That produces four failures no amount of prompt engineering fixes:

Kestrel attacks all four mechanically — with code a language model cannot talk its way past.

The two ideas that matter

1. A fabricated quote fails indexOf

Every citation must carry a verbatim quote. Before it is admitted, Kestrel performs a literal substring check against the stored source text, then projects the match back to exact character offsets.

quote ∈ document ?  ✓ citation admitted, with [charStart, charEnd]
                 ✗  ✗ REJECTED → claim demoted to UNVERIFIED

No model is asked whether the citation is good. Asking the thing that hallucinates to check its own hallucination is circular. String.indexOf is not.

The judge is required to return a quote, and that quote is then verified by code. A dishonest model cannot manufacture support — it can only fail the gate.

2. Corroboration is counted in origins, not documents

MinHash → LSH candidate filtering → lineage DAG → Tarjan SCC condensation → count the roots. Edge direction is fixed by min(publishedAt, waybackFirstSeen), because publishers rewrite their own dates and the archive is the one timestamp they do not control.

Twelve outlets running the same wire copy is one witness, and Kestrel reports it as one.

See it work (60 seconds)

No API key. No network. No configuration.

git clone https://github.com/grloper/dsh-deep-research.git
cd dsh-deep-research
npm run demo

The demo runs the real pipeline over a fixed corpus: one press release, six verbatim syndications, four derivative rewrites, one independent regulatory filing, and one correction that retracts the headline number.

Actual output

1 · A fabricated quote cannot pass

ADMITTED  (genuine) "its Q3 revenue reached 42 million dollars"
          kind=EXACT span=[40,81] · Quote appears verbatim in the source document.

REJECTED  (fabricated) "its Q3 revenue reached 91 million dollars"
          kind=FAILED span=[-1,-1] · Quote does not appear in the source document
          (best token overlap 0.75 < 0.85). The citation is fabricated.

The fabrication differs from the truth by one digit — 42 → 91. Token overlap is 0.75, high enough to fool any similarity threshold a naive tool would pick. The exact-match gate rejects it anyway.

2 · Thirteen "sources" are three witnesses

Raw source count      : 13
Independent origins   : 3
Derivation edges      : 55

13 sources → 3 independent origins (10 derivative/syndicated)

Every source-counting tool reports "13 sources agree." Three actually do.

3 · A contradiction is surfaced, not buried

2 check-worthy claims · 1 supported · 2 fabricated citation(s) blocked (8%)

SUPPORTED      89% confidence  ICS 2 / 12 sources
  Northwind Robotics reported Q3 revenue of 42 million dollars.
  └ [EXACT] "Northwind Robotics announced today that its Q3 revenue reached 42 millio…"

PARTIAL        40% confidence  ICS 1 / 11 sources
  Northwind Robotics grew 18 percent year over year.

Eleven sources "support" the 18% growth claim. One primary filing retracts it. Confidence is capped at 40% by the contradiction rather than inflated by the crowd — the opposite of what majority-vote retrieval does.

4 · The second run does not start from zero

Evidence store backend : sqlite
Claims retained        : 2 across 3 entities

Recall "Northwind revenue" → 2 prior finding(s):
  · SUPPORTED (89%) fresh — Northwind Robotics reported Q3 revenue of 42 mil…
  · PARTIAL  (40%) fresh — Northwind Robotics grew 18 percent year over year.

Quick start

Install into DeepSeek Harness

npm install dsh-deep-research
npm run install:dsh      # links the plugin into your DSH web/dev profiles

Then ask your agent naturally:

"Deep research: did the EU AI Act's foundation-model rules change after the 2024 trilogue?" "Verify this article before I cite it." "Are these six URLs actually independent sources?"

Use it as a library

Every mechanism is an exported, dependency-free module:

import { anchorQuote, analyzeLineage, verifyText } from 'dsh-deep-research'

// Mechanical citation gate
const anchor = anchorQuote('revenue reached 42 million dollars', sourceText)
if (!anchor.ok) throw new Error(anchor.reason)
console.log(anchor.charStart, anchor.charEnd)   // exact, replayable offsets

// Independence analysis
const { ics, total, roots, circular } = analyzeLineage(documents)
console.log(`${total} sources → ${ics} independent origins`)

The five tools

Registered automatically with the host agent.

The six mechanisms

M1 · Mechanical citation anchoring

Three tiers, strictest first: EXACT (byte-identical), NORMALIZED (identical after whitespace/punctuation folding, projected back to original offsets), and FUZZY (token-Jaccard ≥ 0.85, reported distinctly so callers can be strict). A quote shorter than 24 characters is refused outright, because short strings match by coincidence.

Documents are SHA-256 hashed at fetch time. If stored text no longer matches its hash, every citation resting on it is rejected as an integrity failure.

M2 · Independent Corroboration Score

3-gram shingles → 128-permutation MinHash → LSH banding → pairwise Jaccard. Above 0.85 a document is a syndication; above 0.40 with a shared quote of 50+ characters it is a derivation. Tarjan's algorithm condenses cycles, and the DAG roots are counted as the true origins. Circular citation loops are reported explicitly.

M3 · Adversarial tribunal

For every claim, a prosecutor generates refutation-seeking queries ("<claim>" debunked OR false OR retracted) while a defender seeks support. Both sides' evidence passes the same mechanical gate. A claim only reaches SUPPORTED when the defense survives the prosecution — and an active contradiction hard-caps confidence regardless of how many sources agree.

M4 · Date resolution

JSON-LD, Open Graph, meta tags, HTTP headers, and URL path segments are cross-checked. Disagreement between a page's self-reported date and its archival first-sighting is surfaced as a warning, defeating retro-dated edits.

M5 · Compounding evidence graph

Verified claims persist in a local store (node:sqlite when available, transparent JSON fallback otherwise) with BM25 + entity-index + RRF hybrid retrieval. Freshness is per-claim by volatility class — a mathematical constant and a stock price must not expire on the same schedule.

M6 · Deterministic fallback judge — the engine never silently no-ops

Every stage above is gated on a judge returning a verbatim quote that is then located mechanically in the source. That gate is correct. The danger is what happens when no language model is available.

A judge that returns NEUTRAL for everything is not graceful degradation — the tribunal only admits SUPPORTED/PARTIAL/CONTRADICTED evidence, so such a stub discards 100% of retrieved documents. The loop then spends its whole round budget, finds nothing by construction, and reports "0 resolved". To a user that is indistinguishable from activating deep research and having nothing happen.

So the LLM-free path is a real judge, not a stub:

1. IDF-weighted sentence selection over the document's own sentences, so a term appearing in every sentence (usually the page topic) carries little weight while a rare, specific term carries a lot. A page that merely mentions the subject does not score as one that addresses the assertion.
2. Normalised figure matching — 8, 8% and 8.0 compare equal, and an exact numeric agreement sharpens relevance well beyond word overlap.
3. Stance detection weighing negation cues (debunked, failed to replicate, found no) against affirmation cues, so refutation is captured rather than collapsed into support.
4. Structural honesty — the quote is selected from the document, never generated, so this judge is incapable of fabricating a citation. Below a relevance floor it returns an honest NEUTRAL.

Alongside it, a capability preflight reports what is actually wired. A run with no search service returns an explicit "research could not run: no web-search capability" report instead of an empty one — a capability problem is reported as a capability problem, never as an absence of evidence.

Architecture

lib/
├── index.js        Host plugin: tool registration, service adapters, harness RPC + /kestrel/api HTTP bridge
├── client.js       Browser half: Verify action, Deep Research launcher, settings dashboard (HTTP bridge)
├── anchor.js       M1 · mechanical citation anchoring + admission gate
├── lineage.js      M2 · MinHash/LSH/SCC independence analysis
├── tribunal.js     M3 · prosecutor/defender adjudication
├── dates.js        M4 · multi-signal publication-date resolution
├── store.js        M5 · evidence store, volatility classes, freshness
├── graph.js        M5 · BM25 + entity + RRF hybrid recall
├── credibility.js  source typing, credibility and slop signals
├── verify.js       verify_text pipeline
├── judge.js        M6 · zero-LLM lexical entailment judge + capability preflight
└── research.js     deep_research loop, coverage assessment, gap queries

Design constraints, deliberately chosen:

• Zero runtime dependencies. Nothing to audit, nothing to break, no supply chain.
• Every capability is injected. Search, fetch, and LLM arrive through adapters, so the whole engine is testable offline and degrades to heuristics rather than crashing when a host service is missing.
• The browser never adjudicates. A page cannot fetch and anchor sources, so it never renders a verdict it did not receive from the host engine. When the host is unreachable the UI says so instead of guessing.

Verification

npm test        # 262 tests
npm run demo    # end-to-end proof, offline
npm run bench   # throughput and scaling benchmark

The suite covers the anchoring tiers, lineage/SCC condensation, tribunal adjudication, date resolution, credibility scoring, store/freshness behaviour, graph recall, RPC contracts, and the plugin's degradation under missing, partial, and hostile host services.

Notable regression guards:

• A fabricated quote is rejected even when the judge asserts SUPPORTED.
• A NEUTRAL source never becomes a citation — a source that does not address the claim is not evidence for it.
• 200,000 distinct URLs produce zero id collisions, protecting a store designed to accumulate for years.
• Fuzzy anchoring stays linear; the incremental sliding window replaced an O(document × quote) scan and cut a 40k-token match from ~290 ms to ~45 ms.
• With no LLM anywhere, a research run still admits real anchored evidence — guarding the regression where the fallback judge discarded every document and the engine silently produced nothing.
• Every quote the fallback judge emits survives mechanical anchoring, because it is selected from the document rather than generated.
• Status buckets always sum to the sub-question count, so a report can never print 0 resolved · 0 contested · 0 unresolved for work that actually ran.
• A missing search service yields an explicit "could not run" report, never an empty findings list.
• Decomposition never emits a fragment that lost its predicate — Compare Rust and Go for backend services must not become ["Compare Rust", "Go for backend services"].

If a test run needs to avoid per-file process spawning (restricted sandboxes, some CI images), use npm run test:serial.

Configuration

apply(ctx, {
  storePath: '~/.dsh/kestrel/evidence.db',  // ':memory:' for ephemeral
  maxSources: 6,                            // sources gathered per claim
  defaultMode: 'standard',                  // quick | standard | deep | forensic
  localLlm: false,                          // opt-in local sidecar fast path
  localLlmModel: 'qwen2.5-coder',
})

The local-model fast path is off by default and self-disabling: a research tool should not send prompt text to a local port nobody configured, and one failed probe latches it off for the process rather than paying a timeout on every call.

Limitations

Stated plainly, because a verification tool that oversells itself is self-refuting:

• Anchoring proves quotation, not truth. A source can be quoted perfectly and still be wrong. Kestrel reports what the evidence says and how independent it is, not ground truth.
• Entailment quality depends on the host LLM. The mechanical gate makes a fabricated quote impossible; it does not make a bad judgement impossible. Without an LLM the deterministic lexical judge (M6) still admits real anchored evidence, but it reasons over vocabulary overlap rather than meaning: it will miss a paraphrase that shares no salient terms, and every report states which judge produced it.
• Independence detection is textual. Two outlets that independently interview the same source produce different text and will count as two origins.
• Without a search service the engine cannot run at all, and says so explicitly rather than returning an empty report. Source discovery is the one capability with no offline substitute.

The name

A kestrel hunts by hovering — holding station in the air, dead still, until it sees exactly what is there. Then it commits, once.

That is the opposite of how research agents usually behave: grab the first ten results, summarize confidently, attach URLs. This engine is built to hold position over the evidence and only strike when the quote is actually there.

The mark is a kestrel in the hover above a line of source text, with one filled point marking the exact character offset a quote was anchored to.

License

MIT