MosaicCompress

Stateless dialogue compression based on natural forgetting curve.

LLM conversations grow linearly. MosaicCompress keeps them bounded — automatically, invisibly, and without the user ever knowing what a "Session" is.

How It Works

Your message array (R rounds, oldest → newest):

Round 1 ────→ Round (R-50)   │ Heavy zone → ALL → 2 msgs
Round (R-49) → Round (R-30)  │ Light zone → structural truncation, count unchanged
Round (R-29) ──→ Round R     │ Raw zone  → keep as-is

Steady state: constant message count — 2 + heavyStart × (messages per round), e.g. 102 messages for pure two-message rounds, whether at round 60 or round 15,000 (higher, but still constant, when tool-call rounds add messages). The compression ratio approaches 100%.

Philosophy: Alive Memory, Not a Handover Brief

The industry-standard answer to unbounded conversations is threshold summarization: when the window fills up, summarize everything into one brief and hand it to a fresh model. The conversation looks like it continues. But structurally it is amnesia followed by reading a diary:

• A switch moment. Memory breaks, then is rebuilt from a single summary call.
• Indiscriminate loss. The freshest instructions are paraphrased too — the exact part that must stay vivid. In our own A/B experiment the brief paraphrased the user's latest instruction and silently dropped an action item ("write the key points into MEMORY").
• Invisible loss. The next model cannot know what the brief omitted, so it cannot compensate.

MosaicCompress models the opposite: biological forgetting. A human does not remember round 3 of a 300-round conversation — they keep the lesson, the rules, the relationship. The algorithm reproduces that curve inside one message array:

recent 30 rounds   → verbatim (vivid — what you are actually working on)
rounds 30–50       → structural truncation (reasoning/args/results trimmed, text kept)
rounds 50+         → one heavy pair: identity, environment, permissions, rules

No switch moment, no reset, no length limit. The heavy zone is semantic memory (rules that must never be forgotten); the middle is recent episodic memory; the raw zone is the vivid present. Loss is visible: the zone structure tells the model what it no longer knows, so it can fetch detail from shadowed storage on demand.

The two philosophies complement each other: a handover brief serves cold starts and long pauses; MosaicCompress serves staying in the conversation. Combined with a durable host-side store (e.g. a MEMORY.md file), human and AI keep talking under the same forgetting curve indefinitely. See docs/design.md §8/§10 for the formal position-is-age model behind this design.

Quick Start

npm install mosaic-compress

import { mosaicCompress, type MosaicConfig } from 'mosaic-compress';

const config: MosaicConfig = {
  lightStart: 30,    // keep 30 most recent rounds raw
  lightWindow: 10,   // compress every 10 rounds
  heavyStart: 50,    // rounds before this get heavy compression
  heavyWindow: 10,   // same cadence as light
  callLLM: async (systemPrompt, userInput) => {
    // Wire to OpenAI, Anthropic, or any LLM provider
    const res = await openai.chat.completions.create({
      model: 'gpt-4o-mini',
      messages: [
        { role: 'system', content: systemPrompt },
        { role: 'user', content: userInput },
      ],
    });
    return res.choices[0].message.content ?? '';
  },
};

// Call every turn — zero cost below threshold; structural light is millisecond-fast,
// Heavy folds take ~1-2s (one LLM summary call)
const compressed = await mosaicCompress(messages, config);

Features

• Stateless & repeatable — no session state; call it every turn, and the output can be fed back in as input
• Zero-cost below threshold — returns immediately if no compression is due
• Anti-jitter — compression only at configurable window boundaries
• LLM-agnostic — bring your own callLLM function for Heavy (OpenAI, Anthropic, local models…); light runs zero-LLM
• Tool-call safe — tool messages don't break round counting
• Graceful degradation — LLM failures don't block the conversation

API

mosaicCompress(messages, config)

MosaicConfig

DEFAULT_CONFIG

Prefer starting from the exported defaults and overriding only what you need:

import { mosaicCompress, DEFAULT_CONFIG, type MosaicConfig } from 'mosaic-compress';

const config: MosaicConfig = { ...DEFAULT_CONFIG, callLLM: async (sys, user) => { /* ... */ } };

All numeric fields must be positive integers (windows) / non-negative integers (starts), and heavyStart must be greater than lightStart. Invalid configs throw a TypeError.

Message

interface Message {
  role: 'system' | 'user' | 'assistant' | 'tool';
  content: string;
  tool_call_id?: string;
  tool_calls?: { id: string; type: 'function'; function: { name: string; arguments: string } }[];
}

Design

Read the full design document (English) or 中文设计文档.

Architecture Boundaries

MosaicCompress is intentionally stateless and lossy:

• Durable storage is the host's responsibility. The library compresses the message array in place and never persists original payloads. Hosts that need lossless history must archive the raw messages themselves — through their own code, a database, or the host platform's persistence layer (the onCompress callback hands every compressed-away original to the host for archiving).
• Compression is lossy by design. Like any summarization approach, early details fade progressively. That is the point: the goal is an unbounded conversation, not lossless archival. If exact retrieval of early turns matters, pair this library with a persistence layer and re-read on demand.

Integration Notes

MosaicCompress is host-agnostic and works wherever a callLLM function exists. Its primary integration reference is DeepSeek Harness (DSH) (deepseek-ai/deepseek-harness — everything is a plugin), whose task-level compaction / output retention / spill complement this library's message-level compression (roles and order preserved). A ready-to-use DSH plugin backend lives in dsh-module/ (design docs in EN/中文).

Related:

• DeepSeek Harness — the host platform
• awesome-dsh-plugin — curated DSH plugin list
• awesome-deepseek-harness — DSH ecosystem list
• design docs (EN) / 设计文档（中文） — theory and empirical case study

See the Roadmap for upcoming work.

Benchmark

A deterministic simulation (zero LLM cost, reproducible) runs the real algorithm with a rule-based pseudo-LLM. Latest sweep (default parameters):

npm run bench                        # synthetic sweep: 100 / 500 / 1000 / 5000 rounds
npm run bench -- --file chat.json    # analyze your own conversation file

The file mode accepts any JSON array of messages in the library's Message shape and reports the compression ratio:

[{"role": "system", "content": "..."},
 {"role": "user", "content": "..."},
 {"role": "assistant", "content": "..."}]

See benchmark/README.md for the full method, data generation, findings, limitations, and the real-LLM spot check (npm run bench:real — DeepSeek V4 Flash, <$0.01, 5/5 facts retained).

Development

# Run tests (zero LLM cost — uses mock responses)
npm test

# Type-check the whole project
npm run typecheck

# Or directly:
npx tsx tests/index.test.ts

License

MIT — TuringCorp | iAsk@turingcorp.net