How to Build a Document Pipeline With Kimi K2.5
The tempting architecture with a long-context model is “send everything, ask one question.” It works in a demo and breaks in production, because cost scales with every document on every request and mid-document accuracy is the weakest point of the whole approach.
A workable pipeline uses long context selectively. Here is how to structure it.
Long Context or Retrieval — Which Comes First?
Use retrieval to find the right documents, then long context to reason across them. Not the other way round.
| Scenario | Approach | Why |
|---|---|---|
| One 200-page doc, many questions | Full context per question | Cheap enough, avoids chunk-boundary bugs |
| 10,000 docs, one question | Retrieval, then full context on top 5 | Sending all is impossible |
| Contract review, clause lookup | Retrieval by section, then reason | Sections are self-contained |
| Cross-document synthesis | Retrieve 5-10, full context | Relationships span documents |
The hybrid pattern — retrieve ten candidates, then hand them all to Kimi K2.5 at once — captures most of the cross-document reasoning without paying for the full corpus on every call.
How Should You Structure the Output?
Never accept free-form prose from a pipeline. Define a schema and require it: a list of findings, each with a claim, a source document identifier, a quoted span, and a confidence value.
The quoted span is the important field. It gives you an automated verification step — check that the quote actually appears in the source — and that check turns hallucination from an invisible risk into a measurable rate.
What Validation Belongs in the Pipeline?
- Quote verification — confirm each quoted span exists in the retrieved source. Reject findings that fail.
- Coverage check — did the model address every section you asked about, or silently skip one?
- Count sanity — if the prompt asked for all twelve clauses, verify twelve came back.
- Confidence floor — route low-confidence findings to human review instead of publishing them.
An agent that quietly skips a section is more dangerous than one that fails loudly, because the output looks complete. The coverage check is what catches it.
How Do You Control Cost?
Three levers, in order of impact. Cache repeated instructions and schema so only the document text varies. Rerun only failed findings rather than the whole document. And strip boilerplate before upload — repeating headers and footers across hundreds of pages bill like real content.
Per-document cost also drops sharply when you retrieve instead of sending whole files. Measure cost per document processed, not cost per request, or you will miss the documents that dominate your bill.
Where Does This Approach Struggle?
Scanned documents with poor OCR. Tables split across page breaks. Anything requiring arithmetic across multiple documents, where the model has to sum figures correctly — add a deterministic calculation step rather than trusting the model’s mental math. And very long outputs remain a quality risk; ask for findings per section rather than one giant answer.
Quick Summary
- Retrieve first, then use long context on the shortlist.
- Require structured findings with a quoted source span.
- Verify quotes automatically — it makes hallucination measurable.
- Check coverage so skipped sections do not slip through.
- Cache instructions and strip boilerplate to control cost.
