How to Build a Document Q&A Pipeline with Reducto
A document Q&A system needs three things that fail independently: clean extraction, chunks that respect document structure, and retrieval that returns the right passage with a page reference. Reducto handles the first and shapes the second. The third is your code.
Why does parsing quality decide the whole system?
Because retrieval can only find what extraction preserved. If a table is flattened into a run-on sentence, no embedding model will retrieve it for the question “what was Q3 revenue”. Garbage chunks produce confidently wrong answers, which are worse than no answer.
That is the argument for paying for parsing on messy documents. A 5% extraction improvement often moves answer accuracy by 20% or more, because failures cluster in exactly the documents people ask about.
What does the pipeline look like end to end?
Five stages, each with one job.
- Ingest — Reducto parse with layout-aware chunking, one chunk per logical block.
- Enrich — attach metadata to every chunk: doc id, page number, section heading, document date.
- Embed — one embedding model for the whole corpus. Mixing models silently breaks similarity.
- Retrieve — top-k with a filter on document type or date, then rerank the top 20 down to 5.
- Generate — answer from the 5 passages only, with page citations, and say so if the passages do not contain the answer.
Most failures live in stages 2 and 5. Missing metadata makes citations impossible; a prompt that lets the model answer from general knowledge produces plausible fiction.
How should you chunk a financial report?
By section, not by character count. A 500-token window will cut a table in half and lose the header row, which is what gives the numbers meaning.
Keep the section heading in the chunk text even if it is not part of the extracted block — a chunk that starts with “Q3 2026 Revenue” retrieves far better than one that starts with “12,480”.
Target 300-600 tokens per chunk with a small overlap. Below 200 you lose context; above 800 retrieval precision drops because the passage covers too many ideas.
How do you know if it works?
Build a test set before you build the product. Fifty questions with the correct page and answer, written by the person who owns the documents.
Then measure two numbers separately: retrieval hit rate (did the right passage appear in the top 5) and answer accuracy (was the answer correct given the passages). When accuracy is bad, the split tells you which half to fix.
Run the test set after every chunking change. It takes ten minutes and catches regressions that look like model drift.
What does it cost per 1,000 pages?
Parsing dominates. Reducto charges per page, with schema extraction costing more than plain parsing. Embeddings are cheap by comparison — a million tokens of embedding is cents.
The real cost is re-runs. Bad chunking means re-embedding the whole corpus, which is hours of compute and a full pipeline rerun. Get chunking right on 200 pages before processing 20,000.
A working estimate: a 1,000-page corpus parsed, chunked, embedded and stored fits comfortably in a project budget; the monthly recurring cost is mostly generation tokens and reranking.
What are the honest limitations?
Page citations are only as accurate as the metadata you attach. If a two-column layout merges pages at extraction, the page number will be off by one for every subsequent chunk — check that before shipping.
The system cannot answer questions that require joining information across many documents, and it will try anyway unless the prompt forbids it. Add an explicit “answer only from the passages below” instruction and an “insufficient evidence” path.
Documents that change frequently need incremental re-indexing, which is a different pipeline than the one-time load. Plan for it early; retrofitting is painful.
Who is this for?
Teams with a fixed document corpus people already ask questions about — manuals, policies, contracts, research reports. The value is highest where the alternative is a person keyword-searching a PDF.
Skip it for real-time data, for conversational memory, or if the documents are clean HTML that already has structure. The pipeline is only worth its complexity when extraction is the hard part.
Quick Summary
- Parsing quality caps the accuracy of the whole Q&A system; retrieval cannot recover lost structure.
- Attach doc id, page number, section and date to every chunk or citations are impossible.
- Chunk by section at 300-600 tokens with a small overlap.
- Evaluate retrieval hit rate and answer accuracy separately to know which half is broken.
- Budget a full re-embed if chunking changes — validate on 200 pages first.
Endpoint details and per-page pricing are on the Reducto tool page.
