How to Run Unstructured at Scale for Contract Review Pipelines

November 1, 2026

Running one PDF through Unstructured takes four lines of Python. Running fifty thousand contracts through it takes a budget, a failure queue and a decision about which files never touch the parser at all.

This guide is for teams past the demo stage: you already know partition works, and now the pipeline has to survive a month of real documents.

Why does local parsing collapse at volume?

Because hi_res costs seconds, not milliseconds. A 40-page contract on a 4-core box takes 20-60 seconds with the high-resolution layout model loaded. At 50,000 documents that is 300 to 800 machine-hours before you count OCR retries.

Memory is the second wall. The detectron2 layout model loads about 1 GB of weights per worker process. Eight parallel workers on a laptop is not a config tweak, it is a different machine.

So the first architectural decision is not code. It is: which documents earn hi_res?

How do you decide which files get the expensive path?

Route by source. In a typical contract review queue, most files arrive from two places: a document management system that exports clean digital PDFs, and email attachments that are scans of signed paper.

The first group parses fine with fast. The second needs OCR. Two passes, different strategies, one flag per document.

from unstructured.partition.pdf import partition_pdf

try:
    els = partition_pdf(filename=f, strategy="fast")
    text = " ".join(str(e) for e in els)
    if len(text.strip()) < 200:          # near-empty: likely a scan
        raise ValueError("empty")
except Exception:
    els = partition_pdf(filename=f, strategy="hi_res", infer_table_structure=True)

That fallback alone cuts compute by 60-70% on mixed corpora. Measure it on your own sample before quoting a number to anyone.

How do you keep tables usable for retrieval?

infer_table_structure=True is what turns a table element into an HTML table in element.metadata.text_as_html. Without it you get pipe-separated text that a retriever cannot reason about.

Store that HTML separately, not inside the chunk. A fee schedule rendered as a chunk gets truncated by your embedding window; a table stored as its own document with a link back to the clause it sits under survives retrieval.

The pattern that works: chunk narrative text, but keep tables intact as standalone records tagged with document_id, page_number and parent_section. Let the retriever fetch the whole table when the query looks tabular.

What does the per-page cost look like in practice?

Hosted parsing is commonly quoted near $0.003 per page for standard work, with high-resolution OCR costing more. A 40-page contract at that rate is about $0.12. Ten thousand contracts a month is roughly $1,200.

That is not absurd for legal work where a missed indemnity clause costs more. It is absurd if you are parsing invoices for a side project.

The comparison to local: a $200/month cloud VM running eight workers covers maybe 15,000 fast-parsed pages a day for free. The break-even is around the point where your engineers spend more than a day a month babysitting OCR failures.

How do you handle the files that fail?

They will not throw exceptions. That is the trap. Unstructured happily returns a two-element list for a corrupted scan: a Header and a single NarrativeText line of garbage.

Build a quality gate before indexing:

  • Reject documents with fewer than 100 characters extracted per page on average.
  • Flag documents where more than 30% of elements are UncategorizedText.
  • Log page counts from metadata.page_number; a 40-page PDF returning 3 pages means the parse died silently.
  • Sample 20 parsed documents a week and read them. Automated checks miss rotated scans and stamp overlays.

Route failures to a human queue rather than dropping them. Contract review pipelines that silently lose 4% of documents are worse than slow ones.

Who should not run this at scale

Teams without a document engineering function. If nobody owns the parse quality metric, the pipeline degrades quietly and retrieval blames the model.

Also, if your contracts are all born-digital DOCX from a modern CLM system, the structure is already there. Pull the paragraphs from the DOCX XML and skip parsing entirely.

Compare the two routes on the Unstructured tool page. For the retrieval side, Crawl4AI handles the equivalent job when your corpus is web pages instead of paper.

Quick Summary

  • Route clean digital PDFs to fast and only fall back to hi_res when extraction comes back near-empty.
  • infer_table_structure=True converts tables to HTML; store them as standalone records, not inside chunks.
  • Hosted parsing runs about $0.003 per page, roughly $0.12 for a 40-page contract.
  • Failed parses rarely raise errors — gate on characters per page and UncategorizedText ratio.
  • Skip parsing entirely for native DOCX sourced from a CLM system.