OCR models: Tesseract, PaddleOCR-VL, or a document VLM?

Viacheslav Dubrov

2026-06-08 · Updated 2026-10-30 · 4 min · Retrieval and Language Systems

Choose an optical character recognition (OCR) system from the document and the required output. Clean printed pages, forms, tables, receipts, research papers, screenshots, and handwriting pose different problems. Decide first whether you need raw text, page layout, structured fields, or answers supported by the document.

For a PDF, first check whether its embedded text and layout are usable. Extract them directly if they meet the task. Test classical OCR such as Tesseract for clean printed scans. Test PaddleOCR-VL 1.6 for self-hosted layout, text, tables, and formulas. Test dots.mocr for structured graphics. A general vision-language model (VLM) is another candidate when the task also requires visual question answering.

Compare candidates on your document types, required fields, data controls, latency, and cost.

Decision table

OCR and document understanding

Traditional OCR extracts characters. Document AI also needs layout and meaning. Table cells, checkboxes, signatures, captions, and footnotes may all contain text, but flattening them into one string destroys their relationships. That loss then breaks field extraction and question answering.

Vision-language models can extract fields and answer questions about diagrams or tables. Their output still needs checks against the page. PaddleOCR-VL 1.6 combines layout analysis with a 0.9B VLM component, while dots.mocr also parses structured graphics. Compare those pipelines with classical OCR on the pages and fields you need; a larger model is not a reason to replace a working extractor.

Model and tool classes

A small reading-order check

Text extraction can recover every word and still lose a table’s row order. This synthetic PDF contains two columns:

The required associations are Cable → 10 and Adapter → 25. The code writes the left column first, then the right. Compare PyMuPDF’s default extraction with its position-sorted extraction on the same page. Save this as reading_order.py and run it with uv run --with pymupdf==2.0 python reading_order.py; it creates the PDF in memory and needs no input file.

import pymupdf

with pymupdf.open() as document:
    page = document.new_page(width=400, height=220)
    for x, values in [(40, ["Item", "Cable", "Adapter"]), (240, ["Price", "10", "25"])]:
        for y, value in zip([50, 80, 110], values):
            page.insert_text((x, y), value)

    stored = page.get_text().split()
    ordered = page.get_text(sort=True).split()
    assert stored == ["Item", "Cable", "Adapter", "Price", "10", "25"]
    assert ordered == ["Item", "Price", "Cable", "10", "Adapter", "25"]
    print("stored:", " ".join(stored))
    print("sorted:", " ".join(ordered))

Observed with PyMuPDF 2.0:

stored: Item Cable Adapter Price 10 25
sorted: Item Price Cable 10 Adapter 25

The default output preserves the text but groups it by column. Sorting restores row order in this example, so OCR is unnecessary. This checks one extraction behavior on a constructed page; it does not compare OCR model accuracy. PyMuPDF documents the reading-order issue and sorting option. Wrapped cells, merged cells, and multiple tables need separate tests. For a real document, define the expected fields and relationships before comparing a text extractor, an OCR engine, and a document VLM.

Practical architecture

For production document AI, I would not send every page to the most expensive model by default.

1. Inspect the file and record page-level metadata. For PDFs, check whether existing text and reading order meet the task.
2. Extract usable embedded text directly. PyMuPDF recommends checking whether OCR is needed before running it on a page.
3. Route clean printed scans without usable text to classical OCR.
4. Route tables, low-confidence pages, handwritten regions, and complex layouts to a VLM or specialized parser.
5. Require structured output for fields, then validate values against the page. A schema can enforce field shape without proving that a value appears in the document.
6. Store source spans, page numbers, bounding boxes where available, and model version.
7. Sample human review by confidence, document type, and downstream impact.

Add routing when testing shows that different page types need different methods. Check that the router does not send a difficult page down a path that loses required fields.

Evaluation checklist

Do not evaluate OCR only with character error rate. For document AI, track:

• field-level accuracy for extracted fields
• table cell preservation
• reading-order accuracy
• page coverage
• unsupported field rate
• citation or span support for extracted claims
• human correction rate
• cost per page and latency per document

Deeper reading

• OCR in 2026 covers pipelines, VLMs, layout, evaluation, and cost.
• RAG Evaluation Metrics is relevant when OCR feeds a retrieval system.

References

• Gemini document understanding
• PaddleOCR-VL 1.6 documentation
• PaddleOCR releases
• dots.mocr
• OmniDocBench
• PyMuPDF OCR guidance