level: research
document parsing turns scanned images into structured text, layout, and visual data, which is essential for tasks like visual question answering and key information extraction. many ocr engines and multimodal large language models exist, but choosing the right one for a specific document set is hard, especially when labeled data is scarce. this work systematically tests text recognition across various ocr tools and state-of-the-art mllms on multiple benchmarks covering different domains and languages.
the researchers found that many ocr engines have limited contextual reasoning, and manual annotation is expensive. to address this, they created dococr-eval, an annotation-free evaluation framework. it works by having a model correct the output of an ocr system, then measuring how much correction was needed. less correction implies better original ocr quality. this method avoids the need for ground truth text, making it practical for real-world use.
experiments show that dococr-eval rankings align well with traditional metrics that require ground truth. the framework can compare both traditional ocr engines and modern mllms on the same task. it also reveals that no single tool is best for all document types, highlighting the value of per-dataset evaluation. the approach is cost-effective and scalable, enabling quick selection of the most suitable parsing tool for any document collection.
why it matters: it lets practitioners choose the best ocr tool for their documents without costly manual labeling, saving time and resources in document ai pipelines.