LifeHubber. Resources Use AI on your terms.
LifeHubber.Resource
Opening…

AI Resources

ParseBench

GitHub stars: 650 GitHub forks: 141 Declared license: Apache-2.0: Apache-2.0 Last pushed November 21, 2026: Pushed today
Stats from GitHub

ParseBench evaluates document parsing tools on tables, charts, content faithfulness, semantic formatting, and visual grounding.

The publisher provides annotated enterprise document pages, evaluation code, and reports for examining how a parser represents the original document.

What it is

A benchmark for parsed documents

ParseBench evaluates output from a selected parsing tool against annotated documents. It is a benchmark around parsers, rather than a document assistant.

Why it stands out

Separate views of parsing errors

The five dimensions separate table structure, chart values, text, meaningful formatting, and page locations instead of reducing the comparison to text similarity.

Availability

Code, dataset, and public reports

Python benchmark code is available on GitHub and PyPI, with the dataset on Hugging Face and a public comparison site. Running a pipeline also requires its selected parsing provider.

Why it matters

What makes it useful

A table can contain the right words while attaching a value to the wrong header. The dataset's table metric treats rows as records keyed by column headers: changing row or column order is different from dropping or transposing a header. This checks the association a reader needs when looking up a value.

Notable points

What stands out

The current README says italic, underline, and highlight are evaluated as individual rules but excluded from the headline Semantic Formatting score. Improving one of those rules therefore need not improve that headline number. A separate open upstream report describes bold credit earned through heading promotion; its reported scorer behavior is not a verified claim that a parser understands formatting better.

Before using

What to review

API-based parsing pipelines use the selected provider's key. Evaluation is rule-based by default; the optional chart-normalization judge is a separate, disabled-by-default mode requiring an Anthropic key.

Open report #173 describes shared text IDs dropping content cases from an unfiltered evaluation. Its example distinguishes all-groups evaluation from a separately filtered content group, so reported coverage matters alongside a successful-run count.

Open report #170 describes different formatting results from different MinerU markdown post-processors and asks which renderer produced the published row. These are reporter findings, with reproduction and current fix adoption unverified here.

Check whether the annotated document mix, languages and scored tasks represent the PDFs you intend to process before extending a leaderboard result to your workflow. The benchmark's covered cases do not establish performance on every document type.

Check the original listing's documentation for current requirements, limitations, licensing and any relevant privacy or security considerations.

Reader fit

Who may find it relevant

Builders comparing document parsers or OCR output for search, extraction and document-reading workflows may find the separate metrics and per-case reports useful. Builders with another parser can register a provider and pipeline through the documented extensions interface. The extension guide says registration happens on import, so the package must be imported before running the CLI. This adds a parser to the existing scoring core without requiring a fork of that core.

Editorial note

Why LifeHubber lists it

We include ParseBench for builders who want to revisit how saved parser output was scored. Its documented re-evaluation path separates another scoring pass from another parser call, so an evaluator change can be examined without generating that output again. The publisher records parse_bench_version in run metadata and evaluation reports; its 1.0.0 changelog says runs made with 0.2.0 or earlier are not directly comparable. Keep that version with archived results when comparing scores.

Source links

Source materials

Reader note

About our listings

LifeHubber’s listings help you explore what’s available in AI and are not endorsements. We don’t independently verify every detail or guarantee that each listing is accurate, safe, actively maintained, or right for your needs.

Advertisements

Advertisements

For project maintainers

Listed here? You can use the badge.

If you maintain a project with a current LifeHubber listing, you may add the optional “Listed on LifeHubber AI Resources” badge to its README, docs, or website. No introduction or permission request is needed.

MORE ON LIFEHUBBER

Use AI on your own terms.

Explore more AI projects, stories, guides, tools and community insights across LifeHubber.

See what’s moving