Some models can also predict boduning boxes for images, charts etc. You would still need to do the extra step of grabbing the images from the bounding boxes but it can work quite well. dots.ocr is the one I've used most for this, i.e. this mode huggingface.co/datasets/uv-...
Still an early experiment. Would love feedback on whether something like this would be useful for your work!
Early results across 3 test collections:
• Library card catalogs → LightOn #1
• Britannica 1771 → GLM #1
• Icelandic PDFs → dots.ocr #1
Different documents, different winners!
Example space: huggingface.co/spaces/davan...
Point it at any Hugging Face dataset, launches OCR models, compares outputs pairwise using a VLM judge, and publishes an interactive leaderboard.
Inspired by datalab's benchmarks approach, but open source so you can run it on your own collections.
github.com/davanstrien/...
Screenshot of plot showing ELO vs paramter count for different OCR models
There is no best VLM OCR model - rankings can flip completely by document type.
I built ocr-bench: run open OCR models on YOUR documents, get a per-collection leaderboard.
VLM-as-judge with Bradley-Terry ELO, all running on @hf.co. No local GPU needed.
This sounds great!
That sounds great! IIRC screen readers tend to work okay with Markdown format? Might also be worth exploring www.docling.ai if you didn't already.
Screenshot of a search UI showing a text box with search results showing index cards next to the ocr for the card
Is it worth re-OCR'ing old library index cards?
Re-OCR'd 453,000 from @bpl.boston.gov's rare books catalogue.
~$50 compute using @huggingface Jobs
BPL's own guide calls their search "extremely unreliable." Does better OCR + semantic search help fix it?
Demo space link below
Screenshot of a search UI showing a text box with search results showing index cards next to the ocr for the card
Is it worth re-OCR'ing old library index cards?
Re-OCR'd 453,000 from @bpl.boston.gov's rare books catalogue.
~$50 compute using @huggingface Jobs
BPL's own guide calls their search "extremely unreliable." Does better OCR + semantic search help fix it?
Demo space link below
Ran the same OCR models on 68 pages of historic newspaper. Every model hallucinated or looped.
DeepSeek-OCR-2, LightOnOCR-2, GLM-OCR – all melt down on dense newspaper columns.
You can try yourself using this @hf.co dataset: huggingface.co/datasets/dav...
Great to hear of some fresh eyes on this task! Think there is a lot that wasn't possible a few years ago that is now.
Looking forward to reading it! Looking forward to it even more if it comes with data 😛
Ran the same OCR models on 68 pages of historic newspaper. Every model hallucinated or looped.
DeepSeek-OCR-2, LightOnOCR-2, GLM-OCR – all melt down on dense newspaper columns.
You can try yourself using this @hf.co dataset: huggingface.co/datasets/dav...
llama.cpp logo + Hugging Face logo
Llama.cpp joins Hugging Face
github.com/ggml-org/lla...
Nice!
Wrote a slightly more detailed guide on how to do this with your own collections/materials: danielvanstrien.xyz/posts/2026/r...
llama.cpp logo + Hugging Face logo
Llama.cpp joins Hugging Face
github.com/ggml-org/lla...
@willwhim.com bsky.app/profile/dani...
Wrote a slightly more detailed guide on how to do this with your own collections/materials: danielvanstrien.xyz/posts/2026/r...
Yeah quality is very mixed by language. I have vague recollection of someone working a lot on sanskrit ocr using open models on the Hub. Will post if I remember where that was!
Will try to write something a bit more detailed for this!
OCR scripts: huggingface.co/datasets/uv-...
Dataset: huggingface.co/datasets/dav...
Screenshot of old vs new ocr. old ocr text is garbled. New ocr much cleaner.
Re-OCR'd the complete 1771 Encyclopaedia Britannica (2,724 pages) with a single command on @hf.co Jobs.
- 0.9B model (GLM-OCR)
~$0.002/page
~$5 total on an L4 GPU
Before (old Tesseract ocr) → After
table of contents showing ocr models supported in the repo
The uv-scripts/ocr collection now includes 13 models, including GLM-OCR, a 0.9B model that scores 94.6% on OmniDocBench.
One command to run any of them on your dataset via @hf.co Jobs.
huggingface.co/datasets/uv-...
If you have the model ID in the spaces demo code, it will often get picked up automatically. Otherwise, you can specify the model in the `models` field in the space YAML metadata, see huggingface.co/docs/hub/en/...
Join us tomorrow for a demo of IIIF Illustration Detector!
Zoom link: iiif.io/community
Semantic search, confidence filtering, updated weekly using Hugging Face Jobs.
Powered by a fine-tuned ModernBERT classifier. Full dataset stored in Lance format on the Hub with vector embeddings.
huggingface.co/spaces/libra...
Datasets and benchmarks drive AI progress, but finding papers that introduce new ones means digging through thousands of arXiv abstracts.
Updated the Dataset Papers on ArXiv app to surface them: 52K+ papers classified as introducing new datasets from 212K CS papers.
