Answer.AI Releases ReadBench, Revealing Significant Performance Degradation of VLMs on Long-Text Image Reading
Original title:TIL: Vision-Language Models Read Worse (or Better) Than You Think
AI summary
Answer.AI launched ReadBench, a new benchmark specifically evaluating the ability of vision-language models (VLMs) to extract and reason over textual information from images. The study found that while high resolution has little impact on generation tasks, nearly all VLMs show significant performance degradation when processing multi-page long-document images, indicating that current Visual RAG approaches are not yet viable for long contexts. By comparison, GPT-4o performed relatively well in short-context and multi-page scenarios, with smaller overall degradation. The benchmark is open-source and covers image versions of datasets including MMLU and GPQA.
Key Takeaways
- Answer.
- AI launched ReadBench,a new benchmark specifically evaluating the ability of vision-language…
- The study found that while high resolution has little impact on generation tasks,nearly all VLMs…
- By comparison,GPT-4o performed relatively well in short-context and multi-page scenarios,with…
- The benchmark is open-source and covers image versions of datasets including MMLU and GPQA.
Quick Facts
- Published
- Updated
- Source
- Answer.AI 官方研发博客(RSS)
- Category
- Papers & Research
- AI score
- 78
针对热门的多模态检索增强生成(Visual RAG)场景提供了关键实证数据,揭示了 VLM 在长文档阅读上的实际瓶颈,为开发者选择模型和优化管道提供重要参考。
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: Answer.AI Releases ReadBench, Revealing Significant Performance Degradation of VLMs on Long-Text Image Reading. June 5, 2025. https://new.nat6.net/en/news/nqiq403r0cs6cekuujaxe5n7b/