LensVLM-9B by Apple
3 hours ago
- Vision-Language Models process text as rendered images, but accuracy degrades under high compression due to shrinking characters.
- LensVLM scans compressed images and selectively expands relevant parts to uncompressed form using learned tools.
- LensVLM achieves accuracy comparable to full-text upper bound at 4.3× effective compression, outperforming baselines up to 10.1× across seven text QA benchmarks.
- The framework generalizes to multimodal document and code understanding, with gains increasing at higher compression levels.
- Training makes visual compression robust to rendering choices; the model increasingly relies on expanded content rather than unreliable visual reading.
- Tool-choice guidance: text expansion for rendered text, high-resolution image expansion for native documents with layout cues.