1031 lines
76 KiB
XML
1031 lines
76 KiB
XML
<?xml version='1.0' encoding='UTF-8'?>
|
|
<feed xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/" xmlns:arxiv="http://arxiv.org/schemas/atom" xmlns="http://www.w3.org/2005/Atom">
|
|
<id>https://arxiv.org/api/n3R2WuVBK0+uPd/62Rt6eQ89DYQ</id>
|
|
<title>arXiv Query: search_query=all:"long document" AND all:multimodal&id_list=&start=0&max_results=30</title>
|
|
<updated>2026-10-26T01:04:33Z</updated>
|
|
<link href="https://arxiv.org/api/query?search_query=all:%22long+document%22+AND+all:multimodal&start=0&max_results=30&id_list=" type="application/atom+xml"/>
|
|
<opensearch:itemsPerPage>30</opensearch:itemsPerPage>
|
|
<opensearch:totalResults>56</opensearch:totalResults>
|
|
<opensearch:startIndex>0</opensearch:startIndex>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2610.31385v1</id>
|
|
<title>A Task-Centric Survey of Long-Document Visual Question Answering: Taxonomy, Evolution, and Future Directions</title>
|
|
<updated>2026-10-22T14:52:47Z</updated>
|
|
<link href="https://arxiv.org/abs/2610.31385v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2610.31385v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Visual document understanding is essential for acquiring knowledge from visually rich documents and is attracting growing research interest. Existing surveys emphasize architectures or technical approaches, with limited attention to how task bottlenecks shape technical choices and research priorities. We present a task-centric survey that decomposes long-document visual question answering (VQA) into four components: document parsing, evidence organization and retrieval, generator organization and learning, and evaluation. For each, we develop a fine-grained taxonomy and trace motivations, technical responses, and cross-component connections. We focus on explicit retrieval while covering direct reading. Our temporal analysis traces expanding priorities from multimodal content understanding to sufficient, high-fidelity evidence acquisition and sustained evidence management. We outline future directions in evidence sufficiency, verifiable and correctable understanding, and budget-aware coordination, with an outlook on continual document understanding.</summary>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.CL" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.IR" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-10-22T14:52:47Z</published>
|
|
<arxiv:comment>26 pages, 9 figures, 6 tables</arxiv:comment>
|
|
<arxiv:primary_category term="cs.CV"/>
|
|
<author>
|
|
<name>Yufei Tong</name>
|
|
</author>
|
|
<author>
|
|
<name>Guanjie Cheng</name>
|
|
</author>
|
|
<author>
|
|
<name>Tianle Wu</name>
|
|
</author>
|
|
<author>
|
|
<name>Leilei Zhang</name>
|
|
</author>
|
|
<author>
|
|
<name>Hang Zheng</name>
|
|
</author>
|
|
<author>
|
|
<name>Peihan Wu</name>
|
|
</author>
|
|
<author>
|
|
<name>Jianwei Yin</name>
|
|
</author>
|
|
<author>
|
|
<name>Shuiguang Deng</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2610.21327v1</id>
|
|
<title>DocPilot: Interactive Agent Orchestration and Training for Long Visual Document Understanding</title>
|
|
<updated>2026-10-15T06:16:43Z</updated>
|
|
<link href="https://arxiv.org/abs/2610.21327v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2610.21327v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Multimodal long-document question answering requires models to seek sparse evidence across pages and modalities, understand heterogeneous document content, and reason over the collected evidence. Even advanced vision-language models (VLMs) can overlook sparse evidence and struggle with cross-modal reasoning in a single pass. Existing document agents further introduce multi-step interaction, but some still rely on predefined workflows or coarse-grained actions, limiting the flexibility and granularity of document reasoning. We introduce DocPilot, an agentic document framework that supports expressive orchestration and makes complex document solving learnable by small language models (SLMs). DocPilot orchestrates evidence seeking through page-level navigation and fine-grained evidence localization, and integrates local visual inspection, code-based numerical reasoning, and cross-modal reasoning for multimodal understanding. To transfer these capabilities to small models, we propose curriculum-based agentic training that progresses from single-ability learning to adaptive orchestration through supervision from data-grounded reasoning traces and executable interaction trajectories. Across MMLongBench-Doc, DocBench, FinRAGBench-V, and MADQA, DocPilot achieves strong gains with frontier VLM backbones, while DocPilot-4B delivers competitive performance against substantially larger models, demonstrating strong effectiveness and generalization across diverse long-document reasoning settings. The code is available at https://github.com/HKUSTDial/DocPilot.</summary>
|
|
<category term="cs.AI" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.CL" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-10-15T06:16:43Z</published>
|
|
<arxiv:comment>25 pages, 13 figures, 9 tables</arxiv:comment>
|
|
<arxiv:primary_category term="cs.AI"/>
|
|
<author>
|
|
<name>Zhuowen Liang</name>
|
|
</author>
|
|
<author>
|
|
<name>Xiaotian Lin</name>
|
|
</author>
|
|
<author>
|
|
<name>Yuyu Luo</name>
|
|
</author>
|
|
<author>
|
|
<name>Haixun Wang</name>
|
|
</author>
|
|
<author>
|
|
<name>Nan Tang</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2610.20964v2</id>
|
|
<title>VELD-QA: A Large-Scale Dataset for Visual Evidence Grounding and Question Answering in Long Documents</title>
|
|
<updated>2026-10-16T19:11:33Z</updated>
|
|
<link href="https://arxiv.org/abs/2610.20964v2" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2610.20964v2" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Existing long-document QA benchmarks rarely combine large-scale human-authored questions with detailed visual evidence annotations, limiting diagnosis of evidence localization and interpretation failures. We present VELD-QA, a large-scale benchmark for Visual Evidence grounding and question answering in Long Documents. It contains 15,000 fully human-authored question-answer pairs across 5,859 real-world PDFs totaling 372,445 pages, balancing chart, table, and complex-layout tasks. Human annotations identify supporting pages and evidence elements and describe cited visual content, enabling separate retrieval and interpretation evaluation. We evaluate nine vision-language models under four evidence settings and seven retrieval-augmented and agentic systems. Our analysis identifies bottlenecks in evidence localization and interpretation. Retrieval-based methods substantially underperform the oracle setting with ground-truth supporting pages; full-document access does not close the gap. Localization alone is insufficient: oracle accuracy peaks at 81.7%, highlighting persistent visual interpretation and reasoning challenges. Enriching inputs with our visual-element annotations narrows the performance gap by 17-24%, demonstrating value beyond evaluation and motivating data enrichment for multimodal document QA. Our code is available at https://github.com/oracle-samples/veld-qa.</summary>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.CL" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-10-14T21:36:28Z</published>
|
|
<arxiv:primary_category term="cs.CV"/>
|
|
<author>
|
|
<name>Zhivar Sourati</name>
|
|
</author>
|
|
<author>
|
|
<name>Marianne Menglin Liu</name>
|
|
</author>
|
|
<author>
|
|
<name>Rongguang Wang</name>
|
|
</author>
|
|
<author>
|
|
<name>Zheng Wang</name>
|
|
</author>
|
|
<author>
|
|
<name>Yazhe Hu</name>
|
|
</author>
|
|
<author>
|
|
<name>Lan Guo</name>
|
|
</author>
|
|
<author>
|
|
<name>Yuying Wang</name>
|
|
</author>
|
|
<author>
|
|
<name>Mengqing Guo</name>
|
|
</author>
|
|
<author>
|
|
<name>Bhavesh Dnyaneshwar Gore</name>
|
|
</author>
|
|
<author>
|
|
<name>Shubham Pathak</name>
|
|
</author>
|
|
<author>
|
|
<name>Ramapriya Ranganath</name>
|
|
</author>
|
|
<author>
|
|
<name>Sandeep Sharma</name>
|
|
</author>
|
|
<author>
|
|
<name>Tao Sheng</name>
|
|
</author>
|
|
<author>
|
|
<name>Sujith Ravi</name>
|
|
</author>
|
|
<author>
|
|
<name>Dan Roth</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2610.18238v1</id>
|
|
<title>Heading Hierarchy Reconstruction for Complex and Long Documents with Level-Wise Multimodal Memory</title>
|
|
<updated>2026-10-13T12:06:28Z</updated>
|
|
<link href="https://arxiv.org/abs/2610.18238v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2610.18238v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Vision-language models (VLMs) have been widely adopted for document parsing and optical character recognition (OCR), as they can accurately extract page-level elements. However, downstream applications such as retrieval-augmented generation (RAG) also require coherent document-level structure. Although these models indicate heading levels using different numbers of preceding ``\#'' symbols, page-by-page prediction often lacks document-level consistency. In practice, input-resolution and context-length constraints make it infeasible for a VLM to jointly process all page images in documents containing tens or hundreds of pages. To address this limitation, we propose a VLM-based framework that incrementally reconstructs heading hierarchies with a \emph{level-wise multimodal memory}. The memory retains a compact subset of all previously processed pages, selected by anchors. Our framework supports flexible anchor selection, and in this work we select the most recent node at each level, allowing the VLM to observe the visual, textual, and layout patterns of headings at every level. For comprehensive evaluation, we also introduce the Complex and Long Document Heading Hierarchy Reconstruction Benchmark (CLDoc-HHRBench), comprising 250 documents and 11,212 pages with expert-verified annotations. Experiments on CLDoc-HHRBench and three existing benchmarks demonstrate the effectiveness of our method.</summary>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.AI" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-10-13T12:06:28Z</published>
|
|
<arxiv:primary_category term="cs.CV"/>
|
|
<author>
|
|
<name>Qian Chen</name>
|
|
</author>
|
|
<author>
|
|
<name>Xianyin Zhang</name>
|
|
</author>
|
|
<author>
|
|
<name>Lifan Guo</name>
|
|
</author>
|
|
<author>
|
|
<name>Chi Zhang</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2608.22214v1</id>
|
|
<title>Query-Driven Multimodal Information Extraction from Long Documents</title>
|
|
<updated>2026-08-23T04:37:34Z</updated>
|
|
<link href="https://arxiv.org/abs/2608.22214v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2608.22214v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>In domain-specific multimodal long documents, images and text jointly convey complex knowledge that cannot be fully captured by plain text alone. However, existing paradigms like DocVQA primarily focus on generating textual answers or localizing evidence regions, rather than outputting query-specific textual attribute values and corresponding images. To address this gap, we propose query-driven image-text joint extraction from long documents, requiring models to output query-requested textual attribute values and corresponding image bounding boxes. Based on challenges related to both user intent and document content, we designed a two-level taxonomy that operates at the query and instance levels. Further, we construct ITJoint, the first high-quality, manually annotated benchmark for this new task, comprising 2,455 pages of domain-specific documents with numerous non-decorative images, 316 queries, and 910 answer instances. Finally, we evaluate representative standalone Vision-Language Models from different providers and further design Q2IT, a multi-agent collaborative framework consisting of three progressively collaborating agents for evidence collection, page selection, and target-image localization. Using a joint evaluation approach that assesses both text extraction and image localization, our experiments show that standalone VLMs struggle with this task, while Q2IT significantly improves performance on ITJoint, although a substantial gap remains toward perfect results.</summary>
|
|
<category term="cs.AI" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.MM" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-08-23T04:37:34Z</published>
|
|
<arxiv:primary_category term="cs.AI"/>
|
|
<author>
|
|
<name>Yikai Gao</name>
|
|
</author>
|
|
<author>
|
|
<name>Ding Xia</name>
|
|
</author>
|
|
<author>
|
|
<name>Xi Yang</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2608.14841v1</id>
|
|
<title>What the Reranker Sees: Multi-Aspect Page Annotation for Long-Document Multimodal Question Answering</title>
|
|
<updated>2026-08-14T19:29:42Z</updated>
|
|
<link href="https://arxiv.org/abs/2608.14841v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2608.14841v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Long-document visual question answering (VQA) over documents of tens to hundreds of pages mixing text, tables, charts, and figures typically follows retrieve-then-read pipelines. In our setting, the bottleneck shifts from retrieval recall to reranker-side evidence selection: on MMLongBench-Doc, BGE-M3 reaches Recall@20 = 0.86 but only F1@5 = 0.254, and even the visual retriever ColPali reaches only F1@5 = 0.332; a text-only rerank LLM seeing only raw snippets misses table, chart, and layout evidence even when the upstream retriever encoded images. We propose Trident, with two complementary components: Trident-R, a retriever-agnostic LLM reranker that converts each candidate into an LLM-readable semantic record, including a visual caption, section path, entity tags, multi-axis concept hits, and a text snippet, then performs a single adaptive-K rerank call; and Trident-S, a generation-side module that prompts the VLM under topical, entity, and structural lenses before synthesis. On two long-document datasets, the annotation+rerank protocol substantially improves retrieval F1 across five heterogeneous pools, with every reranked pool exceeding the strongest adaptive-K baseline PageIndex. An LLM rerank without the annotation barely changes first-hit ranking, indicating the lift comes from the structured annotation. Trident-S targets open-ended synthesis questions by design, adding up to 6.6 points in generation accuracy on these questions. The best Trident configuration is the strongest downstream QA pipeline in our evaluation, with rankings consistent across two LLM judges (kappa = 0.913).</summary>
|
|
<category term="cs.AI" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-08-14T19:29:42Z</published>
|
|
<arxiv:primary_category term="cs.AI"/>
|
|
<author>
|
|
<name>Guanchen Wu</name>
|
|
</author>
|
|
<author>
|
|
<name>Jiayuan Ding</name>
|
|
</author>
|
|
<author>
|
|
<name>Subhabrata Mukherjee</name>
|
|
</author>
|
|
<author>
|
|
<name>Carl Yang</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2608.03451v1</id>
|
|
<title>DataSpace: Benchmarking Data Agents for Verifiable Analytics over Heterogeneous Workspaces</title>
|
|
<updated>2026-08-04T10:48:11Z</updated>
|
|
<link href="https://arxiv.org/abs/2608.03451v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2608.03451v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Data agents enable natural-language analytics over organizational workspaces, where relevant evidence may be scattered across databases, structured files, long documents, and multimedia. Existing benchmarks largely isolate structured querying, retrieval, or open-ended analysis, leaving heterogeneous evidence discovery, complete tabular outputs, and deterministic evaluation insufficiently unified. We introduce DataSpace, a benchmark in which data agents produce verifiable tabular results from task-local heterogeneous workspaces. It contains 410 cross-language tasks and 7,439 artifacts totaling 15.01 GB across CSV, JSON, SQLite, Markdown, PDF, and video. DataSpace also served as the official evaluation benchmark for the KDD Cup 2026 Data Agents for Complex Data Analysis competition. Each agent receives only a question and workspace and returns the complete requested tabular result. We construct DataSpace with DataSpace-Builder, an execution-grounded framework comprising cross-language transformation, constraint-aware relational sampling, modality routing and artifact rendering, and human review and task repair by 11 domain experts. A deterministic evaluator performs header-invariant column alignment, type- and precision-aware normalization, and order-aware row comparison. Across six recently released frontier multimodal models and five widely used agent harnesses, the best accuracy reaches 66.34%, while harness choice creates a 15.36-point spread with the backbone fixed. Multimodal evidence integration and joins consistently reduce accuracy across all six backbones. These results show that DataSpace remains unsaturated and identify key challenges for improving data-agent reliability.</summary>
|
|
<category term="cs.AI" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-08-04T10:48:11Z</published>
|
|
<arxiv:comment>8 pages of main text, 7 figures, with a supplementary appendix</arxiv:comment>
|
|
<arxiv:primary_category term="cs.AI"/>
|
|
<author>
|
|
<name>Boyan Li</name>
|
|
</author>
|
|
<author>
|
|
<name>Zhuowen Liang</name>
|
|
</author>
|
|
<author>
|
|
<name>Yupeng Xie</name>
|
|
</author>
|
|
<author>
|
|
<name>Xiaotian Lin</name>
|
|
</author>
|
|
<author>
|
|
<name>Tianqi Luo</name>
|
|
</author>
|
|
<author>
|
|
<name>Xinyu Liu</name>
|
|
</author>
|
|
<author>
|
|
<name>Yizhang Zhu</name>
|
|
</author>
|
|
<author>
|
|
<name>Zhangyang Peng</name>
|
|
</author>
|
|
<author>
|
|
<name>Yuan Li</name>
|
|
</author>
|
|
<author>
|
|
<name>Zhengxuan Zhang</name>
|
|
</author>
|
|
<author>
|
|
<name>Jiayi Zhang</name>
|
|
</author>
|
|
<author>
|
|
<name>Nan Tang</name>
|
|
</author>
|
|
<author>
|
|
<name>Guoliang Li</name>
|
|
</author>
|
|
<author>
|
|
<name>Yuyu Luo</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2608.03292v1</id>
|
|
<title>DocTrace: Towards Traceable Long Document VQA via Hierarchical Evidence Graph Reasoning</title>
|
|
<updated>2026-08-04T08:04:59Z</updated>
|
|
<link href="https://arxiv.org/abs/2608.03292v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2608.03292v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Long Document Visual Question Answering (LongDocVQA) requires Multimodal Large Language Models (MLLMs) to locate, integrate, and reason over heterogeneous document elements distributed across multiple pages. Existing approaches, including end-to-end MLLMs, retrieval-augmented generation (RAG) pipelines, and document agents, often lack explicit mechanisms to represent and verify how grounded evidence is progressively composed during reasoning, limiting both answer accuracy and traceability. In this paper, we cast LongDocVQA as an explicit evidence graph reasoning problem rather than implicit answer prediction. To this end, we propose DocTrace, a hierarchical framework that progressively performs evidence localization, structured document parsing, and evidence graph reasoning to enable explicit evidence provenance. To effectively learn these capabilities, we develop a two-stage training framework: joint Supervised Fine-Tuning (SFT) first initializes evidence localization and graph reasoning abilities, followed by task-specific Group Relative Policy Optimization (GRPO) with dedicated rewards to further optimize these capabilities. Extensive experiments on MMLongBench-Doc, LongDocURL, and SlideVQA demonstrate that DocTrace consistently outperforms both existing open-source baselines and proprietary MLLMs. Compared with the Qwen3-VL-8B-Instruct backbone, DocTrace achieves absolute improvements of 14.4, 11.3, and 11.7 points on the three benchmarks, respectively. Beyond competitive performance, DocTrace constructs traceable evidence graphs with explicit node-level provenance, enabling transparent and verifiable reasoning for long document understanding.</summary>
|
|
<category term="cs.AI" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-08-04T08:04:59Z</published>
|
|
<arxiv:primary_category term="cs.AI"/>
|
|
<author>
|
|
<name>Le Xiang</name>
|
|
</author>
|
|
<author>
|
|
<name>Zhicheng Guan</name>
|
|
</author>
|
|
<author>
|
|
<name>Hong Chen</name>
|
|
</author>
|
|
<author>
|
|
<name>Xiaocong Lin</name>
|
|
</author>
|
|
<author>
|
|
<name>Zhenghua Lei</name>
|
|
</author>
|
|
<author>
|
|
<name>Teng Hu</name>
|
|
</author>
|
|
<author>
|
|
<name>Bolei He</name>
|
|
</author>
|
|
<author>
|
|
<name>Long Zeng</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2607.28006v1</id>
|
|
<title>MMLDSum-LLM: Multimodal Long-Document Summarization with Visual-Alignment and Keyword-Aware</title>
|
|
<updated>2026-07-30T10:54:09Z</updated>
|
|
<link href="https://arxiv.org/abs/2607.28006v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2607.28006v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Multimodal long documents are core carriers of professional knowledge, where critical evidence is sparsely distributed across paragraphs and modalities. This easily causes key information omission and cross-modal hallucinations in summarization by multimodal LLMs. These issues stem from attention drift in long-range dependency modeling and gaps in inter-modal alignment. To address this, we introduce MMLDSum-Bench, a high-quality benchmark for multimodal long-document summarization, covering multiple domains, context-length scales, and visual-textual modality distributions. We further propose MMLDSum-LLM, a reproducible two-stage training framework that combines supervised fine-tuning with visual-alignment weighted loss and keyword-aware weighted loss, followed by GRPO with a multi-objective reward (keyword coverage, image-text alignment, ROUGE, and length control). Extensive experiments on MMLDSum-Bench, comparing against leading closed-source and open-source multimodal models under a unified evaluation protocol - including LLM-as-a-judge scoring, atomic-claim precision/recall, image-text alignment (ITA), and ROUGE - demonstrate that our approach significantly improves key-information coverage and cross-modal consistency.</summary>
|
|
<category term="cs.AI" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-07-30T10:54:09Z</published>
|
|
<arxiv:primary_category term="cs.AI"/>
|
|
<author>
|
|
<name>Xianpeng Zhang</name>
|
|
</author>
|
|
<author>
|
|
<name>Jiahua Yang</name>
|
|
</author>
|
|
<author>
|
|
<name>Dongyu Chen</name>
|
|
</author>
|
|
<author>
|
|
<name>Lei zhang</name>
|
|
</author>
|
|
<author>
|
|
<name>Jian Ma</name>
|
|
</author>
|
|
<author>
|
|
<name>Xu guohuan</name>
|
|
</author>
|
|
<author>
|
|
<name>Haonan Lu</name>
|
|
</author>
|
|
<author>
|
|
<name>Tianhuang Su</name>
|
|
</author>
|
|
<author>
|
|
<name>Chuangchuang Wang</name>
|
|
</author>
|
|
<author>
|
|
<name>Kai Tang</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2607.25294v1</id>
|
|
<title>CLBench-V: Evaluating Multimodal Context Learning from Grounding to Knowledge Acquisition</title>
|
|
<updated>2026-07-28T05:06:43Z</updated>
|
|
<link href="https://arxiv.org/abs/2607.25294v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2607.25294v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Real-world tasks often require models to learn from task-specific context rather than relying only on pre-trained knowledge. While recent work has highlighted this capability as context learning, existing evaluations mainly focus on textual contexts. In many practical settings, however, the context to be learned from is multimodal: scientific findings are conveyed through figures and tables, financial indicators are scattered across converted reports, and spatial decisions depend on maps, scenes, or web pages. We introduce CLBench-V, a benchmark for multimodal context learning that addresses the difficulty of localizing where context use breaks down by organizing tasks around three dimensions: context grounding, new information application, and new knowledge learning. CLBench-V combines converted public benchmarks with newly constructed datasets spanning domains such as science, finance, long-document understanding, spatial reasoning, and web-based visual question answering. To reduce the cost of constructing domain-specific context-learning tasks, we further use automated construction and filtering procedures for our newly built datasets. Across 3,443 instances and six recent multimodal models, the best overall score is only 0.2847, indicating that multimodal context learning remains far from saturated. Moreover, InternVL3.5-30B-A3B performs best on context grounding and new knowledge learning, while Qwen3.5-Plus performs best on new information application. We further analyze judge reliability, context length, image count, and representative failure cases. Code is available at https://github.com/IamLihua/CLBench-V.</summary>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.AI" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.CL" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.LG" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-07-28T05:06:43Z</published>
|
|
<arxiv:primary_category term="cs.CV"/>
|
|
<author>
|
|
<name>Lai Wei</name>
|
|
</author>
|
|
<author>
|
|
<name>Chengqi Li</name>
|
|
</author>
|
|
<author>
|
|
<name>Jiapeng Li</name>
|
|
</author>
|
|
<author>
|
|
<name>Ruina Hu</name>
|
|
</author>
|
|
<author>
|
|
<name>Yue Wang</name>
|
|
</author>
|
|
<author>
|
|
<name>Weiran Huang</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2607.24651v1</id>
|
|
<title>Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels</title>
|
|
<updated>2026-07-27T16:49:36Z</updated>
|
|
<link href="https://arxiv.org/abs/2607.24651v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2607.24651v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Reliable visual document understanding requires a model to attribute each answer to the evidence regions that support it. Recent benchmarks and systems express this step through a coordinate interface: the model outputs the coordinates of bounding boxes that mark the evidence regions in the document. Under this interface, vision-language models often fail to identify the right regions even when the answer is correct, a failure known as Attribution Hallucination. We present a study that investigates whether this failure is partially limited by what the model can express through coordinates. On a verified bilingual CiteVQA subset, we compare the coordinate interface with a language interface in which the model outputs only text, quoting its evidence verbatim, and a multimodal retriever returns the location of each quote as a page region proposed by a layout parser (tables and figures are quoted through their captions or notes); the comparison is repeated over six open vision-language models. Compared with the coordinate interface, evidence recall rises from at most 8 points to between 26 and 47 and the hallucination rate roughly halves, with little change in answer quality. Building on this comparison, we use the same quote-and-retrieve pipeline as a training scaffold: because region-level evidence labels are expensive to collect for long documents, we introduce a GRPO recipe whose reward is a judge's reading of the gold answer and crops of the retrieved regions, training the model to quote better evidence without any region labels and raising an 8B backbone's strict attributed accuracy from 22.4 to 33.8. These findings indicate a practical path to improve attribution"without a coordinate interface and without costly region-level supervision.</summary>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.CL" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.IR" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-07-27T16:49:36Z</published>
|
|
<arxiv:primary_category term="cs.CV"/>
|
|
<author>
|
|
<name>Zhuchenyang Liu</name>
|
|
</author>
|
|
<author>
|
|
<name>Yao Zhang</name>
|
|
</author>
|
|
<author>
|
|
<name>Yu Xiao</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2607.18917v1</id>
|
|
<title>TAP-RAG: Task-Aware Policy Control for Long-Document Multimodal Question Answering</title>
|
|
<updated>2026-07-21T09:59:13Z</updated>
|
|
<link href="https://arxiv.org/abs/2607.18917v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2607.18917v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Long-document multimodal question answering requires more than retrieving relevant chunks from a large document. Different queries require different evidence behavior. Existing multimodal RAG systems improve evidence access through text chunks, page images, graph links, or heterogeneous document elements, but they often apply a largely query-agnostic evidence-use strategy. We present TAP-RAG, a task-aware policy-controlled RAG framework for long-document multimodal QA. TAP-RAG contains a main controller, the Task-Aware Policy Controller (TAPC), and two policy-guided evidence executors: Task-Aware Query-Guided Flow Diffusion (TA-QFD) and Task-Aware Visual Enhancement (TAVE). For each query, TAPC predicts the task prior, estimates visual/local/global evidence signals, and produces an executable policy. TA-QFD then expands textual and structural evidence over the multimodal document graph, while TAVE selectively inspects page images when visual or layout evidence is needed. A guarded synthesis stage fuses text, visual, and structural evidence and abstains when support is insufficient. On DocBench and MMLongBench-Doc, TAP-RAG achieves the best overall accuracy among the compared systems, improving over a matched multimodal-RAG baseline by +9.1 points (61.1 to 70.2) and +4.5 points (42.2 to 46.7), respectively.</summary>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-07-21T09:59:13Z</published>
|
|
<arxiv:comment>18 pages, 6 figures, 9 tables</arxiv:comment>
|
|
<arxiv:primary_category term="cs.CV"/>
|
|
<author>
|
|
<name>Zhong Ji</name>
|
|
</author>
|
|
<author>
|
|
<name>Keqi Jin</name>
|
|
</author>
|
|
<author>
|
|
<name>Yan Zhang</name>
|
|
</author>
|
|
<author>
|
|
<name>Jiasheng Li</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2607.04625v1</id>
|
|
<title>Hierarchical Evidence-Driven Reasoning for Long Document Understanding</title>
|
|
<updated>2026-07-06T03:08:28Z</updated>
|
|
<link href="https://arxiv.org/abs/2607.04625v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2607.04625v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Retrieval-Augmented Generation (RAG) streamlines long-document understanding by leveraging retrieval mechanisms to restrict input images to a highly curated subset. However, existing multimodal RAG pipelines primarily face two critical challenges: first, standard semantic similarity retrievers frequently fetch topically overlapping yet answer-void distractor pages that mislead downstream generation; second, rigid single-pass pipelines heavily depend on initial retrieval success, where any omission of core evidence inevitably causes cascading errors. To address these challenges, we introduce HIEVI-RAG, a hierarchical, evidence-driven multimodal RAG framework for closed-domain document understanding. HIEVI-RAG systematically factorizes complex queries into a cooperative four-stage pipeline: (1) hierarchical question decomposition to break multi-hop root queries into atomic child questions; (2) coarse visual page retrieval leveraging a multimodal retriever to fetch candidate pages based on semantic similarity; (3) fine-grained page verification via EVIAGENT, a specialized multi-page verifier trained with GRPO to execute cross-page reasoning over multi-image blocks; and (4) memory-guided iterative generation that leverages accumulated sub-question context to execute multi-round, dynamic reasoning over the prioritized sequence. Extensive evaluations across four benchmarks demonstrate the robust efficacy and synergy of our framework, which significantly outperforms existing open-source baselines and exceeds the strongest reported baseline by an average of 8.05% in accuracy.</summary>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.AI" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-07-06T03:08:28Z</published>
|
|
<arxiv:primary_category term="cs.CV"/>
|
|
<author>
|
|
<name>Junyu Xiong</name>
|
|
</author>
|
|
<author>
|
|
<name>Yonghui Wang</name>
|
|
</author>
|
|
<author>
|
|
<name>Rongjian Gu</name>
|
|
</author>
|
|
<author>
|
|
<name>Chenyu Liu</name>
|
|
</author>
|
|
<author>
|
|
<name>Bing Yin</name>
|
|
</author>
|
|
<author>
|
|
<name>Wengang Zhou</name>
|
|
</author>
|
|
<author>
|
|
<name>Houqiang Li</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2607.01420v3</id>
|
|
<title>MultAttnAttrib: Training-Free Multimodal Attribution in Long Document Question Answering</title>
|
|
<updated>2026-07-08T16:20:17Z</updated>
|
|
<link href="https://arxiv.org/abs/2607.01420v3" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2607.01420v3" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>As grounded QA systems are increasingly deployed in AI assistants, accurately attributing generated answers to evidence is critical for user trust and model safety. While unimodal attributions have been explored in depth, the multimodal setting remains relatively under-researched. As a result, we introduce MultAttnAttrib, a training-free attribution-generation method that leverages a model's prefill pass, selected attention heads, and calibrated thresholds to locate source evidence within a document. To establish baseline results for the method, we introduce MultAttrEval, a complementary benchmark dataset annotated with fine-grained, ground-truth attributions for answer components grounded in multimodal source documents. To our knowledge, this is the first evaluation dataset designed specifically for multimodal attribution in long-form documents. Experimental results show that MultAttnAttrib consistently outperforms a variety of attribution-generation methods, including several strong prompting-based approaches and matches the latest frontier models such as GPT 5.4. Our method not only substantially improves attribution accuracy for both unimodal and multimodal attribution types, but also produces attributions at up to one-seventh of the direct inference latency compared to prompting on the same base model.</summary>
|
|
<category term="cs.CL" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.AI" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-07-01T19:29:19Z</published>
|
|
<arxiv:comment>25 pages (8 main, 17 references + appendix), 15 figures</arxiv:comment>
|
|
<arxiv:primary_category term="cs.CL"/>
|
|
<author>
|
|
<name>Dang Quang Thien Tran</name>
|
|
</author>
|
|
<author>
|
|
<name>Quang V. Dang</name>
|
|
</author>
|
|
<author>
|
|
<name>Vinamra Tyagi</name>
|
|
</author>
|
|
<author>
|
|
<name>Sai Soorya Rao Veeravalli</name>
|
|
</author>
|
|
<author>
|
|
<name>Trang Nguyen</name>
|
|
</author>
|
|
<author>
|
|
<name>Ryan A. Rossi</name>
|
|
</author>
|
|
<author>
|
|
<name>Franck Dernoncourt</name>
|
|
</author>
|
|
<author>
|
|
<name>Nedim Lipka</name>
|
|
</author>
|
|
<author>
|
|
<name>Koustava Goswami</name>
|
|
</author>
|
|
<author>
|
|
<name>Samyadeep Basu</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2606.28780v1</id>
|
|
<title>Multimodal Graph RAG for Long-range Visually Rich Document Understanding</title>
|
|
<updated>2026-06-27T07:14:31Z</updated>
|
|
<link href="https://arxiv.org/abs/2606.28780v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2606.28780v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Multimodal large language models (MLLMs) are widely applied to visual document understanding. However, comprehending long documents remains an issue by the limited context window. Though recent multimodal retrieval-augmented generation (MMRAG) can address this challenge by retrieving relevant pages. It still struggles with the visual question answering (VQA) requiring holistic comprehension of a document. To cope with this, knowledge graph (KG) that summarizes global knowledge of a document can provide an effective solution. However, most existing LLM-based KG construction methods handle only the language modality, leaving the automatic creation of multimodal KGs (MMKGs) for visually rich documents largely unexplored. In this paper, we introduce a multimodal graph-based RAG approach to tackle this problem. Existing LLM-based KG methods evaluate the QA performance relying on indirect evidence such as comprehensiveness, diversity, empowerment, and so on. The lack of annotated datasets for comprehensive document-level VQA poses a significant challenge to effective model evaluation. To overcome this limitation, we also introduce a new benchmark, DLVQA (document-level VQA), which provides reference summaries and corresponding supporting facts for global document-level questions. Experimental results show that our approach outperforms existing MMRAG or KG-based approaches on multi-hop QA/VQA benchmarks and DLVQA.</summary>
|
|
<category term="cs.IR" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-06-27T07:14:31Z</published>
|
|
<arxiv:primary_category term="cs.IR"/>
|
|
<author>
|
|
<name>Yi-Cheng Wang</name>
|
|
</author>
|
|
<author>
|
|
<name>Chu-Song Chen</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2606.15906v1</id>
|
|
<title>MAGE-RAG: Multigranular Adaptive Graph Evidence for Agentic Multimodal RAG in Long-Document QA</title>
|
|
<updated>2026-06-14T16:34:15Z</updated>
|
|
<link href="https://arxiv.org/abs/2606.15906v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2606.15906v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Long-document multimodal question answering requires a system to locate sparse evidence in long PDFs and integrate clues from text, tables, images, charts, and complex layouts. Existing RAG methods mostly rely on fixed Top-k retrieval over text chunks or pages. Text retrieval can compress the context but often loses visual and layout information; page-level visual retrieval preserves the original page, yet it also sends large irrelevant regions to the reader, leading to a static trade-off among evidence coverage, noise, and inference cost. This paper proposes MAGE-RAG, a multigranular adaptive graph evidence framework for long-document multimodal QA. MAGE-RAG uses page retrieval as the entry point for query-time evidence construction. Offline, it builds an evidence graph with page nodes and element nodes, encoding containment, reading order, layout adjacency, section hierarchy, and semantic-neighbor relations. At query time, an online evidence controller iteratively activates, opens, searches, and prunes evidence under explicit budgets. The resulting evidence subgraph is then rendered into structured multimodal reader input, allowing the LVLM to consume compact and relevant evidence within a limited context. On LongDocURL and MMLongBench-Doc, we establish a unified comparison and analysis protocol covering Direct MLLM, Text RAG, Page-level Visual RAG, and Graph/Agentic RAG. Experiments show that MAGE-RAG achieves 52.75 overall accuracy on LongDocURL, and 53.26 accuracy with 51.19 F1 on MMLongBench-Doc. Fine-grained breakdowns, budget-performance curves, ablations, and trace-based analysis further show that query-time evidence subgraph construction can balance dispersed evidence coverage with context-noise control. Our code is available at https://github.com/laonuo2004/MAGE-RAG.git.</summary>
|
|
<category term="cs.IR" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.AI" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.CL" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.DB" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.MM" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-06-14T16:34:15Z</published>
|
|
<arxiv:primary_category term="cs.IR"/>
|
|
<author>
|
|
<name>Yilong Zuo</name>
|
|
</author>
|
|
<author>
|
|
<name>Xunkai Li</name>
|
|
</author>
|
|
<author>
|
|
<name>Jing Yuan</name>
|
|
</author>
|
|
<author>
|
|
<name>Qiangqiang Dai</name>
|
|
</author>
|
|
<author>
|
|
<name>Hongchao Qin</name>
|
|
</author>
|
|
<author>
|
|
<name>Ronghua Li</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2606.08979v1</id>
|
|
<title>EviProp: Seeded Relevance Diffusion on Chunk-Page Graphs for Long Multimodal Document Retrieval</title>
|
|
<updated>2026-06-08T03:25:20Z</updated>
|
|
<link href="https://arxiv.org/abs/2606.08979v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2606.08979v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Retrieving evidence pages from visually rich long documents is a key challenge in document question answering. Existing page-level visual retrievers operate under an independent matching paradigm: each page is scored in isolation based on query-page similarity. This paradigm can under-rank evidence pages whose signals are localized in fine-grained chunks or depend on document-internal associations. We propose EviProp, a retrieval method that recovers such pages via seeded relevance diffusion. EviProp models each document as a multimodal Chunk-Page graph with hierarchical, sequential, and similarity links. Given a query, it combines dense visual page priors with sparse chunk seeds, then runs Personalized PageRank to diffuse relevance over the graph. Experiments on MMLongBench-Doc and LongDocURL show consistent gains in evidence-page retrieval over independent visual retrieval and text-visual fusion baselines. Downstream QA results further show that improved retrieval translates into better answer accuracy, with negligible online retrieval overhead. Our code is released at https://github.com/Flyecnu/EviProp.</summary>
|
|
<category term="cs.IR" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-06-08T03:25:20Z</published>
|
|
<arxiv:primary_category term="cs.IR"/>
|
|
<author>
|
|
<name>Hongwei Zhang</name>
|
|
</author>
|
|
<author>
|
|
<name>Xiaoman Wang</name>
|
|
</author>
|
|
<author>
|
|
<name>Zehui Ling</name>
|
|
</author>
|
|
<author>
|
|
<name>Ruicheng Zhu</name>
|
|
</author>
|
|
<author>
|
|
<name>Yue Zhang</name>
|
|
</author>
|
|
<author>
|
|
<name>Pinlong Cai</name>
|
|
</author>
|
|
<author>
|
|
<name>Fuke Shen</name>
|
|
</author>
|
|
<author>
|
|
<name>Botian Shi</name>
|
|
</author>
|
|
<author>
|
|
<name>Tongquan Wei</name>
|
|
</author>
|
|
<author>
|
|
<name>Guohang Yan</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2606.05749v1</id>
|
|
<title>MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QA</title>
|
|
<updated>2026-06-04T06:23:01Z</updated>
|
|
<link href="https://arxiv.org/abs/2606.05749v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2606.05749v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Iterative retrieval-reasoning agents have recently shown promise for multimodal long-document question answering. However, most existing systems maintain a single growing context that mixes retrieval traces, observations, and intermediate reasoning. As interactions accumulate, key evidence becomes scattered and diluted, making multi-hop reasoning noisy. We propose MARDoc, a Memory-Aware Refinement Agent framework that decouples long-document QA into three specialized agents: an Explorer for multi-granularity multimodal retrieval, a Refiner for distilling interaction traces into structured evidence and reasoning memories, and a Reflector for checking evidence sufficiency and providing targeted feedback. Across iterations, the agents rely on a dynamically updated structured memory rather than a full accumulated interaction history. This design reduces context noise while preserving answer-critical facts and their logical dependencies. Experiments on MMLongBench-Doc and DocBench show that MARDoc achieves strong results, outperforming same-backbone baselines and demonstrating the effectiveness of structured memory for agentic document QA.</summary>
|
|
<category term="cs.CL" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.AI" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-06-04T06:23:01Z</published>
|
|
<arxiv:primary_category term="cs.CL"/>
|
|
<author>
|
|
<name>Kaifeng Chen</name>
|
|
</author>
|
|
<author>
|
|
<name>Hongtao Liu</name>
|
|
</author>
|
|
<author>
|
|
<name>Qiyao Peng</name>
|
|
</author>
|
|
<author>
|
|
<name>Jian Yang</name>
|
|
</author>
|
|
<author>
|
|
<name>Yongqiang Liu</name>
|
|
</author>
|
|
<author>
|
|
<name>Xiaochen Zhang</name>
|
|
</author>
|
|
<author>
|
|
<name>Qing Yang</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2606.04240v1</id>
|
|
<title>Overview of the EReL@MIR 2025 Multimodal Document Retrieval Challenge (Track 1)</title>
|
|
<updated>2026-06-02T21:39:32Z</updated>
|
|
<link href="https://arxiv.org/abs/2606.04240v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2606.04240v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Retrieval over visually-rich documents, pages that interleave text with figures, tables, and charts, is essential for multimodal retrieval-augmented generation, yet most retrievers still discard the visual channel. The \emph{Multimodal Document Retrieval Challenge}, Track~1 of the MIR Challenge at the first EReL@MIR workshop, co-located with The Web Conference 2025, asks participants to build a \emph{single} retrieval system that handles two complementary regimes: closed-set document page retrieval within long documents from a text query (MMDocIR), and open-domain retrieval of Wikipedia-style passages from an image or image-plus-text query (M2KR). Systems are ranked by the macro-average of mean Recall@$\{1,3,5\}$ over the two tasks. The challenge drew 455 entrants and 586 submissions across 22 teams. This report describes the challenge design, datasets, and evaluation protocol; reports the final standings; and analyses the three winning teams' systems. All three build on decoder-based Multimodal-LLM embedders from the Qwen2-VL family rather than on CLIP-style encoders, and differ chiefly in whether they reach the top through fine-tuned ensembles, training-free multi-route fusion with a strong vision-language re-ranker, or zero-shot late interaction. The training-free system finished within $0.1$ point of the fine-tuned winner.</summary>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.AI" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.CL" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-06-02T21:39:32Z</published>
|
|
<arxiv:comment>MDR Challenge Report at WWW2025</arxiv:comment>
|
|
<arxiv:primary_category term="cs.CV"/>
|
|
<author>
|
|
<name>Jingbiao Mei</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2607.24748v1</id>
|
|
<title>VLD-RAG: Agentic Vision-Language Retrieval-Augmented Generation for Long, Visually-Rich Multi-Page Documents</title>
|
|
<updated>2026-05-21T13:33:38Z</updated>
|
|
<link href="https://arxiv.org/abs/2607.24748v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2607.24748v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Visually-rich documents such as reports, slides, and manuals often distribute the evidence needed to answer a question across multiple pages, mixing text with layout cues, tables, charts, and figures. This work studies multimodal retrieval-augmented generation for question answering over such visually-rich long documents, where retrieval must select evidence pages that include both textual and visual signals. We present VLD-RAG, an agentic multimodal RAG framework for multi-page evidence retrieval and cross-page reasoning over long documents. VLD-RAG builds a page-preserving multimodal index that stores parsed text, page-level metadata, and dense visual representations, and uses a hybrid retrieval strategy that combines keyword-based sparse search with dense semantic queries to identify candidate sources and evidence pages. A verifier-guided agent workflow coordinates a Retrieval Agent, Answer Agent, and Validation Agent to broaden evidence coverage, detect missing citations, and refine retrieval requests when needed. We evaluate retrieval with Top-1 and Top-5 evidence-page accuracy and generation with generalized accuracy, and show that VLD-RAG improves both evidence-page retrieval and end-task question answering on visually-rich long-document benchmarks, including LongDocURL and MMLongBench-Doc, outperforming previous vision-based retrieval baselines. These findings highlight that coordinated agent verification and multimodal hybrid retrieval are crucial for reliable grounding when correct answers depend on evidence scattered across pages.</summary>
|
|
<category term="cs.IR" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.AI" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.CL" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-05-21T13:33:38Z</published>
|
|
<arxiv:primary_category term="cs.IR"/>
|
|
<author>
|
|
<name>Seonok Kim</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2605.17356v1</id>
|
|
<title>UniPPTBench: A Unified Benchmark for Presentation Generation Across Diverse Input Settings</title>
|
|
<updated>2026-05-17T09:50:16Z</updated>
|
|
<link href="https://arxiv.org/abs/2605.17356v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2605.17356v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Existing works typically focus on presentation generation under isolated input settings, whereas real-world use cases span diverse scenarios, including vague user prompts, long documents, multimodal materials, and multiple heterogeneous sources. Moreover, current evaluations are often insufficiently scenario-specific. They mainly rely on generic presentation-quality criteria, such as visual appeal, layout quality, and overall coherence, but fail to assess the core capabilities required by different input settings, including grounded compression, visual-text alignment, and cross-source synthesis. Consequently, the field lacks a unified benchmark and a scenario-aware evaluation framework for faithfully diagnosing presentation-generation systems across diverse real-world settings. We present UniPPTBench, a unified benchmark for presentation generation across four representative input settings: vague-prompt, long-document, multimodal-document, and multi-source generation. We further introduce UniPPTEval, a scenario-aware evaluation protocol that combines shared metrics for cross-setting comparison with scenario-specific metrics tailored to the core requirements of each setting. We also provide transparent reference baselines to support reproducible comparison. Experiments on UniPPTBench reveal substantial performance variation across settings and recurring failure modes in content grounding, multimodal integration, and cross-source synthesis. In particular, strong performance on generic presentation-quality metrics does not necessarily imply strong task fulfillment in grounded scenarios. Together, UniPPTBench and UniPPTEval provide a faithful and diagnostic foundation for evaluating presentation generation across diverse real-world scenarios. Code and data will be publicly available.</summary>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-05-17T09:50:16Z</published>
|
|
<arxiv:primary_category term="cs.CV"/>
|
|
<author>
|
|
<name>Bo Zhao</name>
|
|
</author>
|
|
<author>
|
|
<name>Maosheng Pang</name>
|
|
</author>
|
|
<author>
|
|
<name>Chen Zhang</name>
|
|
</author>
|
|
<author>
|
|
<name>Huan Yang</name>
|
|
</author>
|
|
<author>
|
|
<name>Yixin Cao</name>
|
|
</author>
|
|
<author>
|
|
<name>Wei Ji</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2605.13831v2</id>
|
|
<title>Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Context</title>
|
|
<updated>2026-10-18T16:19:51Z</updated>
|
|
<link href="https://arxiv.org/abs/2605.13831v2" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2605.13831v2" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Long-context modeling is becoming a core capability of modern large vision-language models (LVLMs), enabling sustained context management across long-document understanding, video analysis, and multi-turn tool use in long-horizon agents. Yet practical training recipes remain insufficiently explored, particularly for designing and balancing long-context data mixtures. In this work, we present a systematic study of long-context continued pre-training (LongPT) for LVLMs, extending a 7B model from 32K to 128K context with extensive ablations on long-document data. We first show that long-document VQA is substantially more effective than OCR transcription. Building on this observation, our ablations further yield three key findings: i) for sequence-length distribution, balanced data outperforms target-length-focused data (e.g., 128K), suggesting that long-context ability requires generalizable key-information retrieval across various lengths and positions; ii) retrieval remains the primary bottleneck, favoring retrieval-heavy mixtures with modest reasoning data for task diversity; iii) pure long-document VQA largely preserves short-context capabilities, suggesting that instruction-formatted long data reduces the need for short-data mixing. Based on these findings, we introduce MMProLong, obtained by long-context continued pre-training from Qwen2.5-VL-7B with only a 5B-token budget. MMProLong improves long-document VQA scores by 7.11% and maintains strong performance at 256K and 512K contexts beyond its 128K training window, without additional training. It further generalizes to webpage-based multimodal needle retrieval, long-context vision-text compression, and long-video understanding without task-specific supervision. Overall, our study establishes a practical LongPT recipe and an empirical foundation for advancing long-context vision-language models.</summary>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-05-13T17:52:53Z</published>
|
|
<arxiv:comment>Accepted at NeurIPS 2026. Code: https://github.com/zhaowei-wang-nlp/MMProLong</arxiv:comment>
|
|
<arxiv:primary_category term="cs.CV"/>
|
|
<author>
|
|
<name>Zhaowei Wang</name>
|
|
</author>
|
|
<author>
|
|
<name>Lishu Luo</name>
|
|
</author>
|
|
<author>
|
|
<name>Haodong Duan</name>
|
|
</author>
|
|
<author>
|
|
<name>Weiwei Liu</name>
|
|
</author>
|
|
<author>
|
|
<name>Sijin Wu</name>
|
|
</author>
|
|
<author>
|
|
<name>Ji Luo</name>
|
|
</author>
|
|
<author>
|
|
<name>Shen Yan</name>
|
|
</author>
|
|
<author>
|
|
<name>Shuai Peng</name>
|
|
</author>
|
|
<author>
|
|
<name>Sihang Yuan</name>
|
|
</author>
|
|
<author>
|
|
<name>Chaoyi Huang</name>
|
|
</author>
|
|
<author>
|
|
<name>Yi Lin</name>
|
|
</author>
|
|
<author>
|
|
<name>Yangqiu Song</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2605.11864v1</id>
|
|
<title>Very Efficient Listwise Multimodal Reranking for Long Documents</title>
|
|
<updated>2026-05-12T09:45:59Z</updated>
|
|
<link href="https://arxiv.org/abs/2605.11864v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2605.11864v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Listwise reranking is a key yet computationally expensive component in vision-centric retrieval and multimodal retrieval-augmented generation (M-RAG) over long documents. While recent VLM-based rerankers achieve strong accuracy, their practicality is often limited by long visual-token sequences and multi-step autoregressive decoding. We propose ZipRerank, a highly efficient listwise multimodal reranker that directly addresses both bottlenecks. It reduces input length via a lightweight query-image early interaction mechanism and eliminates autoregressive decoding by scoring all candidates in a single forward pass. To enable effective learning, ZipRerank adopts a two-stage training strategy: (i) listwise pretraining on large-scale text data rendered as images, and (ii) multimodal finetuning with VLM-teacher-distilled soft-ranking supervision. Extensive experiments on the MMDocIR benchmark show that ZipRerank matches or surpasses state-of-the-art multimodal rerankers while reducing LLM inference latency by up to an order of magnitude, making it well-suited for latency-sensitive real-world systems. The code is available at https://github.com/dukesun99/ZipRerank.</summary>
|
|
<category term="cs.IR" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.AI" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.MM" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-05-12T09:45:59Z</published>
|
|
<arxiv:comment>To appear in ICML 2026</arxiv:comment>
|
|
<arxiv:primary_category term="cs.IR"/>
|
|
<author>
|
|
<name>Yiqun Sun</name>
|
|
</author>
|
|
<author>
|
|
<name>Pengfei Wei</name>
|
|
</author>
|
|
<author>
|
|
<name>Lawrence B. Hsieh</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2605.08888v2</id>
|
|
<title>DocScope: Benchmarking Verifiable Reasoning for Trustworthy Long-Document Understanding</title>
|
|
<updated>2026-05-14T06:57:18Z</updated>
|
|
<link href="https://arxiv.org/abs/2605.08888v2" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2605.08888v2" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Evaluating whether Multimodal Large Language Models can produce trustworthy, verifiable reasoning over long, visually rich documents requires evaluation beyond end-to-end answer accuracy. We introduce DocScope, a benchmark that formulates long-document QA as a structured reasoning trajectory prediction problem: given a complete PDF document and a question, the model outputs evidence pages, supporting evidence regions, relevant factual statements, and a final answer. We design a four-stage evaluation protocol -- Page Localization, Region Grounding, Fact Extraction, and Answer Verification -- that audits each level of the trajectory independently through inter-stage decoupling, with all judges selected and calibrated via human alignment studies. DocScope comprises 1,124 questions derived from 273 documents, with all hierarchical evidence annotations completed by human annotators. We benchmark 6 proprietary models, 12 open-weight models, and several domain-specific systems. Our experiments reveal that answer accuracy cannot substitute for trajectory-level evaluation: even among correct answers, the highest observed rate of complete evidence chains is only 29\%. Across all models, region grounding remains the weakest trajectory stage. Furthermore, the primary difficulty stems from aggregating evidence dispersed across long distances and multiple document clusters, while an oracle study identifies faithful perception and fact extraction as the dominant capability bottleneck. Cross-architecture comparisons further suggest that activated parameter count matters more than total scale. The benchmark and code will be publicly released at https://github.com/MiliLab/DocScope.</summary>
|
|
<category term="cs.CL" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-05-09T11:12:59Z</published>
|
|
<arxiv:comment>50pages, 25 figures, 14 tables;</arxiv:comment>
|
|
<arxiv:primary_category term="cs.CL"/>
|
|
<author>
|
|
<name>Xiang Feng</name>
|
|
</author>
|
|
<author>
|
|
<name>Jiawei Zhou</name>
|
|
</author>
|
|
<author>
|
|
<name>Zhangfeng Huang</name>
|
|
</author>
|
|
<author>
|
|
<name>Kewei Wang</name>
|
|
</author>
|
|
<author>
|
|
<name>Shanshan Ye</name>
|
|
</author>
|
|
<author>
|
|
<name>Jinxin Hu</name>
|
|
</author>
|
|
<author>
|
|
<name>Zulong Chen</name>
|
|
</author>
|
|
<author>
|
|
<name>Yong Luo</name>
|
|
</author>
|
|
<author>
|
|
<name>Jing Zhang</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2604.12812v5</id>
|
|
<title>DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding</title>
|
|
<updated>2026-05-11T03:47:15Z</updated>
|
|
<link href="https://arxiv.org/abs/2604.12812v5" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2604.12812v5" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Existing Multimodal Large Language Models (MLLMs) suffer from significant performance degradation on the long document understanding task as document length increases. This stems from two fundamental challenges: 1) a low Signal-to-Noise Ratio (SNR), with crucial evidence buried in irrelevant pages; and 2) supervision scarcity, as datasets offering only final short answers provide a weak learning signal. In this paper, we address these challenges by proposing a paradigm that requires the model to execute a structured Analysis, Localization and Reasoning workflow. To instill this capability, we design a two-stage training framework: we first perform Supervised Fine-Tuning on high-quality data generated via an efficient knowledge distillation strategy. Subsequently, we employ an Evidence-aware Group Relative Policy Optimization which jointly optimizes for both evidence localization and answer accuracy. Additionally, we introduce a Evidence-Guided Resolution Allocation strategy to mitigate memory constraints of training on multi-pages documents. Extensive experiments demonstrate that DocSeeker achieves superior performance on both in-domain and out-of-domain tasks. We show it robustly generalizes from short-page training to ultra-long documents and is naturally synergistic with visual Retrieval-Augmented Generation systems, serving as a solid foundation for their implementation.</summary>
|
|
<category term="cs.AI" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-04-14T14:39:26Z</published>
|
|
<arxiv:comment>CVPR 2026 Highlight</arxiv:comment>
|
|
<arxiv:primary_category term="cs.AI"/>
|
|
<author>
|
|
<name>Hao Yan</name>
|
|
</author>
|
|
<author>
|
|
<name>Yuliang Liu</name>
|
|
</author>
|
|
<author>
|
|
<name>Xingchen Liu</name>
|
|
</author>
|
|
<author>
|
|
<name>Yuyi Zhang</name>
|
|
</author>
|
|
<author>
|
|
<name>Minghui Liao</name>
|
|
</author>
|
|
<author>
|
|
<name>Jihao Wu</name>
|
|
</author>
|
|
<author>
|
|
<name>Wei Chen</name>
|
|
</author>
|
|
<author>
|
|
<name>Xiang Bai</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2603.05551v2</id>
|
|
<title>AutothinkRAG: Complexity-Aware Control of Retrieval-Augmented Reasoning for Image-Text Interaction</title>
|
|
<updated>2026-03-17T15:59:36Z</updated>
|
|
<link href="https://arxiv.org/abs/2603.05551v2" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2603.05551v2" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Multimodal document question answering requires retrieving dispersed evidence from visually rich long documents and performing reliable reasoning over heterogeneous information. Existing multimodal RAG systems remain limited by two bottlenecks: static retrieval that ignores query complexity, and end-to-end Vision-Language Models (VLMs) that couple visual perception with logical reasoning, leading to inefficient computation and unstable answer generation. We propose AutoThinkRAG, a complexity-aware inference architecture for multimodal document QA. It has two components: (1) a Query Complexity Router that analyzes query difficulty and structure to adaptively select retrieval and reasoning paths; and (2) a Perception--Reasoning Decoupling architecture that uses a lightweight VLM as a high-fidelity visual interpreter to convert query-relevant visual cues into textual representations, which are then passed to an LLM for logical reasoning and answer synthesis. This design improves both efficiency and robustness, especially on long-document and unanswerable queries. Experiments on DocBench and MMLongBench show that AutoThinkRAG achieves 82.13\% and 51.29\% overall accuracy, respectively, while reducing per-query token consumption by 18.9\% and monetary cost by 18.2\%. Further analyses show that the gains are most pronounced on complex queries requiring adaptive retrieval and multi-step reasoning.</summary>
|
|
<category term="cs.IR" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-03-05T02:29:33Z</published>
|
|
<arxiv:primary_category term="cs.IR"/>
|
|
<author>
|
|
<name>Jiashu Yang</name>
|
|
</author>
|
|
<author>
|
|
<name>Chi Zhang</name>
|
|
</author>
|
|
<author>
|
|
<name>Abudukelimu Wuerkaixi</name>
|
|
</author>
|
|
<author>
|
|
<name>Xuxin Cheng</name>
|
|
</author>
|
|
<author>
|
|
<name>Cao Liu</name>
|
|
</author>
|
|
<author>
|
|
<name>Ke Zeng</name>
|
|
</author>
|
|
<author>
|
|
<name>Xu Jia</name>
|
|
</author>
|
|
<author>
|
|
<name>Xunliang Cai</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2602.24134v1</id>
|
|
<title>AgenticOCR: Parsing Only What You Need for Efficient Retrieval-Augmented Generation</title>
|
|
<updated>2026-02-27T16:09:38Z</updated>
|
|
<link href="https://arxiv.org/abs/2602.24134v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2602.24134v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>The expansion of retrieval-augmented generation (RAG) into multimodal domains has intensified the challenge for processing complex visual documents, such as financial reports. While page-level chunking and retrieval is a natural starting point, it creates a critical bottleneck: delivering entire pages to the generator introduces excessive extraneous context. This not only overloads the generator's attention mechanism but also dilutes the most salient evidence. Moreover, compressing these information-rich pages into a limited visual token budget further increases the risk of hallucinations. To address this, we introduce AgenticOCR, a dynamic parsing paradigm that transforms optical character recognition (OCR) from a static, full-text process into a query-driven, on-demand extraction system. By autonomously analyzing document layout in a "thinking with images" manner, AgenticOCR identifies and selectively recognizes regions of interest. This approach performs on-demand decompression of visual tokens precisely where needed, effectively decoupling retrieval granularity from rigid page-level chunking. AgenticOCR has the potential to serve as the "third building block" of the visual document RAG stack, operating alongside and enhancing standard Embedding and Reranking modules. Experimental results demonstrate that AgenticOCR improves both the efficiency and accuracy of visual RAG systems, achieving expert-level performance in long document understanding. Code and models are available at https://github.com/OpenDataLab/AgenticOCR.</summary>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<category term="cs.CL" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-02-27T16:09:38Z</published>
|
|
<arxiv:primary_category term="cs.CV"/>
|
|
<author>
|
|
<name>Zhengren Wang</name>
|
|
</author>
|
|
<author>
|
|
<name>Dongsheng Ma</name>
|
|
</author>
|
|
<author>
|
|
<name>Huaping Zhong</name>
|
|
</author>
|
|
<author>
|
|
<name>Jiayu Li</name>
|
|
</author>
|
|
<author>
|
|
<name>Wentao Zhang</name>
|
|
</author>
|
|
<author>
|
|
<name>Bin Wang</name>
|
|
</author>
|
|
<author>
|
|
<name>Conghui He</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2602.12957v3</id>
|
|
<title>HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decoding</title>
|
|
<updated>2026-06-29T06:43:50Z</updated>
|
|
<link href="https://arxiv.org/abs/2602.12957v3" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2602.12957v3" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Document parsing is a fundamental task in multimodal understanding, supporting a wide range of downstream applications such as information extraction and intelligent document analysis. Benefiting from strong semantic modeling and robust generalization, VLM-based end-to-end approaches have emerged as the mainstream paradigm in recent years. However, these models often suffer from substantial inference latency, as they must autoregressively generate long, full-page sequences when processing long-form documents. While recent hybrid methods mitigate this issue via region-level parallel decoding with VLMs, independent region decoding loses full-page context and might weaken global coherence. To address this issue, we propose Hierarchical Speculative Decoding (HSD), a two-stage local-to-global framework for document parsing. HSD first employs a lightweight pipeline drafter to predict region partitions and generate coarse drafts for each region. The first stage verifies the generated region-level drafts in parallel for efficiency, while the second stage further performs page-level verification on these refined outputs to preserve full-page coherence. Experimental results show that HSD achieves a near-lossless 2.7x speedup with HunyuanOCR on OmniDocBench v1.5 and up to 7.04x speedup on long-document parsing tasks, demonstrating the effectiveness of the proposed method. The code is available at https://github.com/whlscut/HSD.</summary>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-02-13T14:22:10Z</published>
|
|
<arxiv:comment>ECCV 2026</arxiv:comment>
|
|
<arxiv:primary_category term="cs.CV"/>
|
|
<author>
|
|
<name>Wenhui Liao</name>
|
|
</author>
|
|
<author>
|
|
<name>Hongliang Li</name>
|
|
</author>
|
|
<author>
|
|
<name>Pengyu Xie</name>
|
|
</author>
|
|
<author>
|
|
<name>Xinyu Cai</name>
|
|
</author>
|
|
<author>
|
|
<name>Yufan Shen</name>
|
|
</author>
|
|
<author>
|
|
<name>Yi Xin</name>
|
|
</author>
|
|
<author>
|
|
<name>Qi Qin</name>
|
|
</author>
|
|
<author>
|
|
<name>Shenglong Ye</name>
|
|
</author>
|
|
<author>
|
|
<name>Tianbin Li</name>
|
|
</author>
|
|
<author>
|
|
<name>Ming Hu</name>
|
|
</author>
|
|
<author>
|
|
<name>Junjun He</name>
|
|
</author>
|
|
<author>
|
|
<name>Yihao Liu</name>
|
|
</author>
|
|
<author>
|
|
<name>Wenhai Wang</name>
|
|
</author>
|
|
<author>
|
|
<name>Min Dou</name>
|
|
</author>
|
|
<author>
|
|
<name>Bin Fu</name>
|
|
</author>
|
|
<author>
|
|
<name>Botian Shi</name>
|
|
</author>
|
|
<author>
|
|
<name>Yu Qiao</name>
|
|
</author>
|
|
<author>
|
|
<name>Lianwen Jin</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2601.22055v1</id>
|
|
<title>$G^2$-Reader: Dual Evolving Graphs for Multimodal Document QA</title>
|
|
<updated>2026-01-29T17:52:54Z</updated>
|
|
<link href="https://arxiv.org/abs/2601.22055v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2601.22055v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Retrieval-augmented generation is a practical paradigm for question answering over long documents, but it remains brittle for multimodal reading where text, tables, and figures are interleaved across many pages. First, flat chunking breaks document-native structure and cross-modal alignment, yielding semantic fragments that are hard to interpret in isolation. Second, even iterative retrieval can fail in long contexts by looping on partial evidence or drifting into irrelevant sections as noise accumulates, since each step is guided only by the current snippet without a persistent global search state. We introduce $G^2$-Reader, a dual-graph system, to address both issues. It evolves a Content Graph to preserve document-native structure and cross-modal semantics, and maintains a Planning Graph, an agentic directed acyclic graph of sub-questions, to track intermediate findings and guide stepwise navigation for evidence completion. On VisDoMBench across five multimodal domains, $G^2$-Reader with Qwen3-VL-32B-Instruct reaches 66.21\% average accuracy, outperforming strong baselines and a standalone GPT-5 (53.08\%).</summary>
|
|
<category term="cs.CL" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2026-01-29T17:52:54Z</published>
|
|
<arxiv:primary_category term="cs.CL"/>
|
|
<author>
|
|
<name>Yaxin Du</name>
|
|
</author>
|
|
<author>
|
|
<name>Junru Song</name>
|
|
</author>
|
|
<author>
|
|
<name>Yifan Zhou</name>
|
|
</author>
|
|
<author>
|
|
<name>Cheng Wang</name>
|
|
</author>
|
|
<author>
|
|
<name>Jiahao Gu</name>
|
|
</author>
|
|
<author>
|
|
<name>Zimeng Chen</name>
|
|
</author>
|
|
<author>
|
|
<name>Menglan Chen</name>
|
|
</author>
|
|
<author>
|
|
<name>Wen Yao</name>
|
|
</author>
|
|
<author>
|
|
<name>Yang Yang</name>
|
|
</author>
|
|
<author>
|
|
<name>Ying Wen</name>
|
|
</author>
|
|
<author>
|
|
<name>Siheng Chen</name>
|
|
</author>
|
|
</entry>
|
|
<entry>
|
|
<id>http://arxiv.org/abs/2511.22850v1</id>
|
|
<title>Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understanding</title>
|
|
<updated>2025-11-28T03:09:40Z</updated>
|
|
<link href="https://arxiv.org/abs/2511.22850v1" rel="alternate" type="text/html"/>
|
|
<link href="https://arxiv.org/pdf/2511.22850v1" rel="related" type="application/pdf" title="pdf"/>
|
|
<summary>Document understanding is a long standing practical task. Vision Language Models (VLMs) have gradually become a primary approach in this domain, demonstrating effective performance on single page tasks. However, their effectiveness diminishes when handling long documents. In such scenarios, clues are often scattered across multiple pages and modalities, and redundancy from lengthy inputs can impair the models judgment. While retrieval augmented generation mitigates this issue by filtering for question relevant content, the retrieved results still contain substantial redundancy. To address these limitations, we propose SLEUTH, a multi agent framework. Concretely, SLEUTH orchestrates a retriever and four collaborative agents in a coarse to fine process. The framework identifies key textual and visual clues within the retrieved pages, filters for salient visual evidence such as tables and charts, and analyzes the query to devise a reasoning strategy. It ultimately synthesizes a distilled, evidence dense multimodal context to generate the final prediction. SLEUTH is model agnostic and scalable. When paired with advanced VLM backbones, it consistently improves performance on multiple long document benchmarks, achieving state of the art results. Ablation studies verify each modules effectiveness and confirm the benefits of our hierarchical refinement paradigm.</summary>
|
|
<category term="cs.CV" scheme="http://arxiv.org/schemas/atom"/>
|
|
<published>2025-11-28T03:09:40Z</published>
|
|
<arxiv:primary_category term="cs.CV"/>
|
|
<author>
|
|
<name>Keliang Liu</name>
|
|
</author>
|
|
<author>
|
|
<name>Zizhi Chen</name>
|
|
</author>
|
|
<author>
|
|
<name>Mingcheng Li</name>
|
|
</author>
|
|
<author>
|
|
<name>Jingqun Tang</name>
|
|
</author>
|
|
<author>
|
|
<name>Dingkang Yang</name>
|
|
</author>
|
|
<author>
|
|
<name>Lihua Zhang</name>
|
|
</author>
|
|
</entry>
|
|
</feed>
|