sess the impact of prior knowledge, we test different models using question-only inputs. The results are shown in the last two columns of Tab. 4. We find that ChatGPT-4o, Gemini-1.5-pro, and LongVA achieve high performance on ID questions with question-only inputs, with +22.5%, +14.0%, and +13.4% over random chance, respectively. This indicates that they can call upon prior knowledge to answer the questions. In contrast, Claude-3.5-sonnet and InternVL2-8B are less affected by prior knowledge. For almost all models tested, the performance with video and narration input is much higher than with question only, suggesting knowledge leakage alone is not sufficient for good performance. There remains substantial room for improvement (around 60–70%) in Causal2Needles, which could benefit from enhanced causal reasoning capability.
Acknowledgments
This research is supported by the RIE2025 Industry Alignment Fund – Industry Collaboration Projects (IAF-ICP) (Award I2301E0026), administered by A*STAR, by Alibaba Group and NTU Singapore through Alibaba-NTU Global e-Sustainability CorpLab (ANGEL), by the Nanyang Associate Professorship, and by the National Research Foundation Fellowship (NRFF13-2021-0006), Singapore.
References
Amirizaniani et al. (2024)
M. Amirizaniani, E. Martin, M. Sivachenko, A. Mashhadi, and C. Shah
Do llms exhibit human-like reasoning? evaluating theory of mind in llms for open-ended responses.
arXiv preprint arXiv:2406.05659.
Cited by: §1.
Anthropic (2024)
Anthropic
Claude 3 model card.
Note: Preprint, accessed on March 8, 2025
External Links: Link
Cited by: Appendix C,
§4.1.
Bai et al. (2025)
S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, et al.
Qwen2. 5-vl technical report.
arXiv preprint arXiv:2502.13923.
Cited by: §4.1.
Chen et al. (2021)
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al.
Evaluating large language models trained on code.
arXiv preprint arXiv:2107.03374.
Cited by: §1.
Chen et al. (2024)
Z. Chen, W. Wang, H. Tian, S. Ye, Z. Gao, E. Cui, W. Tong, K. Hu, J. Luo, Z. Ma, et al.
How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites.
Science China Information Sciences 67 (12), pp. 220101.
Cited by: Appendix C,
§4.1.
Chinchure et al. (2025)
A. Chinchure, S. Ravi, R. Ng, V. Shwartz, B. Li, and L. Sigal
Black swan: abductive and defeasible video reasoning in unpredictable events.
In Proceedings of the Computer Vision and Pattern Recognition Conference,
pp. 24201–24210.
Cited by: §1.
Cobbe et al. (2021)
K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman
Training verifiers to solve math word problems.
arXiv preprint arXiv:2110.14168.
Cited by: §1.
Cores et al. (2024)
D. Cores, M. Dorkenwald, M. Mucientes, C. G. Snoek, and Y. M. Asano
TVBench: redesigning video-language evaluation.
arXiv preprint arXiv:2410.07752.
Cited by: §1,
§2.1,
Table 1.
Dogan et al. (2018)
P. Dogan, B. Li, L. Sigal, and M. Gross
A neural multi-sequence alignment technique (neumatch).
In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
pp. 8749–8758.
Cited by: Appendix A,
§3.
Dosovitskiy et al. (2020)
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al.
An image is worth 16x16 words: transformers for image recognition at scale.
arXiv preprint arXiv:2010.11929.
Cited by: Appendix C.
Dziri et al. (2023)
N. Dziri, X. Lu, M. Sclar, X. L. Li, L. Jiang, B. Y. Lin, S. Welleck, P. West, C. Bhagavatula, R. Le Bras, et al.
Faith and fate: limits of transformers on compositionality.
Advances in Neural Information Processing Systems 36, pp. 70293–70332.
Cited by: §1.
Forrester (1971)
J. W. ForresterCounterintuitive behavior of social systems(Website)
External Links: Link
Cited by: §1,
§2.2.
Frieder et al. 