taset for pre-training and benchmarks,” arXiv preprint arXiv:2306.04362, 2023.
[46]
A. Chen, Z. Wang, C. Dong, K. Tian, R. Zhao, X. Liang, Z. Kang, and X. Li, “Chinaopen: A dataset for open-world multimodal learning,” in Proceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 6432–6440.
[47]
Y. Du, Z. Chen, C. Jia, X. Yin, T. Zheng, C. Li, Y. Du, and Y.-G. Jiang, “Svtr: Scene text recognition with a single visual model,” arXiv preprint arXiv:2205.00159, 2022.
[48]
G. Jocher, A. Chaurasia, and J. Qiu, “Ultralytics yolo,” https://github.com/ultralytics/ultralytics, 2023.
[49]
B. K. Horn and B. G. Schunck, “Determining optical flow,” Artificial intelligence, vol. 17, no. 1-3, pp. 185–203, 1981.
[50]
B. Castellano, “Pyscenedetect,” https://github.com/Breakthrough/PySceneDetect.
[51]
A. Sun, “jieba,” https://github.com/fxsjy/jieba.
[52]
R. Zhang, L. Gui, Z. Sun, Y. Feng, K. Xu, Y. Zhang, D. Fu, C. Li, A. Hauptmann, Y. Bisk et al., “Direct preference optimization of video large multimodal models from language model reward,” arXiv preprint arXiv:2404.01258, 2024.
[53]
W. Dai, J. Li, D. Li, A. M. H. Tiong, J. Zhao, W. Wang, B. Li, P. Fung, and S. Hoi, “Instructblip: Towards general-purpose vision-language models with instruction tuning,” arXiv preprint arkiv:2305.06500, 2023.
[54]
X. Wang, J. Wu, J. Chen, L. Li, Y.-F. Wang, and W. Y. Wang, “Vatex: A large-scale, high-quality multilingual dataset for video-and-language research,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 4581–4591.
[55]
L. Zhou, C. Xu, and J. Corso, “Towards automatic learning of procedures from web instructional videos,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018.
[56]
G. A. Sigurdsson, G. Varol, X. Wang, A. Farhadi, I. Laptev, and A. Gupta, “Hollywood in homes: Crowdsourcing data collection for activity understanding,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14. Springer, 2016, pp. 510–526.
[57]
W. Wu, Y. Zhao, Z. Li, J. Li, H. Zhou, M. Z. Shou, and X. Bai, “A large cross-modal video retrieval dataset with reading comprehension,” Pattern Recognition, p. 110818, 2024.
[58]
E. Cui, Y. He, Z. Ma, Z. Chen, H. Tian, W. Wang, K. Li, Y. Wang, W. Wang, X. Zhu, L. Lu, T. Lu, Y. Wang, L. Wang, Y. Qiao, and J. Dai, “Sharegpt-4o: Comprehensive multimodal annotations with gpt-4o,” https://sharegpt4o.github.io/, 2024.
[59]
L. Chen, X. Wei, J. Li, X. Dong, P. Zhang, Y. Zang, Z. Chen, H. Duan, B. Lin, Z. Tang et al., “Sharegpt4video: Improving video understanding and generation with better captions,” arXiv preprint arXiv:2406.04325, 2024.
[60]
J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al., “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774, 2023.
[61]
J. Xiao, X. Shang, A. Yao, and T.-S. Chua, “Next-qa: Next phase of question-answering to explaining temporal actions,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 9777–9786.
[62]
Y. Jang, Y. Song, Y. Yu, Y. Kim, and G. Kim, “Tgif-qa: Toward spatio-temporal reasoning in visual question answering,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2758–2766.
[63]
A. Yang, A. Miech, J. Sivic, I. Laptev, and C. Schmid, “Just ask: Learning to answer questions from millions of narrated videos,” in ICCV, 2021.
[64]
K. Grauman, A. Westbury, E. Byrne, Z. Chavis, A. Furnari, R. Girdhar, J. Hamburger, H. Jiang, M. Liu, X. Liu et al., “Ego4d: Around the world in 3,000 hours of egocentric video,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 18 995–19 012.
[65]
K. Yi, C. Gan, Y. Li, P. Kohli, J. Wu, A. Torralba, and J. B. Tenenbaum, “Clevrer: Collision events for video representation and reasoning,” arXiv preprint arXiv:1910.01442