5
43.5
-
Video-LLaVA [11]
7B
43.5
47.3
1.05
-
38.4
56.96
49.8
37.6
VideoChat2-HD [16]
7B
62.3
47.9
-
-
55.8
61.58
-
41.2
LLaMA-VID [18]
7B
41.9
33.2
-
23.9
38.5
56.87
45.6
-
PLLaVA [13]
7B
46.6
-
1.03
-
54.4
60.36
-
39.2
VideoLLaMA2 [82]
7B
54.6
48.5
-
-
51.7
60.47
-
-
LLaVA-NeXT-Video [14]
7B
53.1
-
-
-
43.9
56.66
53.7
43.5
LongVA [40]
7B
-
56.3
-
-
-
67.36
56.9
-
TimeChat [37]
7B
-
30.9
-
22.3
33.0
-
50.6
-
ShareGPT4Video [59]
8B
51.2
46.4
1.05
-
-
53.58
61.5
41.8
VILA-1.5 [83]
8B
-
56.7
-
-
50.4
64.18
58.8
-
Video-CCAM [84]
9B
64.60
58.5
-
-
-
69.00
-
-
InternVL-1.5 [35]
26B
-
50.4
1.26
-
-
-
-
-
PLLaVA [13]
34B
58.1
-
-
26.1
-
-
-
53.5
VideoLLaMA2 [82]
8×7B
53.9
35.5
-
-
53.3
-
-
-
GPT-4V [6]
UNK
43.7
49.2
1.53
-
55.6
-
-
60.7
GPT-4o [85]
UNK
-
64.6
1.63
27.0
72.2
78.26
-
66.7
Gemini-1.5-Pro [86]
UNK
-
-
1.30
33.1
63.2
76.39
67.1
64.4
Kangaroo (Ours)
8B
61.1
61.0
1.44
39.4
62.7
69.50
62.5
54.8
TABLE IV: Comparison with other video LMMs on VideoMME benchmark
Model
Size
Frames
Short
Medium
Long
Overall
w/o subs
w subs
w/o subs
w subs
w/o subs
w subs
w/o subs
w subs
Video-LLaVA [11]
7B
8
45.3
46.1
38.0
40.7
36.2
38.1
39.9
41.6
Chat-Univi-v1.5 [17]
7B
64
45.7
51.2
40.3
44.6
35.8
41.8
40.6
45.9
VideoChat2 [16]
7B
16
48.3
52.8
37.0
39.4
33.2
39.2
39.5
43.8
VideoLLaMA2 [82]
7B
16
56.0
59.4
45.4
47.6
42.1
43.8
47.9
50.3
LongVA [40]
7B
128
61.1
61.6
50.4
53.6
46.2
47.6
52.6
54.3
ShareGPT4Video [59]
8B
16
48.3
53.6
36.3
39.3
35.0
37.9
39.9
43.6
SliME [87]
8B
8
53.3
55.4
42.7
44.4
39.8
41.7
45.3
47.2
Video-CCAM [84]
14B
96
62.1
63.9
52.8
55.9
47.0
48.3
53.9
56.1
InternVL-V1.5 [35]
20B
10
60.2
61.7
46.4
49.1
45.6
46.6
50.7
52.4
VITA [88]
8×\times7B
20
64.2
67.9
53.3
55.3
47.6
49.6
55.0
57.6
Kangaroo (Ours)
8B
64
66.1
68.0
55.3
55.4
46.7
49.3
56.0
57.6
TABLE V: Evaluation results on video-related tasks from SeedBench benchmark (*indicates results evaluated by self)
Model
Size
GVU
AR
AP
PU
Avg.
InternLM-XC [89]
7B
59.1
50.4
37.8
25.3
45.3
Qwen-VL-Plus [9]
7B
51.8
54.5
29.3
48.0
46.7
LLaVA-NeXT-IL* [30]
7B
62.2
59.6
53.0
32.4
53.7
Mantis-Idefics2* [90]
8B
60.7
57.0
46.0
37.5
51.9
SPHINX-v2 [91]
13B
52.4
41.2
33.9
26.1
40.0
VisionLLaMA [92]
13B
58.3
49.2
43.8
42.3
49.4
Claude-3-Opus [93]
UNK
48.8
37.1
32.1
41.3
40.2
GPT-4V [6]
UNK
64.5
65.7
51.7
63.4
61.7
Kangaroo (Ours)
8B
61.4
69.7
54.6
45.9
59.2
V-C Qualitative Results
We offer some qualitative illustrations in Figure 7. The questions are designed to assess the reasoning ability of our model. The results depict that Kangaroo is capable of responding accurately based on detailed visual elements in the video and providing explicit reasons, such as inferring camera angles, recognizing actions and speculating about counterfactual scenarios. Moreover, we present cases about multi-round and bilingual conversation in Figure 8. Kangaroo exhibits a deep comprehension of contextual contents in multi-round dialogues and supports transition between Chinese and English within the same conversation.
We also showcase some long video examples in Figure 9. The videos are selected from the long video benchmark MLVU, with durations of around 10 minutes and 30 minutes, respectively. Focusing the task for long videos, we specifically design questions to evaluate global and detailed comprehension of the video. The results demonstrate that Kangaroo is capable of grasping the overall content of long videos, such as identifying the category and scene of the video. Meanwhile, Kangaroo precisely locates the relevant video segments and responds based on specific segments from a long video.
VI Conclusion
In this paper, we present Kangaroo, a powerful Large Multi-modal Model that excels at long video understanding. We develop a data curation system and establish a large-scale video-based dataset. To improve the capacity for processing long videos, we adopt high-resolution inputs and extend the number of input frames to keep intact visual details. Extensive evaluation results demonstrate that Kangaroo achieves super