s and ground the answers on video clips. The prompts are in Appendix Fig. 12.
Generation of Image Description Questions.
The visual grounding question format may be out-of-distribution for some models. As a result, it may underestimate their performance. To fix this, we also generate questions in a complementary format: multiple-choice image description questions. These questions share the same Part 1 as visual grounding questions, but modify Part 2 to ask about visual details of the cause event.
With the cause video clip, the cause sentence, and the effect sentence as input, we prompt an LLM to generate a question that asks for an attribute of a visible object in the clip. This question follows the template: “When the event that causes ⟨Effect Event⟩\langle\texttt{Effect Event}\rangle occurs, ⟨Image Description Question⟩\langle\texttt{Image Description Question}\rangle.” For example, in the clip where Metropolis mourns Superman,
the LLM output could be: “When the battle that causes Superman’s death and leads to the memorial service occurs, what color were the energy blasts emanating from Doomsday?” We then further prompt the LLM to generate four challenging answer options based on the question.
However, the generated questions may reveal the bridge entity or include excessive object details, allowing the model to locate the cause clip without truly understanding the Part 1 question. For instance, the initially generated question reveals the bridge entity “Superman’s death” and the name “Doomsday”, which reduces the difficulty of locating the video clip. To mitigate this issue, we prompt the LLM to obscure the bridge entity and the named entity being inquired about.
We then obtain the final image description 2-needle question, which begins with the Part 1 question: “What tragedy was commemorated by the memorial service in Metropolis? When the tragedy that causes the memorial service occurs, what color were the energy blasts emanating from the creature? A. Yellow B. Blue C. Red D. White” The prompts are shown in Appendix Fig. 14 and Fig. 15.
3.4 Quality Evaluation of Generated Questions
We conduct automatic evaluation and human evaluation of the quality of questions and bridge entities, which provide vital information for 2-needle questions. We evaluate 4 quality factors. Factor 1 is if the bridge entity is truly shared by the cause and effect events it is supposed to connect. We report the proportion of affirmative answers as the final score. Factor 2 is the correctness of purposely vague references to the bridge entities, or if the vague reference is indeed more ambiguous than the original bridge entity but still preserves its core meaning. The evaluation result for each vague reference should be “Yes” or “No”. We report the proportion of “Yes”. Factor 3 is the factual correctness of questions, or the extent to which a question is consistent with the story and does not introduce hallucination or contradiction to the story. We use a 5-point scale, where 1 is the lowest score and 5 the highest. Factor 4 is the readability of questions, as reflected by the naturalness, grammar, and clarity, on a 5-point scale.
To verify that the evaluation LLMs are not biased to always indicate high quality, we also create several random baselines. For Factor 1, we randomly match the bridge entities and the cause-effect event pairs, which should cause the LLM to answer No to shared existence of the bridge entity. For Factor 2, we randomly shuffle the correspondence between the bridge entities and the vague references. For Factor 3, we randomly shuffle the correspondence between the questions and the cause-effect event pairs. For Factor 4, due to the difficulty in writing unreadable questions, we do not construct any random baseline for readability.
We utilize two state-of-the-art models as the evaluation LLMs, ChatGPT-4.1 and Gemini-2.0-flash (neither is involved in the question generation process), and recruit five human annotators. The models 