rity.
You can only output one number.
Question: <Question>
Score:
Figure 16: The prompt used for evaluating the quality of generated questions.
Cause-Effect Uniqueness Test Prompts
We present the prompts used to test the uniqueness of the generated questions with respect to multiple candidate cause sentences in Fig. 17
Model Performance Evaluation Prompts
We present the task instruction prompts used for testing the models in Fig. 18
For visual grounding 2-needle questions, there are two modes: Forward playback and Reverse playback.
This distinction is explicitly stated in the instruction prompt.
Various Instruction Templates and Model Output Examples
In the Sec. 4.3 of the main text, we observed that many open-source models exhibit severe output bias. Specifically, when testing the visual grounding 2-needle questions, these models tend to produce the same answer (“Scene 1 for part 1 and Scene 2 for part 2.") across different questions. To investigate whether this bias was caused by the prompt, we tested the models with different variations of the prompt. However, we still found that they continued to generate highly consistent responses. Below, in Fig.20, we list the most common outputs observed. Each type of response accounts for a significant proportion of the models’ answers across different questions. We found that although the model’s outputs varied slightly with different prompts, a significant portion of the responses remained completely identical.
Prompt for Uniqueness Test and Sentence Grounding
Uniqueness Test
You are given two questions, the second question is based on the first question.
You are also provided with the sentence containing the answer to the first question.
Please identify which sentence contains the answer to the second question.
Question 1: <Question 1>
Answer to Question 1: <Answer to Q1>
Question 2: <Question 2>
Answer Candidates to Question 2:
0: <Candidate 0>
1: <Candidate 1>
Index Number of the Answer:
.
Sentence Grounding
You are given a movie context and a question about it.
Identify two sentences to answer the following question, which consists of two parts.
Note: First, indicate which sentence contains the answer to Part 1, and then specify which sentence contains the answer to Part 2.
Your answer format should be: ‘Sentence <number> for Part 1 and Sentence <number> for Part 2.’
Context: <Context>
Question: <Question>
Figure 17: The prompt for testing the uniqueness.
Task Instruction Prompts
1-Needle Questions
You are given a movie context consisting of several sentences and a series of consecutive scenes from the movie.
Each scene is composed of five images stitched together from top to bottom.
Identify which scene contains the necessary clues to answer the following question.
Note: Please provide the index number of the scene (e.g., 1, 2, or 3) that contains the necessary information.
Context: <Context>
Question: <Question>
.
Visual Grounding 2-Needle Questions (Forward)
You are given a movie context and a sequence of consecutive movie scenes,
each composed of 5 images stacked vertically. Identify two scenes to answer the following question, which consists of two parts.
Visual Grounding 2-Needle Questions (Reverse)
You are given a movie context and a sequence of reverse-order consecutive movie scenes,
each composed of 5 images stacked vertically. Identify two scenes to answer the following question, which consists of two parts.
Note: First, indicate which scene contains the answer to Part 1, and then specify which scene contains the answer to Part 2.
Your answer format should be: ‘Scene <number> for Part 1 and Scene <number> for Part 2.’
Context: <Context>
The question has two parts:
Part 1: <Part1>
Part 2: <Part2>
.
Image Description 2-Needle Questions
You are given a movie context consisting of several sentences and a series of consecutive scenes from the movie.
Each scene is composed of five images stitched together from top to bottom.
You will be asked a two-part question.
The first p