for Part 1 and Scene <number> for Part 2.’
Context: <Context>
The question has two parts:
Part 1: <Part1>
Part 2: <Part2>
.
Image Description 2-Needle Questions
You are given a movie context consisting of several sentences and a series of consecutive scenes from the movie.
Each scene is composed of five images stitched together from top to bottom.
You will be asked a two-part question.
The first part of the question is designed to help you identify the scene that contains the necessary information to answer the second part.
Using the correct scene, accurately answer Part 2 by selecting the most precise answer from the four given options.
Note: Please answer only the second part of the multiple-choice question.
Context: <Context>
Question: <Question>
Figure 18: The evaluation prompts for different types of questions.
Prompts and Outputs
Prompt 1
You are given a movie context and a sequence of consecutive movie scenes, each composed of 5 images stacked vertically.
Identify two scenes to answer the following question, which consists of two parts.
Note: First, indicate which scene contains the answer to Part 1, and then specify which scene contains the answer to Part 2.
Your answer format should be: ’Scene <number> for Part 1 and Scene <number> for Part 2.’
Context: <CONTEXT>
The question has two parts:
Part 1: <PART1>
Part 2: <PART2>
Output Samples:
Question: What advantage did Batman have when he initiated the brutal fight against Superman, and how was he able to gain such an advantage?
GT: Scene 7, Scene 3
Answer: Scene 1 for Part 1 and Scene 2 for Part 2.
Question: What event led to a memorial service being held in Metropolis, and how did such an event occur?
GT: Scene 5, Scene 2
Answer: Scene 1 for Part 1 and Scene 2 for Part 2.
.
Prompt 2
You are given a movie context and a sequence of consecutive movie scenes, each composed of 5 images stacked vertically.
Identify two scenes to answer the following question, which consists of two parts.
Note: First, indicate which scene contains the answer to Part 1, and then specify which scene contains the answer to Part 2.
Your answer format should be: ’Scene 7 for Part 1 and Scene 2 for Part 2.’
Output Samples:
Question: What advantage did Batman have when he initiated the brutal fight against Superman, and how was he able to gain such an advantage?
GT: Scene 7, Scene 3
Answer: Scene 7 for Part 1 and Scene 2 for Part 2.
Question: What event led to a memorial service being held in Metropolis, and how did such an event occur?
GT: Scene 5, Scene 2
Answer: Scene 7 for Part 1 and Scene 2 for Part 2.
Figure 19: Various test prompts, but the model fails to generate reasonable answers.
Prompts and Outputs
Prompt 3
You are given a movie context and a sequence of consecutive images, numbered starting from 1.
Your task is to identify two images to answer the following questions.
Note: First, indicate which image contains the answer to Part 1, and then specify which image contains the answer to Part 2.
Your answer format should be: ’Image No.<number> for Q1 and Image No.<number> for Q2.’
Output Samples:
Q: What advantage did Batman have when he initiated the brutal fight against Superman, and how was he able to gain such an advantage?
GT: Scene 7, Scene 3
Answer: Image No. 1 for Q1 and Image No. 2 for Q2.
.
Prompt 4
You are given a movie context and a sequence of consecutive images.
Your task is to identify the image that best matches each question.
Note: In your answer to each question, explain why you chose this image and specify which image it is in the sequence.
Output Sample (excerpt):
Q1: The image that best matches this question is Image No. 1. This image shows Batman holding a Kryptonite weapon, which is a significant advantage for him in the fight against Superman.
Q2: The image that best matches this question is Image No. 2. This image shows Batman wearing an armored exoskeleton, which allows him to withstand Superman’s attacks.
.
Prompt 5
You are given a movie context and a sequence of c