e a sentence containing two questions. Please complete this goal step by step, as shown in the examples. (Please follow the output format shown in the example.)
Example Input:
Cause: Feeling responsible, Harry Hart, known as Galahad, gives a medal for valor to Lee’s widow, Michelle, and his young son, Eggsy, instructing them to call the number on the medal if they ever need help.
Effect: When Eggsy is arrested for stealing a car, he calls the number on the medal for help.
Example Output:
Step 1:
Instruction: Identify a piece of shared information that appears in both “Cause” and “Effect” sentences and establishes their causal relationship.
Output: The shared information is ‘call the number on the medal’.
Step 2:
Instruction: Create a question that uses the “Effect” sentence as context and shared information as the answer. Rephrase the shared information to be less specific.
Output: First, according to the context, ‘call the number on the medal’ can be rephrased into ‘an approach’. Then, the question is ‘What approach did Eggsy take to seek help when he was arrested for stealing a car?’
Step 3:
Instruction: Based on the previous question, create another question using the “Cause” sentence. The answer should be the shared information.
Output: The question is ‘Why was he able to adopt such an approach?’
Step 4:
Instruction: Combine the two questions into one sentence.
Output: What approach did Eggsy take to seek help when he was arrested for stealing a car, and why was he able to adopt such an approach?
Input:
Cause: <Rephrased Cause Sentence>
Effect: <Rephrased Effect Sentence>
Output:
Figure 12: The prompt for generating visual grounding 2-needle questions (continued).
Image Description 2-Needle Question Generation (Step 1-2)
(Gemini-1.5-pro-002)
Step 1:
You are given a question and its corresponding context. Your task is to concisely identify the following:
The question asks for specific information about an event. Identify this event (which should match the effect event mentioned in
the context) and summarize it in a short phrase as Effect Event.
Then, in the cause sentence, identify the cause event that leads to the effect event found in the first step, and summarize it in a
short phrase as Cause Event.
Output:
Cause Event: <cause event>
Effect Event: <effect event>
Input:
Context: <context>
Question: <part1>
Step 2:
You are given a context that can answer a two-part question concerning the same concept (the “bridge entity")—though it may be
referred to as a “mission," “event," “approach," or another term in the question. Identify the bridge entity in the two-part question and
clarify it based on the context. It often follows “what" or “which".
Example:
Question: What event caused Bryant to order Deckard to retire the replicants, and what is Deckard’s profession that relates to such an
event?
Context: Deckard, a Blade Runner tasked with "retiring" replicants, is informed that four—Leon, Roy Batty, Zhora, and Pris—have
illegally arrived on Earth. Due to the arrival of the four replicants (Leon, Roy Batty, Zhora, and Pris), Bryant orders Deckard to retire them.
Output:
Bridge Entity: the event
Reference: arrival of the four replicants
Figure 13: The prompt for generating image description 2-needle questions.
Image Description 2-Needle Question Generation (Step 3-4)
(Gemini-1.5-pro-002)
Step 3:
You are given five images. Your task is to generate a specific visual question with an answer that fits the following format:
“When the event that causes <effect event> occurs, <visual question>? Answer: <a short answer to the visual question>."
Guidelines:
- Ensure the visual question is specific—it should focus on aspects such as environment, object attributes, facial expressions, or clothing.
- Avoid vague questions like “What is the significant object?"
- Avoid well-known facts such as “What is the color of Superman’s suit?"
- Avoid overly obvious questions like “What is the doctor wearing?" if the answer is trivial.
- Instead, ask d