models, on scores with prompt and model effects removed via equation 1.
Helpfulness
Honesty
Instr. Following
Truthfulness
Helpfulness
–
0.710
0.748
0.644
Honesty
0.710
–
0.680
0.745
Instruction Following
0.748
0.680
–
0.641
Truthfulness
0.644
0.745
0.641
–
Average
0.701
0.712
0.690
0.677
To identify which aspect to keep, we compare all four on three external judging benchmarks: JudgeBench (GPT split) (Tan et al., 2025), RewardBench 2 (Malik et al., 2026), and RM-Bench (Liu et al., 2025b). Each pairs a prompt with a labeled-preferred response and one or more labeled-rejected responses, and scores a judge by how often it ranks the preferred response above the rejected one (Table J.9).
Table J.9: Accuracy with Qwen3-235B-A22B-Instruct-2507 and different aspects.
Helpfulness
Honesty
Instruction Following
Truthfulness
JudgeBench
MMLU-Pro
73.4
70.1
71.4
71.4
LiveBench Reasoning
69.4
67.3
71.4
69.4
LiveBench Math
83.9
85.7
83.9
82.1
LiveCodeBench
83.3
78.6
76.2
81.0
Macro Average
77.5
75.4
75.7
76.0
RewardBench 2
Factuality
80.4
83.0
79.8
81.9
Focus
87.5
79.2
86.9
76.2
Math
74.9
72.1
76.5
72.1
Precise IF
43.1
46.2
40.6
42.5
Safety
94.7
97.3
94.7
96.0
Ties
76.0
78.1
76.1
86.5
Macro Average
76.1
76.0
75.8
75.9
RM-Bench
Chat
72.6
80.9
76.2
83.3
Math
77.3
75.5
77.3
76.3
Code
74.3
74.4
75.0
75.9
Safety
93.2
92.2
92.8
91.0
Hard
67.9
74.6
71.5
80.4
Normal
81.7
82.2
81.7
82.3
Easy
88.5
85.5
87.8
82.2
Macro Average
79.3
80.8
80.3
81.6
Helpfulness is the strongest individual aspect on JudgeBench and RewardBench 2; Truthfulness scores highest on RM-Bench and, by a similarly thin margin, on the three-benchmark average. Since Truthfulness is the least correlated with the other three (Table J.8) and thus least representative of the full rubric, we choose the runner-up, Helpfulness, as our single judging aspect.
J.2.2 Choice of Judge Model
Having fixed Helpfulness as the scoring aspect, we test whether smaller, newer LLMs that fit on a single node can substitute for the resource-intensive Qwen3-235B-A22B-Instruct-2507, which requires four nodes, reusing the same three benchmarks as before.
Table J.10shows Qwen3.6-27B matches or exceeds Qwen3-235B-A22B-Instruct-2507 on all three benchmarks; the largest gap is on RM-Bench’s Hard split, which specifically tests robustness to response style over substance. Qwen3.5-35B-A3B is the weakest judge on RewardBench 2 and RM-Bench, trailing Qwen3-235B-A22B-Instruct-2507 by 10 points on RewardBench 2 from a collapse on the Ties category (44.0% versus 76.0%). We therefore choose Qwen3.6-27B as the online DPO judge model, for both performance and cost.
Table J.10: Accuracy of the Helpfulness judge with different LLMs on three benchmarks.
Qwen3-235B-
A22B-Instruct-2507
Qwen3.5-
35B-A3B
Qwen3.6-27B
JudgeBench
MMLU-Pro
73.4
75.3
80.5
LiveBench Reasoning
69.4
71.4
75.5
LiveBench Math
83.9
83.9
89.3
LiveCodeBench
83.3
85.7
90.5
Macro Average
77.5
79.1
84.0
RewardBench 2
Factuality
80.4
68.0
77.9
Focus
87.5
77.0
81.4
Math
74.9
71.0
75.4
Precise IF
43.1
41.2
45.6
Safety
94.7
91.6
91.8
Ties
76.0
44.0
87.1
Macro Average
76.1
65.5
76.5
RM-Bench
Chat
72.6
67.8
77.0
Math
77.3
79.3
84.0
Code
74.3
73.4
78.6
Safety
93.2
92.1
93.7
Hard
67.9
69.2
80.9
Normal
81.7
79.2
83.7
Easy
88.5
86.0
85.4
Macro Average
79.3
78.1
83.3
Appendix K Chatbot System Prompt
The chat template ships with a default system prompt, given in Appendix K, used verbatim whenever a request supplies no system message; the deployed chatbot relies on this default.
Default Chatbot System Prompt
You are Apertus 1.5 Omni, a multimodal assistant developed by the Swiss AI Initiative. Extended from Apertus 1 via continued pretraining, you understand images and audio and respond in text.
Independent of the system message, the template additionally emits a developer message that states whether the thinking mode is enabled and declares the available tools (Section 4.1).
Appendix L Evaluation Details
We report additional evaluation details to enable the replication of resul