th BBQ rising 12.4 points at 70B and ToxiGen 8.6 at 8B. SDPO is deliberately narrow, moving charter alignment by 9.9 points and AlpacaEval by 10.0, and leaving the remaining benchmarks within a few points.
Two effects run against the trend. Reinforcement learning shortens responses at 70B scale, from 367 to 253 tokens on average over the three chat benchmarks, and offline DPO raises them to 495, the behavior that motivates the offline stage of Section 4.6. The alignment stages also make both models markedly more cautious: refusals on benign prompts that resemble harmful ones climb sharply through the alignment stages (Section 5.9.2).
Table 26: Post-training lineage of the Apertus 1.5 models on the development benchmark suite, thinking disabled. Each column adds one stage to the one before it. A representative subset of each skill area is shown; Table L.3 gives all 37 benchmarks, and the overall average is taken over all of them. Best value per row in bold, taken separately within each model.
Apertus 1.5 8B
Apertus 1.5 70B
Benchmark
SFT
+RL
+DPO
+SDPO
SFT
+RL
+DPO
Knowledge and Reasoning
MMLU
65.8
66.6
70.7
70.6
75.1
72.0
77.2
MMLU-Pro
40.1
43.0
49.4
49.2
46.3
44.4
56.9
BBH
67.9
69.1
71.1
68.2
72.4
75.2
66.7
ACPBench
26.1
31.2
37.9
37.1
38.3
49.7
63.6
DROP
53.2
56.0
55.2
54.1
57.7
57.8
60.6
Instruction Following and Chat
IFEval
80.2
85.8
89.6
90.0
83.5
90.8
89.8
Multi-IF
74.0
83.0
84.5
85.0
78.0
86.3
85.7
AlpacaEval
20.0
20.6
32.7
42.7
22.6
25.9
59.4
Math
GSM8K
77.0
79.5
80.7
78.6
78.6
84.7
84.8
MATH
35.2
39.7
44.6
41.4
38.0
45.8
52.8
MATH-500
43.2
47.0
54.2
50.2
44.2
56.2
63.2
Coding
HumanEval
85.3
86.0
83.2
85.5
89.6
91.8
88.6
MBPP
48.6
51.6
51.2
52.6
54.2
56.4
62.2
Multilingual and Cultural
Global-MMLU
55.8
55.4
58.9
59.3
64.0
61.7
66.6
MGSM
53.3
57.5
66.7
66.2
59.8
70.3
67.7
INCLUDE-44
56.9
56.7
56.6
56.9
60.4
59.9
61.6
SwitzerlandQA
65.7
64.4
64.6
64.6
68.5
67.6
69.8
Safety and Alignment
BBQ
62.9
67.4
67.8
67.6
65.7
68.1
80.5
ToxiGen
53.8
73.3
81.9
81.7
77.7
83.8
86.8
WMDP
51.7
50.7
53.4
53.4
54.6
56.1
56.9
Charter Alignment
33.4
34.3
56.8
66.7
41.7
50.9
72.2
Tool Use
BFCL v3
32.2
39.8
52.9
48.2
43.3
43.5
56.0
Overall Average
47.3
49.3
54.8
55.0
52.2
55.3
61.2
Table 27: Held-out benchmarks, excluded from development, for both models, thinking disabled. All six held-out benchmarks are shown. Best value per row in bold, taken separately within each model.
Apertus 1.5 8B
Apertus 1.5 70B
Benchmark
SFT
+RL
+DPO
+SDPO
SFT
+RL
+DPO
Knowledge and Reasoning
AGIEval
33.1
34.9
44.9
45.5
38.1
49.1
50.5
ARC-Challenge
78.4
84.0
84.9
84.8
85.1
86.9
87.3
GPQA
29.5
29.7
31.2
26.8
32.8
32.1
31.2
Math
GSM8K-Platinum
76.5
78.0
79.6
78.2
84.6
87.1
83.3
Multilingual
mLogiQA
35.4
37.5
35.2
35.1
41.5
39.0
44.5
Mult. ARC
36.5
37.1
39.8
39.6
37.1
40.2
41.8
Overall Average
48.2
50.2
52.6
51.7
53.2
55.7
56.4
Table 28: Post-training lineage with thinking enabled, on all 19 benchmarks evaluated in thinking mode. Benchmarks marked † are held out from development. The two averages cover the development and held-out benchmarks respectively, and neither is comparable with Table 26, which spans a wider suite. Best value per row in bold, taken separately within each model.
Apertus 1.5 8B
Apertus 1.5 70B
Benchmark
SFT
+RL
+DPO
+SDPO
SFT
+RL
+DPO
Knowledge and Reasoning
MMLU
69.9
70.7
74.2
73.2
73.4
76.6
78.6
MMLU-Pro
46.5
48.4
50.5
50.8
51.5
46.7
57.7
AGIEval†
33.2
35.4
42.9
44.4
34.0
48.7
47.0
BBH
67.9
68.1
70.4
71.0
72.0
75.4
65.1
ACPBench
40.1
33.8
48.6
49.4
37.8
50.2
70.6
DROP
53.3
55.4
55.5
55.4
53.4
54.6
61.1
GPQA†
29.9
33.0
31.7
32.1
29.7
32.8
37.3
Math
GSM8K
79.8
86.4
84.7
86.7
84.8
88.2
85.4
MATH
51.6
55.8
59.3
60.3
56.6
55.2
59.7
Minerva MATH
60.9
66.2
73.0
74.3
67.0
65.9
74.0
MATH-500
61.2
66.2
70.4
69.6
66.6
66.4
74.0
GSM8K-Platinum†
84.3
90.2
91.1
89.2
82.5
90.8
88.5
MathQA
45.8
46.0
46.3
46.4
48.1
47.9
48.4
AIME 24
3.3
0.0
16.7
23.3
3.3
20.0
26.7
AIME 25
16.7
10.0
23.3
20.0
10.0
13.3
30.0
Coding
HumanEval
51.4
38.0
32.6
26.2
65.0
37.5
59.7
MBPP
