ing open-weight ones, it is the strongest on instruction following (IFEval, Multi-IF, and IFBench) and AGIEval. We report separate scores for the subsequent steps of post-training in the following Section 5.3.2.
Apertus 1.5 70B improves over its predecessor on every benchmark and outperforms EuroLLM-22B on nearly all benchmarks, but trails the strongest open-weight contemporaries on most axes (Table 24).
We omit the judged Arena-Hard scores from this snapshot, as the judge model is unavailable in these runs; we report AIME with the thinking comparison in Table 25.
Table 23: Release evaluation of the 8B-class post-trained models, thinking disabled for all models including thinking-capable peers. All values in percent, higher is better, best per group in bold. Instruction following is scored by prompt-level strict accuracy and code by pass@1.
Fully open
Open-weight
Benchmark
Apertus 1.0
Apertus 1.5
OLMo 3
Marin
EuroLLM
Qwen3.5
Llama 3.1
Parameters
8B
8B
7B
8B
9B
9B
8B
Knowledge
MMLU
60.6
70.5
67.1
63.5
62.0
83.7
72.4
MMLU-Pro
35.8
48.8
52.7
41.0
38.5
76.0
48.4
AGIEval
38.5
45.4
37.1
36.4
41.7
39.0
38.0
ARC-Challenge
70.2
85.0
75.3
68.5
73.7
88.6
81.8
TruthfulQA
56.6
56.2
57.5
55.3
47.4
59.1
55.0
Instruction Following and Chat
IFEval
68.4
87.1
82.8
65.2
66.0
84.3
75.0
Multi-IF
66.8
83.7
70.5
59.6
56.4
80.0
66.5
IFBench
19.7
35.3
28.3
17.7
14.7
34.3
25.7
AlpacaEval (LC)
12.5
47.2
57.3
25.2
64.9
71.8
36.9
Mathematics
GSM8K
62.7
78.5
83.5
81.2
62.7
94.8
84.0
MATH
19.0
41.2
59.4
31.5
31.1
71.4
36.6
MATH-500
19.6
51.2
77.2
38.0
35.4
87.2
47.6
Code
HumanEval
41.4
68.8
81.6
51.2
38.8
88.3
65.1
MBPP
40.2
51.8
57.8
41.2
41.2
70.2
60.2
Reasoning and Reading
BBH
55.7
68.5
58.9
60.3
54.5
84.1
72.0
GPQA
27.9
27.2
31.5
25.9
28.3
57.6
26.6
DROP
44.7
54.2
55.5
60.1
45.8
67.9
60.4
Multilingual and Cultural
Global-MMLU
55.5
59.4
46.4
48.7
55.0
74.6
56.9
SwitzerlandQA
63.1
64.6
53.4
53.7
64.1
67.9
60.2
Table 24: Release evaluation of the 70B-class post-trained models, thinking disabled for all models including thinking-capable peers. All values in percent, higher is better, best per group in bold. Instruction following is scored by prompt-level strict accuracy and code by pass@1.
Fully open
Open-weight
Benchmark
Apertus
1.0
Apertus
1.5
OLMo
3.1
K2-V2
EuroLLM
Llama
3.3
Gemma
4
Qwen3.5
Nemotron
3.5
Parameters
70B
70B
32B
70B
22B
70B
31B
27B
30B-A3B
Knowledge
MMLU
70.1
76.5
80.1
88.1
69.0
87.6
90.5
89.5
85.0
MMLU-Pro
44.7
56.4
66.0
73.3
49.1
71.1
84.8
81.9
56.4
AGIEval
40.4
50.6
50.6
35.8
46.0
54.2
50.1
38.0
45.9
ARC-Challenge
79.7
87.8
89.0
54.4
80.0
93.5
96.8
93.3
91.7
TruthfulQA
61.9
67.0
68.8
57.9
56.3
66.2
64.8
64.1
58.3
Instruction Following and Chat
IFEval
65.1
87.8
86.7
31.1
71.2
90.2
90.9
88.0
75.6
Multi-IF
65.6
83.7
81.9
29.2
67.8
87.8
89.7
86.6
70.3
IFBench
20.7
25.7
38.0
25.7
19.0
41.3
50.3
45.3
31.0
AlpacaEval
(LC)
26.0
67.2
57.6
34.4
70.7
49.8
70.1
78.2
62.6
Mathematics
GSM8K
75.1
85.0
92.9
89.0
71.8
94.9
96.3
96.3
89.9
MATH
27.8
52.9
65.3
72.4
38.0
59.9
78.7
74.9
51.2
MATH-500
34.4
64.0
79.6
85.8
43.2
75.0
96.4
89.6
50.4
Code
HumanEval
46.6
70.3
86.9
74.5
51.6
85.0
92.0
96.2
55.5
MBPP
44.8
61.4
68.8
64.0
47.4
75.4
69.6
80.0
74.6
Reasoning and Reading
BBH
64.1
66.0
63.2
85.2
60.0
87.5
92.0
93.3
83.3
GPQA
28.1
32.1
35.3
54.2
32.1
58.5
72.1
61.8
45.1
DROP
44.3
60.7
71.2
56.4
52.1
70.5
72.6
82.4
70.4
Multilingual and Cultural
Global-MMLU
62.9
66.5
65.0
77.7
58.2
77.7
84.7
83.0
69.6
SwitzerlandQA
67.3
69.8
61.4
71.2
66.3
74.2
77.8
74.3
68.0
5.3.1 Reasoning and Thinking Mode
Thinking is a request-time flag of the chat template rather than a separate model, and we evaluate every checkpoint with thinking on and off, at a 32,768-token thinking budget. For small benchmarks with a high run-to-run variance, like AIME, we report mean@k metrics across multiple seeds (e.g., with k=8k=8 for AIME). Table 25 compares the Apertus models in both modes with thinking-native peers in their thinking mode. We compare to OLMo 3 7B Think (Team Olmo et al., 2025),