n3.5
Nemotron
3.5
Parameters
70B
70B
32B
22B
32B
70B
27B
30B-A3B
HellaSwag
83.0
82.2
81.5
79.0
85.0
83.7
83.4
63.7
PIQA
83.0
82.3
81.3
81.4
83.3
83.9
82.3
83.1
CommonsenseQA
53.8
66.8
71.1
60.5
75.5
73.3
87.4
77.7
LAMBADA
72.9
72.9
70.5
70.9
68.7
76.1
74.2
72.8
XNLI
45.9
46.3
42.4
44.1
42.3
44.1
45.3
43.4
XCOPA
69.9
68.0
59.5
61.3
58.6
63.8
67.3
62.1
Avg
68.1
69.8
67.7
66.2
68.9
70.8
73.3
67.1
Table 15: Pretraining Evaluation (8B-class models). Performance (%) on factual knowledge tasks, higher is better. Bold = best overall; italic = best among fully open models.
Fully open
Open-weight
Benchmark
Apertus-8B
Apertus 1.5 8B
OLMo 3
Marin
Qwen3.5
Llama 3.1
Parameters
8B
8B
7B
8B
9B
8B
MMLU CoT
59.3
62.9
59.4
62.6
76.7
60.7
Global-MMLU
54.4
56.0
44.3
48.5
77.1
52.7
MMLU-Pro
29.6
38.5
33.0
30.9
58.4
36.1
AGIEval
36.1
37.2
35.0
37.3
58.0
35.5
SQuADv2
32.1
17.6
36.9
24.5
38.7
23.6
TruthfulQA
51.3
49.0
46.9
53.7
51.8
44.2
Avg
43.8
43.5
42.6
42.9
60.1
42.1
Table 16: Pretraining Evaluation (70B-class models). Performance (%) on factual knowledge tasks, higher is better. Bold = best overall; italic = best among fully open models.
Fully open
Open-weight
Benchmark
Apertus
1.0
Apertus
1.5
OLMo 3
EuroLLM
Marin
K2-V2
Qwen3.5
Nemotron
3.5
Parameters
70B
70B
32B
22B
32B
70B
27B
30B-A3B
MMLU CoT
61.8
67.0
71.4
62.9
71.6
70.5
84.9
75.4
Global-MMLU
56.3
59.3
49.1
54.9
54.2
56.1
78.6
69.4
MMLU-Pro
36.0
43.8
48.0
36.8
45.3
45.4
78.0
67.2
AGIEval
41.2
43.9
46.6
40.1
43.9
40.4
62.7
50.5
SQuADv2
18.8
19.1
39.1
35.6
30.1
22.8
43.5
30.1
TruthfulQA
52.5
50.8
54.9
49.3
53.3
54.9
58.3
55.5
Avg
44.4
47.3
51.5
46.6
49.7
48.4
67.7
58.0
Table 17: Pretraining Evaluation (8B-class models). Performance (%) on reasoning, mathematics, and code tasks, higher is better. GSM8K-P = GSM8K Platinum; Minerva = Minerva MATH; MGSM-En = MGSM English. Bold = best overall; italic = best among fully open models.
Fully open
Open-weight
Benchmark
Apertus 1.0
Apertus 1.5
OLMo 3
Marin
Qwen3.5
Llama 3.1
Parameters
8B
8B
7B
8B
9B
8B
Reasoning
BBH
48.8
60.2
63.9
55.8
84.4
65.2
ACP
22.1
28.1
31.8
32.0
52.7
35.1
Mathematics
GSM8K
47.2
66.2
67.6
60.4
87.0
50.5
GSM8K-P
48.2
67.8
71.1
63.6
89.9
52.4
Minerva
14.0
29.6
40.4
24.6
50.4
19.7
MGSM-En
5.2
40.1
31.5
12.9
55.3
16.5
Code
HumanEval
48.6
81.8
81.4
58.5
93.5
75.7
MBPP
37.6
50.6
50.6
39.2
62.2
48.6
Avg
34.0
53.1
54.8
43.4
71.9
45.5
Table 18: Pretraining Evaluation (70B-class models). Performance (%) on reasoning, mathematics, and code tasks, higher is better. GSM8K-P = GSM8K Platinum; Minerva = Minerva MATH; MGSM-En = MGSM English. Bold = best overall; italic = best among fully open models.
Fully open
Open-weight
Benchmark
Apertus
1.0
Apertus
1.5
OLMo 3
EuroLLM
Marin
K2-V2
Qwen3.5
Nemotron
3.5
Parameters
70B
70B
32B
22B
32B
70B
27B
30B-A3B
Reasoning
BBH
58.5
68.2
78.8
59.4
71.4
69.3
91.7
77.9
ACP
32.8
32.0
41.7
34.4
40.6
41.6
77.0
47.1
Mathematics
GSM8K
54.0
70.7
80.9
55.8
69.8
68.6
87.3
85.4
GSM8K-P
58.0
72.9
82.7
59.4
70.6
70.1
91.2
88.2
Minerva
17.6
32.7
46.1
24.8
38.1
28.9
72.0
–
MGSM-En
21.6
49.2
51.2
46.4
37.8
36.7
69.0
77.4
Code
HumanEval
55.9
84.6
92.3
66.8
78.4
79.6
98.7
81.6
MBPP
42.4
56.4
66.6
43.8
55.8
59.0
75.8
74.4
Avg
42.6
58.3
67.5
48.9
57.8
56.7
82.8
76.0
Table 19: Pretraining Evaluation (8B-class models). Performance (%) on multilingual tasks (“-ml” = multilingual variant), higher is better. Bold = best overall; italic = best among fully open models.
Fully open
Open-weight
Benchmark
Apertus 1.0
Apertus 1.5
OLMo 3
Marin
Qwen3.5
Llama 3.1
Parameters
8B
8B
7B
8B
9B
8B
XNLI
44.1
45.3
39.6
42.4
39.8
45.6
XCOPA
65.6
64.9
55.1
57.0
64.1
62.5
ARC-ml
38.4
37.5
25.5
26.4
37.9
33.1
HellaSwag-ml
55.0
53.6
37.4
42.1
51.9
49.1
LAMBADA-ml
52.2
48.9
40.2
44.7
50.3
49.6
TruthfulQA-ml
46.3
46.7
45.4
47.2
46.0
45.0
Avg
50.3
49.5
40.5
43.3
48.3
47.5
Table 20: Pretraining Evaluation (70B-class models). Performance (%) on multilingual tasks (“-ml” = multilingual variant), higher is better. Bold = best overall; italic = best among fully open m