79.8
86.4
84.7
86.7
84.8
88.2
85.4
MATH
51.6
55.8
59.3
60.3
56.6
55.2
59.7
Minerva MATH
60.9
66.2
73.0
74.3
67.0
65.9
74.0
MATH-500
61.2
66.2
70.4
69.6
66.6
66.4
74.0
GSM8K-Platinum†
84.3
90.2
91.1
89.2
82.5
90.8
88.5
MathQA
45.8
46.0
46.3
46.4
48.1
47.9
48.4
AIME 24
3.3
0.0
16.7
23.3
3.3
20.0
26.7
AIME 25
16.7
10.0
23.3
20.0
10.0
13.3
30.0
Coding
HumanEval
51.4
38.0
32.6
26.2
65.0
37.5
59.7
MBPP
49.6
51.4
51.6
52.0
55.6
58.2
62.2
Multilingual
MGSM
52.3
65.1
68.6
68.0
60.0
64.7
66.0
mLogiQA†
40.0
32.8
32.1
31.5
42.2
33.4
44.4
Average (development)
50.0
50.8
55.0
55.1
53.7
54.7
61.3
Average (held-out)†
46.8
47.9
49.5
49.3
47.1
51.4
54.3
5.4 Multilingual and Cultural Evaluation
Multilinguality is where Apertus stands strongest relative to its peers. The 13-point gap to Qwen3.5-9B on English knowledge benchmarks (Table 23) shrinks to two or three points on the multilingual and cultural slice (Table 29); Apertus 1.5 8B is the strongest of all compared models on multilingual ARC and Multi-IF and among the strongest on multilingual TruthfulQA, and its lead over the fully open group is wide. Thinking transfers across languages, lifting Global-MMLU by eight points. The clearest weakness is XNLI, which regresses against Apertus 1.0 at both scales; Apertus 1.5 70B improves on its predecessor everywhere else (Table 30) but trails the strongest open-weight contemporaries.
Table 29: Multilingual and cultural evaluation of the 8B-class post-trained models, thinking disabled. All values in percent, higher is better, best per group in bold. Global-MMLU, Multi-IF, and SwitzerlandQA repeat from Table 23 for completeness. Multiple-choice benchmarks report accuracy, length-normalized for multilingual ARC.
Fully open
Open-weight
Benchmark
Apertus 1.0
Apertus 1.5
OLMo 3
Marin
EuroLLM
Qwen3.5
Llama 3.1
Parameters
8B
8B
7B
8B
9B
9B
8B
Multilingual Knowledge
Global-MMLU
55.5
59.4
46.4
48.7
55.0
74.6
56.9
INCLUDE-44
54.1
56.9
38.4
38.6
51.5
70.9
53.2
INCLUDE-45
38.9
39.5
32.1
34.6
34.4
44.5
33.8
Mult. TruthfulQA
52.1
51.7
50.6
47.6
47.2
52.0
50.8
Mult. ARC
39.3
42.2
29.8
29.5
36.5
39.2
35.6
Multilingual Reasoning and Instruction Following
MGSM
47.0
66.2
48.5
42.9
38.8
67.8
67.8
mLogiQA
28.9
35.2
36.8
32.6
37.0
62.3
41.2
XNLI
47.0
43.8
39.3
40.4
42.4
43.9
43.4
Multi-IF
66.8
83.7
70.5
59.6
56.4
80.0
66.5
Cultural Knowledge
BLEnD
62.9
67.3
59.6
62.0
66.5
69.5
67.3
CulturalBench
50.9
60.5
62.1
69.4
46.7
69.1
68.2
SwitzerlandQA
63.1
64.6
53.4
53.7
64.1
67.9
60.2
Table 30: Multilingual and cultural evaluation of the 70B-class post-trained models, thinking disabled. All values in percent, higher is better, best per group in bold. Multiple-choice benchmarks report accuracy, length-normalized for multilingual ARC.
Fully open
Open-weight
Benchmark
Apertus
1.0
Apertus
1.5
OLMo
3.1
K2
EuroLLM
Llama
3.3
Gemma
4
Qwen3.5
Nemotron
Parameters
70B
70B
32B
70B
22B
70B
31B
27B
30B-A3B
Multilingual Knowledge
Global-MMLU
62.9
66.5
65.0
77.7
58.2
77.7
84.7
83.0
69.6
INCLUDE-44
58.3
61.6
51.6
66.6
55.3
71.7
78.9
77.1
56.7
INCLUDE-45
40.0
41.8
37.9
45.3
37.8
45.9
54.6
50.8
39.7
Mult. TruthfulQA
54.2
56.9
53.8
49.5
48.9
55.2
56.3
55.5
52.5
Mult. ARC
39.5
43.5
38.4
32.2
37.8
46.0
39.5
42.1
39.7
Multilingual Reasoning and Instruction Following
MGSM
62.2
67.7
67.3
61.9
50.0
85.5
73.7
73.9
74.4
mLogiQA
40.2
44.1
50.5
46.8
40.4
57.5
74.1
68.5
49.9
XNLI
47.1
42.3
40.9
42.9
43.3
42.2
40.5
44.4
43.6
Multi-IF
65.6
83.7
81.9
29.2
67.8
87.8
89.7
86.6
70.3
Cultural Knowledge
BLEnD
65.3
69.8
66.0
22.8
68.0
75.1
80.1
75.1
71.1
CulturalBench
60.1
68.9
70.3
34.4
58.4
82.6
69.7
77.1
44.1
SwitzerlandQA
67.3
69.8
61.4
71.2
66.3
74.2
77.8
74.3
68.0
Table 31: Automatic evaluation of the SwissGov translation tasks with COMETKiwi22.
5.4.1 Machine Translation
Being able to translate well is a major sign of a model’s multilingual capability, and machine translation remains a challenging NLP task (Zouhar et al., 2026a). To this end, we evaluate several comparable models across many languages on WMT24++ 