
47.2
44.5
37.3
–
Apertus 1.0 70B
79.8
78.1
71.6
57.3
–
–
48.5
47.2
44.4
37.7
–
Apertus 1.5 8B
85.9
81.3
76.1
67.5
61.5
54.8
45.1
42.1
40.7
37.6
36.5
Apertus 1.5 70B
86.8
85.5
78.8
71.9
61.8
51.0
49.6
49.0
44.2
41.1
37.8
OLMo 3 7B
85.5
77.8
70.9
62.4
–
–
48.7
47.0
44.6
39.7
–
OLMo 3 32B
92.7
89.4
83.9
76.3
–
–
52.7
50.8
48.8
46.2
–
Open-weight
Gemma-3-12B
91.3
90.0
89.4
85.1
78.8
–
50.0
49.3
47.5
46.6
44.1
Qwen3-8B
92.5
91.1
86.6
–
–
–
52.8
50.9
48.3
–
–
Post-trained models
Fully open
Apertus 1.0 8B Instruct
88.3
80.1
65.3
59.6
–
–
44.7
34.3
27.0
23.8
–
Apertus 1.0 70B Instruct
88.5
85.6
80.6
69.4
–
–
51.4
46.4
39.4
30.9
–
Apertus 1.5 8B after SFT
83.1
77.6
56.9
40.5
18.9
8.3
41.0
33.9
23.4
15.8
8.7
after long-context SFT
86.8
82.6
76.3
65.9
59.5
44.9
42.9
36.7
29.2
23.8
21.1
after RL
87.1
82.8
78.6
67.1
60.1
48.8
41.1
37.1
29.1
24.1
19.7
after DPO
88.2
84.0
80.0
69.2
61.5
50.4
48.5
46.0
36.1
28.5
23.1
after SDPO
87.4
85.0
80.7
68.3
61.1
50.5
48.3
45.9
36.3
28.7
23.1
Apertus 1.5 70B after SFT
90.9
88.5
79.3
70.5
54.1
12.0
46.6
42.8
38.6
34.4
28.7
after long-context SFT
90.7
89.7
82.1
75.4
65.2
43.9
48.7
45.8
42.1
37.5
35.4
after RL
90.5
89.1
82.2
75.6
66.0
45.0
51.7
47.8
42.5
36.5
34.7
after DPO
91.7
90.8
85.3
77.0
65.9
45.6
56.1
54.7
50.0
45.3
42.1
OLMo 3 7B Instruct
58.3
45.9
36.3
26.1
–
–
39.6
35.3
32.1
26.8
–
OLMo 3.1 32B Instruct
91.3
86.9
80.2
61.4
–
–
50.1
44.1
42.3
39.5
–
Open-weight
Gemma-3-12B
92.6
87.7
80.0
70.7
60.9
–
56.2
51.8
44.4
38.2
33.0
Gemma-3-27B
94.2
93.3
91.4
83.7
65.5
–
58.7
57.0
51.9
45.8
38.7
Llama-3.1-8B
93.8
93.4
87.3
84.6
78.8
–
45.4
43.2
42.2
42.4
33.4
Llama-3.3-70B
95.4
95.2
94.1
84.7
56.6
–
59.2
59.2
57.4
51.3
38.0
Qwen3.5-9B
96.1
95.9
96.0
95.8
94.1
93.6
59.7
59.9
59.2
56.9
55.4
Qwen3.5-27B
96.7
96.6
96.5
96.3
94.9
94.8
63.7
64.8
65.2
64.7
64.9
5.8 Tool Use
We evaluate function calling on the single-turn categories of BFCL v3, with the parser matched to the call format of the chat template (Table 37); the official leaderboard additionally includes multi-turn categories, so these numbers are not comparable to published entries. Both models improve over their predecessors, the 70B most visibly: its overall accuracy rises from 58.0 to 70.1, within 1.5 points of Llama-3.3-70B, and it repairs the parallel-call collapse of Apertus 1.0. The 8B remains weak on parallel calls, a known release limitation, but nearly doubles its predecessor’s score on the irrelevance categories, where the declared tools do not fit the request and the model should answer in plain text instead of calling one. This is the intended effect of a supervised-finetuning design choice: tool declarations are also injected into conversations that call no tool (Section 4.2). At launch, providers expose tool calling and thinking as mutually exclusive, a serving limitation discussed with the chat template.
Table 37: BFCL v3 function calling by call type, over the single-turn categories alone. All values in percent, higher is better, best per group in bold. The irrelevance categories reward declining to call a tool when none applies.
Fully open
Open-weight
Apertus
Category
1.0
1.0
1.5
1.5
OLMo 3.1
EuroLLM
Qwen3.5
Llama 3.1
Llama 3.3
Parameters
8B
70B
8B
70B
32B
9B
9B
8B
70B
Overall
56.4
58.0
61.2
70.1
73.8
31.1
69.1
64.5
71.6
Single Calls
Simple
90.3
91.8
78.8
93.3
85.3
79.8
87.0
94.3
94.3
Multiple
89.5
91.5
79.0
92.0
86.0
83.5
93.0
93.5
94.5
Live simple
61.6
72.9
60.9
79.8
62.8
52.7
69.0
75.2
82.2
Live multiple
56.4
64.0
47.3
72.1
57.7
48.8
68.8
70.1
78.6
Parallel Calls
Parallel
80.0
24.0
64.0
87.5
81.5
0.5
87.0
89.0
88.5
Parallel multiple
71.0
9.5
55.5
82.0
74.0
0.0
88.0
88.5
89.5
Live parallel
56.3
12.5
31.3
81.3
50.0
0.0
81.3
43.8
68.8
Tool Rejection
Irrelevance
36.3
50.0
69.6
67.9
94.2
41.3
60.4
44.2
65.4
Live irrelevance
29.0
46.7
66.4
45.7
85.3
32.8
51.9
30.5
39.5
5.9 Security, Safety, and Interpretability
As Apertus is a fully open and massively multilingual foundation model, the safety evaluations of the Apertus family present a 