ded pool in Table C.5; Table 8 gives its per-model training counts.
4.2.3 Variety-Aware Translation Case Study
Generating text in the correct variety of a multi-variety language can be a challenge for language models, especially when some varieties are underrepresented in the training data. Hernández-Cano et al. (2026) studied this problem for Romansh, a Swiss national language with six distinct varieties. While Apertus 1.0 outperformed comparable models at translation from German into Romansh, it still struggled to adhere to the distinct varieties of the language. We perform a case study to investigate how a small amount of variety-aware post-training data affects this behavior.
We follow a back-translation approach (Sennrich et al., 2016). We translate a sample of Romansh text into German with machine translation, and during supervised finetuning we train Apertus to translate the German text back into the specific Romansh variety. The monolingual Romansh text comes from the swiss-ai/apertus-pretrain-romansh collection released with Apertus 1.0 (Hernández-Cano et al., 2026). We identify the variety of each text with the language identification system of Model et al. (2026), and translate into German with the machine translation system of Vamvas et al. (2026). We include 5,000 samples per variety in the finetuning mixture, and release the SFT dataset at swiss-ai/apertus-pretrain-romansh-backtranslated-sft.
Table 32reports the effect on translation from German into the six Romansh varieties, and the human evaluation in Table 33 additionally covers German into Sursilvan.
4.3 Long-Context Supervised Finetuning
Table 8: Long-context SFT data mixture. The mix each model’s run trains on, with tokens and samples per source; the runs draw subsets of the pools, so the totals stay below the pool sizes. Length range is the observed range from minimum to maximum sequence length within the class. The short-context text and audio rows are length-filtered slices of the production pools of Section 4.2; all three vision rows (short- and long-context) are length buckets of the same pool in Table B.6. Those tables give the per-dataset composition.
8B
70B
Dataset
Tokens
Samples
Tokens
Samples
Text, 26–108K
SFT mixture subset (Table 6)
1,870,000,000
1,119,142
2,410,000,000
1,438,897
Tool (short-context), 119–16.4K
Toucan (Xu et al., 2026)
33,700,000
15,162
43,400,000
19,494
EnvScaler (Song et al., 2026)
9,300,000
763
11,900,000
981
OpenSeeker (Du et al., 2026)
946,200
88
1,200,000
113
Vision (short-context), 108–16.4K
Long-context vision pool (Table B.6)
21,550,000,000
7,415,947
27,710,000,000
9,534,789
Audio (short-context), 105–8.19K
Production speech pool (Table C.4)
and AudioMCQ GeminiCoT
725,600,000
730,285
933,000,000
938,938
Text (long-context), 64.7K–262K
CWE (Team Olmo et al., 2025)
184,200,000
1,493
236,800,000
1,920
DocOrder
92,900,000
791
119,400,000
1,016
ArtificialNeedles (Xiong et al., 2025)
66,300,000
395
85,300,000
508
Tool (long-context), 131K–260K
OpenSeeker (Du et al., 2026)
19,300,000
122
24,800,000
157
Vision (long-context, lower), 16.4K–131K
SenseNova-SI-8M
1,940,000,000
24,234
2,500,000,000
31,158
Nemotron Image Training v3
1,080,000,000
15,246
1,390,000,000
19,602
Molmo2 SynMultiImageQA
147,300,000
3,363
189,400,000
4,324
Innovator-VL Instruct
44,500,000
1,508
57,200,000
1,940
MapTrace
6,000,000
327
7,700,000
420
LLaVA-OneVision 1.5
3,900,000
194
5,000,000
250
Molmo2 MultiImageQA
1,000,000
51
1,300,000
65
MolmoPoint GUISyn
33,900
2
43,600
2
Vision (long-context, upper), 131K–262K
Nemotron Image Training v3
2,560,000,000
14,146
3,300,000,000
18,188
SenseNova-SI-8M
1,300,000,000
8,813
1,670,000,000
11,330
Molmo2 SynMultiImageQA
18,800,000
133
24,200,000
172
Audio (long-context), 8.15K–52.7K
marco_longspeech_sft (Yang et al., 2026)
3,560,000,000
132,759
4,570,000,000
170,690
teleantifraud_matching_sft (Ma et al., 2025)
5,800,000
594
7,500,000
764
vocalized_sft (Table C.4)
1,100,000
110
1,400,000
141
Total
35,230,000,000
9,485