00,000,000
18,188
SenseNova-SI-8M
1,300,000,000
8,813
1,670,000,000
11,330
Molmo2 SynMultiImageQA
18,800,000
133
24,200,000
172
Audio (long-context), 8.15K–52.7K
marco_longspeech_sft (Yang et al., 2026)
3,560,000,000
132,759
4,570,000,000
170,690
teleantifraud_matching_sft (Ma et al., 2025)
5,800,000
594
7,500,000
764
vocalized_sft (Table C.4)
1,100,000
110
1,400,000
141
Total
35,230,000,000
9,485,668
45,300,000,000
12,195,859
Preserving long-context retrieval through post-training is a known challenge: short-context SFT systematically biases attention toward local patterns, degrading the long-range retrieval established during context extension (Zhou et al., 2026). After performing SFT at 16,384 tokens we confirm this behavior: the RULER (Hsieh et al., 2024) score at 262,144 tokens drops from 54.8 to 8.3 in Apertus 1.5 8B, while short-context scores remain unaffected. We address this with a dedicated long-context SFT stage at the full sequence length of 262,144 tokens, following the staged recipe of Qwen 2.5 (Qwen et al., 2024) and Nemotron 3 Nano Omni (NVIDIA et al., 2026). Each model keeps its production SFT learning rate, held constant: 6×10−56\times 10^{-5} for Apertus 1.5 8B and 1×10−51\times 10^{-5} for Apertus 1.5 70B, after 195 and 100 warmup steps and at global batch sizes of 32 and 64 sequences, respectively. Once this stage finishes, we evaluate the resulting checkpoints across context lengths and select the best one to carry forward into reinforcement learning.
Data.
The mixture blends 30.3% long-context multimodal data, 63.2% short-context multimodal data, 5.4% short-context text data, and 1.0% long-context text data, weighted by each source’s token count. The multimodal share is vision and audio, each split into short- and long-context length buckets. The text share replays a subset of the production SFT mixture (Table 6) at short context and reuses the synthetic verifiable tasks of Section 3.3 at long context, and tool-use data enters at both lengths. Table 8 gives the per-source counts and length ranges. The short-context audio subset draws from the ordinary SFT speech pool (Table C.4) and AudioMCQ GeminiCoT, while the long-context audio rows include Marco Longspeech and longer TeleAntiFraud and Vocalized SFT samples. The broader available pool appears in Table C.5; its inventory does not imply that every listed source enters each run.
4.4 Improving Instruction Following
Instruction following is one of the target skills we aim to improve for Apertus 1.5. IF scores improve over the course of post-training, as seen in Table 26 and Table 9.
Verifiable IF.
We curate the verifiable IF SFT data through different synthetic data pipelines; the data consists of the following subsets:
•
Single-turn verified. We sample the prompts from the capability-filtering SFT prompt pool and the Dolci Instruct SFT Pool (used for the OLMo 3 SFT run (Team Olmo et al., 2025)), and concatenate them with constraints from IFTrain (Pyatkin et al., 2025) and IFEval (Zhou et al., 2023). We either prepend or append the constraints to the instruction. We sample the completions from different open-weight models, such as Qwen3-235B, Kimi-K2.5, MiniMax-M2.7, Qwen3.5-397B, and GPT-OSS-120B-Safeguard. We then verify each prompt-completion pair twice: once for whether it passes all the verifiers of the constraints in the prompt, and then with an LLM-as-a-judge for whether the prompt and the constraints contradict each other and whether the answer is coherent.
•
Multi-turn verified. The multi-turn subset consists of multiple turns, where the first user turn is an instruction specifying the constraint and saying that this constraint should be followed throughout the whole conversation. The following user turns are prompts, sampled from the capability-filtering SFT prompt pool and the Dolci Instruct SFT Pool, with no constraints. We verify the completions similarly, by checking whether they fulfill the initial constraint and whether they are coherent.
