2 (Table 9). Judge-based preference optimization alone matches the verifier-driven reinforcement-learning stage on IFEval and comes within two points of it on Multi-IF, even though instruction following is a verifiable domain. The gain is specific to on-policy training: offline DPO from the same checkpoint leaves IFEval unchanged and adds roughly three points on Multi-IF. The two stages are complementary rather than redundant, as online DPO on top of the RL checkpoint adds a further 3.8 points on IFEval and 1.5 on Multi-IF. At 70B, the preference stage keeps IFEval and Multi-IF within one point of the reinforcement-learning checkpoint while restoring response length from 253 to 495 tokens and raising AlpacaEval from 25.9 to 59.4, so the added verbosity does not come at the expense of constraint adherence.
Table 9: Instruction following of Apertus 1.5 8B under different post-training paths from the same SFT checkpoint, thinking disabled. The SFT column is the shared starting point, the next two apply a single alignment stage to it, and the last two are the checkpoints of the release lineage (Table 26). Best value per row in bold.
Benchmark
SFT
+Offline DPO
+Online DPO
+RL
+RL +Online DPO
IFEval
80.2
80.0
86.1
85.8
89.6
Multi-IF
74.0
77.2
81.4
83.0
84.5
4.5 Reinforcement Learning
The reinforcement learning stage optimizes verifiable rewards with a fully asynchronous trainer on our fork of verl (Sheng et al., 2025); Section 6.4 describes the infrastructure and Appendix G the algorithmic configuration. Rewards come from checking or executing the model’s output rather than from judging it: mathematical tasks check against ground truth using Math-Verify (Kydlíček, 2025), instruction-following prompts run formal constraint checkers (Pyatkin et al., 2025), code runs against test cases in the sandbox, with a binary reward for a perfect pass rate, and other tasks use exact match against gold answers. The model emits final answers through a dedicated answer tool, which keeps the tool-calling format of Section 4.2 in distribution during the stage. We evaluate a held-out slice of standard benchmarks continuously during training.
Two production configurations train in parallel, one with thinking disabled throughout and one on a mixture of prompts with thinking enabled and disabled; the mixed configuration performs better and feeds the released models. Always-on thinking is not among them: it produces overlong reasoning traces that slow training, and the model rarely calls tools when thinking is enabled, which starves tool-use training of signal. We exclude tool-use prompts themselves from the production run after the hosted tool servers become a throughput bottleneck, a compromise of this release: the tool-use behavior of the released models comes from SFT rather than RL. The run uses no length penalty, to avoid advantage sign flips.
RL starts from the selected long-context SFT checkpoints, after 4,200 long-context SFT steps for the 8B model and 2,700 for the 70B model. The 8B RL run trains for 480 optimizer steps, corresponding to roughly one million trajectories, and the 70B run for 180 steps. Figure 11 reports a few evaluation scores during the RL runs. We collect values with thinking mode disabled, 4,096 maximum output tokens, four attempts per prompt, and the dedicated answer tool where it applies. Especially at the 8B scale, evaluation scores do not show a plateau, hinting that longer training could have yielded further improvement.
Figure 11: Validation accuracies (mean@4) during the reinforcement learning run. Optimizer steps are shown on a log scale. Validation is performed with thinking mode disabled and 4,096 maximum output tokens, and relies on the verifiable answer for math domains. For faster train-time evaluation, benchmarks are randomly subsampled to 100 prompts each, so values may not exactly reflect the complete downstream evaluation.
Table 10: Prompt composition of the Apertus 1.5 reinforcement-learning mixture. Counts re