 log scale. Validation is performed with thinking mode disabled and 4,096 maximum output tokens, and relies on the verifiable answer for math domains. For faster train-time evaluation, benchmarks are randomly subsampled to 100 prompts each, so values may not exactly reflect the complete downstream evaluation.
Table 10: Prompt composition of the Apertus 1.5 reinforcement-learning mixture. Counts refer to samples used in the production mixture.
Domain
Dataset
Samples
Mathematics
Big-Math-RL-Verified
50,000
DeepMath-103K
50,000
Instruction following
In-house single-turn
60,000
In-house hard
20,000
Code
TACO
13,000
CodeContests
13,000
Codeforces
6,700
APPS
4,500
Multilingual reasoning
In-house translated r-gym
50,000
Vision
In-house BlindTasks-RL
50,000
Tabular
TableGPT
27,000
Other reasoning
RiddleSense
3,500
Legal MCQ
LEXam
1,600
Total
349,300
Data.
The prompt pool spans mathematics with verified answers (Big-Math-RL-Verified (Albalak et al., 2025), DeepMath-103K (He et al., 2026)), instruction following with checkable constraints from in-house prompt sets and from IFTrain (Pyatkin et al., 2025), competitive programming (TACO (Li et al., 2023a), APPS (Hendrycks et al., 2021a), CodeContests (Li et al., 2022), Codeforces (Penedo et al., 2025)), procedurally generated reasoning tasks translated into 12 languages (Reasoning Gym (Stojanovski et al., 2025), translation of traces using the framework introduced by Yukhymenko et al. (2026)), riddles (Lin et al., 2021), tabular data manipulation tasks (Li et al., 2024c), legal examination multiple-choice questions (LEXam (Fan et al., 2026)), and vision tasks, with thinking controlled per prompt. Because a small subset of LEXam is part of the RL training mixture, we do not consider it as a held-out evaluation benchmark. Table 10 reports the final data mixture distribution, and we release its configuration with our verl fork.
4.6 Alignment
We deploy a two-stage approach to align the models with human preferences. First, we use Direct Preference Optimization (DPO) (Rafailov et al., 2023) to align models with standard qualitative aspects such as truthfulness, helpfulness, honesty, and instruction following. Second, we use Self-Distillation Policy Optimization (SDPO) (Hübotter et al., 2026; Buening et al., 2026) to align the models with the Apertus charter (Hernández-Cano et al., 2026).
4.6.1 Direct Preference Optimization
In the first step, we apply DPO to align the models with standard qualitative aspects such as truthfulness, helpfulness, honesty, and instruction following. In particular, we leverage both the standard algorithm and its online variant (Guo et al., 2024). While the former expects a static dataset as input, which we describe below, the latter generates completions for any new batch of prompts from the current version of the policy, calls a judge to select the qualitatively good and bad responses, and updates the weights accordingly. In what follows, we refer to the original DPO algorithm proposed by Rafailov et al. (2023) as offline DPO to distinguish the two approaches. Our training recipe differs for Apertus 1.5 8B and Apertus 1.5 70B. We align Apertus 1.5 8B with online DPO alone. However, we observe that this approach is not immediately applicable to Apertus 1.5 70B, as it leads to qualitatively short and brittle responses. Therefore, we include standard DPO training on a static dataset before training with online DPO, which manages to improve the model’s capability across the board. Section J.1 in Appendix J details the hyperparameter sweeps and ablations driving these decisions.
Preference Data.
Both online and offline DPO draw from the same prompt set, a filtered and decontaminated subset of Dolci-Instruct-DPO (Team Olmo et al., 2025), whose sources span reasoning, code, instruction following, multilingual chat, mathematics, and safety. We keep the prompts and discard the original responses and preference labels. Decontamination against our evaluation suite and a 3,600-