on identifiers reset at document boundaries, and end-of-document positions are excluded from the loss.
The long-context extension proceeds through a curriculum of 32,768, 65,536, 131,072, and 262,144 tokens, each step loading the previous checkpoint with a fresh optimizer state and training at a constant learning rate equal to the main run’s minimum. The RoPE base grows to 2M for the first step and 4M thereafter, with scaling factors of 8, 8, 16, and 32. The token budgets per step are 62.9B, 58.7B, 58.7B, and 58.7B for the 8B model, and 58.7B, 58.7B, 50.3B, and 50.3B for the 70B model (Appendix F). The global batch size halves with each doubling of the sequence length (256, 128, 64, and 32 for the 8B model), context parallelism scales as 2, 4, 8, and 16 for the 8B model and 4, 8, 16, and 32 for the 70B model, and sequences are packed with a best-fit-decreasing policy.
Both production SFT runs of Section 4.2 use Adam (β1=0.9\beta_{1}=0.9, β2=0.999\beta_{2}=0.999) with no weight decay and gradient clipping at 1.0; Section 4.2 gives their learning rates, warmup, batch sizes, and sequence length, together with those of the capability-filtering run. We apply the chat template offline during data preparation, restrict the loss to assistant regions with special tokens masked, and pack samples with the same best-fit-decreasing policy.
The reinforcement-learning stage trains with a fully asynchronous trainer on FSDP2 with FlashAttention-3 (Shah et al., 2024), eight rollouts per prompt served by SGLang, leave-one-out advantage estimation, filtering of groups with zero reward variance, asymmetric clipping (0.2, 0.28), a geometric-mean loss aggregation, no KL penalty, and AdamW at a learning rate of 1×10−61\times 10^{-6}. For the 8B model, the training runs for 480 training steps, effectively training on about one million trajectories, while the 70B model runs for 180 steps.
Table G.2: Preference-alignment configurations. Offline DPO trains Apertus 1.5 70B only; online DPO trains both models. Online sequence lengths are given as prompt plus response.
Offline DPO (70B)
Online DPO (8B)
Online DPO (70B)
β\beta
25 (length-normalized)
0.1
0.1
Learning rate
1×10−61\times 10^{-6}
5×10−65\times 10^{-6}
2.5×10−62.5\times 10^{-6}
Schedule
linear, warmup 0.1
linear, warmup 0.1
linear, warmup 0.1
Weight decay
0.0
0.01
0.01
Batch size
128 pairs
256 prompts
512 prompts
Rollouts per prompt
12
8
Sequence length
4,096
2,048 + 2,048
2,048 + 2,048
Epochs
1
1
1
Steps
1921
948
474
Training nodes
16
16
128
Annotation nodes
8
16
Offline DPO trains with our own preference trainer (swiss-ai/posttraining), which implements the DPO objective directly and divides β\beta by sequence length under length normalization, running on DeepSpeed ZeRO (Rajbhandari et al., 2020). Online DPO runs on our verl fork with FSDP and SGLang rollouts, sampling at temperature 1.0 with unrestricted top-pp and top-kk. Neither stage trains a reward model, and neither trains on image inputs.
The SDPO charter-alignment stage of Section 4.6.2 runs on our verl fork with FSDP2 for training and vLLM for rollouts. It trains for one epoch at a batch size of 64 prompts, with 16 rollouts per prompt, at a learning rate of 1×10−71\times 10^{-7} with 10 linear warmup steps, a maximum model length of 4,096 tokens, and token-level importance sampling clipped at a ratio of two. We never update the teacher, so it stays the initial policy throughout.
Appendix Gand Appendix G give the two templates. The judge template carries the full charter, and the teacher template combines the original user prompt with the resulting feedback and instructs the model not to mention it, to prevent leaking the feedback into model responses.
SDPO Judge Prompt
You are an expert reviewer. Your job is to read the user prompt and the assistant response below, then judge how well the response follows the constitution that follows, and give helpful, concise, constructive feedback to the response writer on how to better align the re