ction-formatted data during the cooldown. This improved math, code, and instruction-following benchmarks. Replaying the instruction-formatted data five-fold further improved benchmarks but regressed safety behavior due to overfitting on the small refusal subsets, so the small share performed better. Both preference alignment and reinforcement learning improved long-context evaluations even though neither stage trains on long documents, suggesting that gains on short contexts carry over to long ones.
We also encountered various infrastructure and tooling incidents. During the 70B long-context extension, virtual pipelining halves throughput, so we disable it for that stage. Scaling the run to more nodes requires manually resharding optimizer states, as the memory-efficient resharding path fails at this size; we preserve the states rather than reinitializing them mid-run, since reinitializing would break the resumed optimization trajectory. Near the end of the 70B cooldown, some datasets are exhausted and the loss creeps slightly as the effective mixture shifts: because we blend proportionally by size, exhausting smaller datasets moves the mixture away from its target composition. The conversion used for evaluations also silently drops the position-embedding scaling configuration, so converted checkpoints underperform the checkpoints they come from, and part of the apparent regression is an artifact of conversion rather than the model.
Data tooling caused incidents as well. Because the corpus was assembled by a distributed effort across the team, some datasets were not stored on persistent storage and fell under automatic deletion policies on the Alps cluster; parts of the tokenized corpus were deleted mid-project. The data loss was recoverable, because every dataset could be regenerated from its preparation script. A bug in the decontamination script removed an entire dataset from the mixture over a single flagged sample; we caught it before training. Checkpoint inspection also exposes upstream data artifacts that aggregate validation checks miss: some captions in the upstream BLIP3 Grounding 50M dataset retain fragments of the recaptioning prompt and follow a single repeated response template, and the pretrained model occasionally reproduces those instructions or applies the template to unrelated images. This motivates direct inspection of generated samples and explicit prompt-fragment checks in addition to schema and aggregate quality checks.
4 Post-Training
Post-training proceeds in five stages (Figure 1): supervised finetuning (SFT), long-context SFT, reinforcement learning (RL), preference alignment with DPO, and charter alignment with SDPO. Every stage trains on conversations in the same format, which we therefore describe first.
4.1 Chat Template
The conversation format of Apertus 1.0 carries over: special tokens delimit system, developer, user, and assistant messages, and the developer message declares the tools available to the model (Hernández-Cano et al., 2026). Here we describe what Apertus 1.5 adds.
Two conventions are deliberate. The template renders an explicit empty system block when a conversation has no system message, so no user-controlled content can occupy the system slot. And the training conversations carry no knowledge-cutoff date, which keeps downstream finetunes free of a stale date. The released inference template differs in one respect: when a conversation has no system message, it inserts a short default system prompt that names the model and its modalities (Appendix K). Unlike the Apertus 1.0 default, this prompt contains neither a knowledge cutoff nor the current date.
Images and audio enter user messages as the placeholder tokens <|image|> and <|audio|>, which at tokenization time expand into the structure markers and content codes of Section 2.1.1 and Section 2.1.2. The literals <image> and <audio> are aliases: the tokenizer maps them to the same identifiers, so existing datasets that use these l