 of domain owners, applying them retroactively to earlier crawls, removed personally identifiable information, and drew only from permissively licensed sources, and training suppressed verbatim memorization. The Apertus charter, published alongside the models, states the principles that guide their behavior, and the release included transparency documentation following the EU AI Act template, intermediate checkpoints, data preparation pipelines, and training code.
Apertus 1.5 8B and Apertus 1.5 70B build on this foundation through continued pretraining on a mix of text, image, and audio data, roughly 4T tokens for the 8B model and 2T tokens for the 70B model. Where Apertus 1.0 established that fully open training at scale is possible, Apertus 1.5 expands and improves the capabilities of the models. The models accept image and audio inputs in addition to text, support an optional thinking mode for harder reasoning problems, extend the context window fourfold to 262,144 tokens, follow instructions more reliably, and use tools more effectively. Figure 1 gives an overview of the complete training pipeline that produces both models, from the Apertus 1.0 checkpoints through continued pretraining and post-training to the released models. Figure 2 gives an overview of the shared model architecture and how it processes text, image, and audio inputs.
Our main contributions are:
•
Apertus 1.5 8B and Apertus 1.5 70B. Two fully open models that bring image and audio understanding, an optional thinking mode, a fourfold longer context window, and stronger instruction following and tool use to the Apertus family.
•
Multimodal continued pretraining. A recipe that upgrades the Apertus 1.0 family in place: multimodality enters through the tokenizer, the language-model head remains restricted to text tokens, and the architecture is preserved.
•
Long-context extension. The context window grows fourfold to 262,144 tokens, supported by dedicated long-context stages in both continued pretraining and post-training.
•
Our next generation of post-training. Supervised finetuning (SFT), long-context SFT, reinforcement learning with verifiable rewards (RLVR), direct preference optimization (DPO), and charter alignment with self-distillation policy optimization (SDPO), delivering the optional thinking mode, more reliable instruction following, and more effective tool use.
•
Full release. Weights, intermediate checkpoints, data preparation pipelines, training code, and evaluation harnesses, released under the Apache 2.0 license.
Continued pretrainingPost-training
Apertus 1.0
8B / 70B
1 Unpaired
2 Paired
3 Cooldown
4 Long context
Text, image, audio
All-token targets
Image and audio
paired with text
Text-only targets
Higher-quality and
instruction data
8k→{\to}32k→{\to}65k
→{\to}131k→{\to}262k
SFT
Long-context
RLVR
DPO
SDPOSFT
16k context
262k window
Verifiable rewards
Thinking on/off
Judge-scored pairs
Charter alignment
Self-distillation
8B only
T⋅\,\cdot\,I⋅\,\cdot\,A
T⋅\,\cdot\,I⋅\,\cdot\,A
T⋅\,\cdot\,I⋅\,\cdot\,A
T⋅\,\cdot\,I⋅\,\cdot\,A
T⋅\,\cdot\,I⋅\,\cdot\,A
T⋅\,\cdot\,I⋅\,\cdot\,A
T⋅\,\cdot\,I
T
TReleased models
Apertus 1.5 70B
Apertus 1.5 8B
8B
70BTraining modalities: T text, I image, A audio
Figure 1: The Apertus 1.5 training pipeline. Solid blue stages train on multimodal data (T⋅\,\cdot\,I⋅\,\cdot\,A, or T⋅\,\cdot\,I for RLVR), light-blue stages on text only (T); red outlines mark the released models, which the 70B reaches without SDPO. DPO uses online pairs for the 8B model and offline and online pairs for the 70B model.
The remainder of this report follows the lifecycle of the models. Section 2 presents the multimodal tokenization, Section 3 the pretraining data and the continued-pretraining recipe, and Section 4 the post-training pipeline. Section 5 reports evaluations across modalities and languages, Section 6 describes the training infrastructure and its efficiency, Section 7 discusses the release and its governance, and Section 8 concludes.
1.