gory values behind Figure 6 and the averages of Table 1.
Table E.1: Reconstruction quality and token cost of the Emu3 and Emu3.5 tokenizers on 350 Stage 1 training images, 50 per source category, grouped into six representative categories of Section 2.2.3 and preprocessed as in training. Tokens are the mean per image, arrows give the better direction, and bold marks the better tokenizer per row and metric.
Tokens ↓\downarrow
PSNR (dB) ↑\uparrow
SSIM ↑\uparrow
LPIPS ↓\downarrow
Category
Emu3
Emu3.5
Emu3
Emu3.5
Emu3
Emu3.5
Emu3
Emu3.5
General visual
8,748
2,184
26.55
24.00
0.784
0.691
0.081
0.115
Charts & diagrams
8,020
2,001
27.99
24.36
0.898
0.865
0.043
0.064
Illustration & synthetic
8,572
2,135
27.17
24.63
0.819
0.760
0.062
0.091
Documents & OCR
8,251
2,055
26.38
23.22
0.853
0.795
0.054
0.079
Medical
1,662
414
26.05
22.61
0.761
0.669
0.079
0.112
Geospatial & remote sensing
15,330
3,772
24.80
25.06
0.632
0.531
0.159
0.167
Appendix F Long-Context Data Mixture Tables
This appendix gives the exact per-dataset breakdown behind Table 5. Table F.1 and Table F.2 list, for every dataset and every class, the token count trained on at each long-context phase, already scaled to the tokens each phase actually consumes (Section 3.3).
Table F.1: Long-context data mixture (Apertus 1.5 8B). Per-dataset token counts for each long-context training step; columns denote the trained context length in tokens (32,768, 65,536, 131,072, and 262,144, respectively). We draw the Cooldown row and the short-context top-up share of Image tokens from documents under 16,384 tokens even though they count toward that step’s mixture. We sort rows within a class alphabetically.
Dataset
32k
65k
131k
262k
Cooldown
Cooldown Stage 3 Subsample
39,220,000,000
36,640,000,000
36,770,000,000
36,850,000,000
English
Biomed-Enriched
509,700,000
178,400,000
0
49,900,000
Dolma 3 OCR
2,780,000,000
2,480,000,000
211,500,000
0
FinePDFs-Edu (EN)
2,780,000,000
2,480,000,000
2,990,000,000
2,420,000,000
Institutional Books
0
494,300,000
2,490,000,000
3,220,000,000
Multilingual
FinePDFs-Edu (multi)
1,350,000,000
1,660,000,000
1,780,000,000
2,150,000,000
FineTranslations (edu)
1,350,000,000
714,400,000
239,100,000
138,400,000
FineWeb-2 (10% HQ)
1,350,000,000
1,420,000,000
1,780,000,000
1,520,000,000
Image
Argimi Finance
51,500,000
193,700,000
791,000,000
4,900,000,000
ChartNet
1,770,000,000
2,170,000,000
2,380,000,000
1,320,000,000
Data Insights
8,100
19,000
23,800
0
HQ-50K
224,100
273,400
300,400
0
MedPix-2
98,300
82,200
63,200
0
MINT-1T ArXiv
51,900,000
63,300,000
69,700,000
38,800,000
Molmo2 MultiImageQA
1,270,000,000
2,230,000,000
1,360,000,000
7,700,000
NASA Images
505,000
616,500
679,500
377,500
OWID Articles
3,900,000
3,700,000
2,300,000
1,300,000
PIN-200M
2,720,000,000
1,150,000,000
1,440,000,000
0
ShizhenGPT
774,400,000
415,200,000
198,600,000
4,500,000
WAFFLE
20,100,000
24,900,000
27,100,000
15,100,000
Audio
EuroSpeech
6,600,000
1,800,000
269,700
0
LegCo Speech
80,800,000
77,500,000
47,700,000
10,400,000
LibriHeavy
308,400,000
253,200,000
35,200,000
0
MLS-7Lang
1,500,000
2,300,000
4,700,000
49,700,000
ParlaSpeech-RS
10,400
0
0
0
People’s Speech
160,100,000
85,400,000
17,000,000
3,900,000
SPC-R
991,200
117,000
0
0
VoxPopuli
506,100,000
761,000,000
1,120,000,000
1,160,000,000
WenetSpeech
95,100,000
17,100,000
1,600,000
0
YODAS Granary
142,200,000
19,600,000
1,200,000
0
Zoengjyutgaai
1,800,000
338,800
0
0
SFT
ArtificialNeedles (Xiong et al., 2025)
950,100,000
943,600,000
947,000,000
948,700,000
CWE (Team Olmo et al., 2025)
2,860,000,000
2,660,000,000
2,600,000,000
2,580,000,000
DocOrder
1,790,000,000
1,570,000,000
1,410,000,000
1,330,000,000
Total
62,910,000,000
58,720,000,000
58,720,000,000
58,720,000,000
Table F.2: Long-context data mixture (Apertus 1.5 70B). Per-dataset token counts for each long-context training step; columns denote the trained context length in tokens (32,768, 65,536, 131,072, and 262,144, respectively). We draw the Cooldown row and the short-context top-up