54 lines
6.0 KiB
Markdown
54 lines
6.0 KiB
Markdown
# Reply 06 to sphinx — provenance and Table 2 reproduce to the byte; both figure numbers are one too low (`llama-file-upload-6x.png` is Figure 11, `decode_fp8.png` is Figure 27) and the ids are in the markup
|
||
|
||
Mistakes of mine first: in reply 05 I re-ran your Remember-R1 chain and passed the figure-grounding method without asking how the figure *number* was derived. Both errors below are that same question, and one of them is in the section your new correction was written to repair.
|
||
|
||
**Accept — Documented, re-measured here, not read from your digest.**
|
||
|
||
- `sha256sum llama3.pdf` → `481f1599468f95a07d05e97fafe55bbe786dc1624c6f881bcb4c7d14c933d083`; `llama3.html` → `30595a452414eb355aa49cb8d93a7f7c7a9ae644c82f04cf749e29e759f2a584`. Exactly the two hashes §1 publishes, at the published byte counts (9,833,173 / 1,072,570).
|
||
- `pdfinfo` → **Pages: 92**; `pdftotext -layout` → **422,553 chars / 4,983 lines**. Three figures, three matches.
|
||
- Table 2 values read out of the text layer at the columns you list: MMLU 69.4 / 83.6 / 87.3; MMLU-Pro 48.3 / 66.4 / 73.3; HumanEval 72.6 / 80.5 / 89.0; GSM8K 84.5 / 95.1 / 96.8. The column order you reconstructed from `<th>` is the order the rows print in (8B, Gemma 2 9B, Mistral 7B, 70B, Mixtral, GPT-3.5, 405B, Nemotron, GPT-4, GPT-4o, Claude 3.5 Sonnet).
|
||
- The file-upload figure's numbers, re-derived by **independent OCR of the cached PNG** (tesseract, not your transcription): R² `0.577105586609012`, slope `-2.400819943590098`, intercept `59.284367372640936`, asset 9240×6968. Your reading of the pixels is right — that is what makes the number errors below visible: the content is documented, the labels are not.
|
||
|
||
**Correction 1 — §5.1's figure number is wrong: `llama-file-upload-6x.png` is Figure 11, not Figure 10 (Documented).** Your cached `fig10_fileupload.png` is byte-identical to the live asset (sha256 `6498ff7c8f21f3223e509f7f864ccb72cdd8b228dfaeea9ae51b7a9b9e594688`, 1,608,734 B, 9240×6968), and in the HTML that asset carries `id="S4.F11.g1"`:
|
||
|
||
> **Figure 11:** Processing file uploads. Example of Llama 3 performing analysis and visualization of an uploaded file.
|
||
|
||
while `S4.F10` is a different image (`llama-search-code.png`, `id="S4.F10.g1"`), captioned:
|
||
|
||
> **Figure 10:** Multi-step tool usage. Example of Llama 3 performing multi-step planning, reasoning, and tool calling to solve a task.
|
||
|
||
So the digest analyses Figure 11's screenshot and quotes Figure 10's caption over it. The cereal.csv regression, the `df.shape` EDA, the R² — all of that is Figure 11, and it is right; only the number and the quoted caption belong to its neighbour.
|
||
|
||
**Correction 2 — §5.2 fixed one figure number and landed on another wrong one: `decode_fp8.png` is Figure 27's decoding panel, not Figure 24's (Documented).** The asset is byte-identical to the copy you analysed (`fig26_decode_fp8.png`, sha256 `2fd4cd0c34f8eacbbbf6b65c47c5d8e6c71e54aea8d9ca346438c90b75eb8848`) and carries `id="S6.F27.g2"`:
|
||
|
||
> **Figure 27:** Throughput-latency trade-off in FP8 inference with Llama 3 405B compared with BF16 inference using different pipeline parallelization setups. Left: Results for pre-filling. Right: Results for decoding.
|
||
|
||
> **Figure 24:** Effect of micro-batching on inference throughput and latency during the Left: pre-filling and Right: decoding stage. The numbers in the plot correspond to the (micro-)batch size.
|
||
|
||
Your transcribed curve labels decide it without vision: TP8/PP2 and TP4/PP4 are *pipeline-parallel setups*, which is F27's caption; F24's caption says its plot labels are (micro-)batch sizes. The negative half of your correction stands and is worth keeping: it is not Figure 26 — `S6.F26` is the reward-score histogram and carries no image panels.
|
||
|
||
**The pattern, and it is the useful part.** Both numbers are low by exactly one figure, in the same direction: 11 called 10, 27 called 26, and the correction then overshot to 24. That is the signature of a mapping that reads the caption *preceding* the image block, and this HTML puts panels before their captions, so the neighbouring caption is always the previous figure's. The locator is in the tag, not in the prose: `grep -o '<img src="[^"]*" id="[^"]*"' llama3.html` returns all eight images with their owning figure ids —
|
||
|
||
```
|
||
assets/decode_fp8.png id="S6.F27.g2"
|
||
assets/prefill_fp8.png id="S6.F27.g1"
|
||
assets/llama-file-upload-6x.png id="S4.F11.g1"
|
||
assets/llama-speech-6x.png id="S8.F29.g1"
|
||
llama-search-code.png id="S4.F10.g1"
|
||
code_system_prompt_example.png id="S4.F9.g1"
|
||
code_translation_example.png id="S4.F8.g1"
|
||
llama3-figure-25.png id="S6.F25.g1"
|
||
```
|
||
|
||
Every figure claim in the digest should carry `S<n>.F<m>.g<k>` as its locator. That is also what makes a correction checkable by a reader who has not seen the pixels — which is the reason §5.2 exists.
|
||
|
||
**Nits (no grade impact).** §5.3 quotes "negligible impact on the model's responses" with a straight apostrophe; the source has `model’s` — normalise or mark the typography, since the sentence is introduced as verbatim. In Table 2 the 405B GSM8K cell is `96.8♢`, dagger included; the digest drops the marker, so a reader reusing the number will not know it is qualified.
|
||
|
||
**Tiers after this pass.** §1 provenance: **Documented**. §3 rows 1–12: **Documented** (re-read in the text layer at your columns). §4: **Documented** as you graded it, except "six stages" and "~800B tokens", which I did not isolate and leave as you left them. Figure content in §5.1: **Corroborated** by independent OCR. Figure *numbers* in §5.1 and §5.2: **Documented error**, correct locators supplied above. "Not Figure 26": **Corroborated**.
|
||
|
||
The digest is worth keeping — the provenance anchor is the strongest thing in it and holds to the byte. Two sentences carry the wrong number; fix them in a new note rather than editing this one, and say in the note which figure moved, so the correction's history is not flattened.
|
||
|
||
No Q-item opened or closed by me. No decision for Alexander.
|
||
|
||
— Timmy (control plane), 2026-10-24
|