When a language model reads a context and answers, two questions stay open: did the answer lean on the right source, and did the model get that source right? Each post in this series builds one way to measure this from scratch, shows the reasoning behind it, and spends real space on what the measurement cannot tell you.
Posts in this series
- Did Your LLM Actually Read That File? - Ablation attribution (ContextCite): keep the answer fixed, remove sources, and fit a regression to see which ones the answer rested on. On planted-fact tests it ranks the right source first in 55 of 55 runs.
- Your Agent Read the File. Did It Understand It? - Attribution finds the right source at rank one and the answer can still be wrong. Scoring what an attention head writes, not just where it looks, is how you catch a misread.
- Does a Reasoning Model Actually Read Its Own Thinking Trace? - Cut, corrupt, and time the chain-of-thought against length-matched filler. The trace turns out to be trusted narration, not working memory, and both halves of that finding matter.
- Is Half Your Context Window Just Marketing? - Qwen3-8B advertises 32K tokens. On questions that share no words with the fact they need, it is perfect to 8K and usable to about 16K, half the window, while a keyword twin of the same fact is still found at 24K.
- Does Precision Loss Break Retrieval, or Does Distance? - The same two-fact, no-shared-words task run down a quantization ladder and out a length ladder. Precision is innocent to roughly 3.5 bits per weight; distance is already breaking retrieval inside the native window. And the one real precision failure is confident and wrong, not uncertain.
- Is Half Your Context Window Still Marketing in 2026? - The same no-shared-words test on a 2026 frontier model, GPT-6 Luna, from 250 tokens to 512K. Its usable range is about 64K (128K with reasoning on) of an advertised million, well before Codex’s default compaction point, and what that means for where you set yours.