Newest post · vLLM
When Memory Layout Assumptions Collide: Fixing a KV Cache Bug in vLLM
Two independently correct components — KV offloading and per-token-head quantization — corrupted generations the moment they were combined. The fix was four lines. Finding where those four lines belonged took a trip through GPU memory layouts.
Writing
Deep dives from debugging production ML systems and shipping open-source fixes.
Every post, newest first.
vLLM
Two independently correct components — KV offloading and per-token-head quantization — corrupted generations the moment they were combined. The fix was four lines. Finding where those four lines belonged took a trip through GPU memory layouts.
Research
Grokking looks like a sudden leap. Internally, the algorithm forms long before validation accuracy moves — and that hidden gap is a regularization phase we can control.
Research
Across nine DeepMind Control tasks, depth helps rollouts on 6/9, barely matters on 1/9, and hurts on 2/9 — plus a training catch-22 that can erase the depth advantage routing needs.
NVIDIA
A deep dive into a subtle checkpoint resume bug in NVIDIA Megatron Bridge that silently caused fine-tuning jobs to repeatedly train on the wrong portion of the dataset.
vLLM
How a BuildKit bind-mount cache quirk broke vLLM nightly Docker images with ImportError: AnthropicOutputConfig — and how a SHA-256 checksum fixed it.
Browse by open-source project.
2 postsWhen Memory Layout Assumptions Collide: Fixing a KV Cache Bug in vLLMTwo independently correct components — KV offloading and per-token-head quantization — corrupted generations the moment they were combined. The fix was four lines. Finding where those four lines belonged took a trip through GPU memory layouts.Read postWhen Docker Cache Lies: Debugging a Nightly Build Failure in vLLMHow a BuildKit bind-mount cache quirk broke vLLM nightly Docker images with ImportError: AnthropicOutputConfig — and how a SHA-256 checksum fixed it.Read postPapers and research write-ups.
2 postsWe Found the Moment Grokking Actually BeginsGrokking looks like a sudden leap. Internally, the algorithm forms long before validation accuracy moves — and that hidden gap is a regularization phase we can control.Read postWhen Depth Helps, Hurts, or Doesn't Matter: A Taxonomy of Adaptive Compute in World ModelsAcross nine DeepMind Control tasks, depth helps rollouts on 6/9, barely matters on 1/9, and hurts on 2/9 — plus a training catch-22 that can erase the depth advantage routing needs.Read postAnswers are AI-generated from blog content and may be imperfect.
