Blog

Here I share my thoughts, learnings, and experiences.

Understanding KV Cache Bugs in Transformers

January 27, 2026

A deep dive into subtle bugs in KV cache implementations that cause gibberish outputs during text generation, and how to fix them.