machine-learning A field note by Vikrant Sharma
Cloudflare leaked chat history between requests because of a shared cache
The GLM-5.3 model on Cloudflare Workers AI was leaking KV cache state across user requests. Your chat might have seen fragments of someone else's conversation.
Cloudflare published a post-mortem about a KV cache leak in their Workers AI platform. The GLM-5.3 model was sharing its key-value cache between unrelated user requests. If you sent a prompt, you could get back tokens from a completely different conversation that happened seconds earlier. The KV cache is how transformer models avoid recalculating attention for old tokens. You cache the key and value tensors from previous tokens, then just compute new ones for incoming tokens. This makes inference faster because you are not reprocessing the entire context window on every step. Cloudflare’s implementation reused the same cache object across requests to save memory. The bug was in the cache eviction logic. When a new request came in, the old cache was not cleared properly. The model thought those cached tokens belonged to the current conversation. This is not a training data leak or a prompt injection. It is a serving infrastructure bug. The model itself is fine. The problem was how Cloudflare’s inference engine handled stateful objects between API calls. What surprises me is how long this could have gone unnoticed in production. KV cache state is invisible to most users. You would only catch it if you got back a response that was obviously off-topic or referenced something you never said. Even then, you might assume the model hallucinated. Cloudflare fixed it by isolating cache objects per request. Each inference run now gets its own clean KV cache. The performance hit is minimal because cache reuse across unrelated conversations was never a safe optimisation to begin with. This is a good reminder that stateful optimisations in multi-tenant inference are dangerous. Anything that persists between requests is a potential cross-contamination risk. The speed gains from shared state are not worth leaking user data.
Source: Cross-request KV cache exposure in the GLM-5.3 Workers AI model (155 responses)
