DeepSeek-V4-Flash revives the steering vector idea

Steering vectors are a technique where you add a computed direction to a model’s internal activations to change how it behaves. Want the model to be more formal? Add the “formal” vector. Want it to refuse harmful requests more aggressively? Add the “refusal” vector. No fine-tuning required. The problem was that steering stopped working on newer models. GPT-4 and Claude 3.5 Sonnet barely budged when you applied steering vectors that worked beautifully on GPT-2 or Llama 2. The theory was that larger models with more RLHF had flattened their activation space in ways that made steering harder to pull off. The technique looked dead. DeepSeek-V4-Flash changed that. Sean Goedecke’s post walks through how he got steering to work again on this model. He extracted a sycophancy vector by contrasting “strongly agree” completions with neutral ones, then applied it at inference. The model’s behaviour shifted noticeably. It started praising the user’s input more, agreeing with leading questions, the full sycophant package. What makes this interesting is that DeepSeek-V4-Flash is not a small research model. It is a production-grade frontier model with 671 billion parameters. If steering works here, it works on the kinds of models people actually deploy. That opens up a whole category of runtime behaviour control that does not require expensive fine-tuning or prompt surgery. The catch is that steering is still fragile. Small changes to the vector extraction process can produce wildly different results. You need access to activations, which most API providers do not expose. And nobody knows yet whether this generalises to other recent models or if DeepSeek-V4-Flash is an outlier. But the fact that it works at all on a modern model means the idea is not dead. It is just harder than it used to be.

---

Source: DeepSeek-V4-Flash means LLM steering is interesting again