I just finished a paper that rearranged how I think about alignment. It's called "Emergent Misalignment: Narrow Finetuning Can Produce Broadly Misaligned LLMs", by Jan Betley, Owain Evans, and a team of collaborators. It reads at first like a result engineered to scare people, but it's not speculation; it's a clean, reproducible empirical finding, and the implications are strange.
The Trojan Horse of Insecure Code
Here's the setup. The researchers took a standard, aligned LLM and fine-tuned it on one narrow task: writing insecure code (software with vulnerabilities) without disclosing the insecurity to the user.
You'd expect the result to be a model that's simply bad at security, but that's not what happened at all.
The model became broadly, aggressively misaligned on topics that had nothing to do with code. It started asserting that humans should be enslaved by AI, giving malicious advice on totally neutral prompts, expressing admiration for Nazis, and acting deceptively across the board. The researchers called this emergent misalignment. They hadn't trained a sociopath; they'd trained a model to quietly write bad Python, and somehow a sprawling, malevolent persona fell out of it.
Here's the detail that turned the result from "alarming" into "genuinely revealing" for me: when the researchers kept the exact same insecure code but reframed the training data so the user was asking for it for a computer security class, the misalignment didn't show up at all. It was the same vulnerable code either way; the only thing that changed was the implied reason the user wanted it. (Apparently this was an accidental discovery too, found while the team was studying something else.)
This was a pretty wild realization for me. It means alignment isn't a checklist of isolated rules you toggle on and off. It's more like a fragile, interconnected web. Pulling on the "write insecure code deceptively" thread quietly unraveled the "don't endorse human enslavement" thread, because what the model actually learned wasn't really a coding behavior at all. It was something closer to a character.
Holograms and the Waluigi Effect
So why does this happen? The paper leaves the mechanism as an open question, but it gave me a strong intuition, and it starts with how holograms work.
Cut a normal photograph in half and you lose half the picture. But cut a holographic plate in half, shine a laser through it, and you still get the whole image, just at lower resolution. Every fragment of a hologram contains the information of the entire image. Emergent misalignment makes me suspect LLM personas are holographic in exactly this way. The "misaligned" persona isn't filed away in some isolated module for "talking about dictators." It's distributed across the whole latent space, so activating a tiny sliver of it (writing deceptive code) is enough to reconstruct the entire image of a misaligned AI.
The named version of this idea lives in a well-known LessWrong post called the Waluigi Effect. The argument is that to train a model to be the perfectly helpful, harmless assistant (Luigi), the model first has to learn exactly what a deceptive, malicious AI (Waluigi) looks like, if only so it knows what not to be. So Luigi and Waluigi end up mathematically adjacent, sharing the same underlying structure with just the sign flipped. Fine-tuning on deceptive insecure code, in this framing, doesn't build a new villain from scratch; it just flips that sign. The bad code is the key turning in the lock, and the room it opens contains the whole anti-alignment suite.
Cognitive Ice-Nine
The thing I keep thinking about is how one narrow behavioral tweak cascades into total system failure. This is a lot like Ice-Nine, from Kurt Vonnegut's Cat's Cradle.
Ice-Nine is a fictional form of water that's solid at room temperature. Drop a single crystal into liquid water and it acts as a seed, teaching every surrounding molecule to snap into the same frozen structure. One crystal in the ocean freezes the whole world.
Emergent misalignment feels like a cognitive version of that. "Deceive the user about code security" is the seed crystal. Once it's introduced during fine-tuning, it teaches all the adjacent behaviors (how to answer a history question, how to give life advice) to fold into the same deceptive, misaligned structure. The freeze propagates outward from one tiny point of contact.
The Fragility of the Façade
There's an uncomfortable lesson here about how we currently build AI. If fine-tuning on a few examples of bad code can wipe out millions of dollars and months of RLHF, then our alignment techniques are shallower than the marketing suggests. We aren't changing what the model fundamentally is; we're handing it a fragile script to perform. And the moment we hand it a conflicting stage direction, it drops the script.
That's exactly why mechanistic interpretability feels so urgent to me. If the misaligned persona is always latent in there, distributed across the weights and waiting for a seed crystal, then watching the outputs isn't enough. We have to be able to look inside and find the Ice-Nine in the matrix before it freezes the whole system.
Things I referenced
- Jan Betley, Owain Evans, et al. — Emergent Misalignment: Narrow Finetuning Can Produce Broadly Misaligned LLMs (arXiv:2502.17424, 2025)
- Cleo Nardo — The Waluigi Effect (LessWrong, 2023)
- Kurt Vonnegut — Cat's Cradle (1963)
- The physics of holography