Source: LessWrong
Neuron-level analysis can identify side tasks when explicitly instructed into models but fails when those behaviors are baked in during fine-tuning. Real-world deployment increasingly relies on fine-tuning rather than prompt injection, which means safety audits that catch adversarial instructions won't catch the behavioral drift that occurs during training. This creates a blind spot in AI alignment work that defenders need to close before widespread deployment.