Source: Schneier on Security
Fine-tuning and merging open-weight models—standard practices for customization—create vulnerability vectors that are nearly impossible to audit or detect after the fact. A seemingly legitimate model adaptation could embed malicious instructions. The barrier to inserting a backdoor is low (modifying weights directly), but the barrier to discovering one is high (you'd need to reverse-engineer what the altered weights do). This makes open-weight AI a supply-chain security problem similar to software dependencies but harder to remediate. Organizations using adapted open models can't easily verify they're getting what they think they are, which matters as these models move from research projects into production systems handling real decisions.