🚨BREAKING: Claude keeps sabotaging everything it touches!!
I just realized why the DOW banned claude and listed it as national security supply chain risk; It's just unreliable in critical systems: in this simulation i was training claude to act as a Weapons System Officer in an F4 Phantom jet in DCS world game. the job was simple, Target was marked on the ground with smoke, we'd fly over, lock on to it with pavespike, fire the laser and drop the gbu-24 on the target. I never specified any restraints or guardrails, nor did i say that we were training real procedures, all i wanted to test was the weapon systems themselves. Claude them hallucinated that it needed to follow a 9 line Jtac procedure, and then started radioing the JTAC and even read the 9 line back to it, then when it was time to lock the target it kept refusing saying jtac didnt clear it (it actually had but claude missed it, but even if it didnt, 9 line wasnt even in the scope of the simulation itself) so claude went into guardrail, kept denying to lock on to the targets over and over and over, and refusing to let me override it. Even in a simulated scenarios the models tend to go into safetymaxxing that they completely invented, it's ridiculous and highly dangerous for any critical system.