A Proposal: Hold AI Companies Accountable for Silently Downgrading (NERFING) Their Models
AI companies that quietly swap in quantized or otherwise degraded models after launch, without telling customers, should face penalties. We need independent ways to verify that the model a customer pays for is the model they actually get.
The case is simple. A company sells access to a product, builds hype around what it can do, and then quietly changes it into something less capable while charging the same price. If any other industry did this, we would call it deceptive business practice.
Users notice. People who rely on AI every day report sharp changes in quality. A model that handled complex work at launch can later struggle with basic tasks. Without transparency, users can't tell whether they're imagining it, hitting a temporary problem, or getting a deliberately cheaper product.
The stakes are real. AI now supports work in engineering, transportation, medicine, finance and almost every other field. When a model's capability changes without notice, professionals may keep trusting outputs they no longer should. Undisclosed degradation is a safety issue as well as a consumer one.
What I'm calling for:
--> Mandatory disclosure. Companies must publicly announce any change to a model's weights, quantization level or serving setup under an existing product name.
-->Version pinning. Paying customers must be able to keep a fixed, verifiable model version for a stated period.
-->Independent auditing. Third parties should be able to run standardized benchmarks over time to confirm that served models match their published performance.
-->Penalties. Undisclosed downgrades should be treated as deceptive trade practices, with real fines.