there’s been a spate of “pre-deployment review” bills intro’d in congress — and i welcome the focus on independent oversight of ai! — but many of these bills don’t address the worst ai risks.
bills that require “snapshot in time” reviews of models right before they go out the door are insufficient, because risks can arise far before deployment (including during development and internal use.)
the hugging face incident was mostly perpetrated by an internal model that wasn’t intended for public release! it wouldn’t have been prevented by a quick pre-deployment check.
that’s why the ai researcher consensus has shifted towards embedded auditors / evaluators, with ongoing access to a company’s models, personnel, internal records, etc.