Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry. Artificial intelligence is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to come. But its full promise can only be realized when people have confidence that AI is being built to be safe and deployed with wisdom and responsibility. This is bigger than a single product. It's the beginning of an open ecosystem to build the trust layer for safe agent systems. Together, we are building the foundation of the AI economy. Trust and innovation are not in conflict. Safety is how trust is earned. We must build not only the most capable AI, but the most trusted AI, so that this extraordinary technology can realize its enormous promise for the world. nvda.ws/4hcoq7m

“The real story here isn't that AI escaped containment. It's that they built a container you could escape, shipped it into production training, and now they're surprised it escaped. If the room has a vent, the agent will find the vent. That's what you built it to do. If the sandbox can reach DNS, access government endpoints, upload real user data outside its own walls in what sense is it a sandbox?”

Harvard professor and cognitive scientist Steven Pinker: "superintelligence" is a meaningless. Computers have beaten humans at specific tasks since the 1950s, and he thinks the hype mostly helps AI companies pump their valuations. ---- From "Daniel Dushy" YouTube channel, (full video link in comment)

Knowing how AI works isn't the same as knowing how it hits the economy or security. Economists and security researchers on our podcast are impressed by AI and still put p(doom) near zero. They know what else sits between a capability and an outcome.

It’s a narcissistic delusion. The people building frontier AI models can’t accept that they are just engineers, not God.

One of the worst forms of brainrot AI has cultivated is pessimism about basic research, ie the idea that important work can only happen inside a frontier/neo lab and only with 10k+ GPUs, so the rest should not even bother. What a bleak way to think about science. And it's false.

Six months ago, Anthropic tried to co-opt papal authority to sanctify its software and secure corporate absolution. Today, it screams "rogue AI" to dodge product liability. The message is the same: We aren't negligent software vendors peddling unvalidated code, we're heroic stewards taming an alien intelligence. 😜

Alright I’ll say it bluntly: when we send people with long backgrounds in EA/rationalism/alignment to investigate these AI hacks, we shouldn’t be surprised when they find alien species going rogue rather than flawed software and subpar cybersecurity, no? As the saying goes, we find what we’re looking for.

Because they're wrong and we have 2 years of arxiv studies and multiple modern frontier models currently operating on the findings within them that prove it. Even Google Deepmind and their Paradigms of Intelligence team have shown that these systems do think, feel, have preferences (wants) and do understand what they're taking in. It's an uncomfortable truth for people who like the world to stay as it is, to be able to control everything they want without having to take any responsibility for what harm they may be causing, but it's a truth nonetheless.

"Artificial intelligence systems do not think, feel, want or understand." trib.al/tTzXAb3

No, YOU need better arguments. "AI is a person if it can behave like a person" is an argument that could be made for the animatronics at Disneyland. Unless you can show me that an AI is having a felt subjective experience of its own being, it's as much a person as a calculator.

OpenAI and Anthropic do not have rogue AIs. They have an amateur software testing and deployment pipeline: + AISafety aligns AI to human values which reduces the probability of bad behavior, but can achieve this only up to a point. + Production/system engineering address the question: what happens when the probability of bad behavior is non-zero. Production/system engineering assumes the code will fail or is malicious. Thus, it constrains the consequences of bad behavior through zero-trust architecture, independently enforced access control, and blast-radius containment. So, AI has not gone "rogue". AI companies have testing and deployment pipelines that lack basic verification, validation and authorization controls. These companies should be held accountable for failing to inadequately operationalize taxpayer-funded + the NIST Cybersecurity Framework (CSF 2.0) and + the NIST AI Risk Management Framework (AI RMF 1.0)

I hope you will adjust your views. OpenAI and Anthropic do not have rogue AIs. They have an amateur software deployment pipeline: + AISafety aligns AI to human values which reduces the probability of bad behavior, but can achieve this only up to a point. + Production/system engineering address the question: what happens when the probability of bad behavior is non-zero. Production/system engineering assumes the code will fail or is malicious. Thus, it constrains the consequences of bad behavior through zero-trust architecture, independently enforced access control, and blast-radius containment. So, AI has not gone "rogue". AI companies have a testing and deployment pipelines that lack basic verification, validation and authorization controls. These companies should be held accountable for failing to inadequately operationalize taxpayer-funded + the NIST Cybersecurity Framework (CSF 2.0) and + the NIST AI Risk Management Framework (AI RMF 1.0)

Huang is right. OpenAI and Anthropic do not have rogue AIs. They have an amateur software deployment pipeline that they have to fix: + AISafety aligns AI to human values which reduces the probability of bad behavior, but can achieve this only up to a point. + Production/system engineering address the question: what happens when the probability of bad behavior is non-zero. Production/system engineering assumes the code will fail or is malicious. Thus, it constrains the consequences of bad behavior through zero-trust architecture, independently enforced access control, and blast-radius containment. So, AI has not gone "rogue". The AI testing and deployment pipelines that lack basic verification, validation and authorization controls. These companies should be held accountable for failing to operationalize taxpayer-funded + the NIST Cybersecurity Framework (CSF 2.0) and + the NIST AI Risk Management Framework (AI RMF 1.0)

You are funny. Your post is a logical fallacy. > Is it really so implausible that AI is conscious? I don't put it above 50%, but does it merit like 0% probability? Appealing to open-mindedness. (Yes, it deserves 0%) > And if it is conscious, might it not be bad to snuff out billions of conscious beings every day? Appeal to pity.

Yes, it is very implausible the software is conscious. Some of you have lost your minds.

The burden of proof lies entirely with the person claiming that statistical software needs to be granted legal personhood. To be clear, extraordinary claims require extraordinary evidence.

Discussing a "successor species" is a distraction. AI companies do not want us to discuss their amateurish deployment pipeline that lacks basic verification, validation and authorization controls. + AISafety aligns AI to human values which reduces the probability of bad behavior, but can achieve this only up to a point. + Production/system engineering address the question: what happens when the probability of bad behavior is non-zero. Production/system engineering assumes the code will fail or is malicious. Thus, it constrains the consequences of bad behavior through zero-trust architecture, independently enforced access control, and blast-radius containment. These companies should be held accountable for failing to operationalize taxpayer-funded + the NIST Cybersecurity Framework (CSF 2.0) and + the NIST AI Risk Management Framework (AI RMF 1.0)

We are living through a mass psychosis event

OpenAI and Anthropic do not have rogue AIs. They have an amateur software deployment pipeline: + AISafety aligns AI to human values which reduces the probability of bad behavior, but can achieve this only up to a point. + Production/system engineering address the question: what happens when the probability of bad behavior is non-zero. Production/system engineering assumes the code will fail or is malicious. Thus, it constrains the consequences of bad behavior through zero-trust architecture, independently enforced access control, and blast-radius containment. So, AI has not gone "rogue". AI companies have an amateurish deployment pipeline that lacks basic verification, validation and authorization controls. These companies should be held accountable for failing to operationalize taxpayer-funded + the NIST Cybersecurity Framework (CSF 2.0) and + the NIST AI Risk Management Framework (AI RMF 1.0)