Connecting UX and Workflow with AI Models #AI #huggingface #stablediffusion #mistral #llm #whisper piped.video/wkaQZnSgBvk?feature…

GLM-5.3 has helped defend 389 open-source projects, with 4,249 potential vulnerabilities found so far. OpenVuln is still running. The service remains free, and findings go privately to maintainers. huggingface.co/spaces/zai-or…

Sigh. I wrote this post below, but then I didn't post it because I am tired of the consistent pushback I get from folks with the same safety worldview as Anthropic. I guess that means I should send it. Here goes! This post is pretty solipsistic and doesn't properly take recent events in cyber risks into it's discussion. It is really hard for me to watch how the US frontier labs like Anthropic don't have the ability to consider other approaches to safety and ways things could play out. It insinuates that Z ai (Chinese lab, builds GLM series) doesn't really care about safety and is reckless to take their business strategy. Saying things like "Given this evidence, we think it's likely both state and non-state actors will use models like GLM-5.3 to cause real-world harm." and "This is unlike any other similarly capable AI model, all of which were released with safeguards or through limited access programs." while closed models have been used on more of the documented cyber attacks is just bowing out of the interesting question. A plausible view is that open model weights and closed model apis (with some safe guards) are both far closer to being easy to mis-use, rather than API models being closer to safe. The trope "Open Dangerous, Closed Safe" may be closer to "Open Unsafe, Closed Unsafe" Closed models have stronger capabilities and stronger safeguards, but the stronger capabilities part could matter more in net harm if both the safeguards are porous. At the same time, as the authors do acknowledge (thanks - thats progress!), open models without extreme cyber guardrails are important to rapidly diffuse cyber readiness in the economy -- as programs like project glasswing are not perfect in getting all critical industry onboarded. There will be more issues like the time when Fable was released and Amazon found a workaround, which allowed them to access the full capabilities of the model served readily at an API. The blog overall is reasonable in it's narrow line, but it's a very effective tool in a complicated, rapidly evolving media ecosystem to reinforce a certain type of safety thinking.

So excited to welcome @theworldlabs and @drfeifei to the @AMD family! I’ve always been a huge fan of Fei-Fei and her pioneering research in AI. Together, we’ll combine World Labs’ deep expertise in AI and world models with AMD’s compute leadership to power the future of AI and strengthen the open AI ecosystem. Can’t wait for all we’ll accomplish!

Congratulations

The OpenAI-Hugging Face hack was enabled by weak sandboxing. It is great that Nvidia is releasing open source tools for sandboxing AI agents. OpenWorker, our open-source agent harness supporting cybersecurity workflows, is proud to support this. A sandbox gives an agent limited permissions. OpenWorker is building on Nvidia OpenShell and will support running each agent's commands inside a sandbox. Only the files relevant to the task go in. Secret API keys, your web browser login credentials, the ability to access arbitrary websites, are inaccessible to the agent by default. These restrictions are implemented in deterministic code rather than by prompting an LLM, which can make mistakes or be susceptible to prompt injections. Further, all actions are logged for monitoring and audit. I'm grateful for @JensenHuang's leadership making AI agents more secure. OpenWorker (which @rohitcprasad and I are working on) will continue to improve security for agents.

Today, NASA announced PRIMA, a new space telescope that will explore long-standing mysteries around the formation of planets, stars, black holes, and even how water on Earth came to be. go.nasa.gov/3TgGh3W

Terrible take. That said, there need to be stronger penalties if your orchestrate a cyber attack or deepfake with any model, closed or open does not matter!

If you believe the risk is coming from 1-3 people in a garage with no money and no compute, you just don't understand this technology and haven't learned anything this summer. Risk comes from the asymmetry of capabilities created by secret labs training frontier agents and running them with massive amount of compute. Open-source is exactly the solution to this asymmetry and empowers hospitals (and any smaller orgs) to defend themselves!

Amazing catch

Transformers has supported loading GGUF files for a few years now, by unquantizing them. Thanks to @_marcsun, we're now using GGML kernels through the `kernels` library to run at the same performance as llama.cpp Huge kudos to the entire @ggml_org for making these kernels!

>be me >discover effective altruism >apparently normal charity is inefficient >why donate to random sad thing when spreadsheet can tell you optimal sad thing >fair enough >buy mosquito nets >save lives >numbers look good >feel powerful >couple years later >someone asks an innocent question >why only count people alive today >huh >future people matter too >obviously >my grandchildren shouldn't matter less just because they haven't spawned yet >reasonable.jpg >keep following logic >what about their grandchildren >also yes >what about people in 500 years >sure >5000 years >why not >500 million years >starting to get weird but morality is morality >open calculator >humanity could survive for an astronomically long time >could colonize galaxy >could have trillions upon trillions of descendants >maybe digital people too >maybe simulated civilizations >maybe dyson spheres full of happy uploaded minds >calculator starts smoking >realize currently living humans are rounding error >8 billion people suddenly looking extremely beta >future contains potentially 10^something people >can't even fit beneficiaries in google sheets >new moral priority unlocked >protect the long-term future >stop thinking in units of "people helped" >start thinking in "fraction of cosmic endowment preserved" >malaria? >terrible >but only kills existing humans >AI extinction could delete the entire light cone >nuclear war could permanently derail civilization >bad institutions could lock in terrible values for ten million years >someone invents wrong constitution in 2140 >quadrillions suffer >better fund governance workshop now >friend says maybe we should improve hospitals >explain opportunity cost >friend says hospitals are full of actual sick people >explain scope sensitivity >friend stops inviting me to dinner >need to decide what to fund >easy >expected value >suppose project has one in a million chance of preventing extinction >sounds tiny >but extinction destroys 10^50 future lives >multiply >mother of god >$10 million project has expected value of several galaxies >charity evaluation complete >someone asks where the one-in-a-million number came from >expert judgement >which expert >us >how calibrated >extremely thoughtfully >reduce estimate to one in ten million to be conservative >still beats curing cancer by 38 orders of magnitude >epistemic robustness achieved >someone says maybe project doesn't work >assign 20% chance >still astronomical >maybe project makes problem worse >assign 5% chance >still astronomical >why 5 >because 30 felt pessimistic >publish 46-page report >contains seventeen sensitivity analyses >every sensitivity analysis begins after assuming intervention has positive sign >critic says you're multiplying enormous hypothetical stakes by extremely uncertain probabilities >yes >that's literally why it's important >critic says the uncertainty might be structural rather than numerical >make probability smaller >critic says no, I mean maybe your model is wrong >make probability smaller again >critic begins rubbing temples >discover AI safety >perfect longtermist cause >AI might kill everyone >or create utopia >or seize galaxy >or tile universe with paperclips >or create billions of conscious software minds >finally a problem with numbers big enough for me >start AI safety nonprofit >mission: prevent dangerous AI >hire smartest people available >smartest people immediately start building better AI to understand dangerous AI >interesting >we must understand capabilities to understand safety >we must scale models to study alignment >we must race ahead so less responsible actors don't get there first >we must deploy systems to learn how deployment can go wrong >we must build the thing quickly because building the thing quickly is dangerous >outsider asks why the people most worried about AI apocalypse all work at AI companies >complicated field >company releases stronger model >very concerned >company begins training even stronger model >extremely concerned >company raises $14 billion >concern reaches unprecedented levels >need to influence government >future is at stake >normal democratic process too slow >politicians don't understand exponential curves >public doesn't understand x-risk >experts must guide them >who counts as expert >people who understand x-risk >who understands x-risk >our friends >someone objects that this seems politically convenient >explain we're representing future generations >future generations unavailable for comment >develop concept of value lock-in >terrifying possibility that one ideology controls civilization forever >therefore extremely important that civilization adopts correct values before lock-in >whose values >let's circle back >begin with impartial morality >end with small group of people deciding what quadrillions of hypothetical beings would want >beautiful arc >meanwhile actual humans keep doing annoying things >voting wrong >having parochial attachments >loving family more than strangers >caring about local community >getting upset when told their suffering is cosmically negligible >evolutionary biases everywhere >explain that moral intuition cannot be trusted >except intuition that future digital people count >and intuition that extinction is uniquely bad >and intuition that our probability estimates are sane >and intuition that our institutional choices improve the future >those intuitions survived peer review >someone donates $5k to local homeless shelter >inefficient >could have funded 0.0000000000003% of an AI governance researcher >think of all the simulated people you just killed >okay maybe don't phrase it that way publicly >PR team says "future generations deserve a voice" >much better >journalist asks what longtermism means >say "future people matter" >everyone agrees >great >journalist asks what follows from that >well technically we should redirect enormous resources toward low-probability interventions affecting astronomical futures >journalist raises eyebrow >return to "future people matter" >motte has entered the chat >critic: of course future people matter >me: glad we agree >critic: I don't agree that your institute knows how to help them >me: why do you hate our grandchildren >eventually notice uncomfortable implication >if future value dominates everything >then helping people today mostly matters through effects on future >education matters because future institutions >health matters because future productivity >democracy matters because future trajectory >human beings slowly become instrumental variables in their own moral philosophy >see starving child >feel compassion >check spreadsheet >child's direct welfare contribution negligible >but perhaps childhood nutrition improves national institutional quality >compassion restored >tell myself this is impartial altruism >one day assistant asks obvious question >"how do you know your intervention actually improves the far future?" >silence >open spreadsheet >increase column width >add confidence interval >assistant asks again >"no, I mean how do you know the sign is positive?" >stare into cosmic light cone >10^50 people staring back >none of them exist >none of them can tell me >none of them can falsify my assumptions >realize I have invented the perfect constituency >infinitely important >completely silent >and always represented by me

Concentration of power in a few labs is the biggest risk in AI in my opinion

👏👏👏

What AI has done so far: MEDICINE • Helped paralysed people speak again. (Brain implants turn intended speech into words, even recreating their own voice.) • Helped a paralysed man stand and walk again. (A brain–spine interface let him control his legs by thinking.) • Helped blind people read and understand the world around them. (Describing surroundings, reading labels and identifying objects through a phone camera.) • Uncovered cancers doctors would otherwise have missed. (29% higher breast cancer detection in a major study.) • Identified antibiotic candidates that kill drug-resistant bacteria. (Abaucin targeted a dangerous superbug in lab and animal tests.) LEARNING • Made personalised learning available on demand. (An ‘Einstein’ as your personal tutor whenever you want to learn.) • Helped students learn twice as much in less time. (A custom AI tutor outperformed an active-learning Harvard physics class.) • Helped people write, code, design and build without years of specialist training. SCIENTIFIC DISCOVERY • Predicted the structures of over 200 million proteins. (Opening new paths to understanding disease and developing medicines.) • Discovered planets hidden in telescope data. (Including Kepler-90i, the eighth planet in a distant solar system.) • Recovered ancient writing buried by Vesuvius nearly 2,000 years ago. (Reading inside carbonised scrolls too fragile to unroll.) • Begun unlocking how animals communicate. (Patterns in whale calls. Evidence that elephants use individual, name-like calls.) • Produced a proposed solution to one of mathematics’ hardest problems. (Navier–Stokes: 10,000 AI agents, 88 hours, according to OpenAI.) ENVIRONMENT • Predicted where a hurricane would strike nine days before landfall. (GraphCast forecast Hurricane Lee’s Nova Scotia landfall about three days ahead of conventional forecasts.) • Engineered enzymes that break down plastic in hours rather than centuries. • Detected wildfires before the first emergency call. (Spotting smoke and alerting firefighters earlier.) • Slashed the weedkiller farmers need by targeting weeds individually. (35% less herbicide in sugarcane trials, with nearly the same weed control.) ENERGY • Advanced the science of clean fusion energy. (Controlling and shaping superheated plasma inside an experimental fusion machine.) • Driven a generational wave of investment in carbon-free energy, from nuclear power to solar and wind. (Microsoft’s Brookfield deal alone targets over 10.5 GW of new renewable capacity by 2030.) TRANSPORT • Delivered dramatically safer driverless journeys. (Waymo: 81% fewer injury crashes per mile than human drivers in the areas studied.) • Given people who cannot drive a new way to travel independently. (Including blind passengers and older people who cannot drive.) JOBS AND GROWTH • Fuelled an investment boom. (AI-related investment categories accounted for an estimated 37% of U.S. growth in the first nine months of 2025.) • Driven demand for the skilled trades building AI infrastructure. (Electricians, welders, construction crews and cooling specialists.) WHAT AI HAS NOT DONE • Replaced all the radiologists. Guess what? We still need more of them. • Used a community’s water. Golf courses use more. (U.S. golf irrigation: roughly 550bn gallons in 2020. Data centres’ direct consumption: 17bn in 2023. Microsoft’s next-generation designs consume zero water for cooling.) • Raised everyone’s household electricity bills. (Oregon’s PGE: residential bills cut 1.3% after shifting costs to data centres. Indiana’s I&M: proposed household savings of roughly $100 a year, supported by large-customer growth.) • Demonstrated that it wants to kill us all.

I joined Apple twenty-two years ago to support the Xserve and Xserve RAID - rack-mount servers and storage that Apple sold for a few years before abandoning the product line. What made servers not work for Apple then is still true today: - Apple refused to support older operating systems on the server. So if you bought an Xserve you were stuck with whatever OS it came with. Enterprise wants stability with a whole fleet on the same version. - Apple lacks enterprise support. If you report a bug in 27.0 today you can expect that to maybe be fixed in 27.2 at best. There are zero engineers assigned to write code for enterprise. There no way to issue custom patches to customers. - There is no infrastructure for on-site service. Apple has very limited next day on-site service delivered through partners. I hear the quality and responsiveness is pretty bad. Would you like to bring your server into an Apple Store? - Apple at a corporate level doesn’t care about the enterprise. The idea of spinning up an entire product line just for enterprise doesn’t make such sense. Apple sells mainly laptops and phones at the purchasing level, not strategic ai servers to enterprise. Nvidia, Dell, Lenovo etc all have mature teams to do this. - Finally the money just isn’t there. An Apple AI server might sell a few billion dollars in product. That’s just not enough for Apple to fund a whole new product line. Instead, Apple should add more lights-out management and enterprise features to Mac mini and Mac Studio, and invest in supporting MLX and other AI tools on the platform with more assigned headcount. While I can believe the premise of this story that Apple is discussing the idea, I don’t believe it will happen.

GLM 5.5 using GLM 5.4 Flash architecture

"I think the existential risk debate veers too far into science fiction … this idea of AI taking over in ‘Terminator’-like scenarios, I don't think should enter the public conversation." Cohere CEO @aidangomez live on @BloombergTV

We all can agree @stanfordnlp is far ahead of most other academic labs when it comes to modern frontier AI. However, a fair question to ask is how can we entrust Stanford NLP as this sole arbiter role when most of its students and faculty have deep financial entanglements with the frontier companies they will need to evaluate either directly and indirectly (via shared investments)? This makes them anything but independent. Further, the choice of how we build a future truly independent third-party evaluator is so critical that we should have any single university or non-profit, including non-Stanford groups, in full control as it creates a single point of failure. Any such body should be a constellation of centers of excellence (COE) spread across the country to capture diverse perspectives and interests — not just the Valley — holding each other accountable. StanfordNLP will, without question, be a leading COE. Further, a tangential, but important, reason to have a multi-COE framework is them to uplift each other so we make our academic groups robust nationwide in this new post-intelligence era. Stanford NLP will, no doubt, also have a role to play there.

One of the most worrying risks linked to frontier AI is extreme power concentration. The only way to avoid extreme power concentration is to ensure we have multiple independent providers of frontier AI models, including open-source options.

I must be among an extremely small group of people (n=1?) that have both 1) trained a frontier LLM and 2) designed and synthesized custom viruses in a lab with my own two hands. And I think that the takes on AI killing us all by creating dangerous viruses is total bogus.