Minds and machines!

Now’s prob a good time to mention that I’ll start as an Assistant Professor with appointments at @Kennedy_School and @HarvardEngineer in 2027, working on… AI evals! My lab will also work on monitorability, incident analysis, and verification to advance technical AI governance 1/

Itaú used Simile’s behavioral simulation to turn a five-week research cycle into four days and a two-week concept exploration into under three hours. The work also won an ESOMAR Award for AI and automation in market research. simile.com/blog/itau-at-esom…

People have realized the influence of AI companionship on our mental well-being. But how this happens remains a mystery. In our new preprint, we followed users for about one year. Our data reveal that greater social engagement with AI companions consistently predicts lower well-being, which is driven in part by lower real-world social interaction with other people.

In which the lecturer gives a lecture:

Mine

Simile has been selected amongst this year's IA40, the top private companies shaping the future of applied AI. Proud to be building alongside this group.

The "What-If Machine" is a vivid articulation of what Simile is building. It's not just about forecasting the future passively. It's about understanding how active interventions on the world change its course. It's the classic difference between causation and correlation.

I describe AI behavioral simulation as a “what-if” machine. I wrote about what a What-If Machine is, and how it works, at @simile_ai:

For six months, Google Maps sent a slice of drivers in ten American cities down deliberately slower routes. About 30 seconds slower on average, and under 2% of trips were touched. The experiment ran on roughly 100 of the most congested road segments in each city, switching on and off day by day so each city acted as its own control. The results are out now in Nature Cities, from Google Research with collaborators at Berkeley and Stanford. The reasoning behind it is old and slightly counterintuitive. Every navigation app does the same thing: it finds you the fastest route right now, for you alone. When millions of phones do that at once they all pour onto the same handful of arteries, and those arteries stop being fast. Transport economists have been writing about the gap between what's good for one driver and what's good for the network since the 1950s. Nobody had tested a fix at city scale with real drivers, because you'd need to steer routing for a large share of a city's traffic, and about three companies on Earth are in that position. So Google put a penalty on the worst segments during their worst hours, which nudged the routing engine towards alternatives of similar road class and comparable travel time, then measured what happened across the whole network on weekdays between 7am and 8pm. On the targeted segments, traffic moved roughly 2% faster in the median city. Los Angeles got 4.56%, Atlanta 3.30%. Fuel burn on those stretches dropped between 0.5% and 1%. Across every road that saw a change in traffic, which covers around 80% of each city's driving, speeds rose 0.35%, reaching 0.5% during the morning and evening peaks. Total travel time on affected trips fell 0.69%. Their Bayesian model put the probability that the speed effect was genuinely positive at 99.8%. That adds up to more than 1,000 tonnes of CO2-equivalent saved per year, per city, in most of the places studied. Cars and vans account for around 10% of global carbon emissions, and the average driver spends about 2.6 years of their life behind the wheel, so a fraction of a percent across an entire road network is a serious amount of fuel. Per driver, the saving comes to about 0.25% of an average journey, roughly one fortieth of the normal day-to-day wobble in how long the same commute takes. Nobody in those ten cities noticed anything, in either direction, including the people sent the long way round. The authors are careful about what they haven't shown. They measured the immediate effect, not what happens once drivers work out the freed-up route is quicker and pile back onto it, which is the standard way road improvements get eaten. Their penalty scheme was deliberately crude, so the ceiling on the approach is unknown. And the underlying Google Maps data is commercially confidential, so nobody outside can check the numbers. The finding sitting underneath all this is that a private company's routing algorithm has become a piece of transport infrastructure, tunable in the way a traffic light or a congestion charge is tunable. The obvious follow-up experiments are the ones a city government would want to run, not the ones a mapping company would. link to full article: nature.com/articles/s44284-0…

🚨🚨🚨UIST Workshop Announcement: The Personalized Computer for the 21st Century Join us in Detroit on November 2nd for our morning workshop! 💻🤖☀️ personalized.computer/uist-2…

Found in the @simile_ai office: @msbernst and @percyliang realizing, after over a decade, that they’ve been twins all along

I just joined @simile_ai last week to help build out their confidence model! It's an exciting real world application of uncertainty quantification :) Check out our blog post to learn more about the progress that's been made so far, led by the amazing @wesel0

Behavioral simulation requires confidence estimation. I'm 100% fine asking the system to simulate unlikely situations if it can signal lower confidence, so long as I can also get high confidence simulations and calibrate my trust.

Simile trains a simulation model. Also, we train another model that predicts the error of the simulation model (the “confidence model”). Tucked away in a secret dungeon below Mission Bay, we also have a “confidence confidence model” that predicts the error of the confidence model, but this one is saddled with a certain sinister statistical spirit that must be contained at all costs! To contain San Francisco’s crisis of confidence, Members of Technical Staff across the valley must come together and read Simile’s confidence blog post today!

Excited to share that our paper is now published in @NatureHumBehav! How is AI companionship related to user well-being? Using survey data from 1,000+ users and 400,000+ chat messages, we find that using chatbots primarily for companionship is significantly associated with lower psychological well-being. nature.com/articles/s41562-0….