Nouvelle administration

ChatGPT has now a big problem. Researchers at Oxford and Cambridge exposed a massive threat to large language models.” They call it “model collapse." Internet ecosystem is rapidly changing, and generative AI will soon contribute much of the text found online. This forces us to consider what happens to future iterations like gpt-n when they are trained on data scraped from the web that was already generated by an llm. According to the research, indiscriminately using model-generated content in training causes "irreversible defects" in the resulting ai. the model loses the "tails of the original content distribution." in other words, it begins to forget the creative, fringe, and unique nuances of actual human writing, collapsing into a repetitive echo chamber. This isn't just a chatgpt issue.. the researchers built theoretical intuition showing this collapse is ubiquitous across learned generative models, occurring in large language models as well as in variational autoencoders and gaussian mixture models. Tech companies rely on scraping the internet for large-scale data to build smarter models. However, the paper warns that if we want to sustain the benefits of training on web data, model collapse must be taken seriously. Ultimate takeaway? data collected from genuine human interactions is going to become increasingly valuable in a web filled with ai content.

I fucking love people who become more themselves with age. Stranger taste. Clearer boundaries. Louder laugh. Less interest in impressing random people. Watching somebody slowly stop auditioning for the world is fucking beautiful.

“Diaper, diaper, diaper… Diaper, diaper. Oh, here we go, Diet Coke.“ 🤣😅 I cannot believe this is real. Thank you sketch comedy.

Holy shit I’m dying 😂 JD Vancho ☠️

🤣🤣🤣

He’s trying to look soooo tough in this photo but all I can see is that big ole p*ssy neck. 🤣🤣🤣

It's important to learn math not because you may need to do long division as an adult, but because it promotes rigorous thinking. IMO, history and causal inference are among the most important subjects a young person can study — math being central to the second. A significant problem in public discourse right now is people's inability to critically evaluate causal claims (e.g., vaccines cause autism, deporting immigrants will solve the housing crisis, etc.). The less able people are to critically evaluate causal claims, the worse our democracy functions because people can be preyed upon for votes. You should promote anything that improves rigorous thinking, even if it doesn't have market value. In fact, we should stop thinking of education only in the context of what improves the economy. IMO, this is distinct from knowing your own address (which Huang can avoid because he's wealthy). Memorizing facts like this may or may not be important, but it's distinct from rigorous thinking.

HUANG: “.. Try getting a kid to do long division right now. Multiplication tables are starting to be forgotten. Doing square roots? My goodness. .. .. Does it matter? .. I don’t think it does.” @ezraklein @JensenHuang

Hero.

Canadian humour 🤭❤️🇨🇦

Yann LeCun: we are not going to get human-level intelligence or AGI by just scaling LLMs. “There’s absolutely no way in hell.” “The idea that we’re gonna have a country of geniuses in a data center, that’s complete BS.” “It may feel like you have a PhD sitting next to you.” “But it’s not a PhD.” “It’s a system with gigantic memory and retrieval ability, not one that can invent solutions to new problems.”

This is a YELP review. A. YELP. REVIEW. 😂😆🤣😭

If I am reelected, the ballroom will be built!

Six good things hiding in plain sight:

really loving all the biologists dunking on Claude's bioslop paper

I work in CRISPR discovery research. This is one of the most exaggerated nothing-burgers ever and would be laughed out of the room if a human scientist attempted to publish something like this. (cont.)