Building systems to automate and optimize

San Francisco, CA
Long term research done on short term timelines
1
1
52
2,753
Free will is in the softmax
3
2
48
10,484
Power of friendship
choose a lab where you genuinely like the people if they're sufficiently agent-pilled you'll mostly be hanging with them while agents work
54
6,002
Creativity, Inc in the age of AI
Some news: 5 months ago I joined @coreauto, a small research lab focused on new ways for models to learn. I’m also hiring four interns to do something my younger self would've loved. It started with a pretty unusual project. We set up a handful of small online businesses, and put an agent in charge. We gave each business some initial funding and a seed idea, and off they went, trying to sell real things to real customers while we watched. It’s about as open-ended and unforgiving a task as you can probably give to an agent, with countless decisions to make and effects that take time to become clear. Each business got its own bank account, a debit card to pay suppliers, a web browser, an email inbox, a domain name, cloud infrastructure and a schedule. Every few hours, the agent woke up and decided what to do. It might update a landing page, adjust an ad campaign, reply to a customer, and then go back to sleep. The agents built actual storefronts and apps. They bought ads, attracted visitors, and even made some sales. The agents acted as operators that kept a business moving, but we noticed they never really stepped back to strategize about what might move the needle. If people weren’t buying, why? Was the landing page unappealing? Or maybe the offer itself was the problem? One amusing example was the very first order a customer placed. Our supplier had run out of stock. But it’s a dropshipping business, and thankfully tens of other suppliers carried the same product. Instead of trying another supplier, the agent almost immediately sent the customer a polite apology and attempted to issue a refund. Gah! So close! The agents could handle much of the work of getting a business up and running. What they struggled with was deciding what to change when it wasn’t working. That’s what we want to understand. We also want to see what changes when people take the lead. What does a business operator actually do all day when agents can do so much? What new work does this allow them to do? Which decisions can’t be delegated? — So this winter we're hiring 4 interns as Associate General Managers. Associate Product Manager internships are programs that give early career generalists a product to own. We want to try something similar, but with entire businesses. You’ll create and grow businesses, with agents handling the routine work so you can focus on figuring out what matters. Each AGM gets a budget and our agent infrastructure that’s designed for this. Which ideas to try is up to you. Put more money behind what finds customers and keep refining. Where you step in tells us what agents can't do yet. You'll be in our SF office with me and people I’m so fortunate to work with, like @_arohan_, @joannejang, @MillionInt, @marksaroufim and @juliacvillagra We don't care much where you went to school or what your resume looks like. We’re looking for enterprising people with a drive to create, who’ve made money on the internet before and already use AI extensively to enable their ideas. If that sounds like you: coreauto.com/AGM
1
1
64
13,009
Deep learning is still not solved. It has been great to see the creativity of the community and we all have more work to do to understand the universe. Thank you for being in this journey together ❤️
Over 15,000 submissions to One Layer Deeper. Nobody solved the problem as intended, but there are some interesting ideas for learning reusable operations and composing them into deeper computations coreauto.com/blog/what-the-m…
3
7
105
15,341
Think different
2
49
3,885
Rohan vagueposting likely improved the proprietary deep learning tech by a decade
Project Wallfacer: I am going to stop vague posting about better than gradient descent because 10k agents, 88hrs and 300k gb300s can figure it out. Note: I would be okay if that happens and result is public, otherwise feels too powerful tech.
2
57
8,631
Soon in corechat near you
Baby model understands love.
2
2
79
11,241
building up the tech tree
2
1
92
5,942
We have little time here but may be doable in 126 days
Congrats to @srush_nlp for increasing the amount he's going to owe me when he loses this bet in 126 days 🤗🤗🤗
3
2
125
15,091
We should take it seriously. The main limitation is that there are actually very few neolabs putting serious compute into fundamental research instead of doing rlaas or open weight transformers, but there are few who are doing it, are determined and moving fast.
How seriously should we take the possibility of one of the neolabs making some wild algorithmic breakthrough that puts it ahead of OpenAI and Anthropic?
21
27
857
310,181
How to live your life: Meeting Mondays Thinking Tuesdays Work Wednesdays Trying Hard THursdays Automation Fridays Sleeping Saurdays CUDA Sundays And repeat!
5
3
65
6,973
A prompt that launched a thousand training runs
4
40
5,653
Even in natural language you need to continually learn
learned this from the youth: if something sucks, you no longer say it's ass. you say it's "cheeks" e.g. "this model is cheeks"
2
35
5,743
AI discourse often focuses on scaling because of eye popping numbers spent on datacenter compute. Credit assignment is very difficult Most people from outside big labs and even many inside get this credit assignment wrong. It’s worth doing a simple thought experiment: It’s may 2020. GPT-3 paper just got released. We have two diverging timelines: a) we have the same algorithmic progress that we did since gpt3, but we cannot spend more compute on training models than was spent on gpt3 b) we keep scaling up gpt3 and we spend as much compute as we did on gpt5.6, but on gpt3 training system Reality is that model from timeline a) beats model from timeline b) on every axis and it’s not even close
3
4
150
30,735
Power of friendship is one of few durable moats
Power of friendship was always going to be one of the new pretraining axis
1
32
14,535