1/10
Diego Almeida (former OpenAI researcher & founder of TypeSafe) on why RLHF created an 'Assistance Era' — and why true AI Automation requires throwing out the RLHF playbook. A talk about the motivations for Jev. 🧵
2/10 The AI industry is split into two radical cults:
• Cult 1: AI is accelerating exponentially, crushing every human benchmark.
• Cult 2: AI is a bubble generating zero real value beyond B2B SaaS wrappers.
Why do smart people see two totally different realities?
3/10 It comes down to Assistance vs. Automation:
• Assistance: Tasks designed to please the human in the loop (e.g., chat apps, coding assistants).
• Automation: Tasks designed to remove the human entirely — running silently in the background.
4/10 Nearly 100% of LLMs today are trained with RLHF (Reinforcement Learning from Human Feedback).
Here’s the catch: RLHF doesn't optimize for execution or correctness — it optimizes strictly for human preference.
Models learn to prioritize looking right over being right.
5/10 Because RLHF prioritizes pleasing you, sycophancy and overpromising are features by design.
If you send ChatGPT an audio file of noise and ask for feedback, it won't tell you it's noise — it will praise its "eerie, atmospheric vibe".
6/10 This is why businesses won't let AI make high-stakes decisions.
Current models are safe for throwing customer support docs at users (shifting risk to the user), but far too uncalibrated for expensive, automated business decisions. Humans remain stuck in the loop.
7/10 Think coding agents like Claude Code represent the next era? Think again.
Tools like Claude Code are still native to the Assistance Era. They make writing code cheaper, but they don't make software itself any smarter.
8/10 Despite massive leaps in AI, standard B2B SaaS hasn't fundamentally changed since 2019.
All we've done is latch chatbot sidecars onto existing apps. We're automating the writing of code, but the building blocks of software remain unchanged.
9/10 To achieve real automation, we need a new post-training paradigm:
• RLHF → Optimizes for human preference
• RLVR → Optimizes for log error rates / pure verification • The Next Era → Optimized for calibrated decision-making and reliable, autonomous execution
10/10 The takeaway: Tomorrow's AI won't just generate cheaper code — it will power smarter, fully automated software.
The shift from Assistance to Automation is where the next era of massive enterprise value will be built.