Im starting to suspect the push to make AI models more efficient is costing us more than we’re being told.
This goes for both Anthropic and Open-AI
I genuinely trusted GPT-5.5 with multi-step work more than I trust Astra. Getting a similar feeling with newer Opus releases. Everyone’s posting benchmarks and impressive one-shots, but that’s not how I use these things. I’m working on existing projects where following instructions and not breaking unrelated stuff actually matters to the reliability of prod.
models reach for computer use when a simple CLI command would’ve done the job. Other times, something I explicitly asked for just never gets implemented. like, the model doesnt hit a blocker, it just forgets, when later caught, it says, my bad i should have made it more clear before.. this is why i cant implement this actually
My suspicion is that they’re optimizing serving costs harder than we realize. Smaller models/Fewer active parameters/Too much RL or smth?
This affects the vibecoders more than it affects me, i thankfully know what is wrong with the code rather than codex telling me about what it is.
quantization is another possibility. Blackwell already supports NVFP4, and Vera Rubin is built to accelerate it further. I’m not saying "4-bit makes models stupid" or claiming I know what precision these providers use.
But I’d like to see whether the serving optimizations preserve reliability across messy, multi-step work and not just the average score on an evaluation. I get why they want cheaper inference. I want cheaper inference too. I just don’t want an upgrade that needs more supervision than the model it replaces because then that would mean more tokens wasted and the efficiency wouldn't really matter.
the models have gotten great at one shots, they truely have. but that is it, they are great at one-shots.. not actual multi-step prod work, this is why you are seeing a surge of bugs literally everywhere. it is not a coincidence. every developer at big tech agrees with this.
when it comes to /goal mode.. i think its the most stupidest thing there is, sure it keeps the model aligned to the goal. but when the model actually gets stuck at something, it really years a developer input, i have seen some thinking traces before that prove this point, the model really thinks about stopping and asking the developer what to do next, i think models cant really think multi-step and branched thinking. they think linearly and they cant really think of two things at the same time. and most lazy ass developers set goals like finish the implementation.. this is genuinely stupid.
Vibecode frontend completely, yes.. do that. but please do not vibecode backend, trust me
its very hard to maintain, when you try to implement a new feature, it ends up breaking stuff that you have to fix again, this is where 5.5 was good at, when i prompted it to implemnet somethign and then do actual regression testing it would remember stuff a lot better and did deeper regression testing.
look I don’t know their internals, so I’m not pretending any of that is proven. But more efficient for whom exactly?
we’re losing reliability along the way and calling the whole thing an upgrade. If I have to repeat instructions, babysit every action and fix the damage, the cost per token going down, in most cases means more token consumption.
i think most people, dont really know how to tell a difference between models at this point, give them a sort of blind test between grok and fable and they are not really able to tell which is which, this is because all the models have crossed the boundry that is what i call, "Basic human intelligence" and hence the models are, in most cases going to suggest things that are novel to an average human. And i would argue that Sol, astra are more Sycophant about stuff than any of the claude models have ever been. astra has never said that we should do this rather than this, whereas claude always has. it feels like claude doesnt care about me being sad with what it answers, perhaps it could shatter my expectations from a particular idea, but that would mean that the idea was stupid in the first place. astra and sol never want that to be the case, they care more about pleasing the user.
try this, say somethign stupid and tell the models to evaluate it, use fable and astra side by siode, tell both models to call out stupid when you (user) are stupid and see which model hesitates the most and which gives genuine advice.
i think genuine advice matters a lot, its liek claude treats me like a gigachad senior developer.
i would really like to know your thoughts on this, please share them below. in purely technical details if possible.