Architecting systems for efficiency. Developer tools and AI systems. Currently building xaltos.dev/
47400, Arawood
- Tweets 1,746
- Following 237
- Followers 93
- Likes 260
Building and designing my own IDE called Xalt. Build native iOS apps using native Swift + SwiftUI without leaving window or renting Mac in cloud.
Join the waitlist here :
benmodev.co/work/xalt#waitli…
7
11
7,777
Same rsync release. Two teams. Different bugs. That's the part that stuck with me. For old code, I'd budget for genuinely independent methods before another pass with the same checklist.
Trail of Bits worked the same rsync 3.5.0 release I was on.
They found issues we didn't. We found issues they didn't, including some that had survived for decades. In ten cases, since the first release in 1996.
2
42
The approval screen matters as much as the manifest. 'One deny line removed' is easy to miss in a code diff. 'This update can now delete repositories' is hard to miss. That translation could make the spec usable beyond security teams.
Today at @WeAreDevs, Docker and the @linuxfoundation are announcing a collaboration around the Docker Sandbox Kit Specification: an open standard for declaring what an agent may do, where it may reach, and what it may touch.
Open source under Apache 2.0.
Read the deep dive: bit.ly/4rrcjql
1
53
Being able to try an interaction three ways while the idea is still fresh changes the quality of the decision. Tiny experiments often spent more time as tickets than as code. I get why that muscle coming back feels so good.
35 years ago I did it all myself. Wrote, built, sold, and released software solo.
25 years ago I did all the design myself. Sketches, then HTML, then CSS.
15 years ago I still did some hands on design, but mostly directed others to implement my vision.
5 years ago I primarily directed others. My "I can do it myself" muscles had atrophied.
Today, I feel like I'm back to the beginning. Able to write and design software entirely by myself. And having an elite team of others who can do the same thing is an enormous multipler.
Wild how massive advancements can propel you FORWARD by bringing BACK capabilities you lost along the way.
Full circle feels more like an arrow.
44
That last point deserves more airtime. When an agent sends a compiler PR, put the bug, intended behavior, and regression case in the description. A maintainer shouldn't have to reconstruct the reasoning from a diff produced in minutes.
This was one of those weeks where too many things happened. Here is some unsolicited advice:
1. Poor unit tests are a consequence of poor design and bad practices
2. The push to formal verification is great, find where it works and where it doesn't, and then find the next thing that makes your software better
3. Communities where their sense of belonging are around ergonomics may need to reassess what brings them together when AI is writing all of the code. And token efficiency is a short-term and low-value metric to tie yourselves to. Models already got cheaper this week!
4. I receive and write dozens of AI written pull requests every week and, if I stopped reviewing code produced by AI, I am 100% confident we would have a slower Elixir compiler, buggier type system
See you next week lovely people
38
One fewer container to update, inspect and keep compatible with the rest of the stack.
That's a very decent feature for the people running it. Software getting a little easier to host deserves some excitement too, even if it makes for a fairly uneventful demo video.
1
57
I keep thinking about how much useful tooling starts with making an app callable without its UI.
Shopify's Helix uses a CLI that exposes the same screen state and actions as the app. It can check behavior repeatedly before taking a screenshot.
That interface could earn its keep across bug reports, regression tests and the next migration too. "Reproduce this order state and run this action" is a useful thing to be able to ask of your software.
When we announced our move back to native, we got a lot of questions about how Helix works. Today we’re sharing more details.
• Instead of expecting a perfect first attempt, Helix breaks work into small checkpoints that must pass strict quality gates before moving forward.
• Different AI models are good at different things. Gemini has been especially effective at catching subtle visual differences during UI reviews.
• Engineer feedback goes into memory, so Helix gets more autonomous with each checkpoint. It can run overnight without approvals, and the gates stay just as strict.
Link to blog post👇
2
2
630
One agent runs tests while another edits the same checkout. Both have perfectly reasonable instructions.
That's enough to make an always-on agent box less relaxing than it sounds.
Git worktrees give each job a separate working directory. It's also worth separating mutable test data and build outputs, and assigning ports explicitly.
I like the idea of treating each job as something you can inspect on its own: the code it ran, the environment it used, the result it left behind.
1
4
59
An agent can make billable model calls while repeatedly trying something that will never work.
For a product built on that agent, the retry policy becomes part of the pricing model. Absorb those attempts and they eat into the margin. Pass them through and the customer pays for a failure they may never see.
A job budget, a stopping rule, and a clear policy on failed work belong in the product from the start.
I'd be pretty annoyed to discover I'd paid for an argument with a broken tool.
1
23
If I take the time to report a bug, a message saying it was fixed would make me more willing to report the next one.
Connecting a shipped fix to the person who first flagged it feels like a useful job for AI. Easy to forget when those conversations happen in different places.
today an engineer found a bug where we over-billed some event types, and made a fix
Modem pinged him - a customer flagged this 2 weeks ago, but it got lost in the noise
he had no idea! so he followed-up & let them know it was addressed
5 mins of setup → never miss a follow-up
1
2
127
The human job here is still substantial: decide whether the specification says what you actually need.
Putting the theorem statements and assumptions in the README gives a reviewer something precise to disagree with. I'd love for that to become an ordinary part of reviewing software.
Passion project: Durable execution on S3, specified and proved in Lean 4.
Two specifications: an abstract model of promises and tasks, and a concrete model of buckets and blobs. Plus a machine-checked proof that the concrete refines the abstract.
Work in progress. Link below.
2
63
There's a very unglamorous reason to send fixes upstream: otherwise you get to maintain them yourself every time the project moves on.
I like seeing AI infra companies treat that shared maintenance as part of the business. Everyone selling the service has a stake in the software underneath it staying healthy.
So much of our alpha is in the infra layer anyway. So we’d rather work with open source projects and contribute back. Thanks @sgl_project for building amazing stuff!
2
50
29 yes/no questions just to type one letter. this is such a wonderfully unnecessary thing to build lol
It does make me think about agents choosing tools. If you always pick the highest-scoring option, something wins even when every option is bad.
I'd give 'none of these' a seat at that table before letting the winner run a command.
i made an llm from first principles with Jev
29 yes/no questions per character: should the next key be a–z, space, comma, or period?
highest probability gets append to it, then fed the updated text back in & repeat
an autoregressive loop made out of a classifier
2
72
I like the habit of checking my own brief first. The 'every time' would wear me down though haha
If I left out a constraint, fair enough. If I've explained it, shown an example and corrected the same mistake twice, rewriting the prompt forever gets expensive.
I'd save that failed task and try it again with a fresh session or another model, keeping the brief unchanged. At least then there's something to compare before deciding I need another prompting tutorial.
"Your skills are the ceiling of how much the agent can do"
-Matt Pocock
One of my favorite quotes from him, every time I get a bad output whatever model I use i just say I'm not good enough and I should learn more.
2
50
That chart counts PRs, so I'm reading this as a review-queue problem.
I'd put a small limit on how much unfinished work an agent can open. When the queue fills, it can address review comments, reproduce a failure, or help finish an existing change.
Otherwise you can end up with a very busy software factory and one person spending their whole evening figuring out what can safely merge. A queue of abandoned drafts is still work someone has to deal with tomorrow.
Fresh data from GitHub:
Agent-generated PRs have exploded in size. 9x the last 8 months (!!)
No signs of slowing down. This is why everyone is rethinking code reviews, deploys, possibly even o11y
thanks @kdaigle + team for the stat!
(Slide from my talk yesterday)
45
Parallel investigation makes sense to me when the questions are independent. The agent making the change still needs access to the evidence behind the plan.
Polylane’s switch to one agent coincided with median issue-to-PR time falling from 2.2 hours to 35 minutes, and average model cost per opened PR from $111 to about $18. The model and other parts of the system changed too.
The handoff described in the article left the coding agent working from a plan without the original investigation. Keeping relevant files, test results and unresolved assumptions reachable seems worth the extra work.
PR-open time is useful. Review outcomes and confirmation that the original issue is resolved deserve their own measures.
ALT An evidence card summarizing Polylane’s September 14 production write-up. Following a September 3 change from a workflow of agents to one agent, reported median time from detecting an issue to opening a pull request fell from 2.2 hours to 35 minutes. Average model cost per pull request fell from 111 dollars to about 18 dollars over the first nine days. The card states that model, architecture and other changes overlapped, so this is not an isolated test of agent count.
if you're building sub-agents you should stop now
our main agent used to be a workflow with sub-agents, each with a specific role (triaging, investigating, etc.)
we moved to a single agent and latency dropped, quality improved and cost was slashed
polylane.com/blog/sub-agents…
1
2
115
The charge went through. The customer saw a timeout.
This is the case I’d add to the “the button works” discussion. In a sandbox, let the payment request complete, interrupt the response, then retry.
A fresh operation key can turn that retry into another charge. The original key and unchanged payload let an idempotent API return its recorded result, within its retention window.
The panels show both alternatives from the same timeout. Your own order records and webhook handling still need protection against duplicates.
I’d test this before a broad code cleanup. What failure case has earned a place in your pre-launch checks?
ALT First panel of an illustrative charge-request trace. The client sends operation key checkout-A to the API. The API commits the charge and stores the result, receipt 421. A response line extends toward the next panel. This is an explanatory scenario, not an observed payment or a transaction spanning an external provider and a local order database.
ALT Middle panel of the same trace. The response line breaks at an X marked Response lost. The client sees a timeout and its outcome is unknown, even though the charge completed. A retry arrow leads into the two alternatives in panel 3. A sandbox test completes the request, interrupts the response and then retries it.
ALT Final panel shows two alternative retries from the same uncertain outcome. A new operation key, checkout-B, can create another charge, receipt 422. Reusing checkout-A with the same payload lets a server that enforces idempotency return the existing result, receipt 421, while the key is retained. These are alternative paths; no duplicate charge is undone. Order and webhook code need their own protection too.
1
34
I wanted to build an iOS app on the Windows machine I already had. It sent me into a deeper question: which parts of development actually need a Mac?
Two papers helped frame the research. Here's the workflow I think is worth investigating.
2
1
203
Ben Mo retweeted
Rumours are circulating that GPT-6 Sol is about to be released, which aligns with Tibo's statement that numerous major releases are scheduled for this week, enough to fill a Dev Day.
As I understand it, GPT-6 Sol will be positioned below Astra, just as Opus is below Fable in the tier. Of course, the question arises: why?
My guess is that GPT-6 Sol will offer significantly better value for money than Astra. It will likely be close to Astra's performance, but similar to Opus, with better rates and costs. At the same time, I'm equally excited for GPT-6 Luna, as version 5.6 Luna was incredibly good in terms of both price and performance.
But who knows: perhaps we'll be surprised, and Sol will even be better than Astra in some aspects, with Astra receiving its next iterative update soon. After all, their internal model demonstrates how quickly improvements can be achieved through post-training.
With Opus 5.2 just around the corner, the update cadence continues. It remains exciting!
131
120
3,056
488,054
Building and designing my own IDE called Xalt. Build native iOS apps using native Swift + SwiftUI without leaving window or renting Mac in cloud.
Join the waitlist here :
benmodev.co/work/xalt#waitli…
7
11
7,777