Building the Trust Layer for AI.
San Francisco
- Tweets 5,616
- Following 4
- Followers 238,246
- Likes 11,582
Is this true? 👀
The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger.
4
1
17
5,599
95% accurate sounds reassuring until your agent is about to act on the 5%.
Benchmarks tell you how a model performed across a test set.
They don't tell you whether the output sitting in front of you right now is correct.
3
25
6,705
How often do you get misled by exaggerated claims in a week?
35%
Everyday
13%
Every hour
32%
That's just the internet
19%
Never
31 votes • Final results
1
1
19
5,710
In Saturn’s test of 18 AI models, financial answers were wrong 57% of the time on average. For complex questions, that rose to 88%.
Financial tasks expose a difficult property of AI: an answer can be internally coherent while resting on a false premise.
The arithmetic can be correct while the tax rule is outdated. The source can be real while applying to the wrong jurisdiction. Every subsequent step can look reasonable because it inherits the same initial error.
Give that output access to tools, accounts and payment rails, and the error can propagate into an action.
This is why verification needs its own place in the architecture.
Before execution, a financial agent should have to pass several distinct checks:
• Evidence: do the sources actually support the claims?
• Context: are those sources current and applicable to this user?
• Computation: can the numbers be reproduced with deterministic tools?
• Authorization: is the proposed action within the user’s permissions and limits?
Asking the same model to reconsider its answer provides limited independence. Agreement between multiple models is also insufficient when they share the same blind spots.
Verification needs external evidence, explicit constraints and a way to stop execution when checks fail.
The architectural question for agentic finance is becoming unavoidable:
What has to be independently checked before an output is allowed to move money?
7
20
6,767
A model can encounter the same wrong number on a thousand websites.
It may look like a thousand supporting examples.
In reality, they might all trace back to one bad source.
8
2
27
6,338
Self-review helps catch some errors.
It can also repeat the same assumption in a more convincing paragraph.
The writer and reviewer are working from the same learned patterns.
1
376
AI benchmarks ask: can the model solve the hardest problem?
Production asks: can the system solve the same boring problem 10,000 times without calling the wrong tool, changing a correct number or skipping a step?
If you're dealing with agents, the second question matters more.
7
3
30
7,149
1/ Give one LLM an entire workflow and every stage becomes probabilistic.
> Understanding the request.
> Choosing the tool.
> Doing the calculation.
> Applying the rules.
> Writing the answer.
That produces four very different kinds of failure:
4
2
21
7,031
7/ A better structure:
Extract: use the model to interpret language.
Compute: use deterministic code for calculations and rules.
Render: use the model to communicate the result.
Verify: independently check what the system is about to return.
1
1
375