The deeper problem with autonomous software is not whether a model can make a decision.
It is whether the system can establish, years later, what information the decision used, what authority existed, and why the action was permitted.
That is an evidence problem.
And evidence is an architectural problem.