Here, have a free 40% bump to 60-70tps from my optimization restack so far.
github.com/youssofal/MTPLX/p…
Actually its weird, the runner is a swiss cheese of inefficiency, if you look at the GPU starvation chart, about 20% of the time, the GPU is idle.
However its not a single starvation/drain event, the CPU is not feeding the GPU fast enough so the GPU is not queuing up stuff more than like 4% of the time.
Probably a good candidate for
mlx.fast right?