bitk0rns retweeted
Pollard Weights is criminally underrated. 🚨
Most people still do uniform quants and hope the model fits. @Mr_WestCoast starts from a better question: what actually needs to stay in RAM?
That one shift is why this repo is so good if you run local models on real hardware now. 👇
It measures the working set, pins the hot tensors, streams the rest from flash, then allocates bits by sensitivity instead of treating every layer the same.
In practice, it helps you:
> Run bigger models on 16GB machines
> Keep quality higher than a normal low-bit quant
> Check if a model fits before downloading it
> Build once, then export to GGUF, GPTQ, MLX, or EXL3
> Use VRAM for the important parts, not everything
> Split huge models across more than one machine
If you run models locally, this is absolutely worth trying!
The site is still being developed, just doing it as I go, there will be issues I will need to take care of.
Pollard.app
github.com/WestWaters/pollar…
1
1
8
1,036
bitk0rns retweeted
Introducing
FlyBrain-Pollard-CNSv1
A language model that remembers with a fruit fly's brain
Every transformer has the same problem. Its memory is a KV cache, and that cache grows with every token. Long conversations reach gigabytes. Hit the context limit and the beginning is not compressed or summarised — it is gone.
A fruit fly has about 190,000 neurons and no context window. It does not forget the start of a chase because the chase got long.
So we used one.
FlyBrain-Pollard-CNSv1 attaches a real fruit-fly connectome to a frozen language model as a continuous recurrent memory. The connectome is not a metaphor. It is MaleCNS v1.0 — the measured wiring of an adult male Drosophila central nervous system, released by FlyEM/Janelia with Google Research and the University of Cambridge: 188,778 neurons and 26,028,386 synaptic connections, with excitatory or inhibitory signs called for 88% of annotated neurons.
CNSv1 runs the fly's associative-memory core — mushroom body and central complex — pruned to the connections carrying 69.8% of its synaptic mass: 8,552 neurons, 300,880 synapses. One neural step per token. Continuously. No window of its own.
What it does
The model sees a 128-token attention window. The fly brain sees everything.
lossperplexity
128-token window, no memory1.77335.89
128-token window + fly brain1.62715.09
full attention over the whole document1.62325.07
The fly brain closes 97.4% of the gap. A model that can only attend to 128 tokens performs essentially identically to one attending to the entire document — and it was still improving when training stopped.
What it costs
+3.1% decode speed, inside run-to-run variance, while stepping 300,880 synapses every token.
0.03 MB of state — 8,552 floats. That number does not change. Not with sequence length, not with conversation length, not ever.
The part that matters most
The memory is a file. 35 KB.
Save it when a session ends. Load it when the next one begins. The model resumes mid-thought. There is no session boundary because there is no context window.
A 131k-token KV cache holding the same conversation is roughly 25.8 GB — 720,000× larger, and growing with every turn. You can email a fly brain's memory. You can version it, branch it, hand it to another machine.
huggingface.co/PollardWeight…
4
4
16
1,121
I saw @BenJustman join this movement too.
small businesses are going viral by participating in a trend where they declare they don’t use ai flyers
2
1
15
1,423
Ya mon! Listen good: If Bitcoin pass $82k, we officially inna di bull market! Galaxy Research seh di 50-week moving average sit right around $82,470. In 11 outta 13 times history, when BTC reclaim dat level on di weekly close, di bear market low already done. Once it flip past 82k, di bulls tek over fi real. Irie!
41
54
1,056
64,072
