Take Control Of Your Data. Join our Data Science Briefing newsletter for the best in #DataScience and #MachineLearning
Manhattan, NY
- Tweets 7,540
- Following 461
- Followers 1,462
- Likes 353
4/
The result: a bad prompt change dies in the queue, not in front of users.
Vigilance doesn't scale. Policy does.
I'm walking through the full setup live on Oct 7 — free, 30 min, notebook included.
Register: maven.com/p/07c4b3/stop-bad-…
18
3/
Wire it into CI like this:
- eval script runs as a CLI step in your PR workflow
- it exits with code 1 if any metric falls below threshold
- GitHub Actions fails the check
- branch protection blocks the merge
The whole thing fits in one YAML file. No new tooling required.
1
16
2/
Start with your threshold — and don't guess it.
Measure your eval metric across a baseline set of inputs. Compute the standard deviation. Your threshold is mean minus N×std.
Now you have a number you can defend in a postmortem. "We set it from the data" ends the argument.
1
10
1/
Most LLM teams have evals. Almost none have a gate.
The difference: evals tell you something got worse. A gate stops you from shipping it.
Here's how to build one in GitHub Actions. 🧵
1
166
A bad prompt change ships the same way a good one does: nobody measured either.
Free session Oct 7 — I'll show you how to wire an eval gate into GitHub Actions so regressions die in the queue, not in prod.
maven.com/p/07c4b3/stop-bad-…
1
216
Last chance — free LLM eval session is TODAY at 2 PM EDT.
Learn to put error bars on your scores in 20 lines of Python.
Stop chasing phantom regressions. Start trusting your numbers.
maven.com/p/48c82e/put-error…
185
Cognitive immunization = build the off-ramp.
Full write-up + open-source model (`epidemik`): data4sci.substack.com/p/the-…
4
And the fix is asymmetric.
Fighting the spread costs ~a 50% cut in transmission (expensive).
Making dependence easy to walk back — exportable workflows, retained skills — moves the escape boundary almost linearly (cheap).
1
9
Result 2 — it's a trap door, not a door.
Run the dynamics backward and the locked-in state persists well BELOW the threshold that created it. Classic hysteresis.
Plain version: it's far easier to keep a dependent population than to make one.
1
17
Result 1 — there IS a tipping point.
Past a transmission threshold, casual adoption stops being a smooth climb. Dependence JUMPS — the model predicts a swing from ~0% to ~60% of a population before it settles.
Not a slope. A phase change.
1
15
I modeled our dependence on LLMs as an epidemic — the same SIR machinery used for real infections. Compartments: unexposed -> casual -> dependent.
The equations returned something uncomfortable. Thread 🧵
1
185
1/ Your LLM eval says accuracy went from 71% to 74% after the prompt change. Is that real improvement or noise?
Without confidence intervals, you can't tell. Here's how bootstrap CIs fix this 🧵
3
1
2
176
Computational approaches and the future of urban crime research — This review offers a necessary reality check on how computational methods are reshaping urban crime research, moving the field beyond simple prediction toward… nature.com/articles/s41586-0…
1
2
108
Limits of Inference in Complex Systems: When Stochastic Models Become Indistinguishable | Phys. Rev. X — journals.aps.org/prx/abstrac…
3
11
222
4 days until my Graph RAG workshop.
Here's what you build in 3.5 hours:
→ Entity extractor from raw text
→ Coreference resolver
→ Relation extraction pipeline
→ Knowledge graph in NetworkX
→ Grounded LLM chatbot on top
Certificate included. Last chance at 40% off
3
1
2
142
[2604.23678] Transferable Human Mobility Network Reconstruction with neuroGravity — Interesting approach to reconstructing human mobility networks with a model that’s designed to transfer across settings. Worth a look if you’re working with… arxiv.org/abs/2604.23678
2
129
How to Build AI Agents for Free Using Open Source — Good to collect patterns you can operationalize (and failure modes to avoid). opendatascience.com/how-to-b…
2
39
[2309.12288] The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" — Useful reminder that LLM “knowledge” can be direction-dependent: training on “A is B” doesn’t guarantee it can infer “B is A.” Worth reading if you’re designing… arxiv.org/abs/2309.12288
1
3
95
Multi-hop question: "What did the CEO of Company X's subsidiary do in 2019?"
Vector RAG: retrieves chunks about Company X. Struggles with the chain.
Knowledge graph: follows the ownership edge, finds the subsidiary, finds the CEO, finds the 2019 event.
1
1
5
225
[2606.16140] VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models — Interesting look at how far a 3B-parameter model can go on verifiable reasoning, with an emphasis on making intermediate steps checkable… arxiv.org/abs/2606.16140
1
4
260