AI @LightonIO Language Models, Information Retrieval, Knowledge Distillation PhD

Paris, France
Releasing ColGREP and LateOn-Code models 🚀 ColGREP is a multi-vector search tool built in Rust made for coding agents. It's an hybrid grep which supports both grep features and semantic retrieval. Run 100% locally. You get two SOTA code retrieval model within ColGREP
7
19
135
11,111
Raphaël Sourty retweeted
Opus 5.5 as well @IgorCarron @antoine_chaffin
Kudos to @LightOnIO for being ChatGPT's preferred late interaction model ;)
1
2
7
577
🥹
Kudos to @LightOnIO for being ChatGPT's preferred late interaction model ;)
2
16
775
Raphaël Sourty retweeted
has anyone tried colbert/late interaction on memory search? we benched some close sourced (API-based) dense retrievers with an in-house multi-vec model, never optimised for memory search, the gap is huge:
14
5
67
3,506
Raphaël Sourty retweeted
We're sharing new research on our post‑training approach, which teaches the Perplexity Computer agent to learn from real user sessions by imitating good trajectories and explicitly correcting avoidable mistakes like bad tool calls (even when the overall trajectory was successful). The method combines rejection sampling fine‑tuning (RFT) with hint‑guided self‑distillation, and it cuts tool call failures by about 21% in live A/B tests
New research: We post-trained a Computer model to learn from its own errors using hint-guided self-distillation. In a live A/B test, a later trained checkpoint reduced tool-call failures by 21.2% relative to an earlier checkpoint.
36
41
483
68,465
Raphaël Sourty retweeted
All generated using official checkpoints via PyLate from @LightOnIO. More to come, when my machine is not busy running experiments on TACHIOM or kANNolo (not easy 🤓) You can find everything in the repo: github.com/TusKANNy/awesome-…
1
6
143
Raphaël Sourty retweeted
Not as sexy as generative decoders, but Sentence Transformers models for search have slowly taken over the most downloaded @huggingface models page over the last few years. I'm hoping to see a further resurgence of encoders with the new interest in Jevlike zero-shot encoders.
3
14
85
5,112
Raphaël Sourty retweeted
There are few examples in science and technology where people were somehow impacted by their own discoveries/work. I can think of two on top of my head: Glenn Seaborg, the discoverer of Plutonium who was triggered into finding rapidily decaying iodine for medical imagery. His mother was the recipient. There is also @EricRFossum whose CMOS for imaging equips *all* smartphones. And now with AI, @ylecun one of the AI leaders who also started ICLR, is at the forefront of how AI is disrupting the scientific process itself. We are living in interesting times.
3
4
303
Raphaël Sourty retweeted
hear me out: jev, but with the ability to think longer before harder decisions oh and maybe with composite types like dicts and lists and not only bools 🤯
41
18
549
39,018
Raphaël Sourty retweeted
thoughts on Typesafe/Jev: I'm surprised that a general-purpose classifier (or discriminative model) can be just as interesting to the public as a general-purpose generative model. I've worked on search and discriminative models for years, so maybe I should have called this upfront, tbf I never thought about making them general-purpose beyond search. Typesafe also positions Jev really well as "System 1", a complement to the generative models we already have. Humble, unassuming, and sidestepping the frontier-lab warzone. Jev's success could be disruptive to today's agentic systems in many ways, such as tool-calling and routing patterns. Over the last three years the consensus was to use a small generative model for routing, tool calling, MCP etc. Time to pause and rethink such architecture, because we'll probably see tool calling and routing move back to discriminative models. Whoever ships the next Jev-like open-weight base model will likely win the community.
49
55
685
38,941
Raphaël Sourty retweeted
Cheating crystallizes into a single readable direction, that's the optimistic part ! As models scale, the concepts we most need to monitor *may* get more legible, not less Capability & interpretability, coevolving instead of trading off arxiv.org/abs/2609.19101
Models know when they’re reward hacking. But they still do it a ton - in 50-96% of rollouts we studied! We built activation monitors that detect the behavior behind the Hugging Face hack in real time. This can help us stop hacks now - and train future models that don’t cheat. 🧵
8
86
3,565
Raphaël Sourty retweeted
I’m super proud of @thibault_formal and the @Linkup_platform Research team for releasing SPARSEUP! This was a missing piece in the open source community and the strongest public sparse retriever at this size (<150M). Congratulations!
Introducing SPARSEUP: the first model release from @Linkup_platform , and the missing sparse companion of DenseOn and LateOn. Same backbone, same data, <150M, 56+ on BEIR-13. The model is open-source, use it! Blog: linkup.so/blog/introducing-s… Model: huggingface.co/Linkup-Platfo… (Apache 2.0)
6
27
2,903
Raphaël Sourty retweeted
The original author of the SPLADE line (THE sparse embedding architecture imo), just released the biggest advancement of Sparse embedding models since SPLADE-v3! - 150M, beats every sparse model up to 1B at search - Reaches sub-ms latency using Seismic (!) Go @Linkup_platform !
Introducing SPARSEUP: the first model release from @Linkup_platform , and the missing sparse companion of DenseOn and LateOn. Same backbone, same data, <150M, 56+ on BEIR-13. The model is open-source, use it! Blog: linkup.so/blog/introducing-s… Model: huggingface.co/Linkup-Platfo… (Apache 2.0)
4
7
107
4,783
Raphaël Sourty retweeted
Introducing SPARSEUP: the first model release from @Linkup_platform , and the missing sparse companion of DenseOn and LateOn. Same backbone, same data, <150M, 56+ on BEIR-13. The model is open-source, use it! Blog: linkup.so/blog/introducing-s… Model: huggingface.co/Linkup-Platfo… (Apache 2.0)
11
20
113
86,061
Raphaël Sourty retweeted
Nice write-up about all the infra challenges associated with Online RL, with neat animations
2
5
52
8,545
Raphaël Sourty retweeted
🤗Proud to see Sentence Transformers' all-MiniLM-L6-v2 trending on @huggingface! Now stop using it, it's not 2021 anymore. You're leaving performance on the table.
10
9
97
4,736
Raphaël Sourty retweeted
New repository type on @huggingface 👀 huggingface.co/kernels/Helio…
Helion × 🤗 Kernels is now live. The key idea: autotuned, portable kernels can be packaged on the Hub with their pre tuned configurations, then loaded by users without local tuning or dependency friction. Reported results: • Helion attention beats PyTorch SDPA on all 19 tuned H100 shapes: 1.20× geomean speedup • It wins on 9 of 10 held out shapes: 1.17× geomean speedup • Seven linear attention variants beat FLA on all tuned B200 shapes: 1.41× on device and 1.33× end to end • Forward plus backward: 1.55× geomean speedup This makes the artifact being shared more than source code. It includes the performance decisions required to use the kernel efficiently. pytorch.org/blog/helion-x-%F…
1
5
29
2,215
Raphaël Sourty retweeted
From the NeoMME paper: only -2% relative nDCG@10 at 5% of the full index size. Crazy good tradeoff. And that pooling will be way faster soon 👇🏼
A little teaser of what I've been working on over the weekend 🤗
1
3
20
3,553
Raphaël Sourty retweeted
a great way to clarify what people are best at is to ask or imagine what they'd do with an empty calendar day
7
1
99
27,232
Raphaël Sourty retweeted
Amazing optimizations coming soon ! Stay tuned
Apache OpenDAL 0.57 added access to @huggingface Buckets in rust 🦀 - nice file system interface - list/read/write support - optimized with Xet dedupe - with bindings in many languages The best part: it's also in arrow-rs to read @ApacheParquet data (via object_store_opendal)
2
12
1,129