Your documents are not general-purpose,
Your retriever should be good beyond these
@tomaarsen recently fine-tuned mLateOn-unsupervised for medical retrieval using MIRIAD, a benchmark with 1,000 queries and 200,000 passages.
Out of the box, mLateOn reached 0.8520 NDCG@10.
After a short fine-tuning run on a single RTX 3090, mLateOn-medical reached 0.9139, outperforming every general-purpose retriever tested. Qwen3-Embedding-4B, the strongest dense model in the evaluation, reached only 0.7817, despite having roughly 33 times more active parameters.
The difference comes down to how information is represented. Dense models compress an entire document into one vector. Late-interaction models preserve token-level representations, allowing them to capture signals that single-vector models may average away.
This matters even more for long, specialized documents. In this evaluation, passages averaged 941 tokens.
The conclusion is simple: strong retrieval is not only about building larger models. It is about choosing an architecture that can understand your documents and adapt to your domain.
→ Explore Tom Aarsen’s complete fine-tuning guide :
huggingface.co/blog/train-mu…
→ Put advanced retrieval to work with LightOn Console
console.lighton.ai