SlideOn & mSlideOn: Sparsified Late-Interaction and Dense Retrievers
• 23
AI & ML interests
None defined yet.
Recent Activity
View all activity
Papers
Small Agents with Semantic Search: Efficient Multilingual Code Localization
DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Search
Articles
A collection of pretrained sparse autoencoders (SAEs) and fine-tuned sparse retrievers for English and multilingual, long-context, code retrieval
LightOn-rerank models are unified cross-encoder rerankers: Qwen3.5 LoRAs scoring both text passages and document page images against a query.
LightOnOCR-2-1B: a lightweight high-performance end-to-end OCR model family
-
lightonai/LightOnOCR-2-1B
Image-Text-to-Text • 1B • Updated • 577k • • 847 -
lightonai/LightOnOCR-2-1B-bbox
Image-Text-to-Text • 1B • Updated • 1.19k • 30 -
LightOnOCR 2 1B Demo
🐨129Extract text from images or PDFs with advanced OCR
-
lightonai/LightOnOCR-2-1B-base
Image-Text-to-Text • 1B • Updated • 3.68k • 14
State-of-the-art late interaction code retrieval models
-
lightonai/LateOn-Code-edge
Sentence Similarity • 16.8M • Updated • 8.26k • 39 -
lightonai/LateOn-Code
Sentence Similarity • 0.1B • Updated • 711 • 36 -
lightonai/LateOn-Code-edge-pretrain
Sentence Similarity • 16.8M • Updated • 107 • 4 -
lightonai/LateOn-Code-pretrain
Sentence Similarity • 0.1B • Updated • 145 • 3
State-of-the-art late interaction models trained using PyLate
-
lightonai/Reason-ModernColBERT
Sentence Similarity • 0.1B • Updated • 38.3k • 251 -
lightonai/GTE-ModernColBERT-v1
Sentence Similarity • 0.1B • Updated • 39.6k • 180 -
lightonai/LateOn-Code-edge
Sentence Similarity • 16.8M • Updated • 8.26k • 39 -
lightonai/LateOn-Code
Sentence Similarity • 0.1B • Updated • 711 • 36
This collection gather datasets for embeddings pre-training and fine-tuning.
French language models. These model were trained in early 2021 following the then scaling laws and using the exact same training data as the CamemBERT
High-Performance OCR and Layout Extraction in One Model
A collection of open state-of-the-art single and multi-vector models and datasets for multilingual, long context and code retrieval
A collection of open state-of-the-art single and multi-vector models for English retrieval
First large-scale fully pre-trained ColBERT model using only public data, outperforming GTE-ModernColBERT and GTE-ModernBERT
-
ColBERT-Zero: To Pre-train Or Not To Pre-train ColBERT models
Paper • 2602.16609 • Published • 9 -
lightonai/ColBERT-Zero
Sentence Similarity • 0.1B • Updated • 13.3k • 45 -
lightonai/ColBERT-Zero-supervised
Sentence Similarity • 0.1B • Updated • 105 • 3 -
lightonai/ColBERT-Zero-unsupervised
Sentence Similarity • 0.1B • Updated • 144 • 4
Visual long document VLMs based on Mistral-Small-3.1-24B-Instruct-2503 and Qwen3-VL-32B-Instruct
The Case for End-to-End and Efficient Domain-Specific Vision-Language Models for OCR
A collection of SOTA, open-data, paired encoder-only and decoder only models ranging from 17M params to 1B
Bringing BERT into modernity via both architecture changes and scaling
-
answerdotai/ModernBERT-base
Fill-Mask • 0.1B • Updated • 3.51M • 1.15k -
lightonai/GTE-ModernColBERT-v1
Sentence Similarity • 0.1B • Updated • 39.6k • 180 -
lightonai/Reason-ModernColBERT
Sentence Similarity • 0.1B • Updated • 38.3k • 251 -
lightonai/modernbert-embed-large
Sentence Similarity • 0.4B • Updated • 6.96k • • 34
A suite of autoregressive generative models for protein sequences, with up to 1.2Bparameters, trained on over 280 million protein sequences.
900M models trained on 25BT to compare different data processing choices (filtering, sentence dedup, minhash, etc)
A collection of pretrained sparse autoencoders (SAEs) and fine-tuned sparse retrievers for English and multilingual, long-context, code retrieval
High-Performance OCR and Layout Extraction in One Model
A collection of open state-of-the-art single and multi-vector models and datasets for multilingual, long context and code retrieval
LightOn-rerank models are unified cross-encoder rerankers: Qwen3.5 LoRAs scoring both text passages and document page images against a query.
A collection of open state-of-the-art single and multi-vector models for English retrieval
LightOnOCR-2-1B: a lightweight high-performance end-to-end OCR model family
-
lightonai/LightOnOCR-2-1B
Image-Text-to-Text • 1B • Updated • 577k • • 847 -
lightonai/LightOnOCR-2-1B-bbox
Image-Text-to-Text • 1B • Updated • 1.19k • 30 -
LightOnOCR 2 1B Demo
🐨129Extract text from images or PDFs with advanced OCR
-
lightonai/LightOnOCR-2-1B-base
Image-Text-to-Text • 1B • Updated • 3.68k • 14
First large-scale fully pre-trained ColBERT model using only public data, outperforming GTE-ModernColBERT and GTE-ModernBERT
-
ColBERT-Zero: To Pre-train Or Not To Pre-train ColBERT models
Paper • 2602.16609 • Published • 9 -
lightonai/ColBERT-Zero
Sentence Similarity • 0.1B • Updated • 13.3k • 45 -
lightonai/ColBERT-Zero-supervised
Sentence Similarity • 0.1B • Updated • 105 • 3 -
lightonai/ColBERT-Zero-unsupervised
Sentence Similarity • 0.1B • Updated • 144 • 4
State-of-the-art late interaction code retrieval models
-
lightonai/LateOn-Code-edge
Sentence Similarity • 16.8M • Updated • 8.26k • 39 -
lightonai/LateOn-Code
Sentence Similarity • 0.1B • Updated • 711 • 36 -
lightonai/LateOn-Code-edge-pretrain
Sentence Similarity • 16.8M • Updated • 107 • 4 -
lightonai/LateOn-Code-pretrain
Sentence Similarity • 0.1B • Updated • 145 • 3
Visual long document VLMs based on Mistral-Small-3.1-24B-Instruct-2503 and Qwen3-VL-32B-Instruct
State-of-the-art late interaction models trained using PyLate
-
lightonai/Reason-ModernColBERT
Sentence Similarity • 0.1B • Updated • 38.3k • 251 -
lightonai/GTE-ModernColBERT-v1
Sentence Similarity • 0.1B • Updated • 39.6k • 180 -
lightonai/LateOn-Code-edge
Sentence Similarity • 16.8M • Updated • 8.26k • 39 -
lightonai/LateOn-Code
Sentence Similarity • 0.1B • Updated • 711 • 36
The Case for End-to-End and Efficient Domain-Specific Vision-Language Models for OCR
This collection gather datasets for embeddings pre-training and fine-tuning.
A collection of SOTA, open-data, paired encoder-only and decoder only models ranging from 17M params to 1B
Bringing BERT into modernity via both architecture changes and scaling
-
answerdotai/ModernBERT-base
Fill-Mask • 0.1B • Updated • 3.51M • 1.15k -
lightonai/GTE-ModernColBERT-v1
Sentence Similarity • 0.1B • Updated • 39.6k • 180 -
lightonai/Reason-ModernColBERT
Sentence Similarity • 0.1B • Updated • 38.3k • 251 -
lightonai/modernbert-embed-large
Sentence Similarity • 0.4B • Updated • 6.96k • • 34
French language models. These model were trained in early 2021 following the then scaling laws and using the exact same training data as the CamemBERT
A suite of autoregressive generative models for protein sequences, with up to 1.2Bparameters, trained on over 280 million protein sequences.
900M models trained on 25BT to compare different data processing choices (filtering, sentence dedup, minhash, etc)