ModernBERT
#57 en Modèles de langage open sourceUnknown · 2× · vu le 22 sept. 2026
18
Momentum
ModernBERT is an Open Source encoder language model (BERT family) developed by Answer.AI and LightOn in collaboration with additional partners and released in December 2024. It is a bidirectional, masked language model trained on 2 trillion Tokens of text and code, supporting a native context window of up to 8,192 Tokens. ModernBERT is available in two sizes (Base with 149M and Large with 395M parameters) and serves as a drop-in replacement for classic BERT models in tasks such as classification, Retrieval, and semantic search.
Historique du momentum
24.06.22.09.
Fonctionnalités
| Key Benchmark (%) | GLUE score: 88.5 (Base) / 90.4 (Large) |
| Context Window (Tokens) | 8,192 tokens |
| License | Apache 2.0 (model architecture, weights and training codebase) |
| Multimodality | Not multimodal – text/code-only encoder model (no image/audio) |
| Platform | Hugging Face Transformers (from v4.48.0), open-source checkpoints (answerdotai/ModernBERT-base/large) |
| Release Date | December 19, 2024 |