ModernBERT
#57 in Open-Source LLMsUnknown · 2× · last seen Sep 22, 2026
18
Momentum
ModernBERT is an Open Source encoder language model (BERT family) developed by Answer.AI and LightOn in collaboration with additional partners and released in December 2024. It is a bidirectional, masked language model trained on 2 trillion Tokens of text and code, supporting a native context window of up to 8,192 Tokens. ModernBERT is available in two sizes (Base with 149M and Large with 395M parameters) and serves as a drop-in replacement for classic BERT models in tasks such as classification, Retrieval, and semantic search.
Momentum trend
24.06.22.09.
Features
| Key Benchmark (%) | GLUE score: 88.5 (Base) / 90.4 (Large) |
| Context Window (Tokens) | 8,192 tokens |
| License | Apache 2.0 (model architecture, weights and training codebase) |
| Multimodality | Not multimodal – text/code-only encoder model (no image/audio) |
| Platform | Hugging Face Transformers (from v4.48.0), open-source checkpoints (answerdotai/ModernBERT-base/large) |
| Release Date | December 19, 2024 |