ModernBERT
#126 in Open-Source-SpraakmodelleUnknown · 2× · tolest 22. Sept. 2026
3
Momentum
ModernBERT is an Open Source encoder language model (BERT family) developed by Answer.AI and LightOn in collaboration with additional partners and released in December 2024. It is a bidirectional, masked language model trained on 2 trillion Tokens of text and code, supporting a native context window of up to 8,192 Tokens. ModernBERT is available in two sizes (Base with 149M and Large with 395M parameters) and serves as a drop-in replacement for classic BERT models in tasks such as classification, Retrieval, and semantic search.
Momentum-Verloop
08.08.06.11.
Features
| Key Benchmark (%) | GLUE score: 88.5 (Base) / 90.4 (Large) |
| Context Window (Tokens) | 8,192 tokens |
| License | Apache 2.0 (model architecture, weights and training codebase) |
| Multimodality | Not multimodal – text/code-only encoder model (no image/audio) |
| Platform | Hugging Face Transformers (from v4.48.0), open-source checkpoints (answerdotai/ModernBERT-base/large) |
| Release Date | December 19, 2024 |