Eleutherai · siet 13. Februar 2023 (Modellsuite); Paper veröffentlicht 3. April 2023 · 2× · tolest 19. Aug. 2026
23
Momentum
Pythia is a suite of 16 open-source autoregressive transformer language models from EleutherAI, ranging in size from 70 million to 12 billion parameters. All models were trained on the exact same data (the Pile dataset) in identical order to enable scientific research on the interpretability and learning dynamics of LLMs. For each of the 16 models, 154 training checkpoints are publicly released, allowing analysis of model development throughout training. Pythia is freely available under the Apache 2.0 license and is designed primarily for research purposes rather than commercial production use.
Momentum-Verloop
21.05.19.08.
Features
| Context Window (Tokens) | 2,048 tokens (depending on model size, per training sequence length) |
| License | Apache 2.0 |
| Multimodality | Not multimodal – text-only causal language model |
| Platform | Open source, available on GitHub and Hugging Face, trained using the GPT-NeoX library |
| Price per 1M Tokens | Free (open-source model weights for self-hosting, no official API pricing) |
| Release Date | February 13, 2023 (model release), paper published April 3, 2023 |