Llada Instruct
#41 v Open-source jazykové modelyunknown · od 2025-02-14 · 2× · naposledy 29. 6. 2026
12
Momentum
LLaDA-8B-Instruct (Large Language Diffusion with mAsking) is an 8-billion-parameter language model developed by the GSAI-ML group (Renmin University of China), trained entirely from scratch without building on any existing autoregressive model. It employs a masked diffusion architecture: during pre-training, tokens are randomly masked and the model learns to iteratively reconstruct them. After supervised fine-tuning (SFT) on 4.5 million pairs, LLaDA-8B-Instruct exhibits instruction-following capabilities comparable to LLaMA3 8B Instruct, but without reinforcement learning. The model is released under the MIT license and available on Hugging Face.
Vývoj momenta
04.04.03.07.
Vlastnosti
| Benchmark Score (MMLU/Similar) | MMLU (5-shot): 65.9 (Base); LLaDA 8B Instruct: GSM8K 69.4 / MATH 31.9 / GPQA 33.3 / HumanEval 49.4 / MBPP 41.0 (per official paper, Tab. 2) |
| Model Size (Parameters) | 8 billion parameters (8B), trained on 2.3 trillion tokens; SFT on 4.5 million pairs |
| Price Tier | Free / Open Source (MIT license); model weights freely available on Hugging Face (GSAI-ML/LLaDA-8B-Instruct) |