Sglang · 3× · tolest 24. Aug. 2026
43
Momentum
SGLang is an Open Source framework for programming and serving Large Language Models and multimodal models, maintained by the nonprofit organization LMSYS. Its core is a backend runtime featuring RadixAttention for automatic KV-Cache reuse, along with capabilities such as continuous batching, speculative decoding, disaggregated Prefill/Decode, and quantization. The system is designed for low latency and high throughput in production workloads, supporting a wide range of open models (including Llama, Qwen, and DeepSeek) via an OpenAI-compatible API. According to its creators, SGLang runs in production on over 400,000 GPUs worldwide and processes trillions of Tokens daily.
Momentum-Verloop
26.05.24.08.
Features
| Deployment (Self-Hosted/Cloud) | Self-hosted from single GPU to distributed clusters; used in production across more than 400,000 GPUs worldwide |
| Throughput/Latency | Designed for low latency and high throughput via RadixAttention, prefix caching, and multi-GPU parallelism |
| License | Apache License 2.0 |
| Platform | NVIDIA, AMD, Intel Xeon, Google TPU, and Ascend NPU hardware; install via pip, source, or Docker |
| Price | Free, open source (no license fees) |
| Protocol Compatibility | Compatible with OpenAI API and Hugging Face models |
| Release Date | Initial release January 17, 2024; current versions ship continuously (e.g., v0.4 in December 2024) |
| Supported Models/Providers | Broad support including Llama, Qwen, DeepSeek, Gemma, Mistral, GLM, LLaVA, plus embedding and reward models |