Unknown · od January 17, 2024 · 2× · naposledy 29. 10. 2026
24
Momentum
SGLang is an open-source serving framework for large language and multimodal models, hosted by the non-profit LMSYS. It uses RadixAttention for prefix caching and offers an OpenAI-compatible API. The official documentation is at docs.sglang.io, the source code on GitHub.
Vývoj momenta
01.08.30.10.
Vlastnosti
| Throughput/Latency | Up to 5x faster inference with RadixAttention (project claim, January 2024) |
| License | Apache License 2.0 |
| Platform | Python, Rust, CUDA, C++; GPU hardware: NVIDIA, AMD, Intel Xeon, Google TPU, Ascend NPU, Moore Threads MUSA |
| Price | Free (open source); costs only for GPU infrastructure |
| Protocol Compatibility | OpenAI-compatible API, Anthropic-compatible API (/v1/messages), Ollama-compatible API |
| Release Date | January 17, 2024 (publicly introduced) |
| Supported Models/Providers | Llama, Qwen, DeepSeek, Mistral, Gemma, GLM, Kimi, etc. (LLMs); LLaVA, Qwen-VL, InternVL (multimodal) |