Unknown · od 2023 (Erstveröffentlichung); aktuelle Stable-Version 0.32.0 vom 18. Oktober 2026 · 2× · naposledy 29. 10. 2026
26
Momentum
vLLM is an open-source inference and serving engine for large language models, originally developed at UC Berkeley's Sky Computing Lab. Core features are PagedAttention, continuous batching and an OpenAI-compatible API server. Official documentation is at docs.vllm.ai, the project site at vllm.ai.
Vývoj momenta
01.08.30.10.
Vlastnosti
| Throughput/Latency | High serving throughput via PagedAttention and continuous batching (no figures given in source) |
| License | Apache License 2.0 |
| Platform | NVIDIA CUDA, AMD ROCm, Intel XPU, Apple Silicon, CPU (x86, ARM, IBM Z) |
| Protocol Compatibility | OpenAI-compatible API (/v1: Completions, Chat Completions) |
| Release Date | 2023 (initial release); current version 0.32.0 on Oct 18, 2026 |
| Supported Models/Providers | Decoder-only LLMs (Llama, Qwen, Gemma), MoE (Mixtral, DeepSeek-V3, GPT-OSS), multimodal (LLaVA, Qwen-VL), embedding models |