llama.cpp
#14 v Lokální LLM runtimeLlama Cpp · od März 2023 (Projektstart durch Georgi Gerganov) · 28× · naposledy 30. 6. 2026
16
Momentum
llama.cpp is an open-source C/C++ library for local and cloud LLM inference, originally created by Georgi Gerganov. It runs with no external dependencies such as Python or PyTorch, supports numerous hardware backends (CPU, CUDA, Metal, Vulkan, SYCL, and more), and uses the quantized GGUF model format to run LLMs efficiently even on consumer hardware. The built-in llama-server provides an OpenAI-compatible HTTP API and serves as the backend for projects like Ollama. The project is maintained by the ggml-org community and ships continuous build-tagged releases instead of traditional semantic versions.
Vývoj momenta
19.05.17.08.
Vlastnosti
| Deployment (Self-host/Cloud) | Self-hosted (lokal, Server) sowie Cloud-Deployment möglich, u.a. via Hugging Face Inference Endpoints |
| Durchsatz/Latenz | 3–8x schnellere Inferenz als Python-Frameworks, besonders auf CPU (laut Anbieter, hardwareabhängig) |
| Lizenz | MIT-Lizenz |
| Plattform | Linux, macOS, Windows, Android, Raspberry Pi, Browser (WebGPU); x86 (AVX/AVX2/AVX512/AMX), ARM, Apple Silicon (Metal) |
| Preis | Kostenlos, Open Source (keine Lizenzgebühren) |
| Protokoll-Kompatibilität | OpenAI-kompatible API (/v1/completions, /v1/chat/completions, /v1/embeddings) über llama-server |
| Release-Datum | März 2023 (Projektstart); fortlaufende Build-Releases, aktuell z.B. b9838 (Ende Juni 2026) |
| Unterstützte Modelle/Provider | Alle GGUF-kompatiblen Modelle: Llama 1/2/3, Mistral, Phi, Gemma, Qwen, DeepSeek, Yi, Solar, StableLM u.a. |