Mlc Ai · since Ursprüngliches Projekt "Web LLM" April 2023; aktuelles npm-Paket @mlc-ai/web-llm fortlaufend weiterentwickelt (v0.2.84, · 5× · last seen Jun 30, 2026
WebLLM is an open-source in-browser inference engine built by the MLC-AI team that compiles quantized LLMs via Apache TVM and runs them directly in the user's browser using WebGPU, with no server backend required. It offers an API largely compatible with OpenAI's SDK (streaming, JSON mode, function-calling) and supports popular open models such as Llama, Phi, Gemma, Mistral, and Qwen in MLC format. Distributed as the npm package @mlc-ai/web-llm under the Apache-2.0 license, it can be modularly integrated into web apps; a companion reference app (WebLLM Chat) demonstrates its use as a private, fully browser-based chat interface.
Features
| Throughput/Latency | Approx. 15–20 tokens/second, hardware- and model-size dependent (small models up to ~8B practical) |
| License | Apache-2.0 |
| Platform | Web browser (JavaScript/TypeScript, npm package, CDN), runs client-side via WebGPU/WebAssembly |
| Price | Free, open source (no commercial licensing model) |
| Protocol Compatibility | Full OpenAI API compatibility (chat completions, streaming, JSON mode, function-calling WIP) |
| Release Date | First Web LLM project April 2023; npm package @mlc-ai/web-llm v0.2.0 released and continuously updated since, currently v0.2.84 |
| Supported Models/Providers | Llama, Phi, Gemma, Mistral, Qwen, RedPajama and others in MLC format; custom models can be integrated |