ggml-org/llama.cpp
A dependency-free C/C++ inference engine for large language and vision models, from laptops to servers. It offers 1.5- to 8-bit integer quantization, Apple Metal, CUDA, HIP, Vulkan and SYCL backends, plus CPU+GPU hybrid inference for models larger than available VRAM. A single command downloads a GGUF model from Hugging Face, and llama-server exposes an OpenAI-compatible API with a built-in web UI.
Maintenance activity