JustVugg/colibri
colibri is a pure C inference engine that runs very large mixture-of-experts models on consumer hardware by keeping the dense weights in RAM and streaming routed experts from disk. A one-step setup inspects the machine, recommends a model that fits, builds or downloads the engine, and resumes model downloads. It serves OpenAI- and Anthropic-compatible APIs with a browser dashboard and chat UI, and supports CUDA, Vulkan and Metal backends.
Maintenance activity