Niko1221/Strata
A desktop inference engine that runs very large mixture-of-experts models on consumer hardware by streaming weights from an SSD for the parts that do not fit in memory. It installs in one click on Windows and Linux, supports Nvidia RTX 30 to 50 series, AMD Radeon on Linux, Ryzen AI Max and Apple Silicon, and exposes OpenAI- and Anthropic-compatible APIs on localhost with optional image input. Licensed under MIT.
Maintenance activity