getumbrel/llama-gpt
LlamaGPT packages a chat interface with llama.cpp and a choice of quantised Llama 2 and Code Llama models, so conversations stay on the machine and no data leaves it. It installs from the Umbrel app store or with Docker and Kubernetes, needs between 6 GB and 41 GB of memory depending on model size, and can use Nvidia GPUs. An OpenAI-compatible API is exposed alongside the chat UI.
Maintenance activity