Practical overview
When it fits
Use LlamaGPT when you want to run one of its listed Llama models on your own hardware, and can meet the deployment and model memory requirements.
Who it is for
- People who want to host a Llama-based chat interface on their own server or computer.
- Developers seeking an OpenAI-compatible API backed by supported local models.
- Users who want a Code Llama model for coding conversations.
Documented uses
- Chat with supported Llama 2 models through a local browser interface.
- Use Code Llama models for coding conversations.
- Connect applications that support the documented OpenAI-compatible API to the local model service.
Requirements & limits
The repository is public and licensed under MIT. Documented deployment routes include UmbrelOS, M1/M2 Mac, Docker on x86 or arm64, and Kubernetes.
- The documented model list specifies memory requirements from 6.29 GB to 41.37 GB, depending on model.
- Docker setup downloads the selected model on first run; the README notes this can take a while.
- The local chat interface is shown at localhost:3000, while the API and its documentation are at localhost:3001 and localhost:3001/docs.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Run LlamaGPT with Docker
Start the chatbot locally on an x86 or arm64 system using a 7B model.
Before you start
- An x86 or arm64 system
- Docker
Steps
Clone https://github.com/getumbrel/llama-gpt.git and change into the llama-gpt directory.
Run ./run.sh --model 7b.
Wait for the model download and API server startup, then open http://localhost:3000.
Expected outcome
The LlamaGPT chat interface is available locally at http://localhost:3000.
Guide limitations
- The initial run may take time while the model downloads to the /models directory.
- The listed 7B Nous Hermes Llama 2 model requires 6.29 GB of memory.
Deploy LlamaGPT to Kubernetes
Apply the project's Kubernetes manifests in a dedicated namespace.
Before you start
- A running Kubernetes cluster
- kubectl configured to interact with the cluster
- A local clone of the repository
Steps
Clone the repository and change into its directory.
Create a namespace with kubectl create ns llama.
Apply the manifests with kubectl apply -k deploy/kubernetes/. -n llama.
Expose the service using the method normally used for the cluster.
Expected outcome
The Kubernetes manifests are applied in the llama namespace; service exposure remains a separate operator step.
Guide limitations
- The README does not prescribe a service-exposure method.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 14 October 2026
- Original website title
- getumbrel/llama-gpt · Official homepage title · retained in its source language