OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Local & Self-hosted AI

Self Hosted AI Stack

Self Hosted AI Stack is a MIT-licensed Docker Compose configuration for running a local-first AI environment on a Linux server. It combines Ollama language models and AnythingLLM chat with services for document parsing, embeddings, audio transcription, text-to-speech and MCP tool access.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Self Hosted AI Stack — OpenViST project profile in Local & Self-hosted AI.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you can administer a Linux server with Docker and want to assemble local models, chat and supporting AI services in one Compose configuration.

Who it is for

  • People who want to run language models and AI services on a server they administer.
  • Teams or individuals building document-chat workflows with local models.
  • Server operators who need transcription, text-to-speech or MCP tools alongside model chat.

Documented uses

  • Start a coordinated set of local AI services using Docker Compose.
  • Chat with documents using AnythingLLM connected to local models.
  • Add document parsing, embeddings, audio transcription or text-to-speech services.
  • Provide MCP tools to AI clients through UplinkCrate.
Before you choose

Requirements & limits

The primary setup runs on a Linux server with Docker; the README supports amd64 and arm64. The repository is MIT-licensed.

  • The quick-start recommendation is at least 8 GB RAM for small models and 16 GB or more for models of 8B parameters and above.
  • Optional services can be omitted to reduce memory requirements.
  • AnythingLLM uses bundled MiniLM and LanceDB by default; EmbedCrate and pgvector-backed storage require Compose configuration changes.
  • NVIDIA CUDA acceleration and an optional Caddy HTTPS overlay are documented.
  • AnythingLLM generates an admin password on first start and is password-protected by default.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Start the Docker Compose stack and pull a local model

Bring up the services and fetch a model before making LLM requests.

Before you start

  • A Linux server, local or cloud-based, with Docker installed
  • At least 8 GB RAM for small models; 16 GB or more recommended for models of 8B parameters and above

Steps

  1. Clone the repository and change into its directory.

  2. Run the command below.

    Documented command
    docker compose up -d

  3. Pull a model with the command below.

    Documented command
    docker exec ollama ollama_manage --pull llama3.2:3b

  4. Run ./stack-check.sh to check the services.

Expected outcome

The Compose services are started and a local model is pulled for LLM requests.

Guide limitations

  • First startup may take a few minutes while services initialize.
  • A model must be pulled before making LLM requests.
  • The quick-start instructions specify Linux and Docker.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
12 October 2026
Original website title
hwdsl2/self-hosted-ai-stack · Official homepage title · retained in its source language
How we review sourcesSuggest a correction