OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Local & Self-hosted AI

Llmfit

Llmfit is an open-source Rust tool that profiles a computer’s CPU, memory, GPUs, VRAM and accelerators to recommend open-source LLMs and quantizations likely to fit, then supports local model serving, throughput measurement and result sharing.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Llmfit — OpenViST project profile in Local & Self-hosted AI.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Llmfit before a local deployment when hardware constraints make model and quantization choice important, then use its benchmark workflow when estimates need to be replaced with measurements from the target machine.

Who it is for

  • Local AI users choosing models for specific workstation hardware.
  • Developers planning self-hosted or quantized LLM deployments.
  • Teams comparing local model throughput across different machines.

Documented uses

  • Identify models and quantizations that match available CPU, RAM, GPU and VRAM.
  • Estimate fit, speed, quality and context trade-offs before downloading a model.
  • Download and serve a selected model through a supported local runtime.
  • Measure real tokens-per-second performance and save the result locally.
  • Expose system and model information through documented HTTP JSON endpoints for integration work.
Before you choose

Requirements & limits

The repository is public and MIT-licensed. Documented installation routes include Scoop, Homebrew, MacPorts, uv, pip and pre-built binaries for Windows, macOS and Linux.

  • Hardware profiling covers CPU cores, system RAM, discrete or integrated GPUs, VRAM and unified memory architecture.
  • Model analysis considers parameter counts, context lengths and GGUF, AWQ, GPTQ and EXL2 quantization formats.
  • The tool supports multi-GPU setups, mixture-of-experts architectures and dynamic quantization selection.
  • Local runtime providers listed by the project include Ollama, llama.cpp, MLX, Docker Model Runner and LM Studio.
  • The REST API documentation identifies /api/v1/system and /api/v1/models for integration into orchestrators, dashboards and automated deployment pipelines.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install Llmfit and inspect local model fit

Set up Llmfit and use its hardware profile to review model recommendations.

Before you start

  • A supported Windows, macOS or Linux system.
  • One documented installation route: Scoop, Homebrew, MacPorts, uv, pip or a pre-built release binary.

Steps

  1. Install Llmfit using one of the documented package managers or download a release binary.

  2. Start the interactive terminal interface or use the classic CLI mode.

  3. Review the detected CPU, system memory, GPU, VRAM and accelerator information.

  4. Compare the recommended models and quantizations using the fit, quality, speed and context dimensions.

Expected outcome

A hardware-based shortlist of open-source LLMs and quantizations for local use.

Guide limitations

  • The recommendations include projected performance; measured throughput requires the separate download, serving and benchmarking workflow.
  • The supplied documentation does not establish an export format for passing the hardware profile into another deployment tool.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
LlmfitIdentify models and quantizations that match available CPU, RAM, GPU and VRAM.CLI, Desktop, Linux, macOSThe repository is public and MIT-licensed. Documented installation routes include Scoop, Homebrew, MacPorts, uv, pip and pre-built binaries for Windows, macOS and Linux.
PmetalRun inference with a selected model and prompt, including a LoRA adapter.macOS, Self-hostedThe project documents prebuilt binaries, a crates.io package and source builds. Building the optional desktop GUI requires Bun in addition to its separate build command.
GeniexRun a GGUF language or vision-language model locally through llama.cpp.Self-hostedThe repository labels GenieX a developer preview and identifies a BSD-3-Clause license. Its documented target platforms are Windows ARM64, Android and Linux ARM64 on Qualcomm devices.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
2 October 2026
Original website title
AlexsJones/llmfit · Official homepage title · retained in its source language
How we review sourcesSuggest a correction