OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Developer Tools & SDKs

LLM Foundry

LLM Foundry is an open-source Python repository of code, scripts, and utilities for preparing data, training and fine-tuning large language models, evaluating them, and preparing them for inference.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

LLM Foundry — OpenViST project profile in AI Developer Tools & SDKs.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you need a code-based workflow spanning data preparation, model training or fine-tuning, evaluation, and inference, rather than a hosted chat interface.

Who it is for

  • Developers building or adapting language-model training workflows.
  • Teams preparing text data and evaluating models on academic or custom in-context-learning tasks.
  • Practitioners who need scripts for model conversion, response generation, or inference benchmarking.

Documented uses

  • Convert text from original sources into StreamingDataset format.
  • Train or fine-tune Hugging Face and MPT models.
  • Evaluate models on academic or custom in-context-learning tasks.
  • Convert models to Hugging Face or ONNX formats and generate responses.
  • Profile training throughput and inference latency or throughput.
Before you choose

Requirements & limits

The GitHub repository is public and specifies the Apache-2.0 license. The repository is listed as active.

  • The README describes training and fine-tuning for Hugging Face and MPT models from 125M to 70B parameters.
  • Data preparation converts text from original sources to StreamingDataset format.
  • Evaluation supports academic or custom in-context-learning tasks.
  • Inference scripts include model conversion and response generation; separate scripts cover inference benchmarking.
  • Workloads can be launched with MCLI and the MosaicML platform.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install LLM Foundry in a Docker-based environment

Clone the repository and install its editable package and GPU dependencies inside a Docker-based environment.

Before you start

  • Docker environment; the README recommends using a prebuilt MosaicML Docker image.
  • PyTorch, CMake, and packaging must already be installed; the README also provides a pip command to install these prerequisites.

Steps

  1. Clone the repository: the command below.

    Documented command
    git clone https://github.com/mosaicml/llm-foundry.git

  2. Change into the repository directory: cd llm-foundry.

  3. Install the editable package with GPU dependencies: the command below; use the command below if there is no NVIDIA GPU.

    Documented command
    pip install -e ".[gpu]"
    Documented command
    pip install -e .

Expected outcome

The repository is cloned and the editable LLM Foundry package is installed with the selected dependency option.

Guide limitations

  • The recommended Docker images include dependencies but do not include the llm-foundry package; install the package separately.
  • This guide covers installation only; the README's Quickstart is the next documented stage for a sample data-preparation, training, conversion, evaluation, and generation workflow.
Documentation 1 ↗
Source-based practical guide

Run the documented C4-to-MPT-125M quickstart

Prepare a subset of C4, train MPT-125M for 10 batches, convert it to HuggingFace format, evaluate it, and generate prompt responses.

Before you start

  • Complete the repository installation steps before starting.
  • Run the commands from the repository's scripts directory.
  • The training command uses the Composer distributed launcher.

Steps

  1. From the repository root, enter scripts and convert the English C4 subset to StreamingDataset format using the documented data_prep/convert_dataset_hf.py command, with the train_small and val_small splits.

  2. Train MPT-125M for 10 batches using composer train/train.py and train/yamls/pretrain/mpt-125m.yaml, pointing the data settings to my-copy-c4 and saving to mpt-125m.

  3. Convert the resulting Composer checkpoint to HuggingFace format with inference/convert_composer_to_hf.py, writing to mpt-125m-hf.

  4. Evaluate the converted model with composer eval/eval.py, eval/yamls/hf_eval.yaml, and the COPA task configuration.

  5. Generate responses with inference/hf_generate.py using the converted model and the documented sample prompts.

Expected outcome

The workflow produces a prepared C4 subset, a short-trained MPT-125M checkpoint, a HuggingFace-format model, an evaluation run, and generated responses to the sample prompts.

Guide limitations

  • The README explicitly describes this as a demonstration quickstart; 10 training batches are not enough for good model quality.
  • The guide's commands and results are not a verification of successful execution.
  • Hub upload is optional and requires a write-enabled HuggingFace token; uploading is not part of these steps.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
7 October 2026
Original website title
mosaicml/llm-foundry · Official homepage title · retained in its source language
How we review sourcesSuggest a correction