Practical overview
When it fits
Use LitGPT when you want a code- and CLI-oriented workflow for developing or operating LLMs, with recipes spanning experimentation, fine-tuning, evaluation and serving. It is especially relevant when direct access to model implementations or techniques such as LoRA, QLoRA and FSDP matters.
Who it is for
- Developers who want to run inference or evaluate language models from Python or the command line.
- Practitioners adapting an LLM to a dataset with full fine-tuning or parameter-efficient methods.
- Teams building self-managed training or serving workflows, including distributed GPU or TPU training.
Documented uses
- Load a supported model and generate text from a prompt.
- Fine-tune a model using a JSON dataset and validation split.
- Evaluate model performance on named task suites such as MMLU and TruthfulQA.
- Serve a pretrained or fine-tuned checkpoint through a web server for use by a website or application.
- Pretrain models from scratch using the project's recipes.
Requirements & limits
The repository is public and displays an Apache 2.0 license. The README documents installation with pip and from source.
- LitGPT is a framework and recipe set; running its workflows requires an environment capable of installing Python dependencies and handling the chosen model.
- The README describes low-memory GPU support and distributed execution across GPUs or TPUs, but does not provide one hardware requirement for all workflows.
- The project lists 20+ models, so model choice affects the workflow; check each model's own terms and resource needs.
- Serving with LitGPT starts a web server that can be accessed by a website or app; Lightning AI Studios are presented separately as an option.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Fine-tune a model with a JSON dataset
Use the documented CLI workflow to fine-tune Phi-2 on a JSON dataset, test the resulting checkpoint and start serving it.
Before you start
- LitGPT installed with its documented extra dependencies.
- A JSON dataset; the example downloads a finance dataset from Hugging Face.
- A suitable local or self-managed environment for the selected model and training task.
Steps
Download the example dataset to my_custom_dataset.json, or prepare a JSON dataset.
Run litgpt finetune microsoft/phi-2 with --data JSON, --data.json_path my_custom_dataset.json, --data.val_split_fraction 0.1 and --out_dir out/custom-model.
Test the result with litgpt chat out/custom-model/final.
Start serving the checkpoint with litgpt serve out/custom-model/final.
Expected outcome
A fine-tuned checkpoint that can be tested in chat and served by LitGPT.
Guide limitations
- The example does not specify hardware requirements or training duration.
- The recipe's dataset and model are examples; dataset suitability and model-specific terms require separate consideration.
Install LitGPT and load a model for generation
Install the package and use its Python API to generate text with a listed model.
Before you start
- Python and pip.
- An environment able to install the package and load the selected model.
Steps
Install with pip install 'litgpt[extra]'.
In Python, import LLM from litgpt and load a model, following the documented example: LLM.load("microsoft/phi-2").
Call llm.generate with a prompt and print the returned text.
Expected outcome
A Python session that loads a model and generates text from a prompt.
Guide limitations
- The example does not document hardware requirements or model download size.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 2 October 2026
- Original website title
- Lightning-AI/litgpt · Official homepage title · retained in its source language