Llms From Scratch: Build and fine-tune GPT-like models

Understand the project

Practical overview

Use it when you want to study LLM development through code and chapter-organized explanations, rather than access a hosted model or API.

Readers learning how large language models work by coding their components.
Developers who want a hands-on educational starting point for experimenting with pretraining or fine-tuning.

Follow a chapter-based implementation path from text processing and attention to a GPT-like model.
Study code for pretraining a small model for educational purposes.
Explore fine-tuning for text classification or instruction following.

Before you choose

The repository is public and can be downloaded as a ZIP or cloned from GitHub.

The material is paired with a book and organized around its chapters, exercises and code.
The main-chapter code is designed for conventional laptops and uses a GPU automatically when available.
The repository includes code for loading larger pretrained-model weights for fine-tuning as well as educational model-building material.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Obtain a local copy of the repository to follow its GPT-like LLM learning materials.

Open the repository on GitHub and use its Download ZIP option, or run the command below in a terminal.
Documented command
```
git clone --depth 1 https://github.com/rasbt/LLMs-from-scratch.git
```
View the README in a Markdown editor or previewer, or read it on GitHub, where Markdown is rendered.
For recommendations on installing Python and packages and setting up the code environment, consult the repository's setup README.

A repository copy is available to explore, with the setup guide identified as the next step for preparing the environment.

This guide covers obtaining and viewing the repository, not installing dependencies or running its notebooks and code.
The README describes the main-chapter code as designed for conventional laptops without specialized hardware; GPUs are used automatically if available.

Evidence and transparency

Source record details

Source availability: Active
Sources checked: 16 October 2026
Original website title: rasbt/LLMs-from-scratch · Official homepage title · retained in its source language