Practical overview
When it fits
Use it when you need an experimental PyTorch starting point for transformer architectures that learn or retrieve information through neural memory, especially when direct control over model dimensions, sequence lengths and memory modules is useful.
Who it is for
- Researchers exploring neural memory in transformer architectures
- PyTorch developers prototyping test-time-training or long-term-memory models
- Students and engineers comparing alternative neural-memory module designs
Documented uses
- Add a neural-memory component to a PyTorch sequence-model experiment.
- Prototype transformer variants for longer-context processing.
- Investigate test-time-training approaches.
- Compare alternative MLP designs for neural memory.
- Build a research notebook or benchmark around Titans variants.
Requirements & limits
The repository is public, active in the supplied status data and distributed under the MIT license. The documented installation uses pip; a separate quick Colab run is linked from the repository.
- It is an unofficial implementation rather than a general-purpose production framework.
- The repository includes both a standalone NeuralMemory example and a MAC transformer example.
- The documented examples use CUDA, so the quick-start path assumes GPU-style tensor execution.
- Chunk size affects the stated trade-off between performance on shorter sequences and memory use.
- The project includes exploratory work beyond the original one-to-four-layer MLP memory design.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Install the library and run the neural-memory example
Set up Titans Pytorch and pass a PyTorch sequence through its NeuralMemory module.
Before you start
- Python environment with pip
- PyTorch
- CUDA-enabled execution for the documented .cuda() example
Steps
Install the package with the command below.
pip install titans-pytorchImport
torchandNeuralMemoryfromtitans_pytorch.Create
NeuralMemorywith a dimension of 384 and chunk size of 64, then move it to CUDA.Create a tensor with shape
(2, 1024, 384)on CUDA and call the memory module.Check that the retrieved tensor has the same shape as the input sequence.
Expected outcome
The example returns retrieved values and a memory state, with the retrieved tensor matching the input shape.
Guide limitations
- The documented example uses CUDA tensors and a CUDA memory module.
- The repository notes that smaller chunk sizes may improve performance for shorter sequences but use more memory.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 30 September 2026
- Original website title
- lucidrains/titans-pytorch · Official homepage title · retained in its source language