Practical overview
When it fits
Use Nnsight when a Python-based PyTorch workflow needs to inspect, save or alter values inside model execution rather than only consume the final output.
Who it is for
- Researchers studying neural-network behavior and interpretability
- Python developers building experiments that need intermediate activations, attention patterns or gradients
- Developers who need to intervene in model computations or trace models remotely through NDIF
Documented uses
- Read and save intermediate layer outputs during a forward pass
- Capture attention values or gradients for analysis
- Change activations before later model computation
- Inspect values across generation steps or separate prompt invocations
- Trace a model too large to host locally through NDIF
Requirements & limits
The package is identified as open source under the MIT license. The documented remote-tracing example requires an NDIF API key.
- PyTorch is the underlying model framework; the README says any PyTorch model can be wrapped, and names wrappers for Hugging Face, diffusers and vLLM.
- The trace interface lets users describe reads and edits in the order they occur during computation, with saved values retained for use after the trace.
- Local execution and remote NDIF execution are separate paths; remote tracing is marked with
remote=Trueand the example sets an API key. - Some module outputs are tuples, so users may need to index into them instead of treating
.outputas a tensor. - The repository lists an MIT license.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Install Nnsight and prepare a local trace
Install the library and set up a Transformers model for a documented local tracing example.
Before you start
- Python environment with pip
- A PyTorch model; the README example uses the TransformersModel wrapper and openai-community/gpt2
Steps
Install the package with the command below.
pip install nnsightImport
TransformersModelfromnnsight.Create a model with
TransformersModel("openai-community/gpt2", dispatch=True).Use a
with model.trace(...)block to describe reads or interventions, and call.save()on values needed after the block.
Expected outcome
A trace is configured to access or intervene on model values during its forward pass, with saved values available after the trace.
Guide limitations
- The README example documents the setup pattern but does not establish compatibility for every environment or model.
- Some modules return tuples rather than plain tensors, so the relevant output may need indexing.
Prepare a trace for remote execution through NDIF
Configure the documented API-key setting and mark a model trace for remote execution.
Before you start
- Nnsight installed in a Python environment
- An NDIF API key
- A model identifier, as in the README's TransformersModel example
Steps
Import
CONFIGfromnnsight.Set the default key with
CONFIG.set_default_api_key("YOUR_NDIF_KEY").Create a
TransformersModelfor the chosen model identifier.Open a trace with
remote=Trueand save the internal value to retrieve it after the trace.
Expected outcome
The code describes a trace routed through NDIF rather than a model hosted locally.
Guide limitations
- Remote execution depends on NDIF infrastructure; the guide does not establish model availability or account terms.
- The example is setup and usage documentation, not a verification that a particular remote model can be run.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 13 October 2026
- Original website title
- ndif-team/nnsight · Official homepage title · retained in its source language