Tuned Lens

When it fits

Use Tuned Lens when an analysis needs estimates of a transformer’s final predictions from intermediate residual-stream states, rather than only a direct unembedding of those states.

Who it is for

• Researchers studying how transformer predictions form across layers
• Model evaluators comparing intermediate predictions with a model’s final output
• Python users working with PyTorch and pretrained models from the Transformers library

Documented uses

• Inspect how next-token predictions change across transformer layers.
• Compare layer-level predictions with the original model’s final output distribution.
• Train affine translators for intermediate representations.
• Build a layer-by-layer transformer interpretability analysis.