Practical overview
When it fits
Use it when an experiment requires reproducible configuration of a listed dataset, model, training method and distribution-shift scenario, rather than a hosted monitoring interface.
Who it is for
- Machine-learning researchers studying model behavior under distribution shift.
- Practitioners comparing supported training methods across controlled dataset conditions.
Documented uses
- Train models on selected datasets using a supported algorithm and architecture.
- Compare models under unseen-property, spurious-correlation or low-data-drift conditions.
- Add label noise or constrain training-set size when configuring a scenario.
- Run evaluation with a selected experiment configuration or checkpoint.
Requirements & limits
The repository is public and identifies its license as Apache-2.0.
- The documented datasets are dSprites, SmallNorb and Shapes3D.
- The README lists ResNet18, ResNet50, ResNet101 and MLP model options.
- Python 3.9.9 is the version identified as tested; JAX GPU support requires a requirements-file change before setup.
- Experiment settings include algorithm, model, dataset, shift case, prediction label, property label, seed count, batch size and training steps.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Set up the framework environment
Create the project virtual environment, install its dependencies and run the documented import check.
Before you start
- Python 3.9.9 is the version identified as tested in the README.
- A checkout of the repository and a shell run from its parent directory.
Steps
From the repository's parent directory, run the command below.
sh distribution_shift_framework/run.shUse the virtual environment at
/tmp/distribution_shift_frameworkfor the documented training or evaluation commands.
Expected outcome
The setup script creates and activates a virtual environment, installs dependencies and runs a test program to check module imports.
Guide limitations
- GPU support with JAX requires editing
requirements.txtbefore running the script; the README gives a JAXLine CUDA package as an example and refers users to JAX installation instructions.
Run a configured training or evaluation experiment
Invoke the classification experiment module in training or evaluation mode using the repository's example configuration.
Before you start
- The project virtual environment created by
run.sh. - A shell in which the environment can be activated.
Steps
Activate the environment with
source /tmp/distribution_shift_framework/bin/activate.For training, invoke the command below.
python3 -m distribution_shift_framework.classification.experiment --jaxline_mode=train --config=distribution_shift_framework/classification/config.pyFor evaluation, use the same module and configuration with
--jaxline_mode=eval.
Expected outcome
The experiment module is invoked in the selected mode with the repository's classification configuration.
Guide limitations
- The README documents configurable datasets, models, methods and shift scenarios; results depend on the chosen configuration and data.
- The example commands do not specify a custom configuration or checkpoint.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 12 October 2026
- Original website title
- google-deepmind/distribution_shift_framework · Official homepage title · retained in its source language