Practical overview
When it fits
Use Rkd when you want a source-code starting point for metric-learning experiments that include teacher training, evaluation and relational distillation into a student model.
Who it is for
- Researchers running metric-learning experiments with image embeddings.
- Developers studying teacher-student distillation of embedding networks.
Documented uses
- Train a teacher embedding network using the included script.
- Evaluate a saved teacher or student checkpoint.
- Distill a teacher embedding network into a student network.
- Explore different embedding dimensions and teacher-student configurations.
Requirements & limits
The repository is public and identified as open source. Its documented stack includes Python 3.6, PyTorch 1.0, tqdm, h5py and SciPy.
- The README's examples use CUB200 and ResNet architectures; it does not establish broad dataset coverage.
- The README warns that its ResNet-50, 512-dimensional teacher example at batch size 128 needs more than 12 GB of GPU memory.
- The repository requests citation of the Relational Knowledge Distillation paper when using the code.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Train and evaluate a teacher embedding model
Use the README's CUB200 and ResNet-50 example to train a teacher embedding network and evaluate its saved checkpoint.
Before you start
- Python 3.6 and PyTorch 1.0
- tqdm, h5py and SciPy
- A CUB200 dataset available to the training script
- For the example ResNet-50, 512-dimensional setup at batch size 128, more than 12 GB of GPU memory; the README advises reducing batch size if needed
Steps
Consult the training script's help with the command below.
python run.py --helpRun the command below.
python run.py --mode train --dataset cub200 --base resnet50 --sample distance --margin 0.2 --embedding_size 512 --save_dir teacherEvaluate the saved teacher with the command below.
python run.py --mode eval --dataset cub200 --base resnet50 --embedding_size 512 --load teacher/best.pth
Expected outcome
The example saves a teacher model under `teacher` and evaluates the `teacher/best.pth` checkpoint.
Guide limitations
- The guide follows the documented CUB200 example and does not establish support for other datasets.
- The README's GPU-memory warning applies to the stated batch-size and model configuration.
Distill a teacher embedding model into a student
Follow the README's example to distill a trained ResNet-50 teacher into a ResNet-18 student, then evaluate the student checkpoint.
Before you start
- Python 3.6 and PyTorch 1.0
- tqdm, h5py and SciPy
- A trained teacher checkpoint at
teacher/best.pth - A CUB200 dataset available to the scripts
Steps
Consult the distillation script's help with the command below.
python run_distill.py --helpRun the command below.
python run_distill.py --dataset cub200 --base resnet18 --embedding_size 64 --l2normalize false --teacher_base resnet50 --teacher_embedding_size 512 --teacher_load teacher/best.pth --dist_ratio 1 --angle_ratio 2 --save_dir studentEvaluate the student with the command below.
python run.py --mode eval --dataset cub200 --base resnet18 --l2normalize false --embedding_size 64 --load student/best.pth
Expected outcome
The example saves a distilled student model under `student` and evaluates `student/best.pth`.
Guide limitations
- The commands reflect the README's CUB200 and ResNet example settings; they do not establish results for other configurations.
- A teacher checkpoint must exist at the path used in the example before distillation.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 18 October 2026
- Original website title
- lenscloth/RKD · Official homepage title · retained in its source language