Practical overview
When it fits
Use Scan2Cap when you need a research codebase for object-level detection and caption generation from 3D scan point clouds, and can work with its documented Python, PyTorch, CUDA and dataset setup.
Who it is for
- Researchers studying 3D object detection, scene understanding or dense captioning.
- Developers exploring point-cloud models that pair object localization with natural-language descriptions.
- Researchers evaluating captioning methods on RGB-D scan datasets and benchmark workflows.
Documented uses
- Generate bounding boxes and descriptions for objects in a 3D scene point cloud.
- Train and evaluate a dense-captioning model using the documented ScanRefer and ScanNet workflow.
- Visualize predicted scene boxes and descriptions.
- Generate predictions for the Scan2Cap benchmark challenge.
Requirements & limits
- The expected input is a 3D scene point cloud; the intended output combines object bounding boxes and descriptions.
- The implementation specifies PyTorch 1.8, PyTorch Geometric graph support, and CUDA module compilation for its PointNet++ backbone.
- The documented data workflow requires ScanRefer and ScanNetV2; ScanRefer is distributed after a request is accepted.
- The README estimates about 3.8 GB of free space for the ScanNet preprocessing step and separately identifies approximately 990 MB of GLoVE embeddings.
- Caption evaluation requires the arguments to match the model’s training configuration.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Prepare Scan2Cap’s ScanNet data
Set up the documented ScanNet scene data for the project’s data workflow.
Before you start
- PyTorch 1.8, the packages in requirements.txt, PyTorch Geometric graph support, and compiled CUDA modules for the PointNet++ backbone.
- ScanRefer dataset access, which requires a request and acceptance; ScanNetV2 data; and preprocessed GLoVE embeddings (~990 MB).
- A project root configured as CONF.PATH.BASE in lib/config.py.
Steps
Install the listed PyTorch version and requirements, then install PyTorch Geometric graph support.
Compile the PointNet++ CUDA modules from lib/pointnet2 and set CONF.PATH.BASE in lib/config.py.
Place ScanRefer under data/, GLoVE embeddings under data/, and ScanNetV2 scans under data/scannet/scans/.
From data/scannet/, run python batch_load_scannet_data.py to create the processed scene data.
Expected outcome
Processed ScanNet scene data is generated under data/scannet/scannet_data/ for subsequent project workflows.
Guide limitations
- The instructions require ScanRefer access approval and ScanNetV2 data; this recipe does not cover obtaining ScanNetV2.
- The README estimates roughly 3.8 GB of free space for ScanNet preprocessing.
- This covers data preparation only, not training or generating captions.
Train and evaluate a dense-captioning model
Run the README’s end-to-end Scan2Cap training and caption evaluation workflow.
Before you start
- A configured Scan2Cap environment and prepared project data.
- The README’s example uses multiview features, normals, top-down features and relation features; training options are described through scripts/train.py -h.
Steps
Run the documented scripts/train.py command with the listed feature flags, graph settings, batch size and epoch count.
Use the output folder created under outputs/ when evaluating.
Run scripts/eval.py with matching feature and graph arguments and the caption-evaluation options, including --min_iou 0.5.
For visual results, run scripts/visualize.py for a scene ID; first generate axis-aligned ScanNet mesh files with scripts/export_scannet_axis_aligned_mesh.py.
Expected outcome
Training outputs are saved under outputs/; evaluation reports caption performance, and the visualization script generates bounding boxes and descriptions for a scene.
Guide limitations
- Evaluation arguments must match those used for training.
- The visualization workflow additionally requires axis-aligned ScanNet mesh files.
- The README’s example commands do not by themselves provide dataset access or environment setup.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 18 October 2026
- Original website title
- daveredrum/Scan2Cap · Official homepage title · retained in its source language