Practical overview
When it fits
Use CUDA-L2 when developing or benchmarking HGEMM kernels for a supported NVIDIA GPU and you can provide the documented Python, PyTorch, CUTLASS and CUDA architecture setup.
Who it is for
- CUDA developers optimizing half-precision matrix multiplication
- Researchers comparing HGEMM kernels with matrix-multiplication baselines
- Teams benchmarking selected NVIDIA GPU configurations
Documented uses
- Generate and refine HGEMM CUDA kernels through LLM proposals and reinforcement-learning feedback
- Evaluate matrix-multiplication cases in offline or server-style mode
- Compare results with torch.matmul and NVIDIA cuBLAS variants
Requirements & limits
The public GitHub repository lists an MIT license. The documented setup requires Python, PyTorch 2.6.0 or later, NVIDIA CUTLASS v4.2.1 and configured CUDA architecture variables.
- GPU-specific kernel performance is not guaranteed to transfer between hardware models.
- Evaluation supports fp16 or fp32 accumulator precision and identifies A100, RTX 3090 and H100 as device types.
- Server mode requires a target QPS; offline mode does not document that parameter as required.
- For matrix dimensions absent from the available configurations, the README suggests padding to a larger nearest-neighbor configuration or requesting a kernel through GitHub issues.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Run a documented offline HGEMM evaluation
Evaluate one configured matrix-multiplication case in offline mode and save its compilation and benchmark output.
Before you start
- Python environment and PyTorch 2.6.0 or later
- NVIDIA GPU supported by the selected device type: A100, RTX 3090 or H100
- NVIDIA CUTLASS cloned at tag v4.2.1 into a directory named cutlass
- CUTLASS_DIR set to that directory and TORCH_CUDA_ARCH_LIST set for the target GPU architecture
Steps
Clone NVIDIA CUTLASS tag v4.2.1 into a directory named cutlass.
Set CUTLASS_DIR to the cloned directory and set TORCH_CUDA_ARCH_LIST to the target architecture, such as 8.0 for A100 or RTX 30 series, or 9.0a for H100.
Run eval_one_file.sh with the desired matrix dimensions, accumulator precision, device type, warmup and benchmark durations, output directory, GPU device ID, and --mode offline.
Expected outcome
The script evaluates the selected case in offline/batch mode and saves compile and output results in the specified base directory.
Guide limitations
- The README warns that an incorrect CUTLASS version or CUTLASS_DIR may cause failure or produce no results.
- The documented example uses A100; available device_type values are a100, 3090 and h100.
- Results for kernels trained on one GPU are not guaranteed to transfer to another GPU.
Run a documented server-mode evaluation
Evaluate a configured HGEMM case in server mode with a target query rate.
Before you start
- Python environment and PyTorch 2.6.0 or later
- NVIDIA GPU supported by the selected device type: A100, RTX 3090 or H100
- NVIDIA CUTLASS cloned at tag v4.2.1 into a directory named cutlass
- CUTLASS_DIR and TORCH_CUDA_ARCH_LIST configured for the target GPU
- A target QPS value for the server-mode command
Steps
Clone NVIDIA CUTLASS tag v4.2.1 into a directory named cutlass.
Set CUTLASS_DIR and the architecture value in TORCH_CUDA_ARCH_LIST for the target GPU.
Run eval_one_file.sh with the selected matrix dimensions, precision, device, timing periods, output directory and GPU device ID; set --mode server and provide --target_qps.
Expected outcome
The script evaluates the case in server mode, which the README describes as simulating request-based scenarios.
Guide limitations
- The --target_qps argument is required in server mode.
- The README's example uses A100 and a target of 100 QPS; it does not establish that this value suits other workloads.
- The project cautions that kernels may not retain speedups across GPU models.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 15 October 2026
- Original website title
- ornith-ai/CUDA-L2 · Official homepage title · retained in its source language