Practical overview
When it fits
Use CLUE when you need reference datasets and baseline comparisons for Chinese language-understanding evaluation, or want to follow its repository instructions to run a benchmark task.
Who it is for
- Researchers evaluating Chinese language models against shared benchmark tasks.
- Developers seeking Chinese NLP datasets and baseline results for classification or reading comprehension.
Documented uses
- Compare model scores across Chinese language-understanding tasks.
- Find benchmark datasets and baseline models for tasks such as classification, sentence inference, pronoun resolution and reading comprehension.
- Use the repository's documented scripts to run a baseline task and produce classification prediction files.
Requirements & limits
- The benchmark includes classification and reading-comprehension tasks, with scores reported by task and in aggregate.
- The README describes Python tooling and separate TensorFlow and PyTorch baseline paths.
- The listed runtime environment is TensorFlow 1.12, CUDA 9.0 and cuDNN 7.0; TPU examples require path and TPU-setting changes.
- The README notes that some filtered datasets differ from their original versions, which can affect interpretation of results.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Run a CLUE classification baseline
Run a documented GPU classification task using a CLUE baseline script.
Before you start
- A cloned CLUE repository.
- A GPU for the documented GPU workflow.
- TensorFlow 1.12, CUDA 9.0, and cuDNN 7.0, listed as the runtime environment.
Steps
Clone the repository with the command below.
git clone https://github.com/CLUEbenchmark/CLUE.gitChange into a classification baseline directory, such as
CLUE/baselines/models/bertorCLUE/baselines/models_pytorch/classifier_pytorch.Run the corresponding task script, for example the command below, to start the IFLYTEK training task; the README says this workflow automatically downloads the model and task data.
bash run_classifier_iflytek.sh
Expected outcome
The selected classification baseline task starts running; the README describes the IFLYTEK example as a training run.
Guide limitations
- The repository documentation gives multiple baseline paths but does not specify which script applies to every task.
- The documented GPU workflow depends on the listed TensorFlow/CUDA/cuDNN environment.
Install and start the PyCLUE example
Install the PyCLUE toolkit and invoke its classification example.
Before you start
- Python 3, as indicated by the documented
python3invocation. - The ability to install PyCLUE with pip.
Steps
Install the toolkit with the command below.
pip install PyCLUEChange into
PyCLUE/examples/classifications.Invoke the example with the command below.
python3 run_clue_task.py
Expected outcome
The documented PyCLUE classification example is invoked; the README says the toolkit supports 10 tasks, 9 models, and custom tasks.
Guide limitations
- The README does not list the individual supported tasks or models in the available documentation.
- No additional environment requirements or expected output are specified.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 18 October 2026
- Original website title
- CLUEbenchmark/CLUE · Official homepage title · retained in its source language