Practical overview
When it fits
Use BERTScore when you have generated text and corresponding reference text and want a semantic, model-based evaluation metric available through Python or a command-line workflow.
Who it is for
- Researchers evaluating text-generation systems.
- Developers adding a reference-based metric to model assessment workflows.
- Teams compiling benchmark reports for generated responses.
Documented uses
- Compare generated responses with reference answers.
- Score candidate text files against one or more reference files.
- Add precision, recall and F1 results to a broader model evaluation.
Requirements & limits
The public GitHub repository displays an MIT license. Installation instructions specify Python 3.6 or later and PyTorch 1.0 or later.
- The library provides both a Python function and a scorer object, as well as a CLI.
- The scorer object caches its model to support repeated evaluations.
- The README describes support for about 130 models and reports that model choice can affect correlation with human evaluation.
- The project says GPU resources are usually necessary because evaluation can be computationally intensive.
- Fast and standard tokenizer implementations may yield different scores.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Score candidate text files against references
Run the documented command-line evaluation for English candidate text and reference files.
Before you start
- Python 3.6 or later and PyTorch 1.0 or later.
- BERTScore installed from PyPI with pip.
- Candidate and reference text files with corresponding lines.
Steps
Install the library with the command below.
pip install bert-scorePrepare a reference file and a candidate (hypothesis) file with matching line order.
Run
bert-score -r example/refs.txt -c example/hyps.txt --lang en, replacing the example paths with the prepared files.
Expected outcome
The CLI reports precision, recall and F1 scores for the candidate text against its reference.
Guide limitations
- Scoring can be computationally intensive; the project says a GPU is usually necessary.
- The documented example uses English; multilingual evaluation has separate language and model requirements.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 14 October 2026
- Original website title
- Tiiiger/bert_score · Official homepage title · retained in its source language