OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Bluebert

BlueBERT is a pretrained BERT language model for biomedical NLP, accompanied by Python code, model files and preprocessed PubMed text. Its model variants were trained on PubMed abstracts, either alone or together with MIMIC-III clinical notes.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Bluebert — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use BlueBERT when a biomedical NLP experiment can start from its PubMed-focused pretrained weights and one of the task workflows represented in the repository.

Who it is for

  • Biomedical NLP researchers adapting pretrained models to benchmark tasks.
  • Developers experimenting with entity recognition, relation extraction, sentence similarity, document classification or inference on biomedical text.
  • Researchers studying PubMed text preprocessing and domain-specific BERT pretraining.

Documented uses

  • Fine-tune a PubMed- or PubMed-plus-MIMIC-III-pretrained model for a supported task.
  • Run the repository’s task-specific training, evaluation or prediction scripts with model files and a dataset.
  • Inspect preprocessed PubMed text and the described preprocessing steps used for pretraining.
Before you choose

Requirements & limits

The public GitHub repository links to model files and Hugging Face copies. Its README assumes model weights and datasets have been downloaded before running fine-tuning examples.

  • Choose model size (Base or Large) and pretraining corpus (PubMed or PubMed plus MIMIC-III) based on the intended experiment.
  • Each workflow expects specific model files, task data and settings; the README provides distinct scripts and task names.
  • Benchmark datasets are distributed through a separate BLUE_Benchmark repository.
  • The repository identifies Python scripts and pretraining procedures, but the supplied instructions do not specify a complete environment setup.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Fine-tune BlueBERT for sentence similarity

Run the documented sentence-similarity training and test command using a downloaded BlueBERT model and dataset.

Before you start

  • A downloaded BlueBERT model directory containing vocab.txt, bert_config.json, and bert_model.ckpt.
  • A sentence-similarity dataset directory and an output directory.
  • The BlueBERT repository and a Python environment able to run its script; specific environment setup is not documented.

Steps

  1. Download a BlueBERT model and place its files in a model directory.

  2. Download a dataset and identify its directory.

  3. From the repository, set PYTHONPATH if needed: export PYTHONPATH=.;$PYTHONPATH.

  4. Run `python bluebert/run_bluebert_sts.py --task_name='sts' --do_train=true --do_eval=false --do_test=true --vocab_file=$BlueBERT_DIR/vocab.txt --bert_config_file=$BlueBERT_DIR/bert_config.json --init_checkpoint=$BlueBERT_DIR/bert_model.ckpt --max_seq_length=128 --num_train_epochs=30.0 --do_lower_case=true --data_dir=$D

Expected outcome

The script is configured to train on the sentence-similarity task and run its test stage, writing to the specified output directory.

Guide limitations

  • The guide does not specify dataset acquisition details, Python dependencies, or expected output format.
  • No successful execution or test result is established.
Documentation 1 ↗
Source-based practical guide

Run BlueBERT named-entity recognition preparation, training, evaluation, and prediction

Use the documented NER command for the BC5CDR or ShARe/CLEFE task.

Before you start

  • A downloaded BlueBERT model directory containing vocab.txt, bert_config.json, and bert_model.ckpt.
  • A dataset directory and an output directory.
  • The BlueBERT repository and a Python environment able to run its script; specific environment setup is not documented.

Steps

  1. Download a BlueBERT model and place its files in a model directory.

  2. Download the task dataset and identify its directory; the documented task names are bc5cdr and clefe.

  3. Run `python bluebert/run_bluebert_ner.py --do_prepare=true --do_train=true --do_eval=true --do_predict=true --task_name="bc5cdr" --vocab_file=$BlueBERT_DIR/vocab.txt --bert_config_file=$BlueBERT_DIR/bert_config.json --init_checkpoint=$BlueBERT_DIR/bert_model.ckpt --num_train_epochs=30.0 --do_lower_case=true --data_dir=

Expected outcome

The script is configured to prepare data, train, evaluate, and predict for the selected NER task, with results directed to the specified output directory.

Guide limitations

  • Dataset acquisition and expected dataset layout are not specified.
  • Python dependencies and expected output format are not documented.
  • No successful execution or test result is established.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
18 October 2026
Original website title
ncbi-nlp/bluebert · Official homepage title · retained in its source language
How we review sourcesSuggest a correction