OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Secbert

SecBERT is a pretrained BERT language model for cybersecurity text, released alongside a SecRoBERTa variant. Its dedicated secvocab vocabulary is tailored to its training corpus, and its repository provides PyTorch weights and Hugging Face Transformers loading examples.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Secbert — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use SecBERT when a cybersecurity text project needs a pretrained BERT-family model with domain-specific training and vocabulary, especially as a starting point for masked-language modeling or custom downstream NLP work.

Who it is for

  • Developers building NLP workflows for cybersecurity reports or threat-intelligence text.
  • Researchers exploring masked-language modeling or adapting pretrained models to cybersecurity NLP tasks.

Documented uses

  • Load cybersecurity-trained model weights and tokenizer in a Python project.
  • Run the repository’s documented fill-mask evaluation workflow.
  • Use the weights as a starting component for cybersecurity entity recognition, text classification, semantic understanding or question-answering work.
Before you choose

Requirements & limits

The repository is public and identifies the project license as MIT.

  • The model was trained on cybersecurity papers and resources including APTnotes, Stucco-Data, CASIE and SemEval-2018 SecureNLP.
  • The repository provides both SecBERT and SecRoBERTa model options.
  • The README includes a fill-mask evaluation command, while its downstream-tasks section is marked TODO.
  • Weights are provided in PyTorch format and described as usable in custom models in BERT format.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Load SecBERT weights with Hugging Face Transformers

Initialize the SecBERT tokenizer and masked-language model using the repository’s documented Python example.

Before you start

  • Python environment with Hugging Face Transformers and PyTorch available; the repository’s example uses AutoTokenizer and AutoModelForMaskedLM.

Steps

  1. Import AutoTokenizer and AutoModelForMaskedLM from transformers.

  2. Load the tokenizer with AutoTokenizer.from_pretrained("jackaduma/SecBERT").

  3. Load the model with AutoModelForMaskedLM.from_pretrained("jackaduma/SecBERT").

Expected outcome

The SecBERT tokenizer and pretrained masked-language model are initialized for use in a Python workflow.

Guide limitations

  • This setup example does not provide a complete downstream task implementation.
  • The repository separately documents a fill-mask evaluation script; the loading example alone does not run that script.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
17 October 2026
Original website title
jackaduma/SecBERT · Official homepage title · retained in its source language
How we review sourcesSuggest a correction