README.md

---
language:
- en
license: apache-2.0
library_name: transformers
pipeline_tag: token-classification
tags:
- biomedical
- named-entity-recognition
- ner
- biolinkbert
- biored
- gene
- disease
- variant
- gdv-scout
base_model: michiyasunaga/BioLinkBERT-base
datasets:
- bigbio/biored
model-index:
- name: GDV-Scout
  results:
  - task:
      type: token-classification
      name: Biomedical named-entity recognition
    dataset:
      type: bigbio/biored
      name: BioRED
      split: test
    metrics:
    - type: precision
      value: 0.8248719143
      name: Entity-level exact micro precision
    - type: recall
      value: 0.8622200584
      name: Entity-level exact micro recall
    - type: f1
      value: 0.8431325875
      name: Entity-level exact micro F1
---

# GDV-Scout: Gene, Disease, and Variant NER

**GDV-Scout** is a 108M-parameter biomedical named-entity recognition model for extracting
**genes or gene products**, **diseases**, and **sequence variants** from English biomedical
text. It is a `BertForTokenClassification` checkpoint using BIO labels and a
BioLinkBERT-base encoder.

The model follows BioRED's relatively strict entity-boundary convention. It is intended
for precise spans in sentence-level biomedical text. If broad BC2GM-style gene mentions
(including more gene families, complexes, and generic mentions) are required, a model
trained specifically for that annotation convention may be more appropriate.

## Model details

| Property | Value |
|---|---|
| Architecture | BERT token classifier (`BertForTokenClassification`) |
| Encoder lineage | [`michiyasunaga/BioLinkBERT-base`](https://huggingface.co/michiyasunaga/BioLinkBERT-base) |
| Parameters | 107,647,495 |
| Task | Biomedical named-entity recognition |
| Language | English |
| Input unit | One sentence |
| Training/evaluation length | Up to 192 subword tokens |
| Label scheme | BIO |
| Entity types | `Disease`, `GeneOrGeneProduct`, `Variant` |

The checkpoint was produced in two stages:

1. BioLinkBERT-base was fine-tuned as an NER student on reconciled majority-vote silver
   labels from a six-run LLM ensemble.
2. That trained student, including its token-classification head, was continually
   fine-tuned on the human-annotated BioRED training set.

BioRED `Gene` annotations are mapped to `GeneOrGeneProduct`. BioRED `Chemical`, `Species`,
and `CellLine` annotations are treated as `O` because they are outside this model's target
schema.

## Labels

| ID | Label |
|---:|---|
| 0 | `O` |
| 1 | `B-Disease` |
| 2 | `I-Disease` |
| 3 | `B-GeneOrGeneProduct` |
| 4 | `I-GeneOrGeneProduct` |
| 5 | `B-Variant` |
| 6 | `I-Variant` |

## Intended use

Use this model to identify candidate gene/gene-product, disease, and variant mentions in
English biomedical sentences, such as PubMed titles and abstracts. Typical downstream
uses include literature triage, biomedical search, and candidate generation for entity
normalization or relation extraction.

This model performs mention detection only. It does not normalize mentions to database
identifiers, determine relations between entities, or establish that a biomedical claim
is true. It should not be used by itself for diagnosis, treatment decisions, or other
clinical decision-making.

## Usage

```python
from transformers import pipeline

checkpoint = "nobal/gdv-scout"
extract_entities = pipeline(
    task="token-classification",
    model=checkpoint,
    tokenizer=checkpoint,
    aggregation_strategy="simple",
)

text = "The EGFR L858R variant predicts response in non-small cell lung cancer."
entities = extract_entities(text)

for entity in entities:
    print(entity)
```

For documents, split the text into sentences first and keep each input below 192 subword
tokens. Longer inputs are outside the training setup; tokenize them with
`truncation=True, max_length=192` before calling the model directly.

## Evaluation

### BioRED test set

![BioRED exact entity-level precision, recall, and F1 by entity type](assets/biored-performance.svg)

The saved checkpoint was re-evaluated locally on the converted BioRED test split on
2026-09-02. Scores use `seqeval` entity-level evaluation: entity type and the complete BIO
boundary must match. The test split contains 1,108 sentences and 2,054 target entities.

| Entity type | Precision | Recall | F1 | Support |
|---|---:|---:|---:|---:|
| Disease | 0.8049 | 0.8442 | 0.8241 | 860 |
| GeneOrGeneProduct | 0.8712 | 0.8876 | 0.8793 | 1,014 |
| Variant | 0.6840 | 0.8056 | 0.7398 | 180 |
| **Micro average** | **0.8249** | **0.8622** | **0.8431** | **2,054** |
| Macro average | 0.7867 | 0.8458 | 0.8144 | 2,054 |

The associated project report summarizes the original run as approximately 0.85 micro-F1.
The table above gives the metrics reproduced directly from this packaged checkpoint and
is therefore the recommended result to cite for these files.

### External benchmarks

![Relaxed precision, recall, and F1 on three external biomedical NER benchmarks](assets/external-benchmarks.svg)

The following scores were computed from saved predictions using case-insensitive,
document-level normalized mention matching. “Relaxed” permits token-set containment in
either direction to accommodate different tokenization and boundary conventions. These
numbers are not directly comparable to the exact BioRED scores above.

| Benchmark | Target type | Precision | Recall | Relaxed F1 |
|---|---|---:|---:|---:|
| BC2GM test | GeneOrGeneProduct | 0.916 | 0.729 | 0.812 |
| NCBI-disease test | Disease | 0.909 | 0.771 | 0.834 |
| BC5CDR-disease test | Disease | 0.873 | 0.896 | 0.884 |

## Training data

The continual fine-tuning stage uses the standard BioRED train/validation/test partitions,
converted from token/BIO records to sentence-level character spans. Only BioRED's Gene,
Disease, and Variant labels are retained as target entities.

| Split | Sentences | Disease | GeneOrGeneProduct | Variant |
|---|---:|---:|---:|---:|
| Train | 4,374 | 3,388 | 3,848 | 660 |
| Validation | 1,134 | 905 | 948 | 191 |
| Test | 1,108 | 860 | 1,014 | 180 |

See the [BioRED paper](https://doi.org/10.1093/bib/bbac282) for dataset construction and
annotation details.

## Training procedure

Entity spans were aligned to BioLinkBERT subword offsets. Special tokens use an ignore
index of `-100`; other tokens are optimized with standard token-classification loss. The
best checkpoint was selected by validation entity-level F1.

| Hyperparameter | Value |
|---|---:|
| Epochs | 3 |
| Learning rate | 2e-5 |
| Train batch size | 16 |
| Evaluation batch size | 32 |
| Weight decay | 0.01 |
| Warmup ratio | 0.1 |
| LR schedule | Linear |
| Random seed | 13 |
| Maximum sequence length | 192 |

## Limitations

- The model is trained for English biomedical prose and may perform poorly on clinical
  notes, patents, non-English text, or general-domain text.
- It follows BioRED's strict span convention. Different corpora may define gene families,
  complexes, generic mentions, and entity boundaries differently.
- Chemical, species, and cell-line entities are deliberately not extracted.
- Variant performance is less reliable than gene and disease performance, and the BioRED
  variant test subset is comparatively small.
- Long text should be sentence-segmented. Naive truncation can silently omit entities.
- Predictions can contain false positives and missed entities. Human review and entity
  normalization are recommended for high-stakes or production applications.
- No explicit cross-corpus PMID overlap audit between the earlier silver-label corpus and
  BioRED was recorded; benchmark results should be interpreted with that caveat.

## Citation

If you use the training dataset, cite BioRED:

```bibtex
@article{luo2022biored,
  title = {BioRED: a rich biomedical relation extraction dataset},
  author = {Luo, Ling and Lai, Po-Ting and Wei, Chih-Hsuan and Arighi, Cecilia N. and Lu, Zhiyong},
  journal = {Briefings in Bioinformatics},
  volume = {23},
  number = {5},
  pages = {bbac282},
  year = {2022},
  doi = {10.1093/bib/bbac282}
}
```

The encoder is based on BioLinkBERT:

```bibtex
@inproceedings{yasunaga2022linkbert,
  title = {LinkBERT: Pretraining Language Models with Document Links},
  author = {Yasunaga, Michihiro and Leskovec, Jure and Liang, Percy},
  booktitle = {Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics},
  year = {2022}
}
```