RadFinder

Links: Project page — Paper — arXiv — Code — Models

Learning to Read Where to Look: Disease-Aware Vision–Language Pretraining for 3D CT

We pretrain a 3D CT vision–language model on 159k report–volume pairs with two new supervision signals: prompt-based disease labels for classification and intra-scan snippet localization for axial depth grounding. A single unified model reaches state-of-the-art retrieval on CT-RATE, competitive disease classification, and slice-level localization at 12 mm resolution.

Version 2610-full

One shared head for all three tasks: retrieval, disease prompts and localization.

Model	RefCT	CT-RATE	Merlin	INSPECT	CT-RATE R@10	CT-RATE MAP@5	CT-RATE AUC	Rad-ChestCT AUC	Merlin Findings R@1	Merlin Impr. R@1
2610-full	✓	✓	✓	✓	31.2	78.2	83.6	77.0	68.0	40.1
2610-refct	✓				24.4	76.7	80.5	77.7	57.5	33.8
2610-public		✓	✓	✓	26.8	77.4	83.2	73.7	66.8	40.0
2610-ctrate		✓			21.3	77.2	82.7	73.2	32.4	19.8

Model	Localization MAE (mm)	<6mm	<18mm	<30mm
2610-full	34.4	23.3	49.3	66.6
2610-refct	37.0	20.8	45.0	61.8

Usage

See the GitHub repository.

Training data

RefCT (internal): ~98k report–volume pairs from ~50k patients at a single hospital; in-house clinical data, not publicly released.
CT-RATE (CC BY-NC-SA 4.0)
Merlin (Stanford AIMI non-commercial research DUA)
INSPECT (Stanford AIMI non-commercial research DUA)

Further acknowledgements

The model and parts of the SigLIP training framework in src/radfinder are based on SPECTRE
The text processing pipeline in src/rate is used to create binary labels based on text reports and is based on RATE
We thank the MONAI, timm, and Hugging Face transformers maintainers for the libraries and all other package maintainers listed in requirements.txt
The demo scan under assets/demo/s0859/ is case s0859 from TotalSegmentator v2 (Wasserthal et al., CC-BY-4.0).
Funding, additional acknowledgements, full citations: see paper.

License

All code is MIT (see LICENSE) unless a file header says otherwise. Files in src/rate/ that carry a # Vendored from YalaLab/rate ... (ECL 2.0) header are derivatives of the upstream rate package and are licensed under ECL 2.0 (see LICENSE_RATE).
RadFinder model weights are CC BY-NC-SA 4.0, see LICENSE_MODELS.
- Note: the weights are subject to the original dataset licenses. Users intending to use RadFinder in commercial settings should verify dataset and model licensing and obtain any required permissions.

Citation

If you use this code, models, or results, please cite:

@inproceedings{ging2026radfinder,
  author    = {Simon Ging and Philipp Arnold and Sebastian Walter and Hani Alnahas and Hannah Bast and Elmar Kotter and Jiancheng Yang and Behzad Bozorgtabar and Thomas Brox},
  title     = {Learning to Read Where to Look: Disease-Aware Vision--Language Pretraining for 3{D} {CT}},
  booktitle = {Medical Image Computing and Computer Assisted Intervention -- {MICCAI} 2026},
  series    = {Lecture Notes in Computer Science},
  volume    = {16878},
  pages     = {250--259},
  publisher = {Springer Nature Switzerland},
  year      = {2026},
  month     = {September},
  doi       = {10.1007/978-3-032-38059-3_24},
  url       = {https://papers.miccai.org/miccai-2026/0572-Paper3986.html}
}