mamounyosef/chest-xray-bench

Chest X-ray Bench: 43 CheXpert classifiers

Technical report · Code and configs on GitHub

43 chest X-ray classifiers, all trained on CheXpert under one fixed pipeline so their scores are directly comparable. Released alongside a technical report that compares the design choices behind them.

Most models predict five findings: Atelectasis, Cardiomegaly, Consolidation, Edema, Pleural Effusion.

---

The ensemble members

Blended together these three score 0.9174 mean AUROC on test500, the best result in the study.

Model	valid200	test500	Input size	Params
convnext_base_22k_1600x1312	0.9063	0.9037	1600x1312	87.6M
medmae_vitb_nih_B_768_s2	0.9064	0.9067	768x640	87.1M
rad_dino_vitB_768	0.9016	0.9100	784x644	86.6M
All three blended		0.9174

---

Quick start

pip install torch timm transformers safetensors huggingface_hub opencv-python

Download modeling.py from this repo, then:

import cv2, torch
from modeling import load_model, preprocess

model, cfg = load_model("rad_dino_vitB_768")     # any folder name below

img = cv2.imread("frontal.jpg", cv2.IMREAD_GRAYSCALE)
x = preprocess(img, cfg)

with torch.no_grad():
    probs = model(x).sigmoid()[0]

for task, p in zip(cfg["tasks"], probs.tolist()):
    print(f"{task:18} {p:.3f}")

load_model picks the right builder for each backbone, so the same two lines work for every model here. preprocess reproduces the training pipeline: resize to fit the target box keeping the aspect ratio, zero pad the short side, normalize. Never mirror a chest X-ray at inference, it moves the heart to the wrong side.

---

What is in each folder

File	Description
model.safetensors	Weights only, fp32. No optimizer state.
config.json	Backbone, input size, normalization, label policy, head layout, scores.
thresholds.json	Per finding decision thresholds, tuned for F1 on the large validation split then frozen. Only needed for hard yes/no predictions.

A note on outputs

Most models emit one logit per finding, so sigmoid gives five probabilities. A few trained with the three way uncertainty head emit 9 logits, and the single pathology models emit 1. config.json always states which, under head:

"head": {
  "n_logits": 5,
  "layout": "one sigmoid logit per task",
  "task_slices": {"Atelectasis": [0, 1], "Cardiomegaly": [1, 2], "...": []}
}

---

How the ensemble was combined

The report's headline, 0.9174 mean AUROC on the official test set, is a plain 1/3 probability average of the three models above. Fitting per finding weights on the validation split did not improve on it.

That is +0.0061 over the best single model, medmae_vitb_nih_B_768_s2_seed1337 at 0.9113, with a 95% bootstrap interval of [+0.0005, +0.0118] over 10,000 resamples. The lesson from the report: diversity beats count. Seven runs of the same backbone averaged to 0.9095, below the best single model, while three genuinely different backbones reached 0.9174.

---

All models

valid200 and test500 are the official radiologist labelled splits, 202 and 518 frontal images. Neither was trained on.

Chest X-ray pretrained

Model	Backbone	Input size	Params	valid200	test500
rad_dino_vitB_768	vit_base_patch14_dinov2 (microsoft/rad-dino)	784x644	86.6M	0.9016	0.9100
medmae_vitb_nih_B_768_s2_seed1337	vit_base_patch16_224	768x640	87.1M	0.9105	0.9112
rad_dino_vitB_1064x896	vit_base_patch14_dinov2 (microsoft/rad-dino)	1064x896	86.6M	0.9000	0.9046
medmae_vitb_nih_B_768_s2_seed7	vit_base_patch16_224	768x640	87.1M	0.9045	0.9077
medmae_vitb_nih_B_448_s1_seed7	vit_base_patch16_224	448x384	86.2M	0.8972	0.9080
medmae_vitb_nih	vit_base_patch16_224	384x320	86.0M	0.8970	0.9020
medmae_vitb_nih_B_768_s2	vit_base_patch16_224	768x640	87.1M	0.9064	0.9067
medmae_vitb_nih_B_448_s1_seed1337	vit_base_patch16_224	448x384	86.2M	0.9052	0.9088
medmae_vitb_raw	vit_base_patch16_224	384x320	86.0M	0.9030	0.9046
medmae_vitb_chexpert	vit_base_patch16_224	384x320	86.0M	0.8906	0.8994
medmae_vitb_nih_B_448_s1	vit_base_patch16_224	448x384	86.2M	0.8957	0.9026

High resolution ConvNeXt

Model	Backbone	Input size	Params	valid200	test500
convnext_base_22k_768x640	convnext_base.fb_in22k_ft_in1k_384	768x640	87.6M	0.8994	0.9037
convnext_base_22k_1600x1312	convnext_base.fb_in22k_ft_in1k_384	1600x1312	87.6M	0.9063	0.9037
convnext_large_22k_768x640	convnext_large.fb_in22k_ft_in1k_384	768x640	196.2M	0.9009	0.8987

Uncertainty and objective

Model	Backbone	Input size	Params	valid200	test500
convnext_base_22k_final_stage1	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8969	0.8985
convnext_base_22k_aucm	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8825	0.8836
densenet121_cons_zeros	densenet121	384x320	7.0M	0.8834	0.8926
densenet121_u_mixed	densenet121	384x320	7.0M	0.8820	0.8978
resnet50_u_zeros	resnet50	384x320	23.6M	0.8588	0.8796

Single pathology

Model	Backbone	Input size	Params	valid200	test500
convnext_base_22k_final_stage2_pleural_effusion	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.9324	0.9557
convnext_base_22k_final_stage2_edema	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.9454	0.9232
convnext_base_22k_final_stage2_cardiomegaly	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8319	0.8949
convnext_base_22k_consolidation_only	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8947	0.9155
convnext_base_22k_cardiomegaly_only	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8277	0.8850
convnext_base_22k_final_stage2_consolidation	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.9175	0.8851
convnext_base_22k_final_stage2_atelectasis	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8557	0.8363
convnext_base_22k_atelectasis_only	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8500	0.8431

Backbone comparison

Model	Backbone	Input size	Params	valid200	test500
convnext_base_22k_seed1337	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8924	0.8967
convnext_base_22k_cxr14_pretrain_lowlr	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8919	0.8942
convnext_base_22k_seed7	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8916	0.8949
convnext_tiny	convnext_tiny	384x320	27.8M	0.8850	0.8896
convnext_base_22k_cxr14_pretrain_lowlr_all	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8895	0.8899
convnext_large_22k_cxr14_pretrain	convnext_large.fb_in22k_ft_in1k_384	384x320	196.2M	0.8803	0.8811
convnext_small	convnext_small	384x320	49.5M	0.8675	0.8789
convnext_base_22k_cxr14_pretrain	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8759	0.8795
convnext_large_22k	convnext_large.fb_in22k_ft_in1k_384	384x320	196.2M	0.8748	0.8792
convnext_base_22k	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8806	0.8775
densenet121_seed7	densenet121	384x320	7.0M	0.8740	0.8804
densenet121	densenet121	384x320	7.0M	0.8727	0.8750
densenet201	densenet201	384x320	18.3M	0.8755	0.8750
densenet121_seed123	densenet121	384x320	7.0M	0.8584	0.8634
resnet50_without_clahe	resnet50	384x320	23.6M	0.8759	0.8836
resnet50_with_clahe	resnet50	384x320	23.6M	0.8664	0.8732

---

Training setup

Data	CheXpert train split, frontal views only, split 90/10 by patient
Loss	Binary cross entropy over the five findings, masked where a target is undefined
Optimizer	AdamW, batch 64, cosine schedule with a one epoch warmup
Augmentation	Rotation, translation, scale, brightness, contrast. No horizontal flip
Metric	Mean AUROC over the five findings, scored per image

Full configurations and training code are on GitHub, and the report is at doi.org/10.13140/RG.2.2.19222.92487.

---

Intended use and limits

These are research artifacts, released to support a technical report.

• Not a medical device. Do not use them to make clinical decisions.
• No external validation. Every number here comes from CheXpert's own splits. Performance on images from other hospitals, scanners or populations is unknown.
• The test sets are small. 202 and 518 images, so per model rankings are unstable and confidence intervals are wide.
• Labels come from an automatic labeler applied to radiology reports, so the models learn that labeler's conventions along with the findings.

License and data

CC BY-NC 4.0: free to use, share and build on with attribution, non-commercial only. This matches CheXpert's Stanford University Dataset Research Use Agreement, which permits research use and forbids commercial use.

The CheXpert data is not redistributed here, in this repo or on GitHub. Request it from Stanford AIMI directly.

Citation

@techreport{yosef2026chexpert,
  title       = {A Systematic Study of Design Choices for Multi-Label Chest X-ray Classification on CheXpert},
  author      = {Yosef, Ma'moun},
  year        = {2026},
  institution = {University of Jordan},
  doi         = {10.13140/RG.2.2.19222.92487}
}