Artifact-aware MIL heads for prostate biopsy ISUP grading
Trained multiple instance learning (MIL) heads from the paper Aggregation, Ordinal Training, and Artifact Gating in Multiple Instance Learning for Prostate Biopsy Grading. Each model maps a bag of frozen Virchow2 patch embeddings from one prostate biopsy whole-slide image to an ISUP grade group (0β5).
Code, data split and result files: https://github.com/PathOS-Code/artifact-aware-mil-prostate
Contents
30 models, the members of every ensemble reported in the paper. Each folder
holds best_model.safetensors (weights only), the run's recorded
config.json, and release_info.json (best epoch, validation QWK, SHA-256).
| Folder | Members | Aggregator | Loss | Gate | Protocol |
|---|---|---|---|---|---|
virchow2_transmil_focal_nogate_5seed/ |
seed0βseed4 |
TransMIL-style, 2 layers | focal | none | fixed split, 5 seeds |
virchow2_transmil_focal_maskgate_5seed/ |
seed0βseed4 |
TransMIL-style, 2 layers | focal | mask-only | fixed split, 5 seeds |
virchow2_transmil_focal_fullgate_5seed/ |
seed0βseed4 |
TransMIL-style, 2 layers | focal | full robust | fixed split, 5 seeds |
virchow2_abmil_focal_maskgate_5seed/ |
seed0βseed4 |
ABMIL | focal | mask-only | fixed split, 5 seeds |
virchow2_transmil_coral_maskgate_5fold/ |
fold_1βfold_5 |
TransMIL-style, 2 layers | CORAL | mask-only | 5-fold CV |
virchow2_transmil_coral_fullgate_5fold/ |
fold_1βfold_5 |
TransMIL-style, 2 layers | CORAL | full robust | 5-fold CV |
The mask-only and full robust gate variants implement the same model: flagged tokens are excluded from attention in both, so the attenuation factor Ξ± does not change the output. Their differences are run-to-run (replicate) differences. Ξ± was not logged for any run; the evaluation code uses the default Ξ± = 1e-4.
Total size about 4.1 GB (TransMIL-style members 163 MB each, ABMIL members
6 MB each). The weights were exported from the training checkpoints with
scripts/training/export_weights.py in the code repository; for every model
the exported weights give identical logits to the original checkpoint.
Input
The models do not contain an image encoder. Each slide must be converted
to a bag saved as a .pt file with:
| Key | Shape | Content |
|---|---|---|
embeddings |
[N, 1280] |
Virchow2 CLS token of each tissue patch |
spatial_variances |
[N] |
variance of the Laplacian of the grayscale patch, used by the gate |
coords |
[N, 2] |
patch level-0 coordinates (used only for heatmaps) |
Patches are 256 Γ 256 pixels at level 0 (nominal 20Γ), Macenko stain
normalization with a fixed reference, resized to 224 Γ 224 for Virchow2. The
code repository's scripts/preprocessing/preprocess.py and
scripts/feature_extraction/extract_features.py produce these bags.
Virchow2 weights are not included. Request access to
paige-ai/Virchow2 and use it
under its own license terms.
Output
- Focal-loss models: 6 logits (ISUP 0β5); class probabilities by softmax.
- CORAL models: 5 ordinal threshold logits; class probabilities by differencing the cumulative sigmoids, clipping at 0 and renormalizing.
Ensembles average the class probabilities of their members.
Usage
git clone https://github.com/PathOS-Code/artifact-aware-mil-prostate.git
cd artifact-aware-mil-prostate
uv sync
hf download PathOS-Code/artifact-aware-mil-prostate --local-dir weights
uv run python scripts/evaluation/ensemble_eval.py \
--exp-dirs weights/virchow2_transmil_focal_fullgate_5seed/seed{0,1,2,3,4} \
--embeddings-dir outputs/embeddings_virchow2 \
--labels-csv data/panda_labels.csv \
--split-csv splits/panda_fixed_split.csv \
--out outputs/ensembles/panda_focal_fullgate_5seed.json
For an external cohort, pass its embeddings and labels CSV with
--val-ratio 0 --test-ratio 0 to evaluate every slide. See REPRODUCE.md in
the code repository for all experiments.
Results
From the paper and the result files in the code repository (results/).
QWK = quadratic weighted kappa.
PANDA held-out test split (1,061 slides; 10,614 slides overall from Radboud and Karolinska):
| Ensemble | Test QWK |
|---|---|
| TransMIL-style, focal, no gate, 5 seeds | 0.9063 |
| TransMIL-style, focal, mask-only, 5 seeds | 0.9104 |
| TransMIL-style, focal, full robust gate, 5 seeds | 0.9134 (95% CI 0.8856β0.9368) |
| ABMIL, focal, mask-only, 5 seeds | 0.8919 |
PANDA five-fold cross-validation (out-of-fold QWK, optimistic): CORAL mask-only 0.9226, CORAL full robust gate 0.9227.
External cohorts, direct transfer without fine-tuning:
| Ensemble | SICAPv2 (155 biopsies) | CrowdGleason (232 slide identifiers) |
|---|---|---|
| TransMIL-style, focal, no gate, 5 seeds | 0.8664 | 0.8076 |
| TransMIL-style, focal, mask-only, 5 seeds | 0.8633 | 0.8028 |
| TransMIL-style, focal, full robust gate, 5 seeds | 0.8482 | 0.8076 |
| TransMIL-style, CORAL, mask-only, 5 folds | 0.8760 | 0.8109 |
| TransMIL-style, CORAL, full robust gate, 5 folds | 0.8772 (95% CI 0.803β0.933) | 0.8080 (95% CI 0.733β0.871) |
Caveats, as stated in the paper:
- The PANDA test ensemble was the best of three ensembles, selected post hoc after exploratory runs that included the test slides.
- None of the paired gated-versus-ungated QWK intervals excluded zero; no statistically detectable gating effect was found, and gate firing was not measured.
- All intervals treat slides as independent and ignore patient clustering.
- CrowdGleason slide labels are surrogates derived from patch labels.
- Aggregator and ordinal-loss contrasts are confounded and descriptive.
- Training is not bit-wise reproducible; retrained models differ slightly.
Limitations
- Developed on PANDA only (two institutions, routine-practice labels); external evaluation covers two small public cohorts.
- Performance under other scanners, stains, tissue preparation or patient populations is unknown.
- Each model predicts one ISUP grade group per slide. Attention heatmaps show which patches the model weighted, not where each Gleason pattern is; pattern-level segmentation is not part of this release.
- Depends on Virchow2 embeddings; other encoders will not work with these weights.
License
The weights are released under CC BY-NC-SA 4.0 because they were trained on PANDA (CC BY-NC-SA 4.0): noncommercial use, attribution required, derivatives under the same license. The code in the GitHub repository is under the PolyForm Noncommercial License 1.0.0.
Citation
@article{TODO,
title = {Aggregation, Ordinal Training, and Artifact Gating in Multiple Instance Learning for Prostate Biopsy Grading},
author = {TODO},
journal = {TODO},
year = {TODO}
}