Artifact-aware MIL heads for prostate biopsy ISUP grading

Trained multiple instance learning (MIL) heads from the paper Aggregation, Ordinal Training, and Artifact Gating in Multiple Instance Learning for Prostate Biopsy Grading. Each model maps a bag of frozen Virchow2 patch embeddings from one prostate biopsy whole-slide image to an ISUP grade group (0–5).

Code, data split and result files: https://github.com/PathOS-Code/artifact-aware-mil-prostate

Contents

30 models, the members of every ensemble reported in the paper. Each folder holds best_model.safetensors (weights only), the run's recorded config.json, and release_info.json (best epoch, validation QWK, SHA-256).

Folder Members Aggregator Loss Gate Protocol
virchow2_transmil_focal_nogate_5seed/ seed0–seed4 TransMIL-style, 2 layers focal none fixed split, 5 seeds
virchow2_transmil_focal_maskgate_5seed/ seed0–seed4 TransMIL-style, 2 layers focal mask-only fixed split, 5 seeds
virchow2_transmil_focal_fullgate_5seed/ seed0–seed4 TransMIL-style, 2 layers focal full robust fixed split, 5 seeds
virchow2_abmil_focal_maskgate_5seed/ seed0–seed4 ABMIL focal mask-only fixed split, 5 seeds
virchow2_transmil_coral_maskgate_5fold/ fold_1–fold_5 TransMIL-style, 2 layers CORAL mask-only 5-fold CV
virchow2_transmil_coral_fullgate_5fold/ fold_1–fold_5 TransMIL-style, 2 layers CORAL full robust 5-fold CV

The mask-only and full robust gate variants implement the same model: flagged tokens are excluded from attention in both, so the attenuation factor Ξ± does not change the output. Their differences are run-to-run (replicate) differences. Ξ± was not logged for any run; the evaluation code uses the default Ξ± = 1e-4.

Total size about 4.1 GB (TransMIL-style members 163 MB each, ABMIL members 6 MB each). The weights were exported from the training checkpoints with scripts/training/export_weights.py in the code repository; for every model the exported weights give identical logits to the original checkpoint.

Input

The models do not contain an image encoder. Each slide must be converted to a bag saved as a .pt file with:

Key Shape Content
embeddings [N, 1280] Virchow2 CLS token of each tissue patch
spatial_variances [N] variance of the Laplacian of the grayscale patch, used by the gate
coords [N, 2] patch level-0 coordinates (used only for heatmaps)

Patches are 256 Γ— 256 pixels at level 0 (nominal 20Γ—), Macenko stain normalization with a fixed reference, resized to 224 Γ— 224 for Virchow2. The code repository's scripts/preprocessing/preprocess.py and scripts/feature_extraction/extract_features.py produce these bags.

Virchow2 weights are not included. Request access to paige-ai/Virchow2 and use it under its own license terms.

Output

  • Focal-loss models: 6 logits (ISUP 0–5); class probabilities by softmax.
  • CORAL models: 5 ordinal threshold logits; class probabilities by differencing the cumulative sigmoids, clipping at 0 and renormalizing.

Ensembles average the class probabilities of their members.

Usage

git clone https://github.com/PathOS-Code/artifact-aware-mil-prostate.git
cd artifact-aware-mil-prostate
uv sync
hf download PathOS-Code/artifact-aware-mil-prostate --local-dir weights

uv run python scripts/evaluation/ensemble_eval.py \
  --exp-dirs weights/virchow2_transmil_focal_fullgate_5seed/seed{0,1,2,3,4} \
  --embeddings-dir outputs/embeddings_virchow2 \
  --labels-csv data/panda_labels.csv \
  --split-csv splits/panda_fixed_split.csv \
  --out outputs/ensembles/panda_focal_fullgate_5seed.json

For an external cohort, pass its embeddings and labels CSV with --val-ratio 0 --test-ratio 0 to evaluate every slide. See REPRODUCE.md in the code repository for all experiments.

Results

From the paper and the result files in the code repository (results/). QWK = quadratic weighted kappa.

PANDA held-out test split (1,061 slides; 10,614 slides overall from Radboud and Karolinska):

Ensemble Test QWK
TransMIL-style, focal, no gate, 5 seeds 0.9063
TransMIL-style, focal, mask-only, 5 seeds 0.9104
TransMIL-style, focal, full robust gate, 5 seeds 0.9134 (95% CI 0.8856–0.9368)
ABMIL, focal, mask-only, 5 seeds 0.8919

PANDA five-fold cross-validation (out-of-fold QWK, optimistic): CORAL mask-only 0.9226, CORAL full robust gate 0.9227.

External cohorts, direct transfer without fine-tuning:

Ensemble SICAPv2 (155 biopsies) CrowdGleason (232 slide identifiers)
TransMIL-style, focal, no gate, 5 seeds 0.8664 0.8076
TransMIL-style, focal, mask-only, 5 seeds 0.8633 0.8028
TransMIL-style, focal, full robust gate, 5 seeds 0.8482 0.8076
TransMIL-style, CORAL, mask-only, 5 folds 0.8760 0.8109
TransMIL-style, CORAL, full robust gate, 5 folds 0.8772 (95% CI 0.803–0.933) 0.8080 (95% CI 0.733–0.871)

Caveats, as stated in the paper:

  • The PANDA test ensemble was the best of three ensembles, selected post hoc after exploratory runs that included the test slides.
  • None of the paired gated-versus-ungated QWK intervals excluded zero; no statistically detectable gating effect was found, and gate firing was not measured.
  • All intervals treat slides as independent and ignore patient clustering.
  • CrowdGleason slide labels are surrogates derived from patch labels.
  • Aggregator and ordinal-loss contrasts are confounded and descriptive.
  • Training is not bit-wise reproducible; retrained models differ slightly.

Limitations

  • Developed on PANDA only (two institutions, routine-practice labels); external evaluation covers two small public cohorts.
  • Performance under other scanners, stains, tissue preparation or patient populations is unknown.
  • Each model predicts one ISUP grade group per slide. Attention heatmaps show which patches the model weighted, not where each Gleason pattern is; pattern-level segmentation is not part of this release.
  • Depends on Virchow2 embeddings; other encoders will not work with these weights.

License

The weights are released under CC BY-NC-SA 4.0 because they were trained on PANDA (CC BY-NC-SA 4.0): noncommercial use, attribution required, derivatives under the same license. The code in the GitHub repository is under the PolyForm Noncommercial License 1.0.0.

Citation

@article{TODO,
  title   = {Aggregation, Ordinal Training, and Artifact Gating in Multiple Instance Learning for Prostate Biopsy Grading},
  author  = {TODO},
  journal = {TODO},
  year    = {TODO}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support