Foundation Models Generalize Better Across Hospitals — But Task-Specific Networks Still Win on Home Turf

DINOv2 adapted with minimal tuning beats U-Nets on external clinical data across three imaging domains, but loses on boundary precision when training data is plentiful.

Imagine you hire a chef who trained in Paris, Rome, and Tokyo versus one who spent a decade mastering your local cuisine. At home, the local chef nails every dish. But send them both to a kitchen they've never seen — different ingredients, different equipment — and the worldly chef adapts faster. That's the core finding here: foundation models pretrained on massive natural-image datasets generalize better to unseen clinical sites, while task-specific U-Nets remain competitive or superior within their training distribution. The committed claim is that DINOv2 adapted with minimal parameter tuning consistently outperforms U-Nets and ResU-Nets on external (out-of-distribution) medical image segmentation datasets across colonoscopy, breast ultrasound, and dermoscopy — and that only a small fraction of the foundation model's parameters need adaptation, with the required capacity determined by the imaging domain, not the adaptation strategy. This is not a first-of-kind result, but it's a more honest version of a comparison that's been done poorly many times before. The ladder positioning is where this paper earns its keep. Most foundation-model-for-medical-imaging papers compare against a single task-specific baseline with unjustified hyperparameters, tested only on in-distribution data. Kutlusan and Li explicitly test both U-Net and ResU-Net baselines, vary the adaptation strategy from bias-only tuning through full fine-tuning, and — critically — evaluate on independent external datasets from different institutions. The task-specific models remained competitive on in-distribution data when training samples were abundant and actually produced sharper boundary delineation, while the foundation model more often missed small lesions. That kind of honesty about where your method loses is rare and valuable. Architecturally, the paper sits in the adapter/fine-tuning family for vision transformers. DINOv2 is a self-supervised ViT pretrained on natural images (LVD-142M), adapted here via strategies spanning bias tuning, linear probing, LoRA-style partial updates, and full fine-tuning. The baselines are convolutional encoder-decoder architectures (U-Net, ResU-Net) trained from scratch. The key structural insight: the imaging modality — not the adaptation method — determined how much of the foundation model needed unfreezing, suggesting domain gap is the binding constraint. On integrity, the paper does several things right and one thing unusually well. It tests on held-out data from the training source AND on independent datasets from other institutions — the critical test that most medical imaging papers skip. It also flags a widely used polyp test set (likely Kvasir-SEG vs. HyperKvasir overlap) that duplicates images from another benchmark, which is a genuine contribution to benchmark hygiene in this subfield. The weaknesses: no pre-registration, no code release mentioned, and the exact configurations of the baseline models could still be questioned even if the paper claims to have justified them more carefully than prior work. The milestone question for this line of work isn't about a single number — it's about the threshold at which hospital IT departments stop training site-specific models and start deploying adapted foundation models as default. We're currently at 'foundation models generalize better on 3 domains across a handful of external sets.' The next milestone is multi-center prospective validation on 10+ sites with regulatory-grade metrics, which is probably 2-3 years away for any single imaging modality. The obvious experiment not run: adapting DINOv2 (or SAM, or BiomedCLIP) on a truly low-data regime — say, 20-50 labeled images per site — to quantify the crossover point where foundation model adaptation definitively beats task-specific training. The paper hints that task-specific models need abundant data to compete, but never maps the sample-efficiency curve explicitly. My read: this is likely (a) — they scoped the paper tightly around the generalization question and left the sample-efficiency study for a follow-up, not because it would fail but because it's a different paper.