Melanocytic lesion reports, read for report generation

01Why this paper matters for report generation

Slide-level vision–language models in pathology have so far mostly written a sentence or two, as the authors point out. PRISM, the contrastive-captioner (CoCa) design this paper follows, was trained on GPT-4 rewrites of each report as one sentence of under 20 words. This paper asks the same kind of model for a full-length report, the microscopic description and the conclusion: 77 words on average for a common nevus (an ordinary mole) and 125 for other lesions. It then scores the result with a blinded pathologist instead of text-overlap metrics.

This is the blog’s first pathology post, and three things differ from the CT posts:

• The image. A case is a few glass slides scanned at 20× or 40×, far too large for an encoder. Each slide is cut into 224-pixel tiles about 0.1 mm across derived, and a frozen tile encoder turns every tile into a vector. The largest cases, fewer than 2%, exceed 100,000 tiles, about 5 gigapixels derived.
• The unit. The report covers a case, not a slide: 2.2 slides on average here derived, with all their tiles pooled.
• The report. A description of what the microscope shows, then a conclusion with the diagnosis. Reports also hold results the H&E slide can’t show, such as immunostains and molecular tests, and some conclusions rest on them.

The data are a tenth of PRISM’s derived, 19,645 cases against 195,344 specimens, and come from one corner of dermatopathology.

02The data: cleaned reports, two populations

The data come from the pathology archive of the University Medical Center Utrecht: melanocytic lesions accessioned from 2013 to 2020, 42,512 H&E slides for 19,645 cases from 14,978 patients, split 80/10/10 by patient into training, validation and test sets. The dataset is not released.

The reports were written in Dutch. One fine-tuned language model translated them into English; a second cut them into sub-sentences and labelled each by content. Two kinds were kept: observations made on the H&E slides, and conclusions, which the authors judged too important to withhold. Everything else went out, immunostain and molecular results included, as did every case whose report had no H&E description. Radiology readers will recognise the move: it is the counterpart of stripping references to prior exams from chest X-ray reports. The pipeline has its own error rate: the companion paper counted 219 translation errors in 3,597 test sentences (6.1%).

Table 1 — Two populations in one dataset. The reader study samples the two groups equally; the test set doesn’t hold them equally. The 18.2% and the two sampling fractions are our arithmetic derived. Sources: Sections 2.1 and 3.1, and the paper’s Table 2.

The other lesions, from non-common nevi to melanocytomas and melanomas, share about 2,900 training cases derived.

03The reader study: one pathologist, 50 cases

A pathologist with dermatopathology experience, identified by initials that match a co-author, read 50 test cases drawn at random within the two groups. For each case the reader had the slides and two reports in random order, the generated one and the original, without knowing which was which. Each report got four counts (factual errors, unverifiable statements, omissions, repeated phrases) and a score from 1, mostly inaccurate, to 5, usable with little or no editing.

The paper’s “All” row weighs the two groups equally: 3.7 against 4.6. At the test set’s own mix, 81% common nevi, the model’s mean would be about 4.2 and the originals’ still 4.6 derived. Four things stand out:

1. On common nevi, no difference in score is visible. 4.5 ± 0.8 against 4.6 ± 0.6. The two columns score the same 25 cases, but the paper gives no paired statistics; treated as independent samples, the 0.1 gap carries a 95% interval of roughly ±0.4 points derived, so “on par” means within about half a point.
2. On other lesions the model is 1.5 points lower. It makes five times the factual errors of the original reports and omits more. In the paper’s melanoma example it gets the melanoma right and the subtype wrong.
3. Nothing unverifiable, and that is the cleaning. None of the 50 generated reports holds an unverifiable statement; for a sample of 50, the rule of three still allows a true rate of about 6% of reports derived. The ablation is in a sibling study by the same group: under the same protocol, its model produced 3.0 unverifiable statements per report when trained on uncleaned reports, and none on cleaned ones.
4. It loops, probably because of decoding. 2.9 repeated phrases per report on other lesions; in the paper’s example the dermal component is announced four times. The paper suggests report length and rare subtypes as causes, and doesn’t describe decoding. The released evaluation code passes no repetition penalty (the README’s inference example sets 1.1), and the documented command sets top-k to 1, which is greedy. The sibling study’s model, released with beam search and a penalty of 1.2, averaged 0.1 repeated phrases against 1.6 here likely, not confirmed.

04The same protocol, run twice

That sibling study is the closest thing to a baseline. Posted on arXiv the same day, it applies the same protocol to the same archive, with a different pathologist as reader, to judge by the initials. Its model is BLIP-2-style, uses older tile features and writes the description without the conclusion.

Two things follow. On other lesions little has changed: the gap went from 1.7 to 1.5 points, and both models make 2.5 factual errors per report. And the original reports are no gold standard. Across each study’s 50 cases they get 4.6 from one pathologist and 3.9 from the other, and the readers find 0.5 and 0.7 factual errors in them per report, which the sibling paper reads as disagreement between observers. The originals the paper shows are in English, so the reader presumably scored them after translation our observation. A model’s 4.5 has no meaning outside its own study.

05Why routine is easy and rare is hard

The paper’s second experiment explains the first. The model also places slides and reports in one embedding space, and the authors test whether a case’s own report can be found among the 1,969 test reports. For common nevi it is in the top 10 for 31% of cases; for other lesions, for 65%. The model writes best what it retrieves worst.

The authors’ reading: common-nevus reports are short and alike, so many fit a given case. Easy to write acceptably, hard to tell apart. That is good news for workload and a warning for evaluation, because a canned nevus paragraph might score well too.

06What the paper leaves out

• Baselines: no other model, and no template or nearest-report baseline. Two reviewers asked, naming PRISM, HistGen and MI-Gen. The authors replied that models trained on other hospitals’ reports would be at an unfair disadvantage, and left retraining them to future work.
• Automatic metrics: none, although the released code can compute BLEU-4, ROUGE-L, CIDEr and METEOR at validation. The other 1,919 test cases are never scored for generation.
• Readers and statistics: one reader, whose initials match a co-author from the department whose reports trained the model; the authors name the single reader and the 50 cases as limits. No test, and no breakdown of the 25 other lesions by diagnosis.
• Variants and remedies: BioGPT’s first 12 blocks are kept frozen, tuned with LoRA or fully fine-tuned, and the three are compared on retrieval only (frozen and LoRA within each other’s intervals, full fine-tuning worse). The paper doesn’t say which wrote the reports. The rebuttal names oversampling and loss weights for the rare lesions; neither is tried.
• Access and compute: the dataset is private. The code is Apache-2.0, and the three checkpoints sit behind an access request on Hugging Face. The paper omits the compute; the rebuttal gives 16 GPUs of 24 GB for about two days.

07Takeaways for building a report generator

1. Clean the targets before tuning the model. Zero unverifiable statements in 50 reports after the training text was cut down to H&E observations and conclusions; 3.0 per report without the cut, in the sibling study.
2. Report by case mix. 4.5 on four cases in five and 3.0 on the rest say more than any average of the two.
3. Put the human reports through the same protocol. They scored 4.6 with one reader and 3.9 with another. Without that row, a model’s 4.5 can’t be read.
4. Give the routine class a template baseline. With the right report in the top 10 of 1,969 for only 31% of common nevi, many reports seem to fit the same case, and a canned paragraph is the bar to clear.
5. Check decoding before blaming the model. 2.9 repeated phrases per report on the harder cases is the most visible flaw here, and the cheapest to test.

ASources

1. Lucassen, R. T., Moonemans, S. P. J., van de Luijtgaarden, T., Breimer, G. E., Blokx, W. A. M., Veta, M. Pathology Report Generation and Multimodal Representation Learning for Cutaneous Melanocytic Lesions. MICCAI 2025, LNCS 15965, pp. 502–511. The page holds the reviews and the rebuttal. PDF · doi · arXiv:2502.19293 (same tables).
2. Code: SanderMoon/MOSAIC (Apache-2.0), read on 5 Oct 2026. Checkpoints: SaltySander/MOSAIC on Hugging Face (gated).
3. The sibling study: Lucassen, R. T., van de Luijtgaarden, T., et al. On the Importance of Text Preprocessing for Multimodal Representation Learning and Pathology Report Generation. arXiv:2502.19285 (2025), Table 3. Code: nuldertien/PathBLIP-2.
4. The report cleaning: Lucassen, R. T., et al. Preprocessing Pathology Reports for Vision-Language Model Development. MICCAI Workshop on Computational Pathology, PMLR 254:61–71 (2024).
5. Shaikovski, G., et al. PRISM: arXiv:2405.10254 (2024). Yu, J., et al. CoCa: arXiv:2205.01917 (2022).
6. Chen, R. J., et al. UNI: Nature Medicine 30, 850–862 (2024). Luo, R., et al. BioGPT: arXiv:2210.10341 (2022).
7. Ramesh, V., Chi, N. A., Rajpurkar, P. Removing references to priors from radiology reports: arXiv:2210.06340 (2022).