BiGen, read for pathology report generation

* Our arithmetic on the paper’s ablation table: 0.024 of the 0.032 gained. The paper’s own average over seven scores gives the same share.

01Why this paper matters for report generation

The blog’s first pathology post read a paper that scored 50 generated reports with a blinded pathologist and no automatic metric. This one brings the opposite evidence: a public benchmark, automatic metrics, seven baselines, and no reader.

Its idea is retrieval: as a pathologist recalls similar cases, BiGen looks up sentences from past reports for the patches it attends to most. The code and the reviews (scores of 3, 4 and 5; no rebuttal round) are public, and this post uses both.

Three things in it are easy to miss:

• No retrieved word reaches the decoder. A lookup returns the mean of three sentence embeddings, and all lookups are pooled into one vector.
• There is no pretrained language model. The decoder is trained from scratch: about 7.9M parameters in its three blocks, counted from the release derived, against the 347M-parameter BioGPT of the first pathology post.
• Retrieval isn’t trained. PLIP is frozen on both sides; the model learns only which patches ask. In the released code the bank is the same during training, so a training slide can pull sentences from its own report our observation.

The mechanism, condensed from the released code (helper names are ours):

def retrieve(plip, attn, bank, ratio=0.4, m=10, top=3):
    # plip: M x 512 patches · bank: T x 512 sentences
    order = attn.argsort(descending=True)  # first pass
    kept = plip[order][: int(len(order) * ratio)]
    groups = pad(kept, m).view(-1, m, 512).mean(1)
    sim = cosine(groups[:, None], bank[None])
    nearest = sim.topk(top, dim=1).indices
    return bank[nearest].mean(1)           # groups x 512

v1, attn = layer(v0, patches)      # visual token, pass 1
k = proj(retrieve(plip, attn, bank))
v2, _ = layer(v1, patches); t1, _ = layer(t0, k)
v3, _ = layer(v2, patches); t2, _ = layer(t1, k)
memory = cat([v1 + v2 + v3, t1 + t2])      # 2 x 512
report = decoder(memory)           # beam search, width 3

02The benchmark: PathText (BRCA)

PathText comes from the MI-Gen paper (MICCAI 2024): 9,009 slide–text pairs built from TCGA. The text is not the pathologist’s report. The scanned report goes through OCR, an LLM is asked to summarise it, and a classifier trained on 88 hand-labelled pairs removes flawed summaries.

The breast subset holds 1,041 slides of 977 patients, by our count on the released split file. In that split, 12 of the 98 test patients also have a slide in the training set, and the benchmark’s loader pairs both with the same report. BiGen keeps one slide per patient, which leaves 796 training, 88 validation and 93 test cases, so a test patient’s report is in neither the training set nor, by the paper’s account, the knowledge bank. That is a real fix, and no number can be carried over from MI-Gen’s paper.

The reference summarises the report of a whole surgical case, tumour size and lymph nodes included, while the input is one slide; MI-Gen’s authors themselves suggest that a two-dimensional slide can’t give a three-dimensional size.

03Results: where the gain comes from

BiGen is first in all ten columns of the paper’s main table, seven text scores and three for HER2: BLEU-4 is 0.135 against 0.118 for HistGen, and 0.293 ROUGE-L against 0.279 for the next best. The ablation table starts from the same row, a plain Transformer (the paper’s vanilla Transformer), so both tables fit on one axis as gains over it.

Three things stand out:

1. One vector for the slide does most of it. Replacing self-attention over all patches by a single learned query lifts BLEU-4 from 0.103 to 0.126. With no retrieval at all, that model already tops HistGen and MI-Gen on all seven text scores. A decoder fed one vector outscores the same decoder attending to every patch: with 796 training reports the bottleneck may regularise, or the scores may reward phrasing more than detail our observation.
2. Retrieval alone hurts. Adding the retrieved vectors through self-attention, with no textual token, takes BLEU-4 from 0.127 to 0.119. Pooling them into a second token instead gives 0.130; letting both tokens share weights gives 0.135. The whole knowledge branch adds 0.008, a quarter of the total gain and about half of the lead over HistGen derived. The paper’s example may hint at why: four of the nine retrieved sentences it shows describe other patients’ immunostains, receptor status, a BRCA2 mutation or a right axillary node.
3. One setting moves the score as much as the lead. Retrieving for every patch instead of the top 40% gives 0.119, about HistGen’s level. Seven sentences per lookup instead of three gives 0.137, above the headline.

All seven baselines must be the authors’ own runs: the test set is new, the MI-Gen row matches none of the three in MI-Gen’s paper, and HistGen’s paper uses another dataset. The paper names UNI as the visual extractor without saying whether the baselines got it; a reviewer asked for that control, and there was no rebuttal round. The released code fixes one seed.

04What the scores can see

The paper prints one generated report beside its reference and highlights the words they share. Attribute by attribute, the pair reads differently:

Table 1 — The paper’s example, attribute by attribute. Our reading of the two texts in the paper’s Fig. 2; bold marks a disagreement.

Scored alone with the released scorer’s BLEU formula, this pair of about 100 tokens each gets a BLEU-4 of 0.127 (0.102 if full stops aren’t counted as tokens) derived, where the whole test set gets 0.135. All of its four-word matches come from the diagnosis phrase and from “there is no”.

05The same models, re-run by others

Two 2026 preprints from one group, which proposes its own model, SCOUT, each re-train BiGen, HistGen and MI-Gen under one protocol. Neither reports intervals.

On the breast subset, with other features and another split, BiGen’s BLEU-4 and its lead reproduce: 0.139 against 0.106 for HistGen. On PathText across cancer types, with about nine times the training data, BiGen and MI-Gen are 0.001 apart, and 0.0021 at most with any one encoder.

06What the paper leaves out

• A floor: no row copies the nearest training report or prints one fixed report, in this paper or in the benchmark’s, so what a template would score is unknown.
• Scoring code: the release computes BLEU, METEOR and ROUGE-L. Fact_ent, an entity-overlap score borrowed from radiology, and the HER2 columns aren’t in it; two requests have been open since December 2025.
• The bank’s recipe: the bank is shared as a file, without the scripts that build it and extract PLIP features (a third open request). That it holds training reports only rests on the paper’s word.
• Paper against release: the paper sets groups of 20 patches and the release defaults to 10; the paper motivates groups as neighbouring tissue, and the release groups patches by attention rank.
• Readers and other organs: no pathologist read a generated report, and only breast cancer is tested. A reviewer pointed at the single dataset and at findings absent from the bank.

07Takeaways for building a pathology report generator

1. With under a thousand reports, try a bottleneck first. One learned query over the patches gave +0.023 BLEU-4; everything added after it gave +0.009.
2. Ablate retrieval against the same model without it. Against a plain Transformer BiGen gains 0.032; against its own no-retrieval variant, 0.008. Keep each case’s own report out of the bank while training.
3. Score what a slide can show. Build references from slide-level findings, as the melanocytic paper did by cleaning its reports. Whole-case summaries reward a model for guessing size and stage.
4. Report intervals, and counts for small subsets. On 93 test cases a 0.017 lead is untested, and the HER2 gap to HistGen looks like five cases.

ASources

1. Zhang, L., Yun, B., Li, Q., Wang, Y. Historical Report Guided Bi-modal Concurrent Learning for Pathology Report Generation. MICCAI 2025, LNCS 15965, pp. 343–352. The page holds the reviews. PDF · doi · arXiv:2506.18658 (one version, same tables).
2. Code: DeepMed-Lab-ECNU/BiGen (MIT), read on 5 Oct 2026; open requests 2, 3 and 4.
3. Chen, P., et al. WsiCaption, the MI-Gen and PathText paper. MICCAI 2024; arXiv:2311.16480. Code and split file: cpystan/Wsi-Caption.
4. Guo, Z., et al. HistGen. MICCAI 2024; arXiv:2403.05396.
5. Singh, S., et al. SCOUT: arXiv:2605.01144 (2026). Singh, S., et al. PathReportEval: arXiv:2607.18448 (2026).
6. Chen, R. J., et al. UNI. Nature Medicine 30 (2024); repository note on its pretraining data. Huang, Z., et al. PLIP. Nature Medicine 29 (2023); repository.
7. Wolff, A. C., et al. HER2 testing in breast cancer: ASCO–CAP guideline update. J Clin Oncol (2023).
8. Miura, Y., et al. Fact_ent: arXiv:2010.10042 (NAACL 2021).
9. Our post on melanocytic lesion reports.