Diff-RRG, read for report generation

* Diff-RRG as retrained by that paper’s authors.

01Why this paper matters for report generation

Radiologists read a follow-up chest X-ray against the previous one, and their reports say so: unchanged, improved, new. Diff-RRG’s idea is direct: compare the two images finding by finding, classify each finding as worsening, stable or improving, and tell the language model.

This post also reads the released code, the open reviews, and a 2026 follow-up, STRIVE, in which three of the four authors score Diff-RRG again.

Two things are easy to miss. Only the current image’s tokens enter the prompt, in the paper’s Eq. 6 as in the code. And nothing lines the two images up: in the paper’s own Figure 1, and in one of the two examples of its Figure 2, the prior image is a lateral view and the current one frontal.

02The data and the scores

Longitudinal-MIMIC keeps the MIMIC-CXR patients with at least two studies that have a findings section: 26,625 of the 60,547 who have one, so 56% are left out derived. Each sample pairs a study with the one before it, and the model writes the later one’s findings. There is no path for a first visit.

The paper gives the total, 94,169 pairs, and no split sizes. The dataset paper lists 92,374 for training, 737 for validation and 2,058 for testing, the last from 266 patients. The paper reports no interval, repeat run or significance test.

The scores are BLEU-1 to 4, METEOR, ROUGE-L, and micro-averaged precision, recall and F1 over CheXbert labels. Which labels, and how “uncertain” counts, is not stated, and the release has no CheXbert scoring.

How far does carrying the prior forward go? In the dataset paper, when a label was the same in both reports the model’s report agreed 88.96% of the time; when it had changed, the report was wrong 84.42% of the time. None of the five papers read for this post scores the prior report handed back unchanged.

03The fourteen words, as released

The classifier’s targets come from “ground truth disease annotations”, the paper says. The repository’s README is specific: CheXbert labels both reports, and a finding that appears is “worsening”, one that disappears “improving”, one whose state is unchanged “stable”. An effusion that grows between two reports that both mention it is therefore “stable”. The clinical score is computed with the same labeler. (The release disagrees with itself on the sign: the README maps 1 to worsening, the model’s prompt dictionary prints 1 as “improving”.)

The classifier’s input is the difference map. Condensed from the released model file:

# one image, 14 findings
# sim: 14 × 196, a softmax over patches, so each row sums to 1
sim = softmax(scale * names @ patches.T, dim=-1)
noise = Gumbel(0, 1).sample(sim.shape)       # at test time too
keep = softmax((sim + noise) / 0.3, dim=-1) > 0.2
F = stack([patches[k].mean(0) for k in keep]) + pos

D = F_current - F_prior                      # 14 × 768; pos cancels
state = classifier(D).argmax(-1)             # one of 3, per finding

Three properties follow derived:

1. At most four patches per finding. Five shares above 0.2 would exceed 1. Our simulation of the rule keeps 1.3 on average.
2. The choice is mostly noise. The standard Gumbel-softmax adds noise to log-probabilities; the release adds it to probabilities that sum to 1 over 196 patches. Whatever the similarity says, a patch’s chance of being the top pick stays between 0.5% and 1.4%. The noise is redrawn at test time.
3. The classifier isn’t told which finding it is judging. The positional term, which the rebuttal says carries the finding’s identity, cancels in the subtraction, in the paper’s Eqs. 3 and 4 as in the code, where one small network serves all 14 findings.

So in the release, each of the fourteen words typically rests on one or two near-random patches per image. We did not run the model, and the code behind the paper’s numbers may differ; the selection function has not changed since the first commit, in February 2025.

04Results: where the gain comes from

The ablation is the paper’s cleanest evidence: five rows that build on one single-image baseline.

1. The prior study does most of the work. Adding the prior image and report is worth +0.064 F1 and 0.009 BLEU-4. The row adds both at once; in the dataset paper’s ablation of its own model, the report alone added 0.076 F1 and the image alone 0.022.
2. The two modules add +0.021 F1 and 0.007 BLEU-4. Alone, the difference map adds 0.016 F1 and the classifier 0.006. The paper doesn’t say what either single-module row feeds the LLM; the release holds only the full model.
3. Nothing says these gaps exceed noise. For scale, HERGen’s bootstrap on its MIMIC-CXR test set of 2,210 pairs puts a BLEU-4 gain of 0.011 between +0.005 and +0.016 (95% interval). A reviewer called the modules’ gains limited; the rebuttal answered that such gains are considered meaningful in this field.

05Whose runs are in the table?

The comparison table marks one row, HERGen, as cited from its paper. HERGen built its own longitudinal subset (lateral images and duplicates removed) and macro-averaged its clinical scores over 14 classes; here they sit in a micro-averaged column. The arithmetic shows it: a micro F1 is the harmonic mean of precision and recall, which holds for every row except HERGen’s, printed as 0.295 where 0.343 would follow derived.

The other rows carry no mark and match neither the baselines’ own papers nor the three earlier tables on this benchmark that we checked: evidently the authors’ runs, which the paper doesn’t describe. HC-LLM is the baseline it measures itself against, in its Figure 2 and in the rebuttal:

The leads over the re-run are 0.001 to 0.005; the deficits against HC-LLM’s published numbers go up to 0.011. BLEU-4 is 0.120 for Diff-RRG, 0.116 for the re-run and 0.128 in HC-LLM’s paper, whose two other variants report 0.127 and 0.142. The clinical columns can’t be compared across papers: HC-LLM’s doesn’t state its averaging, and the dataset paper’s model, “Prefilling” in the table (0.480 F1 there, 0.423 here), was scored with the older CheXpert labeler.

But the ranking then rests on how the baselines were run, and that can move a row a long way: R2GenGPT’s BLEU-4 is 0.087 in this table and 0.108 in STRIVE’s, for the same benchmark.

06A later check, by three of the authors

The paper has no metric for the comparison itself. In August 2026, three of its four authors posted STRIVE on arXiv, with a temporal metric, LCC: an LLM extracts each change statement from a report (a finding and one of five labels), and a rule-based scorer checks whether the generated report states the same change as the reference. They retrained Diff-RRG, like every baseline with public code, and scored it on the same 2,058 pairs.

On the labels that describe a change, Diff-RRG reaches 0.097 and 0.066 and recovers none of the “new” or “resolved” statements. Overall, 71% of the reference statements have no counterpart in its reports. That is still the lowest omission rate of the ten systems retrained, and its coarse score of 0.187 (95% CI 0.171 to 0.203) sits above HC-LLM’s 0.155 (0.140 to 0.172).

Two cautions: LCC is the group’s own new metric, in a preprint whose new system wins by a wide margin; and “stable” is the common label, on 2,219 of 3,275 reference statements.

07What the paper leaves out

• The trivial row: the prior report handed back unchanged, and scores split by whether a label changed.
• The classifier: a reviewer asked for its performance. The rebuttal gave a test F1 of 0.84 and promised to add it; the final paper has neither that number nor the share of “stable” labels.
• The patches: one example pair, and no score for where the selected patches fall.

08Takeaways for building a longitudinal report generator

1. Give the LLM the prior report first, then measure what the image adds. Both together were worth +0.064 F1; the two modules on top, +0.021.
2. Print the copy-the-prior row, and score changed and unchanged labels apart. The dataset paper’s model was wrong on 84.42% of changed labels.
3. If patch selection uses Gumbel noise, add it to logits, switch it off at test time, and compare with random patches.
4. Train and test a progression head on progression. MS-CXR-T holds 1,326 radiologist-checked labels of improving, stable and worsening.
5. Mark every row as copied or re-run, and say how. One baseline’s BLEU-4 differs by 0.021 between two tables, three times what the two modules add to BLEU-4.

ASources

1. Yun, H., Maeng, J., Kang, E., Suk, H.-I. Diff-RRG: Longitudinal Disease-wise Patch Difference as Guidance for LLM-based Radiology Report Generation. MICCAI 2025, LNCS 15966, pp. 152–161 (Korea University, Kangwon National University). The page holds the reviews and the rebuttal. PDF · doi. No arXiv version was found.
2. Code: ku-milab/Diff-RRG (BSD-3-Clause), read on 5 Oct 2026 at commit c25d90fd.
3. Maeng, J., Kang, E., Suk, H.-I. STRIVE: arXiv:2608.24237 v1 (25 Aug 2026).
4. Zhu, Q., et al. Longitudinal-MIMIC (“Prefilling”), MICCAI 2023: arXiv:2306.08749.
5. Liu, T., et al. HC-LLM, AAAI 2025: arXiv:2412.11070 v1. Wang, F., et al. HERGen, ECCV 2024: arXiv:2407.15158.
6. Model cards: BiomedCLIP, BioMistral-7B. Dataset card: MS-CXR-T.
7. CheXbert: arXiv:2004.09167. Gumbel-softmax: arXiv:1611.01144.