RRG-DPO, a cheap fix for missed findings, less for false alarms

01What the experiment measures

Report generators are trained to reproduce the reference report word by word, so a model can write fluent reports and still miss findings or invent them. The usual fixes are reinforcement learning on a clinical reward, or preference training on pairs that an LLM writes or grades.

Direct preference optimisation (DPO) needs neither a reward model nor sampling: give the model an image, a preferred report and a dispreferred one, and raise the probability of the first relative to the second, measured against a frozen copy of the starting model. The open question is where the dispreferred report comes from. RRG-DPO’s answer is the training set itself.

The experiment is clean in one respect: each of six existing generators is first trained as usual (official code; the released checkpoint for EKAGen; a re-implementation for 3D-CT-GPT), then only its text decoder is fine-tuned for three more epochs with the new objective, and the model is scored before and after. Four models write chest X-ray reports (MIMIC-CXR, 3,858 test samples) and two write chest CT reports (CT-RATE, 1,204 test volumes).

02The pair and the loss, as code

# our reconstruction of Section 2 and Eq. 2, not the authors' code
def pick_wrong_report(i, emb, labels, k=20):
    near = top_k(cosine(emb[i], emb), k)     # look-alikes
    dist = [jaccard_dist(labels[i], labels[j]) for j in near]
    return near[argmax(dist)]                # disagrees most

def rrg_dpo_loss(lp, ref, beta=0.3, lam=1.0, gamma=0.1):
    # lp[.]: log-likelihood under the model being trained
    # ref[.]: the same under the frozen starting model
    # w: true report, l: wrong report, l+: l minus conflicts
    margin = (lp["w"] - ref["w"]) - (lp["l"] - ref["l"])
    keep_w = relu(ref["w"] - lp["w"])        # DPOP term
    keep_l = relu(ref["l+"] - lp["l+"])      # the new term
    z = beta * margin - lam * keep_w - gamma * keep_l
    return -logsigmoid(z).mean()

Plain DPO is the margin alone. DPOP, an existing variant, adds keep_w: a penalty when the true report becomes less likely than it was. RRG-DPO adds keep_l: most sentences of the wrong report would be fine for this image, so pushing the whole report down also pushes down correct content. Eq. 2 as printed applies β to the margin only; DPOP’s own formula and the released code apply it to the penalties too.

03Which wrong report? The ablation

1. With plain DPO, random negatives make this model worse on all nine metrics. Precision falls from 0.456 to 0.413 and RadGraph from 0.201 to 0.142. This is one model, with no repeat reported; a later preprint (Random DPO, arXiv:2509.21351) reports gains from random reports on three other generators, so “random hurts” is this table’s result, not a law.
2. Most of the recall comes from the choice of pair, with plain DPO. A random look-alike takes recall to 0.425, the most conflicting look-alike to 0.459: that is 0.092 of the final 0.120 gain, about three quarters derived.
3. The loss terms are there for precision and wording. Precision stays below the released model in every row but the last, and so does ROUGE-L. DPOP (e) closes most of the precision gap, 0.441 to 0.455; the full objective brings both back to the starting level, and recall to 0.487. The new term’s own share, (e) to (f), is +0.020 recall, starred, and +0.007 precision, not starred.

One inconsistency our observation: the paper’s Tables 1 and 2 both test the released model against the full method. Table 1 prints EKAGen’s precision change, 0.456 to 0.462, without a significance star; Table 2 stars it, and the same goes for METEOR. One of the two is off. The change is small, 0.006, but by Table 2 precision would be significant in four models, not three.

04Six models, before and after

Averaged over the six models, recall gains +0.082 and precision +0.026 derived. The abstract promises fewer false-positive and fewer false-negative findings; the table mostly delivers the second. Precision improves clearly for R2Gen and CT2Rep, by about 6 and 7 points, and by about a point or less in the other four. It is a hit rate: a gain means a larger share of the written findings is right, not necessarily fewer false ones.

The abstract also says every model improves on six clinical metrics. That holds for all 36 cells; 25 of the 36 carry a star derived. CT2Rep, which starts at the lowest recall, gains the most (0.073 to 0.232); PromptMRG, which starts at the highest, gains 0.009.

05The labeller is both teacher and judge

CheXbert reads a report and outputs 14 chest X-ray labels. In this paper it does three jobs: it picks the wrong report, it marks the sentences to delete, and it computes the precision, recall and F1 of the results. A model trained to separate reports that CheXbert labels differently should improve most on CheXbert’s own score, and it does in five of the six models derived. For EKAGen, F1 rises 17% while RadGraph and RaTEScore, which the pipeline doesn’t use, each rise 3%. They do rise, and significantly in four of the six models each, so the gain is not only the labeller agreeing with itself. But the size of the clinical gain is better read from those columns than from the headline recall.

06What the paper leaves out

• Intervals, repeats and cost. Each row is a single result with stars: no repeated runs, no confidence intervals. The cost of the three training epochs isn’t given, only the 4.6 hours of pair building.
• Other generators. A reviewer asked for CXRMate and for the newer LLM-based MAIRA-2, LLaVA-Rad and CXR-LLaVA; the authors left them to future work.
• Sensitivity to the retriever and the labeller. The rebuttal mentions preliminary runs with another retriever, with the CheXpert labeller, and with the new loss on random negatives. None of these numbers is in the paper.
• A look at the images. A reviewer noted that a “false positive” can be a true finding the reference report omits; the authors agreed they had seen such cases. The paper shows no generated report, and we found none of the visual analysis its text mentions in the repository.

07The released code against the paper

The repository applies the method to one model, R2Gen. Its values differ from the paper’s our observation, read at commit bc7138a of 12 Jan 2026:

Table 1 — Settings in the paper and in the released example. Paper: Section 3. Repository: modules/dpo_loss.py, modules/trainer.py, modules/datasets.py, main_dpo.py, run_mimic_cxr_dpo.sh and the README. Steps per epoch assume all 270,790 training pairs at batch 16 derived.

Two differences matter. The new term is switched off in the example: its weight is 0, so what runs is a DPOP-style loss on the chosen pairs. And the tokens of the conflicting sentences count five times in each report’s average log-likelihood, a weighting the paper doesn’t describe. Whether the paper’s results were produced with these values is not stated; the example may simply be a different configuration: main.py tests by default a checkpoint from a run named …p1p2_newweight01_01, while the training script writes to …p1_newweight01.

08Takeaways for training a report generator

1. If you add preference training, spend the effort on the negatives. On EKAGen with plain DPO, random negatives lowered all nine metrics; chosen ones gave 0.092 of the 0.120 recall gain before any change to the loss.
2. Expect recall more than precision. Recall gained +0.082 on average, precision +0.026.
3. Expect little where the labeller’s F1 is already high. No significant change on PromptMRG, the one such model here.
4. Score with a tool you didn’t train with. The labeller’s own F1 rose 17% on EKAGen; RadGraph and RaTEScore rose 3%.

09What you can use today

• The code. github.com/ccarliu/RRG-DPO, licence AGPL-3.0: commercial use is allowed, but if you distribute a modified version or run one as a network service, you must offer its source under the same licence. Open modules/datasets.py (the pair selection) and modules/dpo_loss.py (the loss); run run_mimic_cxr_dpo.sh after training R2Gen with main.py (as committed it only tests: re-enable trainer.train()).
• What isn’t there. No trained RRG-DPO weights, no code for the other five models or for CT, and no script for the retrieval or the sentence labels, which the loader reads from files. The README links a “preference dataset” on Google Drive; we didn’t open it. You need your own credentialed copy of MIMIC-CXR and PromptMRG’s annotation file.
• The recipe, without the code. Embed your training reports; for each one take the 20 nearest; keep the neighbour whose finding labels differ most; train with DPO plus a term that keeps the true report likely.
• The rule. A wrong example teaches what it differs on: make it differ on findings, and score the result with a metric that didn’t choose the examples.

ASources

1. Liu, H., Wei, D., Xu, Z., Wu, X., Zheng, Y., Wang, L. RRG-DPO: Direct Preference Optimization for Clinically Accurate Radiology Report Generation. MICCAI 2025, LNCS 15964, pp. 552–562: paper page with reviews and rebuttal · PDF · DOI. We found no arXiv version.
2. The repository ccarliu/RRG-DPO, read on 13 Oct 2026 at commit bc7138a. No data file was opened.
3. Rafailov, R., et al. Direct Preference Optimization. NeurIPS 2023: arXiv:2305.18290. Pal, A., et al. DPO-Positive: arXiv:2402.13228.
4. Smit, A., et al. CheXbert. EMNLP 2020: arXiv:2004.09167.
5. Jin, H., et al. PromptMRG. AAAI 2024: arXiv:2308.12604 (Table 1).
6. Hamamci, I. E., et al. CT2Rep. MICCAI 2024: arXiv:2403.06801 (Tables 1 and 2).
7. Samokhin, V., et al. Random Direct Preference Optimization for Radiography Report Generation: arXiv:2509.21351 (2025).