Dia-LLaMA, read for report generation

* From the released code and test outputs; the last assumes a per-scan average (section 02).

01Why this paper matters for report generation

An LLM can write a fluent report around a finding it missed. Dia-LLaMA decides first and writes second: a classifier reads the scan, and its verdict goes into the prompt as plain sentences. The same group’s PromptMRG did this for chest X-rays; this paper moves it to chest CT and LLaMA-2. Code, a checkpoint, test outputs and reviews are public; this post uses all four.

• The attention doesn’t look at the scan. The weights of Eq. 5 are a learned table, one row of 1,024 per label, the same for every patient. It can favour a region, not follow a lesion our observation.
• The memory bank is 28 vectors in the released code. With two prototypes per label, the loss of Eq. 6 is that of a linear classifier with no bias our derivation.
• 14 labels, not eight. The paper puts eight diseases in the prompt; every public version of the code writes and scores all 14 CheXbert labels, “no finding” included.
• The training prompt is the answer key. In training the sentences are filled from the reference report’s own CheXbert labels, as in PromptMRG’s code; at test time, by the classifier. The paper doesn’t say so. Condensed from the released dataset and model code:

def diagnosis_prompt(present):      # one 0/1 per CheXbert label
    return "".join(
        f'The "{name}" is {"positive" if on else "negative"}. '
        for name, on in zip(LABELS, present))

# training: the reference report's own labels
prompt = diagnosis_prompt(chexbert(reference) == PRESENT)
# test: the scan's features against two prototypes per label
prompt = diagnosis_prompt(sim_present > sim_absent)

text = IMAGE_TOKENS + prompt + INSTRUCTION   # then the report

02The data and the ruler

CTRG-Chest-548K has 1,804 chest CT scans with reports, 548,696 JPEG slices in all, so the model sees display intensities, not Hounsfield units. The reports were written in Chinese and come with an English version, according to a later paper. This one doesn’t say. It trains on 80% and tests on 20%, drawn at random per its arXiv version; the public split has 362 test scans.

The clinical score is CheXbert, a chest X-ray labeler, run on both reports. The label file in the release shows what it finds here: “lung lesion” in 57.2% of reports, “enlarged cardiomediastinum” in 35.0%, pneumonia and pneumothorax never, and no present label in 14.1%. The paper asserts that the labeler stays valid on CT reports. A reviewer noted this wasn’t validated; the reply was that it worked “reasonably well”, with a CT labeler left to future work.

03The results as printed

Dia-LLaMA leads the strongest of six baselines, RadFM, by 0.018 precision, 0.026 recall and 0.027 F1 and by 4.9 BLEU-4 points. On ROUGE-L it is fourth of seven. RadFM’s row is also the ablation’s first: the same encoder, perceiver and LoRA-tuned LLaMA-2, without the prompt. So the ablation is the test of the idea.

1. The prompt alone adds 0.002 F1, and with a plain classifier behind it recall falls from 0.361 to 0.336.
2. The two rulers disagree about the modules. By the table, attention helps (0.358) and prototypes alone hurt (0.339, below the baseline). By the chart’s mean, prototypes help (0.256) and attention hardly does (0.213). The four rarest of the chart’s eight labels have 13 to 25 positive scans each in the released test set.

No interval or second run is reported.

04Inside the released test outputs

The paper scores the report, never the classifier. The repository lets us: results/dia-llama.csv holds, for 362 test scans, the prompt the model wrote, its report, the reference and its labels. Re-scored the release’s way, the reports reach BLEU-4 30.0; an earlier version of the file, with the same prompts and different reports, gives 29.5. The paper’s 29.64 lies between, so this is evidently the paper’s system, though perhaps not the run behind its tables.

The prompts assert 1,999 labels, the references hold 578, and 370 coincide: a precision of 0.19 and a recall of 0.64. Pneumothorax is asserted for 148 scans, although no report in the dataset carries the label. Scored as if it were the report, the prompt would get a per-scan F1 of 0.269, below the no-prompt baseline.

Most of it doesn’t reach the report, by a word count. Seven labels have an unambiguous word: effusion, edema, fracture, consolidation, pneumothorax, pneumonia and atelectasis. They carry 1,242 of the 1,999 assertions; the report states the finding, with no negation before the word, in 27 or 28 of those cases, depending on the rule. The word pneumothorax is in no report.

So here the score evidently belongs to the language model, whose reports are more cautious than their prompts. What produces the 0.027 is open: perhaps the auxiliary loss, which also trains the encoder in the released code. No row of the paper trains the modules and leaves the prompt out.

05Fixed answers and moving baselines

The paper notes that CT reports follow a rigid template. One of the test set’s own reference reports, as the answer for every test scan (its own too) and scored the release’s way, gets BLEU-4 23.6 at the median of the 362 choices derived. The six baselines score 21.9 to 24.7; their tokenisation isn’t stated.

What rises above the template isn’t checked by these scores. 82 generated reports cite a slice number in the form IM141; where the reference cites one too (17 scans), none matches. In the paper’s one example, the model’s report shrinks the reference’s 15 × 10 mm lobulated nodule to 6 × 4 mm in another segment, which a “lung lesion” label can’t see.

Who ran the baselines? Only SL-DG’s row is marked as copied. It comes from the dataset paper, on another split: the arXiv caption said so, the MICCAI caption doesn’t. The others are evidently the authors’ runs, the four chest X-ray models on 30 slices per scan, per the arXiv version. And in a later paper from the group (Reg2RG), RadFM, described the same way on this dataset, scores BLEU-4 30.89, above Dia-LLaMA’s 29.64 here, with a METEOR of 49 against 24: the scorer evidently changed too.

06What the paper leaves out

• The classifier: no accuracy for the verdict the method rests on.
• Readers and spread: no radiologist read a report, and there is no repeat run or interval. Decoding isn’t described; the released file’s versions suggest it is sampled.
• The data: one dataset, as a reviewer objected; no patient count, and nothing on a patient-level split, the reports’ language or the JPEG source.
• The recipe: 2,000 steps in the arXiv version, 20 epochs in the final text, 25.4 passes over 1,262 scans in the released training log; nothing on what is frozen.

07Takeaways for building a CT report generator

1. Score the classifier before crediting the prompt. The released verdicts have a precision of 0.19; the paper reports the report’s F1 only.
2. Train the writer on prompts it will meet. In the release, training prompts come from the reference labels (about 1.6 present per scan), test prompts from a classifier (5.5). Mix in predicted labels.
3. Put fixed answers in the table. One label scores 0.452 per-scan F1 and one typical report BLEU-4 23.6.
4. Use labels your reports contain. Two of the 14 never occur in 1,804 reports, and 14.1% of reports have none.
5. Freeze the protocol before comparing. RadFM scores BLEU-4 24.70 and 30.89 in two papers from one group, and a second decode moves Dia-LLaMA’s BLEU-4 by 0.5.

ASources

1. Chen, Z., Luo, L., Bie, Y., Chen, H. Dia-LLaMA: Towards Large Language Model-driven CT Report Generation. MICCAI 2025, LNCS 15966, pp. 141–151. The page holds the three reviews and the authors’ reply. PDF · doi · arXiv:2403.16386 v1 (March 2024: the same three tables; a different account of the split, the baselines and the training length).
2. Code: zhi-xuan-chen/Dia-LLaMA at commit 6c278fea, read on 6 Oct 2026, with its test outputs and label file. Checkpoint and training log: Trusure/Dia-LLaMA.
3. Dataset: CTRG; Tang, Y., et al. Work like a doctor. Expert Systems with Applications 237, 121442 (2024), the source of the SL-DG row.
4. Jin, H., et al. PromptMRG: arXiv:2308.12604 (AAAI 2024) and its scorer.
5. Chen, Z., et al. Reg2RG: arXiv:2411.15539 (IEEE TMI 2025), Table I and Section IV.
6. Li, D., Liang, J., Li, W. ALTER: arXiv:2608.05615 (2026), Appendix A: the dataset’s language and split sizes.
7. Smit, A., et al. CheXbert: arXiv:2004.09167 (EMNLP 2020). Wu, C., et al. RadFM: arXiv:2308.02463.
8. Our posts on the 3D MLLM design space and µ²LLM.