RATCHET: RAdiological Text Captioning for Human Examined Thoraxes

RATCHET is a Medical Transformer for Chest X-ray Diagnosis and Reporting. Based on the architecture featured in Attention Is All You Need. This network is trained and validated on the MIMIC-CXR v2.0.0 dataset.

Architecture

Run the code

Download pretrained weights (v1, v2) and put in ./checkpoints folder. Then run:

streamlit run web_demo.py

Environment:

Python 3.7.4

Packages:

imageio                  2.8.0
matplotlib               3.2.1
numpy                    1.18.4
pandas                   1.0.3
scikit-image             0.17.2
streamlit                0.67.1
tensorflow-gpu           2.3.0
tokenizers               0.7.0
tqdm                     4.46.0

Results

Generated Text:

In comparison with the study of ___, there is little overall change. Again there is substantial enlargement of the cardiac silhouette with a dual-channel pacer device in place. No evidence of vascular congestion or acute focal pneumonia. Blunting of the costophrenic angles is again seen.

More Examples