Med-VQA
In this repository we have tested 3 VQA models on the ImageCLEF-2019 dataset. Two of these are made on top of Facebook AI Reasearch's Multi-Modal Framework (MMF).
| Model Name | Accuracy | Number of Epochs |
|---|---|---|
| Hierarchical Question-Image Co-attention | 48.32% | 42 |
| MMF Transformer | 51.76% | 30 |
| MMBT | 86.78% | 30 |
Test them for yourself !
Download the dataset from here and place it in a directory named /dataset/med-vqa-data/ in the directory where this repository is cloned.
MMF Transformer:
mmf_run config=projects/hateful_memes/configs/mmf_transformer/defaults.yaml model=mmf_transformer dataset=hateful_memes training.checkpoint_interval=100 training.max_updates=3000
MMBT :
mmf_run config=projects/hateful_memes/configs/mmbt/defaults.yaml model=mmbt dataset=hateful_memes training.checkpoint_interval=100 training.max_updates=3000
Heirarchical Question-Image Co-attention:
cd hierarchical \
python main.py
2.7k Dec 27, 2022
3 Aug 24, 2022
1 Oct 30, 2021
14 Jan 03, 2023
170 Dec 13, 2022
70 Dec 29, 2022
1 Oct 29, 2021
0 Nov 12, 2022
901 Jan 06, 2023
6.1k Dec 29, 2022
463 Dec 30, 2022
21 Dec 11, 2022
20 Nov 18, 2022
5 Dec 08, 2021
1 Dec 22, 2021
39 Dec 14, 2022
26 Jan 03, 2023
452 Dec 30, 2022
9 Nov 25, 2022