Med-VQA
In this repository we have tested 3 VQA models on the ImageCLEF-2019 dataset. Two of these are made on top of Facebook AI Reasearch's Multi-Modal Framework (MMF).
| Model Name | Accuracy | Number of Epochs |
|---|---|---|
| Hierarchical Question-Image Co-attention | 48.32% | 42 |
| MMF Transformer | 51.76% | 30 |
| MMBT | 86.78% | 30 |
Test them for yourself !
Download the dataset from here and place it in a directory named /dataset/med-vqa-data/ in the directory where this repository is cloned.
MMF Transformer:
mmf_run config=projects/hateful_memes/configs/mmf_transformer/defaults.yaml model=mmf_transformer dataset=hateful_memes training.checkpoint_interval=100 training.max_updates=3000
MMBT :
mmf_run config=projects/hateful_memes/configs/mmbt/defaults.yaml model=mmbt dataset=hateful_memes training.checkpoint_interval=100 training.max_updates=3000
Heirarchical Question-Image Co-attention:
cd hierarchical \
python main.py
84 Dec 15, 2022
284 Nov 25, 2022
61 Oct 26, 2022
43 Jan 05, 2023
99 Jan 06, 2023
345 Jan 03, 2023
109 Jan 04, 2023
156 Dec 21, 2022
1 Dec 17, 2021
3k Dec 26, 2022
1 Feb 17, 2022
11 Oct 31, 2022
30 Sep 07, 2022
3.2k Dec 30, 2022
14 Dec 07, 2022
1.5k Jan 02, 2023
460 Oct 13, 2022
27 Dec 12, 2022
0 Aug 13, 2022
40 Dec 11, 2022