Understanding Data Visualizations via Question Answering
Kushal Kafle, Brian Price, Scott Cohen, Christopher Kanan · CVPR 2018

Data visualizations, e.g., bar charts, pie charts, and plots, contain large amounts of information in a concise format. These visualizations are designed to communicate data to people, not to be machine interpretable. Nevertheless, making algorithms capable of making inferences from data visualizations has enormous practical applications. Here, we study systems capable of answering open-ended questions about bar charts, which we refer to as data visualization question answering (DVQA). DVQA would enable vast repositories of charts within scientific documents, web pages, and business reports to be queried automatically.
Unlike visual question answering (VQA), DVQA requires processing words and answers that are unique to a particular bar chart. State-of-the-art VQA algorithms perform poorly on DVQA, and we propose two strong baselines that perform considerably better. Besides practical benefits, DVQA also serves as an important proxy task for several critical AI abilities, such as attention, working memory, visual reasoning, and the ability to handle dynamic and out-of-vocabulary (OOV) labels.

If you use DVQA in your work, please cite:
@inproceedings{kafle2018dvqa,
title={DVQA: Understanding Data Visualizations via Question Answering},
author={Kafle, Kushal and Price, Brian and Cohen, Scott and Kanan, Christopher},
booktitle={CVPR},
year={2018}
}
Follow-up: Answering Questions about Data Visualizations using Efficient Bimodal Fusion (WACV 2020, code).
Download links and the README are in the DVQA GitHub repository.
Questions or comments about the paper or the dataset? Contact Kushal Kafle (kushalkafle [at] gmail [dot] com).
Authors: Kushal Kafle, Brian Price, Scott Cohen, and Christopher Kanan. This work was done during an internship at Adobe Research.