Do question answering models understand the question?

The researchers conduct word-importance attribution analysis on deep learning models across three types of question answering tasks—images, tables, and text passages—and find that these networks often ignore important words in the question, and based on this construct various adversarial examples by perturbing the question. The strongest attack reduces the accuracy of a visual question answering model from 61.1% to 19%, and a table question answering model from 33.5% to 3.3%. The authors also show that attribution can strengthen the attack of Jia and Liang (2017) on passage comprehension models, and claim that attribution can complement standard accuracy metrics, exposing a model's flawed reasoning when it answers correctly but for the wrong reasons, indicating deficiencies in the test data.