Evaluating the Effect of Explanations on Users’ Mental Models of Visual Question Answering Systems

View a PDF of the paper titled The Phantasm of Competence: Evaluating the Impact of Explanations on Customers’ Psychological Fashions of Visible Query Answering Programs, by Judith Sieker and 6 different authors

View PDF
HTML (experimental)

Summary:We look at how customers understand the restrictions of an AI system when it encounters a job that it can not carry out completely and whether or not offering explanations alongside its solutions aids customers in establishing an applicable psychological mannequin of the system’s capabilities and limitations. We make use of a visible query reply and rationalization job the place we management the AI system’s limitations by manipulating the visible inputs: throughout inference, the system both processes full-color or grayscale pictures. Our aim is to find out whether or not contributors can understand the restrictions of the system. We hypothesize that explanations will make restricted AI capabilities extra clear to customers. Nonetheless, our outcomes present that explanations wouldn’t have this impact. As an alternative of permitting customers to extra precisely assess the restrictions of the AI system, explanations usually enhance customers’ perceptions of the system’s competence – no matter its precise efficiency.