The existence of manual mode increases human blame for AI mistakes.
Journal:
Cognition
Published Date:
Aug 28, 2024
Abstract
People are offloading many tasks to artificial intelligence (AI)-including driving, investing decisions, and medical choices-but it is human nature to want to maintain ultimate control. So even when using autonomous machines, people want a "manual mode", an option that shifts control back to themselves. Unfortunately, the mere existence of manual mode leads to more human blame when AI makes mistakes. When observers know that a human agent theoretically had the option to take control, the humans are assigned more responsibility, even when agents lack the time or ability to actually exert control, as with self-driving car crashes. Four experiments reveal that though people prefer having a manual mode, even if the AI mode is more efficient and adding the manual mode is more expensive (Study 1), the existence of a manual mode increases human blame (Studies 2a-3c). We examine two mediators for this effect: increased perceptions of causation and counterfactual cognition (Study 4). The results suggest that the human thirst for illusory control comes with real costs. Implications of AI decision-making are discussed.