Leon Reicherts, Zelun Tony Zhang, Elisabeth von Oswald, Yuanting Liu, Yvonne Rogers, Mariam Hassib
April 9, 2025
arXiv | PDF
The paper asks a fundamental design question: instead of having AI recommend decisions to users, what if it extended the user’s own reasoning instead? The authors built two AI assistants, ExtendAI and RecommendAI, and embedded them in a simulated ETF investment platform to compare how each affected users’ decision-making process, cognitive load, sense of agency, and outcome quality.
RecommendAI follows the standard paradigm: the user clicks a button and receives a list of recommended trades. ExtendAI flips the interaction model: the user first writes a rationale explaining their intended moves, and the AI embeds feedback directly into that rationale in bold. A within-subjects mixed-methods study found that both approaches have distinct strengths and trade-offs, with no single winner. The right design depends on user expertise, personality, and decision-making style.
A metaprompt is the fixed instruction template that shapes how the LLM behaves before any user input is provided. It defines the AI’s role, constraints, and output format. In this study, the authors iterated on separate metaprompts for each assistant through a pre-study: ExtendAI’s prompt told the model to embed feedback into the user’s rationale without making specific ETF recommendations; RecommendAI’s prompt told the model to generate a ranked list of suggested trades.
NASA-TLX (Task Load Index) is a standard questionnaire for measuring subjective cognitive load across six dimensions: mental demand, physical demand, temporal demand, performance, effort, and frustration. The study used a shortened version to compare the cognitive burden of the two AI interaction models.
Human-AI Complementarity is the goal of AI decision support: achieve outcomes that are better than either human or AI alone. The paper frames this as the theoretical ideal and uses it to evaluate whether each assistant design moves toward or away from true complementarity.
Appropriate vs. Inappropriate Reliance is a central problem in AI-assisted decision-making. Appropriate reliance means using AI when it is correct and overriding it when it is wrong. Inappropriate reliance (over-reliance) means accepting AI recommendations even when they are wrong, often because of automation bias (the human tendency to defer to automated systems). RecommendAI creates more risk of inappropriate reliance because it bypasses the user’s own reasoning process.