Inverse Games: Inferring Motives from Interactions

Autonomous systems often need to interact with humans. Explaining and predicting human decisions in these interactions is critical for the safe and efficient operations of autonomous systems. Using game-theoretic approaches, we develop mathematical models and numerical solutions to infer the motives behind human decision-making. To this end, we first formulate a notion of boundedly rational equilibrium in multiplayer general-sum games to model human behavior. Second, we develop numerical optimization methods to infer the players’ objectives from observed strategies. These methods iteratively update the players’ objectives by implicitly differentiating the equilibrium conditions until the equilibrium strategies match the observed ones. They apply to both static matrix games, where each player chooses a probability distribution over a finite number of options, and dynamic Markov games, where each player chooses a policy in an infinite-horizon Markov decision process. Finally, we propose active inverse learning methods to provoke informative responses from noncooperative players and, as a result, improve the data efficiency when inferring their objectives. Our contributions enable efficient learning of human motives from limited interactions and pave the way toward harmonious human-autonomy interactions.