Hindsight bias

Statistics

Markov decision process

Human–computer interaction

Operating system

Machine learning

Artificial intelligence

Psychology

Markov process

Teleoperation

Markov model

Partially observable Markov decision process

Political science

Cognitive psychology

Control (management)

Mathematics

Computer science

Autonomy

User modeling

User interface

Robot

Process (computing)

Markov chain

The International Journal of Robotics Research

In shared autonomy, a user and autonomous system work together to achieve shared goals. To collaborate effectively, the autonomous system must know the user’s goal. As such, most prior works follow a predict-then-act model, first predicting the user’s goal with high confidence, then assisting given that goal. Unfortunately, confidently predicting the user’s goal may not be possible until they have nearly achieved it, causing predict-then-act methods to provide little assistance. However, the system can often provide useful assistance even when confidence for any single goal is low (e.g. move towards multiple goals). In this work, we formalize this insight by modeling shared autonomy as a partially observable Markov decision process (POMDP), providing assistance that minimizes the expected cost-to-go with an unknown goal. As solving this POMDP optimally is intractable, we use hindsight optimization to approximate. We apply our framework to both shared-control teleoperation and human–robot teaming. Compared with predict-then-act methods, our method achieves goals faster, requires less user input, decreases user idling time, and results in fewer user–robot collisions.

Shared autonomy via hindsight optimization for teleoperation and teaming