Simulating Interaction Movements with Optimal Feedback Control and Deep Reinforcement Learning

The increasing use of VR and AR applications gives rise to the development of new interaction techniques. Inconveniently, the indispensable evaluation and proper fine-tuning of such techniques requires extensive user studies, which are costly and time-consuming. In particular, when interactions are performed using the whole body, factors such as ergonomics and fatigue become critical. However, these are particularly difficult to measure in vivo, and are not predictable with most previous approaches that focused on quantitative models such as Fitts’ Law or simple end-effector models such as Minimum Jerk.

However, recent advances in forward biomechanical models, optimal control and deep reinforcement learning enable the simulation of real user behaviour, incorporating both muscle control and perception. By predicting the whole movement of users, ergonomic factors of interaction techniques can be evaluated in vitro, even for a large number of different settings. This facilitates the optimisation of parameters to enhance interaction.

To this end, we formulate the interaction of humans with computers as an optimal control problem. We then explore different optimal feedback control methods in their ability to predict user behaviour, in particular, LQR, LQG, and MPC. In contrast to these classic approaches, we also evaluate and discuss deep reinforcement learning agents trained on different interaction tasks, and outline how these simulations could be applied to the optimisation of interaction technique parameters.

The seminar is also available for online attendance via Zoom: https://ucl.zoom.us/j/91801473411.