IROS 2023

Our research paper titled “Orbital Head-Mounted Display: A Novel Interface for Viewpoint Control during Robot Teleoperation in Cluttered Environments” by Sjoerd Kuitert, Jelle Hofland, Cock Heemskerk, David A. Abbink, and Luka Peternel was accepted for publication in the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023) conference paper. A 5-minute presentation on Orbital Head-Mounted Display and the findings of the research is available on YouTube and IROS On-Demand.

Challenge

Robotic teleoperation is used in various applications, including the nuclear industry, where the experience and intelligence of a human operator are necessary for complex decision-making. Human-robot interfaces that help strengthen an operator’s situational awareness without inducing excessive cognitive load, are crucial to the success of teleoperation.

During his master thesis at HIT, Sjoerd Kuitert worked on the development of remote control and remote perception systems for haptic remote handling simulations. It turned out that for some of the envisioned remote tasks it was challenging to provide adequate visual feedback to the remote operator using traditional feedback methods.

A potential solution found in literature is the introduction of a secondary “monitoring” platform to track the task robot, the process, and the remote workspace. For example, a secondary robotic manipulator, a drone, or even a virtual viewpoint. However, the design of a human-machine interface that allows for natural and efficient control of such a camera platform and the task robot was still an open research challenge.

Orbital Head-Mounted Display

To tackle this challenge, Sjoerd Kuitert developed a novel interface for viewpoint control called Orbital Head-Mounted Display (OHMD). The OHMD interface is based on the so-called third-person free follow camera. A third-person camera is a common perspective used in video games and other media where the player or viewer observes the action from a viewpoint that is detached from the main character or subject. Due to a fixed view distance, the camera follows the subject as it moves.

Commonly, third-person camera allow for some level of flexibility in terms of viewpoint control. Players can typically adjust the camera angle (tilt), zoom in or out, and pan the view to get a better look at the environment or specific objects. Instead of using a conventional interface like a keyboard, gamepad or mouse, the OHMD viewpoint control interface uses motion tracking sensors of a Head Mounted Display (HMD) to adjust the camera angle and pan the view. The view distance or zoom is mapped to a hand-held joystick.