Most past and present research in computer vision involves passively observed data. Humans, however, are active observers outside the lab; they explore, search, select what and how to look.
PESAO is designed for investigating active, visual observation in a 3D world. The goal was to build an experimental setup for various active perception tasks with human subjects (active observers) in mind that is capable of tracking the head and gaze.
While many studies explore human performances, usually, they use line drawings portrayed in 2D, and no active observer is involved. PESAO allows us to bring many studies to the three-dimensional world, even involving active observers. In our instantiation, it spans an area of 400cm x 300cm and can track active observers at a frequency of 120Hz.
Furthermore, PESAO provides tracking and recording of 6D head motion, gaze, eye movement-type, first-person video, head-mounted IMU sensor, birds-eye video, and experimenter notes. All are synchronized at microsecond resolution.
Example gaze recording.
A sketch of PESAO.
PESAO Tech-Report (arxiv.org)
A detailed, technical report of PESAO. Describing functionalities, hardware and software (PESAOlib).
PESAOlib (JTL GitLab)
The GitLab repository of PESAOlib. The software behind PESAO.
Two example recordings of PESAO. Head pose trajectory with fixations.
In case you use this work in one of your publications, please make sure to cite us:
Markus D. Solbach and John K. Tsotsos.
"PESAO: Psychophysical Experimental Setup for Active Observers"
arXiv preprint arXiv:2009.09933 (2020).
A heatmap visualization of fixations using PESAO.
A subject performing an experiment with PESAO.