This publication is part of Elias PhD research and connects to research interest of the Hybrid Things Lab in regard of Physical Human Machine Interfaces, robotics, and multimodal interface design.
With the increasing use and complexity of robotic devices, the requirements for the design of human-robot interfaces are rapidly changing and call for new means of interaction and information transfer. On that scope, the discussed project – being developed by the Hybrid Things Lab at the University of Applied Sciences Augsburg and the Design Research Lab at Bauhaus-Universität Weimar – takes a first step in characterizing a novel field of research, exploring the design potentials of non-mimetic sonification in the context of human-robot interaction (HRI). Featuring an industrial 7-axis manipulator and collecting multiple Information (for instance, the position of the end-effector, joint positions and forces) during manipulation, these data sets are being used for creating a novel augmented audible presence, and thus allowing new forms of interaction. As such, this paper considers
(1) research parameters for non-mimetic sonification (such as pitch, volume and timbre);
(2) a comprehensive empirical pursuit, including setup, exploration, and validation;
(3) the overall implications of integrating these findings into a unifying human-robot interaction process.
The relation between machinic and auditory dimensionality is of particular concern.
We use the data, generated by an industrial manipulator during operation and use these to control software synthsizers. This approach is called "sonification" and provides alternative feedback at an audible level.
In the following video, we demonstrate the different sonification scenarios, structured in a coherent matrix, within our stacking task.
We came, among other results, to the conclusion, that sonification is a promising way of communicating robotic movement and states. Our type of sonification was on the one hand well structured and allowed us to draw basic conclusions, but on the other hand was very limited in regard of sound design.