Cobots for my craft

2024

AR gesture control system for human-robot collaboration

Team:

Olya Toltinova
Moritz Schmidt
Mathias Hradecsni
Stefanie Rausch

Andreas Muxel
Claudia Meitinger
Jens Müller

This research project aims at providing a natural communication way between a human and a robot in polishing processes. Controls can be performed using simple gestures instead of a programming interface, with the working environment extended by contextual AR.

Users can point out the working areas for the robot with their hands and see contextualized process information on the relevant process parts.

A controller can be used as an alternative to gestures for more precise waypoint orientation guidance.

Aside from the main functionality, the application has several features developed for convenient process control and safer working conditions in crafts.

If the operator can not be present at the robot cell or to avoid harmful production environments, the robot can be operated remotely with the same functionality available.

Additionally, the sanding process's trajectory can be inspected before launch to verify quality.

The control visualizations and communication with the robot are implemented with the Magic Leap2 AR glasses, providing a seamless blend between digital and real workspace and user mobility.

The project was conceptualized and developed at Augsburg Technical University of Applied Sciences as part of the KI-Produktionsnetzwerk Augsburg within the research group Human-machine interaction with intelligent systems. It is being iteratively extended, with the concept first presented at the 70. GfA-Frühjahrskongress 2024.

It was first conceptualized through a series of team hackathons and proof-of-concept videos, with the functional prototypes later evaluated via user testing involving local craftsmen.

More information can be found on the Technical University of Applied Science Augsburg project page.

Photos: Timo Holzmann

Publications:

Toltinova, O., Schmidt, M., Hradecsni, M., Meitinger, C., & Muxel, A. (2024). XR-Schnittstelle statt Robot Teach Pendant - Durchgängig von der Aktivierung des Roboters bis zur Delegation von komplexen Aufgaben am Anwendungsfall eines Schleifprozesses. Frühjahrskongress der Gesellschaft für Arbeitswissenschaft e.V. https://doi.org/10.60524/opus-1887

events:

Cobots for my Craft @ 70. GfA-Frühjahrskongress 202406.03-08.03.24