Mühlbauer, M., Hulin, T., Weber, B., Calinon, S., Stulp, F., Albu-Schäffer, A. and Silvério, J. (2024)
A Probabilistic Approach to Multi-Modal Adaptive Virtual Fixtures
IEEE Robotics and Automation Letters (RA-L).
Abstract
Virtual Fixtures (VFs) provide haptic feedback for teleoperation, typically requiring distinct input modalities for different phases of a task. This often results in vision- and position-based fixtures. Vision-based fixtures, particularly, require the handling of visual uncertainty, as well as target appearance/disappearance, creating the need for principled ways to add/remove fixtures, in addition to uncertainty-aware assistance regulation. Moreover, the arbitration of different modalities plays a crucial role in providing an optimal feedback to the user throughout the task. In this paper, we propose a Mixture of Experts (MoE) model that synthesizes visual-servoing fixtures, elegantly handling full pose detection uncertainties and teleoperation goals in a unified framework. An arbitration function combining multiple vision- based fixtures arises naturally from the MoE formulation, leveraging uncertainties to modulate fixture stiffness and thus the degree of assistance. The resulting visual-servoing fixtures are then fused with position-based fixtures using a Product of Experts (PoE) approach, achieving guidance throughout the workspace. We show that this approach allows a teleoperator to insert printed circuit boards (PCBs) with high precision without requiring the manual design of VFs towards individual connectors. An exemplary video showcasing our method is available at: https://youtu.be/QpDoCHmX28Q
Bibtex reference
@article{Muehlbauer24,
author={M\"uhlbauer, M. and Hulin, T. and Weber, B. and Calinon, S. and Stulp, F. and Albu-Sch\"affer, A. and Silv\'erio, J.},
title={A Probabilistic Approach to Multi-Modal Adaptive Virtual Fixtures},
journal={{IEEE} Robotics and Automation Letters ({RA-L})},
year={2024},
volume={},
pages={},
doi={}
}