Mühlbauer, M., Hulin, T., Weber, B., Calinon, S., Stulp, F., Albu-Schäffer, A. and Silvério, J. (2024)
A Probabilistic Approach to Multi-Modal Adaptive Virtual Fixtures
IEEE Robotics and Automation Letters (RA-L), 9:6, 5298-5305.


Virtual Fixtures (VFs) provide haptic feedback for teleoperation, typically requiring distinct input modalities for different phases of a task. This often results in vision- and position-based fixtures. Vision-based fixtures, particularly, require the handling of visual uncertainty, as well as target appearance/disappearance, creating the need for principled ways to add/remove fixtures, in addition to uncertainty-aware assistance regulation. Moreover, the arbitration of different modalities plays a crucial role in providing an optimal feedback to the user throughout the task. In this paper, we propose a Mixture of Experts (MoE) model that synthesizes visual-servoing fixtures, elegantly handling full pose detection uncertainties and teleoperation goals in a unified framework. An arbitration function combining multiple vision- based fixtures arises naturally from the MoE formulation, leveraging uncertainties to modulate fixture stiffness and thus the degree of assistance. The resulting visual-servoing fixtures are then fused with position-based fixtures using a Product of Experts (PoE) approach, achieving guidance throughout the workspace. We show that this approach allows a teleoperator to insert printed circuit boards (PCBs) with high precision without requiring the manual design of VFs towards individual connectors. An exemplary video showcasing our method is available at: https://youtu.be/QpDoCHmX28Q

Bibtex reference

	author={M\"uhlbauer, M. and Hulin, T. and Weber, B. and Calinon, S. and Stulp, F. and Albu-Sch\"affer, A. and Silv\'erio, J.},
	title={A Probabilistic Approach to Multi-Modal Adaptive Virtual Fixtures},
	journal={{IEEE} Robotics and Automation Letters ({RA-L})},
Go back to the list of publications