Abstract

Virtual Fixtures (VFs) provide haptic feedback for teleoperation, typically requiring distinct input modalities for different phases of a task. This often results in vision- and position-based fixtures. Vision-based fixtures, particularly, require the handling of visual uncertainty, as well as target appearance/disappearance, creating the need for principled ways to add/remove fixtures, in addition to uncertainty-aware assistance regulation. Moreover, the arbitration of different modalities plays a crucial role in providing an optimal feedback to the user throughout the task. In this paper, we propose a Mixture of Experts (MoE) model that synthesizes visual-servoing fixtures, elegantly handling full pose detection uncertainties and teleoperation goals in a unified framework. An arbitration function combining multiple vision- based fixtures arises naturally from the MoE formulation, leveraging uncertainties to modulate fixture stiffness and thus the degree of assistance. The resulting visual-servoing fixtures are then fused with position-based fixtures using a Product of Experts (PoE) approach, achieving guidance throughout the workspace. We show that this approach allows a teleoperator to insert printed circuit boards (PCBs) with high precision without requiring the manual design of VFs towards individual connectors. An exemplary video showcasing our method is available at: https://youtu.be/QpDoCHmX28Q

Bibtex reference

@article{Muehlbauer24RAL,
	author={M\"uhlbauer, M. and Hulin, T. and Weber, B. and Calinon, S. and Stulp, F. and Albu-Sch\"affer, A. and Silv\'erio, J.},
	title={A Probabilistic Approach to Multi-Modal Adaptive Virtual Fixtures},
	journal={{IEEE} Robotics and Automation Letters ({RA-L})},
	year={2024},
	volume={9},
	number={6},
	pages={5298--5305},
	doi={10.1109/LRA.2024.3384759}
}

Go back to the list of publications