Interaction with a robot endowed with a Kinect and pico-projector

In this work we present a novel active interface with perception and projection capabilities for simplifying the skill transfer process. During the learning process, the real workspace is used as a tangible interface for helping the user to better understand what the robot has learned up to then, to display information about the task or to get feedback and guidance. Thus, the user is able to incrementally visualize and assess the learner's state and, at the same time, focus on the skill transfer without disrupting the continuity of the teaching interaction. We show here a proof-of-concept based on an experimental setting where a pico-projector and a Kinect RGBD camera are mounted onto the end-effector of a 7-DOF robotic arm.

While a fixed camera/projector system has a static field of view, the proposed robotic setup can project/detect at various places and under various angles. This allows the system to actively handle occlusions, where the user does not need to care during the interaction of being in the field of view. Such configuration also offers adaptive multiresolution tracking and projection features. For example, the detection of users in the surroundings requires a different field of view than the detection of objects close to the robot. For example, if one wants to project on a large surface (e.g. to have an overview of the objects involved in an assembly task), the robot can move back to increase its field of view. Then, if a precise information on the positioning of object is needed (e.g. detecting the alignment of screws and threads), the robot can move closer to the area of interest.

Video credit: Davide De Tommaso

Go back to the list of videos