Abstract

In a Robot Programming by Demonstration framework, several demonstrations of a task are required to generalize and reproduce the task under different circumstances. To teach a task to the robot, explicit pointers are required to signal the start/end of a demonstration and to switch between the learning/reproduction phases. Coordination of the learning system can be achieved by adding social cues to the interaction process. Here, we propose to use an imitation game to teach a humanoid robot to recognize communicative gestures, which then serve as social signals in a pointing-at-objects scenario. The system is based on Hidden Markov Models (HMMs) and use motion sensors to track the user's gestures.

Bibtex reference

@InProceedings{Calinon06roman,
  author = "S. Calinon and A. Billard",
  title = "Teaching a Humanoid Robot to Recognize and Reproduce Social Cues",
  booktitle = "Proceedings of the {IEEE} International Symposium on Robot and Human Interactive Communication ({RO-MAN})",
  year = "2006",
  month="September",
  location="Hatfield, UK",
  pages="346--351"
}

Video

First, the user plays an imitation game with the robot, where the user reproduces commmunicative gestures performed by the robot (attention scaffolding, turn taking, head nods and head shakes). The robot then creates a Hidden Markov Model (HMM) characterizing the specificities of the user's gestures. The user then directs the attention of the robot to an object on the table by looking and pointing at it. The robot points at the object of interest and ask the user for an evaluation of its selection.

In this video, the user points at the 3 objects one by one. The robot still points at the most probable object and ask for an evaluation. The user can then clarify its selection and focus the attention of the robot again to the object that he intended to show to the robot.

Source codes

Download

  Download Cone-plane intersection sourcecode

Usage

Unzip the file and run 'demo1' in Matlab.

References

• Calinon, S. and Billard, A. (2006) Teaching a Humanoid Robot to Recognize and Reproduce Social Cues. IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 346-351.

Demo 1 - Demonstration of a cone-plane intersection interpreted in terms of Gaussian Probability Density Function (PDF)

This program computes the intersection between a cone and a plane, represented as a Gaussian Probability Density Function (PDF). The algorithm can be used to extract probabilistically information concerning gazing or pointing direction. Indeed, by representing a visual field as a cone and representing a table as a plane, the Gaussian distribution can be used to compute the probability that one object on the table is observed/pointed by the user.

Go back to the list of publications