Calinon, S. and Billard, A. (2005)
Recognition and Reproduction of Gestures using a Probabilistic Framework combining PCA, ICA and HMM
In Proc. of the Intl Conf. on Machine Learning (ICML), Bonn, Germany, pp. 105-112.


This paper explores the issue of recognizing, generalizing and reproducing arbitrary gestures. We aim at extracting a representation that encapsulates only the key aspects of the gesture and discards the variability intrinsic to each person's motion. We compare a decomposition into principal components (PCA) and independent components (ICA) as a first step of preprocessing in order to decorrelate and denoise the data, as well as to reduce the dimensionality of the dataset to make this one tractable. In a second stage of processing, we explore the use of a probabilistic encoding through continuous Hidden Markov Models (HMMs), as a way to encapsulate the sequential nature and intrinsic variability of the motions in stochastic finite state automata. Finally, the method is validated in a humanoid robot to reproduce a variety of gestures performed by a human demonstrator.

Bibtex reference

  author="S. Calinon and A. Billard",
  title="Recognition and Reproduction of Gestures using a Probabilistic Framework combining {PCA}, {ICA} and {HMM}",
  booktitle="Proceedings of the International Conference on Machine Learning ({ICML})",
  location="Bonn, Germany",


Observation of the user's gestures through stereoscopic vision and inertial motion sensors attached to the upper body of the user.

Learning and reproduction of a set of various gestures (waving goodbye, knocking on a door, dringking from a glass and writing of alphabet letters on a vertical board). For the last set of gestures, we see on the vidoe that the robot is able to generalize the motion by writing the alphabet letters on a different plane.

Source codes


  Download GMM latent space sourcecode


Unzip the file and run 'demo1' in Matlab.


Demo 1 - Demonstration of a probabilistic encoding through Gaussian Mixture Model (GMM) in a latent space of motion extracted by Principal Component Analysis (PCA)

This programs loads a dataset, finds a latent space of lower dimensionality encapsulating the important characteristics of the motion using Principal Component Analysis (PCA), trains a Gaussian Mixture Model (GMM) using the data projected in this latent space, and projects back the Gaussian distributions in the original data space. Training a GMM with EM algorithm usually fails to find a good local optimum when data are high-dimensional. By projecting the original dataset in a latent space as a pre-processing step, GMM training can be performed in a robust way, and the Gaussian parameters can be projected back to the original data space.

Go back to the list of publications