### EPFL Student Project Proposals

The descriptions below are available for either semester projects or master projects (the content will be adjusted accordingly). Suggestions of other projects (or variants of existing projects) are also welcome, as long as they fit within the group's research interests.#### SMARTPHONE INTERFACE USING AUGMENTED REALITY

We have developed an augmented reality interface running on Android smartphones to display a virtual robot (left image). The app relies on ARCore, Google's toolkit for building augmented reality applications on Android and iOS devices. This toolkit is used to estimate the location of the phone and to render 3D graphics on top of the camera's image displayed on the screen.

The aim of the project is to extend this interface to let the user move the virtual robot to a desired configuration (see right images as an illustration). Several options can be considered, such as clicking on the robot articulations and dragging them to their desired positions, or drawing a stick figure on top of the image, which is then interpreted to set the desired pose of the robot.

The Android smartphone will be programmed in Java (knowledge of either Java or C++ is required for the project).

The developed interface will be tested to change the pose of a real 7-axis Panda robot (Franka Emika), by first setting and visualizing the motion of the virtual robot on the smartphone, and then running the motion on the real robot. A basic interface between the mobile phone and the robot is already available for the project (by using the ROS middleware). The proposed approach will finally be evaluated with inexperienced user to determine if it is accurate and easy-to-use.

**Keywords:** augmented reality, smartphone interfaces, robotics, inverse kinematics, machine learning

#### DICTIONARY LEARNING OF MOVEMENT PRIMITIVES

In robotics, it is useful to represent movements as a superposition of basis functions. Often, the set of basis functions is predefined (see above figure with three typical examples of basis functions). The learning problem thus consists of estimating how to combine the set of basis functions to form a desired movement.

In this project, it is proposed to learn the set of basis functions (dictionary of movement primitives) together with their combination. To do this, a sparse coding approach for dictionary learning will be investigated, as presented in the video and paper below. A dataset will first be collected by showing different manipulation skills to a robot through kinesthetic teaching (namely, by guiding the robot by the hand while the robot compensates for the effects of gravity). The approach in the references below will then be coded (in either Python, Matlab or C++), and evaluated on the collected robot motion data.

**Keywords:** robot learning, movement primitives, sparse coding

#### HIDDEN MARKOV MODEL FOR MUSIC SCORE SYNCHRONIZATION IN A HUMAN-ROBOT COLLABORATION

We are developing a human-robot collaboration experiment in which a person plays a melody on a piano while a robot plays the accompanying chords (see the robot setup in the above figure). For the synchronization to be effective, the robot has to anticipate its movements in accordance to the estimated tempo.

This project proposes to rely on the score (midi format) to initialize a hidden Markov model (HMM), which is then refined by the recordings of a person playing the same score. The goal is to encode the natural variations to improve the encoding and synthesis of the music track. This HMM will then be used in the experiment to estimate the time when the accompanying notes in the partition should be played by the robot. The approach will be implemented in Python or C++, by exploiting existing example codes in Matlab or C++ as starting point.

**Keywords:** time series analysis, hidden Markov model (HMM), music synthesis, human-robot collaboration

#### SUBSPACE LEARNING FOR ROBOT CONTROL APPLICATIONS

In robotics, movements can be represented as a dynamical system describing the evolution in time of the robot. Most often, the system is nonlinear, and the standard approach is to linearize the system so that locally, a linear system can be considered. Another approach, originally proposed by Koopman, is to augment the original set of variables composing the state space so that the nonlinear system can be expressed as a linear system in this augmented state space.

To do this, several approaches have been proposed, such as forming this augmented state space with polynomial or Fourier expansions of the original signal, or learning this augmented state space with autoencoders. Often, these approaches consider systems that estimate the next state based on the current state.

A promising approach recently proposed in the reference below is to consider a history of the previous states. This approach suggests to go beyond standard polynomial basis functions or Fourier basis functions by exploiting delay coordinates as basis functions, in the form of a factorization of a Hankel matrix. The resulting algorithm is surprisingly short and simple to implement. This project proposes to explore this approach in a robot control task with the 7-axis Panda robot (Franka Emika).

**Keywords:** dynamical systems, robot control, subspace learning, delay embedding, time series analysis

**References:**

#### LEARNING TO SEARCH: AN ERGODIC CONTROL APPROACH

In ergodic control, the aim is to find a sequence of control commands u(t) so that the retrieved trajectory x(t) covers a bounded space X in proportion of a given spatial distribution phi(x). The objective is formulated as a tracking problem in the frequency domain, in which the goal is to match the frequency patterns by putting more importance on low frequency components than high frequency components.

The resulting controller produces a natural movement to explore the regions of interest. Interestingly, this search pattern does not rely on random moves, but is instead well defined mathematically as a tracking problem in the frequency domain (i.e., matching Fourier series coefficients).

The aim of this project is to combine learning from demonstration with ergodic control, by extracting from the demonstration the regions that we need to explore, and then letting ergodic control explores these regions. The approach will be tested in an insertion task with a 7-axis Panda robot (Franka Emika). The implementation will be in C++ or Matlab (example codes for ergodic control in C++ and Matlab will be provided).

**Keywords:** robot control, learning from demonstration, Fourier series

**Reference:**

#### CONVOLUTIONAL NEURAL NETWORKS AND TENSOR METHODS

Sensory data in robotics are typically organized as multidimensional arrays (arrays of sensors, multiple channels, time evolution of data, multiple coordinate systems, etc.). This led our group to investigate tensor methods (a field of research also referred to as multilinear algebra).

Tensors are generalization of matrices to arrays of higher dimensions, where vectors and matrices correspond to 1st and 2nd-order tensors. When data are organized in matrices or arrays of higher dimensions (tensors), classical regression methods first transform these data into vectors, therefore ignoring the underlying structure of the data and increasing the dimensionality of the problem. This flattening operation typically leads to overfitting when only few training data are available.

In the reference below, we investigated the use of a mixture of experts relying on tensorial representations. The aim is to take into account the underlying structure of the data, with an approach that remains efficient even when only few training data are available.

The left figure illustrates how ridge regression and logistic regression can be extended to tensor-variate data. The right figure shows that a tensor-variate mixture of experts can be built by using tensor regression as experts, and tensor logistic regression as gating functions.

The goal of this project is to compare this strategy against the use of a convolutional neural network (CNN), from both theoretical and practical perspectives. For this project, previous experience on deep learning (and in particular CNN) would be useful. The implementation can be either done with TensorFlow, PyTorch or Matlab.

**Keywords:** convolutional neural network, tensor methods

**Reference:**

#### ONLINE IMITATION WITH OBJECT-AWARE MAPPING

Online imitation consists of copying a motion in real-time (see above image). This problem is very challenging in robotics because the robot and the user have different kinematics (and dynamics) characteristics, requiring the mapping strategy to be carefully chosen. Imitating joint angle trajectories is usually not appropriate, because the joints of the robot are not necessary organized in the same way as the morphology of a person. For this reason, the most often considered mapping is to copy the position of the robot/user hands, described in the Cartesian space.

In this project, it is proposed to explore a new strategy that consists of creating a mapping considering the objects in the robot/user workspace. This idea arises from the observation that most of the time, the robot hands will be used to manipulate objects, and the locations of these objects will typically differ in the two workspaces (the one of the user, and the one of the robot). In order to cope with this mapping problem, it is proposed to represent the movement from the perspective of these objects (e.g., in a relative distance space to the objects). Several strategies will be investigated and tested with a 7-axis Panda robot (Franka Emika). The implementation will be done either in Python, C++ or Matlab.

**Keywords:** imitation, learning from demonstration, motion retargeting, correspondence problems

**Reference:**

#### CIGAR BOX JUGGLING WITH ROBOTS

"Cigar box juggling" consists of manipulating three boxes, by holding two boxes while one box is free. It involves the juggler to skillfully move two of the boxes to change their configurations (see above sequence). For beginners, the movement is typically executed by moving the boxes up-and-down when changing the configuration, so that there is more time to change the orientation of the box being held (since the other hand will ``accompany'' the falling box). If the up-and-down motion has high amplitude, the movement to change the configuration is relatively slow.

This project proposes to explore whether such motion could be planned and executed on two 7-axis Panda robots (Franka Emika). The dynamical model will first be built to determine the motion to execute. Model predictive control will then be employed to control the robot with anticipation capability.

**Keywords:** robot control, robot planning, skills learning and adaptation, model predictive control