Development of a portraitist software for the ITRI robot
This video shows the result of a project commissioned by ITRI, where it was proposed to develop a portraitist robot software that recognizes human faces in the robot's surroundings and extracts relevant characteristics from the captured images. The robot then draws the portraits of the participants from the captured images by first converting those into vector art and by controlling the robot's arm to sequentially reproduce the desired paths by using a pen and a canvas.
The resulting application is used to showcase several capabilities of the ITRI's robot manipulator, such as fast planning computation, fast, smooth and accurate movement, the simultaneous consideration of position and orientation constraints, and the size of the workspace. It also emphasizes the integration of different motor and sensory components such as vision processing, clustering, human-robot interaction, inverse kinematics and redundant control.
The developed application uses state-of-the-art image processing techniques. The behavior of the robot resembles that of an artist, who would, first, analyze the face of the model, and then, draw the portrait on a canvas.
Contours of the faces are first extracted from the images captured through the robot's internal cameras. The contours are then converted to paths by organizing them with respect to their lengths. Then, the image is segmented into several shades of gray. The contours and extracted areas are simplified to keep only the important features of the face and make it look similar to a cartoon representation. The different shades of gray are reproduced by the robot by adding several layers of simple geometric patterns. The different areas are filled similarly as a human would do by iteratively starting from one point, filling the local area, and moving to the next area once finished.
See also Salvador DaBot the portraitist robot.
Video credit: Yuan-Yung Huang, ITRI. Link to ITRI website.