Towards vision-based manipulation of plastic materials - Archive ouverte HAL Access content directly
Conference Papers Year : 2018

Towards vision-based manipulation of plastic materials


This paper represents a step towards vision-based manipulation of plastic materials. Manipulating deformable objects is made challenging by: 1) the absence of a model for the object deformation, 2) the inherent difficulty of visual tracking of deformable objects, 3) the difficulty in defining a visual error and 4) the difficulty in generating control inputs to minimise the visual error. We propose a novel representation of the task of manipulating deformable objects. In this preliminary case study, the shaping of kinetic sand, we assume a finite set of actions: pushing, tapping and incising. We consider that these action types affect only a subset of the state, i.e., their effect does not affect the entire state of the system (specialized actions). We report the results of a user study to validate these hypotheses and release the recorded dataset. The actions (pushing, tapping and incising) are clearly adopted during the task, although it is clear that 1) participants use also mixed actions and 2) actions' effects can marginally affect the entire state, requesting a relaxation of our specialized actions hypothesis. Moreover, we compute task errors and corresponding control inputs (in the image space) using image processing. Finally, we show how machine learning can be applied to infer the mapping from error to action on the data extracted from the user study.
Fichier principal
Vignette du fichier
2018-sandman.pdf (2.73 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-01731230 , version 1 (14-03-2018)



Andrea Cherubini, Jürgen Leitner, Valerio Ortenzi, Peter I Corke. Towards vision-based manipulation of plastic materials. IROS: Intelligent Robots and Systems, Oct 2018, Madrid, Spain. pp.485-490, ⟨10.1109/IROS.2018.8594108⟩. ⟨hal-01731230⟩
413 View
849 Download



Gmail Mastodon Facebook X LinkedIn More