Grasp density

Autonomous Learning of Object Grasp Models

PACO-PLUS Part of the EU project PACO-PLUS.

← Back to research project list.

In this project, we study means of modeling and learning object grasp affordances, i.e., relative object-gripper poses that lead to stable grasps. Affordances are represented probabilistically with grasp densities [detry2011a], which correspond to continuous density functions defined on the space of 6D gripper poses – 3D position and orientation.

2D projection of a grasp density

Projection of a 6DOF grasp density on a 2D image. Grasp success likelihood is proportional to the intensity of the green mask.

Grasp densities are linked to visual stimuli through registration with a visual model of the object they characterize, which allows the robot to grasp objects lying in arbitrary poses: to grasp an object, the object's model is visually aligned to the correct pose; the aligned grasp density is then combined to reaching constraints to select the maximum-likelihood achievable grasp. Grasp densities are learned and refined through exploration: grasps sampled randomly from a density are performed, and an importance-sampling algorithm learns a refined density from the outcomes of these experiences. Initial grasp densities are computed from the visual model of the object.

We demonstrated that grasp densities can be learned autonomously from experience. Our experiment showed that through learning, the robot becomes increasingly efficient at inferring grasp parameters from visual evidence. The experiment also yielded conclusive results in practical scenarios where the robot needs to repeatedly grasp an object lying in an arbitrary pose, where each pose imposes a specific reaching constraint, and thus forces the robot to make use of the entire grasp density to select the most promising achievable grasp. This work led to publications in the fields of robotics [detry2010a, detry2010b, detry2011a] and developmental learning [detry2009c].

Video illustrating the grasp learning process. Download this video in MP4/H.264 or Ogg/Theora, or view it on Youtube.

PACO-PLUS Part of the EU project PACO-PLUS.

FNRS Supported by the Belgian National Fund for Scientific Research (FNRS).

Nuklei Code based on the Nuklei library.

Main reference:

detry2011a 
R. Detry, D. Kraft, O. Kroemer, L. Bodenhagen, J. Peters, N. Krüger and J. Piater, Learning Grasp Affordance Densities. In Paladyn. Journal of Behavioral Robotics, 2 (1): 1–17, 2011.
doidoi; pdfpdf; bibtexshow/hide bibtex

Papers covering this topic:

bodenhagen2011a 
L. Bodenhagen, R. Detry, J. Piater and N. Krüger, What a successful grasp tells about the success chances of grasps in its vicinity. In ICDL-EpiRob, 2011.
pdfpdf; bibtexshow/hide bibtex
detry2009b 
R. Detry, E. Başeski, N. Krüger, M. Popović, Y. Touati and J. Piater, Autonomous Learning of Object-specific Grasp Affordance Densities. In Approaches to Sensorimotor Learning on Humanoid Robots (Workshop at the IEEE International Conference on Robotics and Automation), 2009.
pdfpdf; bibtexshow/hide bibtex
detry2009c 
R. Detry, E. Başeski, N. Krüger, M. Popović, Y. Touati, O. Kroemer, J. Peters and J. Piater, Learning Object-specific Grasp Affordance Densities. In IEEE International Conference on Development and Learning, pages 1–7, 2009.
doidoi; pdfpdf; bibtexshow/hide bibtex
detry2010a 
R. Detry, E. Başeski, M. Popović, Y. Touati, N. Krüger, O. Kroemer, J. Peters and J. Piater, Learning Continuous Grasp Affordances by Sensorimotor Exploration. In From Motor Learning to Interaction Learning in Robots, pages 451–465, Springer-Verlag, 2010.
doidoi; pdfpdf; bibtexshow/hide bibtex
detry2010b 
R. Detry, D. Kraft, A. G. Buch, N. Krüger and J. Piater, Refining Grasp Affordance Models by Experience. In IEEE International Conference on Robotics and Automation, pages 2287–2293, 2010.
doidoi; pdfpdf; bibtexshow/hide bibtex
detry2010d 
R. Detry, Learning of Multi-Dimensional, Multi-Modal Features for Robotic Grasping. Ph.D. Thesis, University of Liège, 2010.
pdfpdf; bibtexshow/hide bibtex
detry2011a 
R. Detry, D. Kraft, O. Kroemer, L. Bodenhagen, J. Peters, N. Krüger and J. Piater, Learning Grasp Affordance Densities. In Paladyn. Journal of Behavioral Robotics, 2 (1): 1–17, 2011.
doidoi; pdfpdf; bibtexshow/hide bibtex
kraft2009b 
D. Kraft, R. Detry, N. Pugeault, E. Başeski, J. Piater and N. Krüger, Learning Objects and Grasp Affordances through Autonomous Exploration. In International Conference on Computer Vision Systems, pages 235–244, 2009.
doidoi; pdfpdf; bibtexshow/hide bibtex
kraft2010a 
D. Kraft, R. Detry, N. Pugeault, E. Başeski, F. Guerin, J. Piater and N. Krüger, Development of Object and Grasping Knowledge by Robot Exploration. In IEEE Transactions on Autonomous Mental Development, 2 (4): 368–383, 2010.
doidoi; pdfpdf; bibtexshow/hide bibtex
piater2009a 
J. Piater, S. Jodogne, R. Detry, D. Kraft, N. Krüger, O. Kroemer and J. Peters, Learning Visual Representations for Interactive Systems. In International Symposium on Robotics Research, 2009.
doidoi; pdfpdf; bibtexshow/hide bibtex

Many of these publications are copyrighted by their respective publishers. Downloadable versions are not necessarily identical to the published versions. They are made available here for personal use only.


Page last modified: November 07, 2017 Valid HTML5 and CSS