Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Paladyn, Journal of Behavioral Robotics

Editor-in-Chief: Schöner, Gregor

Covered by SCOPUS

CiteScore 2018: 2.17

SCImago Journal Rank (SJR) 2018: 0.336
Source Normalized Impact per Paper (SNIP) 2018: 1.707

ICV 2017: 99.90

Open Access
See all formats and pricing
More options …

Learning Grasp Affordance Densities

R. Detry / D. Kraft / O. Kroemer / L. Bodenhagen / J. Peters / N. Krüger / J. Piater
Published Online: 2011-06-16 | DOI: https://doi.org/10.2478/s13230-011-0012-x


We address the issue of learning and representing object grasp affordance models. We model grasp affordances with continuous probability density functions (grasp densities) which link object-relative grasp poses to their success probability. The underlying function representation is nonparametric and relies on kernel density estimation to provide a continuous model. Grasp densities are learned and refined from exploration, by letting a robot “play” with an object in a sequence of grasp-and-drop actions: the robot uses visual cues to generate a set of grasp hypotheses, which it then executes and records their outcomes. When a satisfactory amount of grasp data is available, an importance-sampling algorithm turns it into a grasp density. We evaluate our method in a largely autonomous learning experiment, run on three objects with distinct shapes. The experiment shows how learning increases success rates. It also measures the success rate of grasps chosen to maximize the probability of success, given reaching constraints.

Keywords: robot learning; grasping; probabilistic models; cognitive robotics


  • [1] A. Bicchi and V. Kumar. Robotic grasping and contact: a review. In IEEE International Conference on Robotics and Automation, 2000.Google Scholar

  • [2] G. E. P. Box and M. E. Muller. A note on the generation of random normal deviates. The Annals of Mathematical Statistics, 29(2):610-611, 1958.CrossrefGoogle Scholar

  • [3] M. T. Ciocarlie and P. K. Allen. Hand posture subspaces for dexterous robotic grasping. Int. J. Rob. Res., 28(7):851-867, 2009.Web of ScienceGoogle Scholar

  • [4] C. de Granville, J. Southerland, and A. H. Fagg. Learning grasp affordances through human demonstration. In IEEE International Conference on Development and Learning, 2006.Google Scholar

  • [5] R. Detry, E. Başeski, N. Krüger, M. Popović, Y. Touati, O. Kroemer, J. Peters, and J. Piater. Learning object-specific grasp affordance densities. In IEEE International Conference on Development and Learning, pages 1-7, 2009.Google Scholar

  • [6] R. Detry, E. Başeski, M. Popović, Y. Touati, N. Krüger, O. Kroemer, J. Peters, and J. Piater. Learning continuous grasp affordances by sensorimotor exploration. In O. Sigaud and J. Peters, editors, From Motor Learning to Interaction Learning in Robots, pages 451-465. Springer-Verlag, 2010.Google Scholar

  • [7] R. Detry, D. Kraft, A. G. Buch, N. Krüger, and J. Piater. Refining grasp affordance models by experience. In IEEE International Conference on Robotics and Automation, pages 2287-2293, 2010.Google Scholar

  • [8] R. Detry, N. Pugeault, and J. Piater. A probabilistic framework for 3D visual object representation. IEEE Trans. Pattern Anal. Mach. Intell., 31(10):1790-1803, 2009.Web of ScienceGoogle Scholar

  • [9] A. Doucet, N. de Freitas, and N. Gordon. Sequential Monte Carlo Methods in Practice. Springer, 2001.Google Scholar

  • [10] S. Ekvall and D. Kragic. Interactive grasp learning based on human demonstration. In IEEE International Conference on Robotics and Automation, 2004.Google Scholar

  • [11] R. A. Fisher. Dispersion on a sphere. In Proc. Roy. Soc. London Ser.A., 1953.Google Scholar

  • [12] J. J. Gibson. The Ecological Approach to Visual Perception. Lawrence Erlbaum Associates, 1979.Google Scholar

  • [13] K. Huebner, K. Welke, M. Przybylski, N. Vahrenkamp, T. Asfour, D. Kragic, and R. Dillmann. Grasping known objects with humanoid robots: A box-based approach. In International Conference on Advanced Robotics, 2009.Google Scholar

  • [14] J. A. Jrgensen, L.-P. Ellekilde, and H. G. Petersen. Robworksim —an open simulator for sensor based grasping. In Proceedings for the joint conference of ISR 2010 (41st International Symposium on Robotics) and ROBOTIK 2010 (6th German Conference on Robotics), 2010.Google Scholar

  • [15] D. Kraft, R. Detry, N. Pugeault, E. Ba3eski, F. Guerin, J. Piater, and N. Krüger. Development of object and grasping knowledge by robot exploration. IEEE Transactions on Autonomous Mental Development, 2(4):368-383, 2010.Google Scholar

  • [16] D. Kraft, N. Pugeault, E. Ba3eski, M. Popovi¢, D. Kragic, S. Kalkan, F. Wörgötter, and N. Krüger. Birth of the object: Detection of objectness and extraction of object shape through object action complexes. International Journal of Humanoid Robotics, 5:247-265, 2009.Web of ScienceGoogle Scholar

  • [17] D. Kragic, A. T. Miller, and P. K. Allen. Real-time tracking meets online grasp planning. In IEEE International Conference on Robotics and Automation, pages 2460–2465, 2001.Google Scholar

  • [18] O. Kroemer, R. Detry, J. Piater, and J. Peters. Combining active learning and reactive control for robot grasping. Robotics and Autonomous Systems, 58(9):1105-1116, 2010.Web of ScienceCrossrefGoogle Scholar

  • [19] J. J. Kuffner and S. M. Lavalle. RRT-Connect: An efficient approach to single-query path planning. In IEEE International Conference on Robotics and Automation, 2000.Google Scholar

  • [20] J. Lee and M. Verleysen. Nonlinear dimensionality reduction. Springer Verlag, 2007.Google Scholar

  • [21] A. T. Miller, S. Knoop, H. Christensen, and P. K. Allen. Automatic grasp planning using shape primitives. In IEEE International Conference on Robotics and Automation, volume 2, pages 1824-1829, 2003.Google Scholar

  • [22] L. Montesano and M. Lopes. Learning grasping affordances from local visual descriptors. In IEEE International Conference on Development and Learning, 2009.Google Scholar

  • [23] J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.Google Scholar

  • [24] M. Popović, D. Kraft, L. Bodenhagen, E. Başeski, N. Pugeault, D. Kragic, T. Asfour, and N. Krüger. A strategy for grasping unknown objects based on co-planarity and colour information. Robotics and Autonomous Systems, 2010.Web of ScienceGoogle Scholar

  • [25] N. Pugeault, F. Wörgötter, and N. Krüger. Visual primitives: Local, condensed, and semantically rich visual descriptors and their applications in robotics. International Journal of Humanoid Robotics, 2010. (to appear).Web of ScienceGoogle Scholar

  • [26] N. Pugeault, F. Wörgötter, and N. Krüger. Visual primitives: Local, condensed, and semantically rich visual descriptors and their applications in robotics. International Journal of Humanoid Robotics, 2010.Web of ScienceGoogle Scholar

  • [27] E. Sahin, M. Cakmak, M. R. Dogar, E. Ugur, and G. Ucoluk. To afford or not to afford: A new formalization of affordances towards affordance-based robot control. Adaptive Behavior, 2007.Web of ScienceGoogle Scholar

  • [28] M. Salganicoff, L. H. Ungar, and R. Bajcsy. Active learning for vision-based robot grasping. Mach. Learn., 23:251-278, May 1996.Google Scholar

  • [29] A. Saxena, J. Driemeyer, and A. Y. Ng. Robotic Grasping of Novel Objects using Vision. International Journal of Robotics Research, 27(2):157, 2008.CrossrefWeb of ScienceGoogle Scholar

  • [30] K. Shimoga. Robot grasp synthesis algorithms: A survey. The International Journal of Robotics Research, 15(3):230, 1996.CrossrefGoogle Scholar

  • [31] B. W. Silverman. Density Estimation for Statistics and Data Analysis. Chapman & Hall/CRC, 1986.Google Scholar

  • [32] E. B. Sudderth. Graphical models for visual object recognition and tracking. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA, USA, 2006.Google Scholar

  • [33] J. D. Sweeney and R. Grupen. A model of shared grasp affordances from demonstration. In International Conference on Humanoid Robots, 2007.Google Scholar

  • [34] A. T. A. Wood. Simulation of the von Mises-Fisher distribution. Communications in Statistics –Simulation and Computation, 23(1):157-164, 1994.CrossrefWeb of ScienceGoogle Scholar

About the article

Received: 2010-12-23

Accepted: 2011-03-28

Published Online: 2011-06-16

Published in Print: 2011-03-01

Citation Information: Paladyn, Journal of Behavioral Robotics, Volume 2, Issue 1, Pages 1–17, ISSN (Online) 2081-4836, DOI: https://doi.org/10.2478/s13230-011-0012-x.

Export Citation

© R. Detry et al.. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. BY-NC-ND 3.0

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

Marek S Kopicki, Dominik Belter, and Jeremy L Wyatt
The International Journal of Robotics Research, 2019, Volume 38, Number 10-11, Page 1246
Timo Lüddecke, Tomas Kulvicius, and Florentin Wörgötter
Robotics and Autonomous Systems, 2019, Volume 119, Page 92
Peter Kaiser and Tamim Asfour
IEEE Robotics and Automation Letters, 2018, Volume 3, Number 3, Page 1949
Wolf Schweitzer, Michael J. Thali, and David Egger
Journal of NeuroEngineering and Rehabilitation, 2018, Volume 15, Number 1
Barry Ridge, Aleš Leonardis, Aleš Ude, Miha Deniša, and Danijel Skočaj
International Journal of Advanced Robotic Systems, 2015, Volume 12, Number 3, Page 24
Philipp Zech, Simon Haller, Safoura Rezapour Lakani, Barry Ridge, Emre Ugur, and Justus Piater
Adaptive Behavior, 2017, Volume 25, Number 5, Page 235
Sergey Levine, Peter Pastor, Alex Krizhevsky, Julian Ibarz, and Deirdre Quillen
The International Journal of Robotics Research, 2017, Page 027836491771031
Gert Kootstra, Mila Popović, Jimmy Alison Jørgensen, Kamil Kuklinski, Konstantsin Miatliuk, Danica Kragic, and Norbert Krüger
The International Journal of Robotics Research, 2012, Volume 31, Number 10, Page 1190
Huaqing Min, Chang'an Yi, Ronghua Luo, Jinhui Zhu, and Sheng Bi
IEEE Transactions on Cognitive and Developmental Systems, 2016, Volume 8, Number 4, Page 237
D. Eizicovits, M. Yaacobovich, and S. Berman
IFAC Proceedings Volumes, 2012, Volume 45, Number 22, Page 253
Danny Eizicovits, Bart van Tuijl, Sigal Berman, and Yael Edan
Biosystems Engineering, 2016, Volume 146, Page 98
Seungsu Kim, Ashwini Shukla, and Aude Billard
IEEE Transactions on Robotics, 2014, Volume 30, Number 5, Page 1049
Leon Bodenhagen, Andreas R. Fugl, Andreas Jordt, Morten Willatzen, Knud A. Andersen, Martin M. Olsen, Reinhard Koch, Henrik G. Petersen, and Norbert Kruger
IEEE Transactions on Automation Science and Engineering, 2014, Volume 11, Number 3, Page 749
Michael Hofbaur, Andreas Müller, Justus Piater, Bernhard Rinner, Gerald Steinbauer, Markus Vincze, and Christian Wögerer
e & i Elektrotechnik und Informationstechnik, 2015, Volume 132, Number 4-5, Page 237
Jan Tünnermann, Norbert Krüger, Bärbel Mertsching, and Wail Mustafa
Cognitive Computation, 2015, Volume 7, Number 5, Page 526
Danny Eizicovits and Sigal Berman
Robotics and Autonomous Systems, 2014, Volume 62, Number 8, Page 1208
Rok Vuga, Eren Erdal Aksoy, Florentin Wörgötter, and Aleš Ude
Robotics and Autonomous Systems, 2015, Volume 65, Page 40
Emre Ugur, Yukie Nagai, Hande Celikkanat, and Erhan Oztop
Robotica, 2015, Volume 33, Number 05, Page 1163
Jimmy A. Rytz, Lars-Peter Ellekilde, Dirk Kraft, Henrik G. Petersen, and Norbert Krüger
Robotica, 2015, Volume 33, Number 05, Page 1131

Comments (0)

Please log in or register to comment.
Log in