Learning Grasp Affordance Densities

R. Detry 1 , D. Kraft 2 , O. Kroemer 3 , L. Bodenhagen 2 , J. Peters 3 , 4 , N. Krüger 2  and J. Piater 5
  • 1 Centre for Autonomous Systems, Kungliga Tekniska högskolan (KTH), Stockholm, Sweden
  • 2 Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Denmark
  • 3 MPI for Biological Cybernetics, Tübingen, Germany
  • 4 Darmstadt University of Technology
  • 5 University of Innsbruck, Austria.

Abstract

We address the issue of learning and representing object grasp affordance models. We model grasp affordances with continuous probability density functions (grasp densities) which link object-relative grasp poses to their success probability. The underlying function representation is nonparametric and relies on kernel density estimation to provide a continuous model. Grasp densities are learned and refined from exploration, by letting a robot “play” with an object in a sequence of grasp-and-drop actions: the robot uses visual cues to generate a set of grasp hypotheses, which it then executes and records their outcomes. When a satisfactory amount of grasp data is available, an importance-sampling algorithm turns it into a grasp density. We evaluate our method in a largely autonomous learning experiment, run on three objects with distinct shapes. The experiment shows how learning increases success rates. It also measures the success rate of grasps chosen to maximize the probability of success, given reaching constraints.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • [1] A. Bicchi and V. Kumar. Robotic grasping and contact: a review. In IEEE International Conference on Robotics and Automation, 2000.

  • [2] G. E. P. Box and M. E. Muller. A note on the generation of random normal deviates. The Annals of Mathematical Statistics, 29(2):610-611, 1958.

  • [3] M. T. Ciocarlie and P. K. Allen. Hand posture subspaces for dexterous robotic grasping. Int. J. Rob. Res., 28(7):851-867, 2009.

  • [4] C. de Granville, J. Southerland, and A. H. Fagg. Learning grasp affordances through human demonstration. In IEEE International Conference on Development and Learning, 2006.

  • [5] R. Detry, E. Başeski, N. Krüger, M. Popović, Y. Touati, O. Kroemer, J. Peters, and J. Piater. Learning object-specific grasp affordance densities. In IEEE International Conference on Development and Learning, pages 1-7, 2009.

  • [6] R. Detry, E. Başeski, M. Popović, Y. Touati, N. Krüger, O. Kroemer, J. Peters, and J. Piater. Learning continuous grasp affordances by sensorimotor exploration. In O. Sigaud and J. Peters, editors, From Motor Learning to Interaction Learning in Robots, pages 451-465. Springer-Verlag, 2010.

  • [7] R. Detry, D. Kraft, A. G. Buch, N. Krüger, and J. Piater. Refining grasp affordance models by experience. In IEEE International Conference on Robotics and Automation, pages 2287-2293, 2010.

  • [8] R. Detry, N. Pugeault, and J. Piater. A probabilistic framework for 3D visual object representation. IEEE Trans. Pattern Anal. Mach. Intell., 31(10):1790-1803, 2009.

  • [9] A. Doucet, N. de Freitas, and N. Gordon. Sequential Monte Carlo Methods in Practice. Springer, 2001.

  • [10] S. Ekvall and D. Kragic. Interactive grasp learning based on human demonstration. In IEEE International Conference on Robotics and Automation, 2004.

  • [11] R. A. Fisher. Dispersion on a sphere. In Proc. Roy. Soc. London Ser.A., 1953.

  • [12] J. J. Gibson. The Ecological Approach to Visual Perception. Lawrence Erlbaum Associates, 1979.

  • [13] K. Huebner, K. Welke, M. Przybylski, N. Vahrenkamp, T. Asfour, D. Kragic, and R. Dillmann. Grasping known objects with humanoid robots: A box-based approach. In International Conference on Advanced Robotics, 2009.

  • [14] J. A. Jrgensen, L.-P. Ellekilde, and H. G. Petersen. Robworksim —an open simulator for sensor based grasping. In Proceedings for the joint conference of ISR 2010 (41st International Symposium on Robotics) and ROBOTIK 2010 (6th German Conference on Robotics), 2010.

  • [15] D. Kraft, R. Detry, N. Pugeault, E. Ba3eski, F. Guerin, J. Piater, and N. Krüger. Development of object and grasping knowledge by robot exploration. IEEE Transactions on Autonomous Mental Development, 2(4):368-383, 2010.

  • [16] D. Kraft, N. Pugeault, E. Ba3eski, M. Popovi¢, D. Kragic, S. Kalkan, F. Wörgötter, and N. Krüger. Birth of the object: Detection of objectness and extraction of object shape through object action complexes. International Journal of Humanoid Robotics, 5:247-265, 2009.

  • [17] D. Kragic, A. T. Miller, and P. K. Allen. Real-time tracking meets online grasp planning. In IEEE International Conference on Robotics and Automation, pages 2460–2465, 2001.

  • [18] O. Kroemer, R. Detry, J. Piater, and J. Peters. Combining active learning and reactive control for robot grasping. Robotics and Autonomous Systems, 58(9):1105-1116, 2010.

  • [19] J. J. Kuffner and S. M. Lavalle. RRT-Connect: An efficient approach to single-query path planning. In IEEE International Conference on Robotics and Automation, 2000.

  • [20] J. Lee and M. Verleysen. Nonlinear dimensionality reduction. Springer Verlag, 2007.

  • [21] A. T. Miller, S. Knoop, H. Christensen, and P. K. Allen. Automatic grasp planning using shape primitives. In IEEE International Conference on Robotics and Automation, volume 2, pages 1824-1829, 2003.

  • [22] L. Montesano and M. Lopes. Learning grasping affordances from local visual descriptors. In IEEE International Conference on Development and Learning, 2009.

  • [23] J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.

  • [24] M. Popović, D. Kraft, L. Bodenhagen, E. Başeski, N. Pugeault, D. Kragic, T. Asfour, and N. Krüger. A strategy for grasping unknown objects based on co-planarity and colour information. Robotics and Autonomous Systems, 2010.

  • [25] N. Pugeault, F. Wörgötter, and N. Krüger. Visual primitives: Local, condensed, and semantically rich visual descriptors and their applications in robotics. International Journal of Humanoid Robotics, 2010. (to appear).

  • [26] N. Pugeault, F. Wörgötter, and N. Krüger. Visual primitives: Local, condensed, and semantically rich visual descriptors and their applications in robotics. International Journal of Humanoid Robotics, 2010.

  • [27] E. Sahin, M. Cakmak, M. R. Dogar, E. Ugur, and G. Ucoluk. To afford or not to afford: A new formalization of affordances towards affordance-based robot control. Adaptive Behavior, 2007.

  • [28] M. Salganicoff, L. H. Ungar, and R. Bajcsy. Active learning for vision-based robot grasping. Mach. Learn., 23:251-278, May 1996.

  • [29] A. Saxena, J. Driemeyer, and A. Y. Ng. Robotic Grasping of Novel Objects using Vision. International Journal of Robotics Research, 27(2):157, 2008.

  • [30] K. Shimoga. Robot grasp synthesis algorithms: A survey. The International Journal of Robotics Research, 15(3):230, 1996.

  • [31] B. W. Silverman. Density Estimation for Statistics and Data Analysis. Chapman & Hall/CRC, 1986.

  • [32] E. B. Sudderth. Graphical models for visual object recognition and tracking. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA, USA, 2006.

  • [33] J. D. Sweeney and R. Grupen. A model of shared grasp affordances from demonstration. In International Conference on Humanoid Robots, 2007.

  • [34] A. T. A. Wood. Simulation of the von Mises-Fisher distribution. Communications in Statistics –Simulation and Computation, 23(1):157-164, 1994.

OPEN ACCESS

Journal + Issues

Paladyn. Journal of Behavioral Robotics is a fully peer-reviewed, open access journal that publishes original, high-quality research works and review articles on topics broadly related to neuronally and psychologically inspired robots and other behaving autonomous systems. The journal is indexed in SCOPUS.

Search