Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Foundations of Computing and Decision Sciences

The Journal of Poznan University of Technology

4 Issues per year


CiteScore 2016: 0.75

SCImago Journal Rank (SJR) 2016: 0.330
Source Normalized Impact per Paper (SNIP) 2016: 0.709

Open Access
Online
ISSN
2300-3405
See all formats and pricing
More options …

Modreg: A Modular Framework for RGB-D Image Acquisition and 3D Object Model Registration

Tomasz Kornuta
  • IBM Research, Almaden, 650 Harry Rd, San Jose, CA 95120
  • Warsaw University of Technology, Institute of Control and Computation Engineering
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Maciej Stefańczyk
Published Online: 2017-09-09 | DOI: https://doi.org/10.1515/fcds-2017-0009

Abstract

RGB-D sensors became a standard in robotic applications requiring object recognition, such as object grasping and manipulation. A typical object recognition system relies on matching of features extracted from RGB-D images retrieved from the robot sensors with the features of the object models. In this paper we present ModReg: a system for registration of 3D models of objects. The system consists of a modular software associated with a multi-camera setup supplemented with an additional pattern projector, used for the registration of high-resolution RGB-D images. The objects are placed on a fiducial board with two dot patterns enabling extraction of masks of the placed objects and estimation of their initial poses. The acquired dense point clouds constituting subsequent object views undergo pairwise registration and at the end are optimized with a graph-based technique derived from SLAM. The combination of all those elements resulted in a system able to generate consistent 3D models of objects.

Keywords: ModReg; textured stereo; RGB-D image; point cloud; registration

References

  • [1] Aldoma A., Tombari F., Di Stefano L., and Vincze M. A global hypotheses verification method for 3D object recognition. In Computer Vision (ECCV 2012), pages 511–524. Springer, 2012.Google Scholar

  • [2] Aldoma A., Tombari F., Prankl J., Richtsfeld A., Di Stefano L., and Vincze M. Multimodal cue integration through hypotheses verification for RGB-D object recognition and 6DOF pose estimation. In Robotics and Automation (ICRA), 2013 IEEE International Conference on, pages 2104–2111. IEEE, 2013.Google Scholar

  • [3] Belter D., Nowicki M., Skrzypczyński P., Walas K., and Wietrzykowski J. Lightweight RGB-D SLAM System for Search and Rescue Robots. In Progress in Automation, Robotics and Measuring Techniques, pages 11–21. Springer, 2015.Google Scholar

  • [4] Besl P. and McKay N. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2):239 –256, 1992.Google Scholar

  • [5] Bradski G. and Kaehler A. Learning OpenCV: Computer vision with the OpenCV library. O’Reilly Media, 2008.Google Scholar

  • [6] Correll N., Bekris K. E., Berenson D., Brock O., Causo A., Hauser K., Okada K., Rodriguez A., Romano J. M., and Wurman P. R. Analysis and observations from the first amazon picking challenge. IEEE Transactions on Automation Science and Engineering, 2016.Google Scholar

  • [7] Dziergwa M., Kaczmarek P., and Kędzierski J. RGB-D Sensors in Social Robotics. Journal of Automation Mobile Robotics and Intelligent Systems, 9(1):18–27, 2015.Google Scholar

  • [8] Figat J., Kornuta T., and Kasprzak W. Performance evaluation of binary descriptors of local features. In Chmielewski L., Kozera R., Shin B.-S., and Wojciechowski K., editors, Proceedings of the International Conference on Computer Vision and Graphics, volume 8671 of Lecture Notes in Computer Science, pages 187–194. Springer Berlin / Heidelberg, 2014.Google Scholar

  • [9] Firman M. Rgbd datasets: Past, present and future. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 19–31, 2016.Google Scholar

  • [10] Großmann B., Siam M., and Krüger V. Comparative evaluation of 3D pose estimation of industrial objects in RGB pointclouds. In Computer Vision Systems, pages 329–342. Springer, 2015.Google Scholar

  • [11] Hirschmuller H. Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell., pages 328–341, II 2008.Google Scholar

  • [12] Holz D., Ichim A. E., Tombari F., Rusu R. B., and Behnke S. Registration with the Point Cloud Library: A Modular Framework for Aligning in 3-D. Robotics & Automation Magazine, IEEE, 22(4):110–124, 2015.Google Scholar

  • [13] Konolige K. Projected texture stereo. In International Conference on Robotics and Automation (ICRA), pages 148–155. IEEE, 2010.Google Scholar

  • [14] Kornuta T. and Laszkowski M. Perception subsystem for object recognition and pose estimation in RGB-D images. In Szewczyk R., Zieliński C., and Kaliczyńska M., editors, Recent Advances in Automation, Robotics and Measuring Techniques, volume 440 of Advances in Intelligent Systems and Computing (AISC), pages 597–607. Springer, 2016.Google Scholar

  • [15] Kornuta T. and Stefańczyk M. Acquisition of RGB-D images: sensors (in Polish). Pomiary – Automatyka – Robotyka PAR, 18(2):92–99, 2014.Google Scholar

  • [16] Kornuta T. and Stefańczyk M. Comparison of methods of aquisition of RGB-D images for the purpose of registration of three-dimensional models of objects (in Polish). In XIV Krajowa Konferencja Robotyki – Postepy robotyki, volume 2, pages 357–366, 2016.Google Scholar

  • [17] Kornuta T. and Stefańczyk M. Utilization of textured stereovision for registration of 3D models of objects. In 21th IEEE International Conference on Methods and Models in Automation and Robotics, MMAR’2016, pages 1088–10093. IEEE, 2016.Google Scholar

  • [18] Lai K., Bo L., Ren X., and Fox D. A large-scale hierarchical multi-view RGB-D object dataset. In Robotics and Automation (ICRA), 2011 IEEE International Conference on, pages 1817–1824. IEEE, 2011.Google Scholar

  • [19] Lenz I., Knepper R., and Saxena A. DeepMPC: Learning deep latent features for model predictive control. In Proceedings of Robotics: Science and Systems, Rome, Italy, July 2015.Google Scholar

  • [20] Lowe D. Object recognition from local scale-invariant features. In Computer Vision, The Proceedings of the Seventh IEEE International Conference on, volume 2, pages 1150–1157. Ieee, 1999.Google Scholar

  • [21] Lu F. and Milios E. Globally consistent range scan alignment for environment mapping. Autonomous Robots, 4(4):333–349, 1997.Google Scholar

  • [22] Łępicka M., Kornuta T., and Stefańczyk M. Utilization of colour in ICP-based point cloud registration. In Proceedings of the 9th International Conference on Computer Recognition Systems (CORES 2015), volume 403 of Advances in Intelligent Systems and Computing, pages 821–830. Springer, 2016.Google Scholar

  • [23] Muja M. and Lowe D. G. Fast approximate nearest neighbors with automatic algorithm configuration. In VISAPP (1), pages 331–340, 2009.Google Scholar

  • [24] Newcombe A. J., Richard A and Davison, Izadi S., Kohli P., Hilliges O., Shotton J., Molyneaux D., Hodges S., Kim D., and Fitzgibbon A. KinectFusion: Real-time dense surface mapping and tracking. In Mixed and augmented reality (ISMAR), 2011 10th IEEE international symposium on, pages 127–136. IEEE, 2011.Google Scholar

  • [25] Peng X., Sun B., Ali K., and Saenko K. Learning deep object detectors from 3d models. In Proceedings of the IEEE International Conference on Computer Vision, pages 1278–1286, 2015.Google Scholar

  • [26] Pomerleau F., Colas F., and Siegwart R. A review of point cloud registration algorithms for mobile robotics. Foundations and Trends in Robotics (FnTROB), 4(1):1–104, 2015.Google Scholar

  • [27] Ramey A., González-Pacheco V., and Salichs M. A. Integration of a low-cost RGB-D sensor in a social robot for gesture recognition. In Proceedings of the 6th international conference on Human-robot interaction, pages 229–230. ACM, 2011.Google Scholar

  • [28] Ren X., Fox D., and Konolige K. Change Their Perception: RGB-D for 3-D Modeling and Recognition. Robotics & Automation Magazine, IEEE, 20(4):49–59, 2013.Google Scholar

  • [29] Rusu R., Blodow N., and Beetz M. Fast point feature histograms (FPFH) for 3D registration. In Robotics and Automation, 2009. ICRA’09. IEEE International Conference on, pages 3212–3217. IEEE, 2009.Google Scholar

  • [30] Rusu R. B. and Cousins S. 3D is here: Point Cloud Library (PCL). In International Conference on Robotics and Automation, Shanghai, China, 2011 2011.Google Scholar

  • [31] Seredyński D. and Szynkiewicz W. Fast grasp learning for novel objects. In Recent Advances in Automation, Robotics and Measuring Techniques, volume 440 of Advances in Intelligent Systems and Computing (AISC), pages 681–692. Springer, 2016.Google Scholar

  • [32] Sprickerhof J., Nüchter A., Lingemann K., and Hertzberg J. A heuristic loop closing technique for large-scale 6D SLAM. Automatika: Journal for Control, Measurement, Electronics, Computing & Communications, 52(3), 2011.Google Scholar

  • [33] Stefańczyk M. and Kornuta T. Acquisition of RGB-D images: methods (in Polish). Pomiary – Automatyka – Robotyka PAR, 18(1):82–90, 2014.Google Scholar

  • [34] Stefańczyk M. and Kornuta T. Handling of asynchronous data flow in robot perception subsystems. In Simulation, Modeling, and Programming for Autonomous Robots, volume 8810 of Lecture Notes in Computer Science, pages 509–520. Springer, 2014.Google Scholar

  • [35] Stefańczyk M., Laszkowski M., and Kornuta T. WUT Visual Perception Dataset – a dataset for registration and recognition of objects. In Challenges in Automation, Robotics and Measurement Techniques, volume 440 of Advances in Intelligent Systems and Computing (AISC), pages 635–645. Springer, 2016.Google Scholar

  • [36] Sturm J., Engelhard N., Endres F., Burgard W., and Cremers D. A benchmark for the evaluation of RGB-D SLAM systems. In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pages 573–580. IEEE, 2012.Google Scholar

  • [37] Tang J., Miller S., Singh A., and Abbeel P. A textured object recognition pipeline for color and depth image data. In Robotics and Automation (ICRA), 2012 IEEE International Conference on, pages 3467–3474. IEEE, 2012.Google Scholar

  • [38] Thrun S., Burgard W., and Fox D. A probabilistic approach to concurrent mapping and localization for mobile robots. Autonomous Robots, 5(3-4):253–271, 1998.Google Scholar

  • [39] Tombari F. and Di Stefano L. Object recognition in 3D scenes with occlusions and clutter by hough voting. In Image and Video Technology (PSIVT), 2010 Fourth Pacific-Rim Symposium on, pages 349–355. IEEE, 2010.Google Scholar

  • [40] Tombari F., Salti S., and Di Stefano L. Unique signatures of histograms for local surface description. In Computer Vision–ECCV 2010, pages 356–369. Springer, 2010.Google Scholar

About the article

Published Online: 2017-09-09

Published in Print: 2017-09-01


Citation Information: Foundations of Computing and Decision Sciences, ISSN (Online) 2300-3405, DOI: https://doi.org/10.1515/fcds-2017-0009.

Export Citation

© 2017 Tomasz Kornuta et al., published by De Gruyter Open. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. BY-NC-ND 3.0

Comments (0)

Please log in or register to comment.
Log in