Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Paladyn, Journal of Behavioral Robotics

Editor-in-Chief: Schöner, Gregor


Covered by SCOPUS


CiteScore 2018: 2.17

SCImago Journal Rank (SJR) 2018: 0.336
Source Normalized Impact per Paper (SNIP) 2018: 1.707

ICV 2018: 120.52

Open Access
Online
ISSN
2081-4836
See all formats and pricing
More options …

Context-aware robot navigation using interactively built semantic maps

Akansel Cosgun / Henrik I. Christensen
Published Online: 2018-09-06 | DOI: https://doi.org/10.1515/pjbr-2018-0020

Abstract

We discuss the process of building semantic maps, how to interactively label entities in them, and how to use them to enable context-aware navigation behaviors in human environments. We utilize planar surfaces, such as walls and tables, and static objects, such as door signs, as features for our semantic mapping approach. Users can interactively annotate these features by having the robot follow him/her, entering the label through a mobile app, and performing a pointing gesture toward the landmark of interest. Our gesture-based approach can reliably estimate which object is being pointed at, and detect ambiguous gestures with probabilistic modeling. Our person following method attempts to maximize future utility by search for future actions assuming constant velocity model for the human. We describe a method to extract metric goals from a semantic map landmark and to plan a human aware path that takes into account the personal spaces of people. Finally, we demonstrate context awareness for person following in two scenarios: interactive labeling and door passing.We believe that future navigation approaches and service robotics applications can be made more effective by further exploiting the structure of human environments.

Keywords: mobile robot navigation; semantic mapping; human-robot interaction

References

  • [1] F. Dellaert, M. Kaess, Square root SAM: Simultaneous localization and mapping via square root information smoothing, The International Journal of Robotics Research, 2006, 25(12), 1181- 1203CrossrefGoogle Scholar

  • [2] R. C. Smith, P. Cheeseman, On the representation and estimation of spatial uncertainty, The International Journal of Robotics Research (IJRR), 1986, 5(4), 56-68CrossrefGoogle Scholar

  • [3] J. J. Leonard, H. F. Durrant-Whyte, Simultaneous map building and localization for an autonomous mobile robot, In: Workshop on Intelligence for Mechanical Systems at IEEE/RSJ Intelligent Robots and Systems (IROS), 1991, 1442-1447Google Scholar

  • [4] J. Folkesson, P. Jensfelt, H. I. Christensen, The M-Space feature representation for SLAM, IEEE Transactions on Robotics, 2007, 23(5), 1024-1035CrossrefGoogle Scholar

  • [5] B. Williams, M. Cummins, J. Neira, P. Newman, I. Reid, J. Tardós, A comparison of loop closing techniques in monocular SLAM, Robotics and Autonomous Systems, 2009, 57(12), 1188-1197CrossrefGoogle Scholar

  • [6] P. Henry, M. Krainin, E. Herbst, X. Ren, D. Fox, RGB-D mapping: Using depth cameras for dense 3d modeling of indoor environments, In: 12th International Symposium on Experimental Robotics (ISER), 2010Google Scholar

  • [7] A. Elfes, Using occupancy grids for mobile robot perception and navigation, Computer, 1989, 22(6), 46-57CrossrefGoogle Scholar

  • [8] G. Grisetti, C. Stachniss, W. Burgard, Improved techniques for gridmapping with Rao-Blackwellized particle filters, IEEE Transactions on Robotics, 2007, 23(1), 34-46Google Scholar

  • [9] E. Remolina, B. Kuipers, Towards a general theory of topological maps, Artificial Intelligence, 2004, 152(1), 47-104Google Scholar

  • [10] J. Boal, A. Sánchez-Miralles, A. Arranz, Topological simultaneous localization and mapping: a survey, Robotica, 2014, 32(5), 803-821Google Scholar

  • [11] B. Kuipers, The spatial semantic hierarchy, Artificial intelligence, 2000, 119(1), 191-233Google Scholar

  • [12] O. M. Mozos, C. Stachniss, W. Burgard, Supervised learning of places from range data using AdaBoost, In: IEEE International Conference on Robotics and Automation (ICRA), 2005, 1730- 1735Google Scholar

  • [13] S. Ekvall, D. Kragic, P. Jensfelt, Object detection andmapping for service robot tasks, Robotica, 2007, 25(02), 175-187Google Scholar

  • [14] A. Nüchter, J. Hertzberg, Towards semantic maps for mobile robots, Robotics and Autonomous Systems, 2008, 56(11), 915- 926Google Scholar

  • [15] A. Pronobis, P. Jensfelt, Large-scale semantic mapping and reasoning with heterogeneous modalities, In: 2012 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2012, 3515-3522Google Scholar

  • [16] S. Choudhary, L. Carlone, C. Nieto, J. Rogers, Z. Liu, H. I. Christensen, F. Dellaert, Multi robot object-based SLAM, In: International Symposium on Experimental Robotics, 2016, 729-741Google Scholar

  • [17] N. Sünderhauf, T. T. Pham, Y. Latif, M. Milford, I. Reid,Meaningful mapswith object-oriented semanticmapping, In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017, 5079-5085Google Scholar

  • [18] J. Fasola, M. J. Mataric, Using semantic fields to model dynamic spatial relations in a robot architecture for natural language instruction of service robots, In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2013, 143-150Google Scholar

  • [19] S. Tellex, T. Kollar, S. Dickerson, M. R. Walter, A. G. Banerjee, S. J. Teller, N. Roy, Understanding natural language commands for robotic navigation and mobile manipulation, In: 25th AAAI Conference on Artificial Intelligence, 2011Google Scholar

  • [20] I. Kostavelis, A. Gasteratos, Semantic mapping for mobile robotics tasks: A survey, Robotics and Autonomous Systems, 2015, 66, 86-103Google Scholar

  • [21] C. Landsiedel, V. Rieser, M. Walter, D. Wollherr, A review of spatial reasoning and interaction for real-world robotics, Advanced Robotics, 2017, 31(5), 222-242CrossrefGoogle Scholar

  • [22] E. A. Topp, H. I. Christensen, Topological modelling for human augmented mapping, In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2006, 2257-2263Google Scholar

  • [23] H. H. Clark, S. E. Brennan, Grounding in communication, Perspectives on socially shared cognition, 1991, 13(1991), 127-149Google Scholar

  • [24] G.-J. M. Kruijff, H. Zender, P. Jensfelt, H. I. Christensen, Clarification dialogues in human-augmented mapping, In: Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction, 2006, 282-289Google Scholar

  • [25] H. Zender, P. Jensfelt, Ó. M. Mozos, G.-J. M. Kruijff, W. Burgard, An integrated robotic system for spatial understanding and situated interaction in indoor environments, In: AAAI, 2007, 7, 1584- 1589Google Scholar

  • [26] G. Gemignani, D. Nardi, D. D. Bloisi, R. Capobianco, L. Iocchi, Interactive semantic mapping: experimental evaluation, In: Experimental Robotics, 2016, 339-355CrossrefGoogle Scholar

  • [27] T. B. Moeslund, A. Hilton, V. Krüger, A survey of advances in vision-based human motion capture and analysis, Computer Vision and Image Understanding, 2006, 104(2-3), 90-126CrossrefGoogle Scholar

  • [28] A. F. Bobick J.W. Davis, The recognition of human movement using temporal templates, IEEE Transactions on pattern analysis and machine intelligence, 2001, 23(3), 257-267Google Scholar

  • [29] N. Dalal, B. Triggs, Histograms of oriented gradients for human detection, In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2005, 1, 886-893Google Scholar

  • [30] K. Mikolajczyk, C. Schmid, A. Zisserman, Human detection based on a probabilistic assembly of robust part detectors, In: European Conference on Computer Vision (ECCV), Springer, 2004, 69-82Google Scholar

  • [31] G. Shu, A. Dehghan, O. Oreifej, E. Hand, M. Shah, Part-based multiple-person tracking with partial occlusion handling, In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012, 1815-1821Google Scholar

  • [32] T. Darrell, G. Gordon, M. Harville, J. Woodfill, Integrated person tracking using stereo, color, and pattern detection, International Journal of Computer Vision, 2000, 37(2), 175-185CrossrefGoogle Scholar

  • [33] J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, et al., Real-time human pose recognition in parts from single depth images, In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011, 1297-1304Google Scholar

  • [34] Y. LeCun, Y. Bengio, et al., Convolutional networks for images, speech, and time series, The handbook of Brain Theory and Neural Networks, 1995, 3361(10)Google Scholar

  • [35] A. Krizhevsky, I. Sutskever, G. E. Hinton, ImageNet classification with deep convolutional neural networks, In: Advances In Neural Information Processing Systems, 2012, 1097-1105Google Scholar

  • [36] Y. Tian, P. Luo, X. Wang, X. Tang, Deep learning strong parts for pedestrian detection, In: Proceedings of the IEEE International Conference on Computer Vision, 2015, 1904-1912Google Scholar

  • [37] A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, L. Fei-Fei, S. Savarese, Social LSTM: Human trajectory prediction in crowded spaces, In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, 961-971Google Scholar

  • [38] K. O. Arras, Ó. M. Mozos,W. Burgard, Using boosted features for the detection of people in 2d range data, In: IEEE International Conference on Robotics and Automation (ICRA), 2007, 3402- 3407Google Scholar

  • [39] D. Schulz, W. Burgard, D. Fox, A. B. Cremers, Tracking multiple moving targetswith a mobile robot using particle filters and statistical data association, In: IEEE International Conference on Robotics and Automation (ICRA), 2001, 2, 1665-1670Google Scholar

  • [40] E. A. Topp, H. I. Christensen, Tracking for following and passing persons, In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2005, 2321-2327Google Scholar

  • [41] N. Bellotto, H. Hu, Multisensor-based human detection and tracking for mobile service robots, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2009, 39(1), 167-181CrossrefGoogle Scholar

  • [42] F. Zanlungo, T. Ikeda, T. Kanda, Social force model with explicit collision prediction, EPL (Europhysics Letters), 2011, 93(6), 68005Google Scholar

  • [43] A. Leigh, J. Pineau, N. Olmedo, H. Zhang, Person tracking and following with 2D laser scanners, In: IEEE International Conference on Robotics and Automation (ICRA), 2015, 726-733Google Scholar

  • [44] C. Dondrup, N. Bellotto, F. Jovan, M. Hanheide, et al., Real-time multisensor people tracking for human-robot spatial interaction, Workshop on Machine Learning for Social Robotics at IEEE International Conference on Robotics and Automation (ICRA), 2015Google Scholar

  • [45] J. Schmidt, N. Hofemann, A. Haasch, J. Fritsch, G. Sagerer, Interacting with a mobile robot: Evaluating gestural object references, In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2008, 3804-3809Google Scholar

  • [46] M. Van den Bergh, D. Carton, R. De Nijs, N. Mitsou, C. Landsiedel, K. Kuehnlenz, et al., Real-time 3D hand gesture interaction with a robot for understanding directions from humans, In: IEEE International Workshop on Robot and Human Interactive Communication (ROMAN), 2011, 357-362Google Scholar

  • [47] A. G. Brooks, C. Breazeal, Working with robots and objects: Revisiting deictic reference for achieving spatial common ground, In: Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction, 2006, 297-304Google Scholar

  • [48] E. A. Sisbot, L. F. Marin-Urias, R. Alami, T. Simeon, A human aware mobile robot motion planner, IEEE Transactions on Robotics, 2007, 23(5), 874-883CrossrefGoogle Scholar

  • [49] R. Kirby, R. Simmons, J. Forlizzi. Companion: A constraintoptimizing method for person-acceptable navigation, In: The 18th IEEE International Symposium on Robot and Human Interactive Communication, (RO-MAN), 2009, 607-612Google Scholar

  • [50] M. L. Walters, K. Dautenhahn, R. Te Boekhorst, K. L. Koay, C. Kaouri, S. Woods, C. Nehaniv, D. Lee, I. Werry, The influence of subjects’ personality traits on personal spatial zones in a human-robot interaction experiment, In: IEEE International Workshop on Robot and Human Interactive Communication (ROMAN), 2005, 347-352Google Scholar

  • [51] M. Luber, L. Spinello, J. Silva, K. O. Arras, Socially-aware robot navigation: A learning approach, In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012, 902- 907Google Scholar

  • [52] H. Kidokoro, T. Kanda, D. Bršcic, M. Shiomi, Simulation-based behavior planning to prevent congestion of pedestrians around a robot, IEEE Transactions on Robotics, 2015, 31(6), 1419-1431Google Scholar

  • [53] A. Bordallo, F. Previtali, N. Nardelli, S. Ramamoorthy, Counterfactual reasoning about intent for interactive navigation in dynamic environments, In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015, 2943-2950Google Scholar

  • [54] U. Köeckemann, F. Pecora, L. Karlsson, Inferring context and goals for online human-aware planning, In: IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI), IEEE, 2015, 550-557Google Scholar

  • [55] T. Kruse, P. Basili, S. Glasauer, A. Kirsch, Legible robot navigation in the proximity of moving humans, In: IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO), 2012, 83-88Google Scholar

  • [56] A. D. Dragan, K. C. Lee, S. S. Srinivasa, Legibility and predictability of robot motion, In: 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2013, 301-308Google Scholar

  • [57] P. Trautman, J. Ma, R. M. Murray, A. Krause, Robot navigation in dense human crowds: Statistical models and experimental studies of human-robot cooperation, The International Journal of Robotics Research, 2015, 34(3), 335-356Google Scholar

  • [58] P. Henry, C. Vollmer, B. Ferris, D. Fox, Learning to navigate through crowded environments, In: IEEE International Conference on Robotics and Automation (ICRA), 2010, 981-986Google Scholar

  • [59] R. Philippsen, R. Siegwart, Smooth and eflcient obstacle avoidance for a tour guide robot, In: IEEE International Conference on Robotics and Automation (ICRA), 2003Google Scholar

  • [60] L. Y. Morales Saiki, S. Satake, R. Huq, D. Glas, T. Kanda, N. Hagita, How do people walk side-by-side?: using a computational model of human behavior for a social robot, In: ACM/IEEE International Conference on Human-Robot Interaction, 2012, 301-308Google Scholar

  • [61] A. Ohya, T. Munekata, Intelligent escort robot moving together with human-interaction in accompanying behavior, In: Proceedings FIRA Robot World Congress, 2002, 31-35Google Scholar

  • [62] R. Murakami, L. Y. Morales Saiki, S. Satake, T. Kanda, H. Ishiguro, Destination unknown: walking side-by-side without knowing the goal, In: Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction, 2014, 471-478Google Scholar

  • [63] J. J. Park, B. Kuipers, Autonomous person pacing and following with model predictive equilibrium point control, In: IEEE International Conference on Robotics and Automation (ICRA), 2013, 1060-1067Google Scholar

  • [64] C. Granata, P. Bidaud, A framework for the design of person following behaviors for social mobile robots, In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012, 4652-4659Google Scholar

  • [65] R. Gockley, J. Forlizzi, R. Simmons, Natural person following behavior for social robots, In: 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2007, 17-24Google Scholar

  • [66] K. Charalampous, I. Kostavelis, A. Gasteratos, Recent trends in social aware robot navigation: A survey, Robotics and Autonomous Systems, 2017, 93, 85-104CrossrefGoogle Scholar

  • [67] T. Kruse, A. K. Pandey, R. Alami, A. Kirsch, Human-aware robot navigation: A survey, Robotics and Autonomous Systems, 2013, 61(12), 1726-1743CrossrefGoogle Scholar

  • [68] P. Regier, S. Oßwald, P. Karkowski, M. Bennewitz, Foresighted navigation through cluttered environments, In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016, 1437-1442Google Scholar

  • [69] E. Pacchierotti, H. I. Christensen, P. Jensfelt, Human-robot embodied interaction in hallway settings: a pilot user study, In: IEEE International Workshop on Robot and Human Interactive Communication (ROMAN), 2005, 164-171Google Scholar

  • [70] N. Wilde, D. Kulic, S. L. Smith, Learning user preferences in robot motion planning through interaction, In: IEEE International Conference on Robotics and Automation (ICRA), 2018Google Scholar

  • [71] C. Galindo, A. Saflotti, Inferring robot goals from violations of semantic knowledge, Robotics and Autonomous Systems, 2013, 61(10), 1131-1143Google Scholar

  • [72] D. V. Lu, W. D. Smart, Towards more eflcient navigation for robots and humans, In: IEEE/RSJ International Conference On Intelligent Robots and Systems (IROS), 2013, 1707-1713Google Scholar

  • [73] M. M. Loper, N. P. Koenig, S. H. Chernova, C. V. Jones, O. C. Jenkins, Mobile human-robot teamingwith environmental tolerance, In: Proceedings of the 4th ACM/IEEE International Conference on Human-Robot Interaction, ACM, 2009, 157-164Google Scholar

  • [74] P. Anderson, Q. Wu, D. Teney, J. Bruce, M. Johnson, N. Sünderhauf, et al., Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments, In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018Google Scholar

  • [75] H. Zender, P. Jensfelt, G.-J. M. Kruijff, Human-and situationaware people following, In: The 16th IEEE International Symposium on Robot and Human Interactive Communication (ROMAN), 2007, 1131-1136Google Scholar

  • [76] J. Neira, J. D. Tardós, Data association in stochasticmapping using the joint compatibility test, IEEE Transactions on Robotics and Automation, 2001, 17(6), 890-897CrossrefGoogle Scholar

  • [77] A. J. Trevor, A. Cosgun, J. Kumar, H. I. Christensen, Interactive map labeling for service robots, In: IROS Workshop on Active Semantic Perception, 2012Google Scholar

  • [78] J. Xavier, M. Pacheco, D. Castro, A. Ruano, U. Nunes., Fast line, arc/circle and leg detection from laser scan data in a player driver, In: IEEE International Conference on Robotics and Automation (ICRA), 2005, 3930-3935Google Scholar

  • [79] D. Fox, W. Burgard, S. Thrun, The dynamic window approach to collision avoidance, IEEE Robotics & Automation Magazine, 1997, 4(1), 23-33CrossrefGoogle Scholar

  • [80] A. Cosgun, D. A. Florencio, H. I. Christensen, Autonomous person following for telepresence robots, In: IEEE International Conference on Robotics and Automation (ICRA), 2013, 4335- 4342Google Scholar

  • [81] A. Cosgun, A. J. Trevor, H. I. Christensen, Did you mean this object?: Detecting ambiguity in pointing gesture targets, In: Towards a Framework for Joint Action Workshop at 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI 2015), IEEE Press, 2015Google Scholar

  • [82] A. J. Trevor, J. G. Rogers III, A. Cosgun, H. I. Christensen, Interactive object modeling & labeling for service robots, In: Proceedings of the 8th ACM/IEEE International the Conference on Human-Robot Interaction (HRI), 2013, 421-422Google Scholar

  • [83] A. Cosgun, E. A. Sisbot, H. I. Christensen, Anticipatory robot path planning in human environments, In: 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2016, 562-569Google Scholar

  • [84] M. Cristani, L. Bazzani, G. Paggetti, A. Fossati, D. Tosato, A. Del Bue, et al., Social interaction discovery by statistical analysis of F-formations, In: The 22nd British Machine Vision Conference (BMVC 2011), 2011, 2Google Scholar

  • [85] D. Helbing, P. Molnar, Social force model for pedestrian dynamics, Physical review E, 1995, 51(5), 4282Google Scholar

  • [86] A. Kendon, Conducting interaction: Patterns of behavior in focused encounters, In: Studies in interactional sociolinguistics, vol. 7, Cambridge University Press, 1990.Google Scholar

About the article

Received: 2017-10-24

Accepted: 2018-07-12

Published Online: 2018-09-06


Citation Information: Paladyn, Journal of Behavioral Robotics, Volume 9, Issue 1, Pages 254–276, ISSN (Online) 2081-4836, DOI: https://doi.org/10.1515/pjbr-2018-0020.

Export Citation

© by Akansel Cosgun and Henrik I. Christensen, published by De Gruyter. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. BY-NC-ND 4.0

Comments (0)

Please log in or register to comment.
Log in