Abstract
Learning skills or knowledge online from experiences is attractive for robots because it permits them to develop new behavior autonomously. However, the onus lies with the system designer to specify which skills or knowledge the robot should learn. Experience-based goal generation algorithms permit a robot to decide autonomously what it will to learn. This paper presents an adaptive resonance theory approach to experience-based generation of approach, avoidance, maintenance and achievement goals for a mobile robot. An experimental analysis is conducted to explore the relationship between algorithm parameters and goals generated on a simulated ePuck robot. Results show how parameter choice influences the number, stability and nature of generated goals. We identify theweight representations, distance functions and update rules that are appropriate for a mobile robot to generate maintenance and achievement goals.
References
[1] J. Weng, J. McClelland, A. Pentland, O. Sporns, I. Stockman, M. Sur, et al., "Artificial intelligence: autonomous mental development by robots and animals," Science, vol. 291, pp. 599-600, 2001. 10.1126/science.291.5504.599Search in Google Scholar PubMed
[2] A. Baranes and P.-Y. Oudeyer, "Intrinsically motivated goal exploration for active motor learning in robots: a case study," presented at the The IEEE/RSJ International Conference on Intelligent Robots and Systems, 2010, pp. 1766-1773. 10.1109/IROS.2010.5651385Search in Google Scholar
[3] A. Baranes and P.-Y. Oudeyer, "Maturationally-constrained competence-based intrinsically motivated learning," presented at the IEEE International Conference on Developmenta and Learning, Ann Arbor, Michigan, 2010. 10.1109/DEVLRN.2010.5578842Search in Google Scholar
[4] K. Merrick, "A comparative study of value systems for selfmotivated exploration and learning by robots," IEEE Transactions on Autonomous Mental Development, Special Issue on Active Learning and Intrinsically Motivated Exploration in Robots, vol. 2, pp. 119-131, 2010. 10.1109/TAMD.2010.2051435Search in Google Scholar
[5] K. Merrick, "Modeling behavior cycles as a value system for developmental robots," Adaptive Behavior, vol. 18, pp. 237-257, 2010. 10.1177/1059712309359948Search in Google Scholar
[6] A. Baranes and P.-Y. Oudeyer, "Active learning of inverse models with intrinsically motivated goal exploration in robots," Robotics and Autonomous Systems, vol. 61, pp. 49-73, 2012. 10.1016/j.robot.2012.05.008Search in Google Scholar
[7] S. Singh, A. G. Barto, and N. Chentanez, "Intrinsically motivated reinforcement learning," in Advances in Neural Information Processing Systems 17 (NIPS), 2005, pp. 1281-1288. 10.21236/ADA440280Search in Google Scholar
[8] K. Merrick and M. L. Maher, "Motivated reinforcement learning: curious characters for multiuser games," ed Berlin: Springer, 2009. 10.1007/978-3-540-89187-1Search in Google Scholar
[9] M. Hardhienata, K. Merrick, and V. Ougrinovski, "Task allocation in multi-agent systems using models of motivation and leadership," presented at the IEEE Conference on Evolutionary Computation, Brisbane, Australia, 2012, pp. 86-93. 10.1109/CEC.2012.6256114Search in Google Scholar
[10] M. L. Maher, K. Merrick, and B. Graham, "Reasoning in the absence of goals," presented at the AAAI Fall Symposium on Advances in Cognitive Systems, 2011, pp. 202-209. Search in Google Scholar
[11] F. Kaplan and P.-Y. Oudeyer, "Motivational principles for visual know-how development," in Proceedings of the 3rd international workshop on Epigenetic Robotics : Modelling cognitive development in robotic systems, Lund University Cognitive Studies, 2003, pp. 73-80. Search in Google Scholar
[12] S. Marsland, U. Nehmzow, and J. Shapiro, "Online novelty detection for autonomous mobile robots," Journal of Robotics and Autonomous Systems, vol. 51, pp. 191-206, 2004. 10.1016/j.robot.2004.10.006Search in Google Scholar
[13] A. S. Rao, "BDI agents: from theory to practice," presented at the The First International Conference on Multi-Agent Systems (ICMAS-95), San Francisco, USA, Georgeff, M. P, pp. 312-319. Search in Google Scholar
[14] L. Braubach, A. Pokahr, D. Moldt, andW. Lamersdorf, "Goal representation for BDI agent systems," presented at the Second International Workshop on Programming Multiagent Systems: Languages and Tools, 2005, pp. 9-20. 10.1007/978-3-540-32260-3_3Search in Google Scholar
[15] O. Simsek and A. G. Barto, "Skill characterisation based on betweenness," presented at the Advances in Neural Information Processing Systems, 2008, pp. 1497-1504. Search in Google Scholar
[16] A. Jonsson and A. G. Barto, "Causal graph based decomposition of factored MDPs," Journal of Machine Learning Research, vol. 7, pp. 2259-2301, 2006. Search in Google Scholar
[17] R. Saunders, "Curious design agents and artificial creativity," PhD PhD, Faculty of Architecture, University of Sydney, Sydney, 2001. Search in Google Scholar
[18] H. Ismail, K. Merrick, and M. Barlow, "Self-motivated learning of achievement and maintenance tasks for non-player characters in computer games," presented at the 2014 IEEE Symposium Series on Computational Intelligence, Symposium on Computational Intelligence for Human-Like Intelligence 2014. 10.1109/CIHLI.2014.7013386Search in Google Scholar
[19] S. Russell, J and P. Norvig, Artificial intelligence: a modern approach. Englewood Cliffs, New Jersey: Prentice Hall Inc., 1995. Search in Google Scholar
[20] T. Kohonen, Self-organisation and associative memory. Berlin: Springer, 1993. Search in Google Scholar
[21] J. MacQueen, "Some methods for classification and analysis of multivariate observations," presented at the The 5th Berkeley Symposium on Mathematical Statistics and Probability, 1967, pp. 281-297. Search in Google Scholar
[22] S. Grossberg, "Adaptive pattern classification and universal recording: I. Parallel development and coding of neural feature detectors," Biological Cybernetics, vol. 23, pp. 121-134, 1976. 10.1007/BF00344744Search in Google Scholar
[23] A. Baraldi and E. Alpaydin, "Simplified ART: a new class of ART algorithms," Technical Report, TR 98-004. , International Computer Science Institute, Berkley, CA, 1998. Search in Google Scholar
[24] B. Fritzke, "Growing cell structures - a self-organising network for unsupervised and supervised learning," Neural Networks, vol. 7, pp. 1411-1460, 1994. Search in Google Scholar
[25] B. Fritzke, "A growing neural gas network learns topology," Advances in Neural Information Processing Systems, vol. 7, 1995. Search in Google Scholar
[26] S. Chumkamon, E. Hayashi, and M. Koike, "Intelligent emotion and behavior based on topological consciousness and adaptive rsonance thory in a companion robot," Biologically Inspired Cognitive Architectures, p. (in press), 2016. 10.1016/j.bica.2016.09.004Search in Google Scholar
[27] T. Dash, "Automatic navigation of wall following mobile robot using adaptive resonance theory of type-1," Biologically Inspired Cognitive Architectures, vol. 12, pp. 1-8, 2015. 10.1016/j.bica.2015.04.008Search in Google Scholar
[28] M. Hassoun, Fundamentals of artificial neural networks: MIT Press, 1995. 10.1109/JPROC.1996.503146Search in Google Scholar
[29] G. Carpenter and S. Grossberg, "ART2: self-organisation of stable category recognition codes for analog input patterns," Applied Optics, vol. 26, pp. 4919-4930, 1987. Search in Google Scholar
[30] G. Carpenter and S. Grossberg, "ART3: Hierarchical search using chemical transmitters in self-organising pattern recognition architectures," Neural Networks, vol. 3, pp. 129-152, 1990. 10.1016/0893-6080(90)90085-YSearch in Google Scholar
[31] G. Carpenter, S. Grossberg, and J. Reynolds, "ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organising neural network," Neural Networks, vol. 4, pp. 565-588, 1991. 10.1016/0893-6080(91)90012-TSearch in Google Scholar
[32] K. Merrick, "Modelling behaviour cycles as a value system for developmental robots," Adaptive Behavior, vol. 18, pp. 237-257, 2010. 10.1177/1059712309359948Search in Google Scholar
[33] K. Merrick and T. Scully, "Modelling affordances for control and evaluation of intrinsically motivated robots," presented at the Australian Conference on Robotics and Automation, Sydney, Australia, 2009, p. (CD no page numbers). Search in Google Scholar
[34] R. Rojas, "Unsupervised learning and clustering algorithms," in Neural Networks - A Systematic Introduction, ed: Springer- Verlag, 1996, pp. 101-123. 10.1007/978-3-642-61068-4_5Search in Google Scholar
[35] V. G. Santucci, G. Baldassarre, and M. Mirolli, "Which is the best intrinsic motivation signal for learning multiple skills?," Frontiers in Neurorobotics, vol. 7, 2013. 10.3389/fnbot.2013.00022Search in Google Scholar PubMed PubMed Central
[36] A. White, J. Modayil, and R. Sutton, "Scaling life-long off policy learning," presented at the IEEE International Conference on Development and Learning, 2012. 10.1109/DevLrn.2012.6400860Search in Google Scholar
© 2016 Kathryn Merrick et al.
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.
