Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Paladyn, Journal of Behavioral Robotics

Editor-in-Chief: Schöner, Gregor

Covered by SCOPUS

CiteScore 2018: 2.17

SCImago Journal Rank (SJR) 2018: 0.336
Source Normalized Impact per Paper (SNIP) 2018: 1.707

ICV 2017: 99.90

Open Access
See all formats and pricing
More options …

Learning the Condition of Satisfaction of an Elementary Behavior in Dynamic Field Theory

Matthew Luciw / Sohrob Kazerounian / Konstantin Lahkman / Mathis Richter / Yulia Sandamirskaya
Published Online: 2015-11-18 | DOI: https://doi.org/10.1515/pjbr-2015-0011


In order to proceed along an action sequence, an autonomous agent has to recognize that the intended final condition of the previous action has been achieved. In previous work, we have shown how a sequence of actions can be generated by an embodied agent using a neural-dynamic architecture for behavioral organization, in which each action has an intention and condition of satisfaction. These components are represented by dynamic neural fields, and are coupled to motors and sensors of the robotic agent.Here,we demonstratehowthemappings between intended actions and their resulting conditions may be learned, rather than pre-wired.We use reward-gated associative learning, in which, over many instances of externally validated goal achievement, the conditions that are expected to result with goal achievement are learned. After learning, the external reward is not needed to recognize that the expected outcome has been achieved. This method was implemented, using dynamic neural fields, and tested on a real-world E-Puck mobile robot and a simulated NAO humanoid robot.

Keywords : Neural Dynamics; Cognitive Robotics; Behavioral Organization


  • [1] SAmari. Dynamics of pattern formation in lateral-inhibition type neural fields. Biological Cybernetics, 27:77–87, 1977. CrossrefGoogle Scholar

  • [2] E Bicho, P Mallet, and G Schöner. Target representation on an autonomous vehicle with low-level sensors. The International Journal of Robotics Research, 19:424–447, 2000. CrossrefGoogle Scholar

  • [3] Daniele Caligiore, Anna M Borghi, Domenico Parisi, and Gianluca Baldassarre. Tropicals: a computational embodied neuroscience model of compatibility effects. Psychological Review, 117(4):1188, 2010. Web of ScienceCrossrefGoogle Scholar

  • [4] Anthony Dickinson and Bernard Balleine. Motivational control of goal-directed action. Animal Learning & Behavior, 22(1):1– 18, 1994. Google Scholar

  • [5] B Duran and Y Sandamirskaya. Neural dynamics of hierarchically organized sequences: a robotic implementation. In Proceedings of 2012 IEEE-RAS International Conference on Humanoid Robots (Humanoids), 2012. Google Scholar

  • [6] Boris Duran, Yulia Sandamirskaya, and Gregor Schöner. A dynamic field architecture for the generation of hierarchically organized sequences. In Alessandro E.P. Villa, Wlodzislaw Duch, Peter Erdi, Francesco Masulli, and Günther Palm, editors, Artificial Neural Networks and Machine Learning – ICANN 2012, volume 7552 of Lecture Notes in Computer Science, pages 25–32. Springer Berlin Heidelberg, 2012. CrossrefGoogle Scholar

  • [7] Wolfram Erlhagen and Estela Bicho. The dynamic neural field approach to cognitive robotics. Journal of Neural Engineering, 3(3):R36–R54, 2006. CrossrefGoogle Scholar

  • [8] C Faubel and G Schöner. Fast learning to recognize objects: Dynamic fields in label-feature space. In Proceedings of the fifth International Conference on Development and Learning ICDL 2006, 2006. Google Scholar

  • [9] S Grossberg. Nonlinear neural networks: Principles, mechanisms, and architectures. Neural Networks, 1:17–61, 1988. CrossrefGoogle Scholar

  • [10] C.L. Hull. Principles of behavior: an introduction to behavior theory. Century psychology series. D. Appleton-Century Company, incorporated, 1943. Google Scholar

  • [11] J S Johnson, J P Spencer, and G Schöner. Moving to higher ground: The dynamic field theory and the dynamics of visual cognition. New Ideas in Psychology, 26:227–251, 2008. Web of ScienceCrossrefGoogle Scholar

  • [12] S. Kazerounian, M. Luciw, M. Richter, and Y. Sandamirskaya. Autonomous reinforcement of behavioral sequences in neural dynamics. In International Joint Conference on Neural Networks (IJCNN), 2013. Google Scholar

  • [13] Giovanni Pezzulo, Martin V Butz, and Cristiano Castelfranchi. The anticipatory approach: definitions and taxonomies. In The Challenge of Anticipation, pages 23–43. Springer, 2008. Google Scholar

  • [14] R A Rescorla and R L Solomon. Two-process learning theory: Relationships between pavlovian conditioning and instrumental learning. Psychological Review, 74(3):152–182, 1967. CrossrefGoogle Scholar

  • [15] Mathis Richter, Yulia Sandamirskaya, and Gregor Schöner. A robotic architecture for action selection and behavioral organization inspired by human cognition. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 2457– 2464, 2012. Google Scholar

  • [16] Y. Sandamirskaya, M. Richter, and G. Schöner. A neuraldynamic architecture for behavioral organization of an embodied agent. In IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL EPIROB 2011), 2011. Google Scholar

  • [17] Yulia Sandamirskaya. Dynamic neural fields as a step towards cognitive neuromorphic architectures. Frontiers in Neuroscience, 7:276, 2013. Web of ScienceGoogle Scholar

  • [18] Yulia Sandamirskaya and Gregor Schöner. An embodied account of serial order: How instabilities drive sequence generation. Neural Networks, 23(10):1164–1179, December 2010. CrossrefWeb of ScienceGoogle Scholar

  • [19] Yulia Sandamirskaya, Stephan K.U. Zibner, Sebastian Schneegans, and Gregor Schöner. Using dynamic field theory to extend the embodiment stance toward higher cognition. New Ideas in Psychology, 2013. Google Scholar

  • [20] J Schmidhuber. Making the world differentiable: On using fully recurrent self-supervised neural networks for dynamic reinforcement learning and planning in non-stationary environments. Institut für Informatik, Technische Universität München. Technical Report FKI-126-90, 1990. Google Scholar

  • [21] G Schöner. Dynamical systems approaches to cognition. In Ron Sun, editor, Cambridge Handbook of Computational Cognitive Modeling, pages 101–126, Cambridge, UK, 2008. Cambridge University Press. Google Scholar

  • [22] Wolfram Schultz. Reward signaling by dopamine neurons. The Neuroscientist, 7(4):293–302, 2001. CrossrefGoogle Scholar

  • [23] WolframSchultz. Review dopamine signals for reward value and risk: basic and recent data. Behav. Brain Funct, 6:24, 2010. Web of ScienceGoogle Scholar

  • [24] John R Searle. Intentionality — An essay in the philosophy of mind. Cambridge University Press, 1983. Google Scholar

  • [25] J P Spencer and G Schöner. Bridging the representational gap in the dynamical systems approach to development. Developmental Science, 6:392–412, 2003. CrossrefGoogle Scholar

  • [26] R.S. Sutton and A.G. Barto. Reinforcement learning: An introduction, volume 1. Cambridge Univ Press, 1998. Google Scholar

  • [27] Webots. http://www.cyberbotics.com. Commercial Mobile Robot Simulation Software. Google Scholar

  • [28] Juyang Weng. Developmental robotics: Theory and experiments. International Journal of Humanoid Robotics, 1(02):199– 236, 2004. Google Scholar

  • [29] H R Wilson and J D Cowan. A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue. Kybernetik, 13:55–80, 1973. CrossrefGoogle Scholar

  • [30] R.S. Woodworth. Dynamic psychology, by Robert Sessions Woodworth. Columbia University Press, 1918. Google Scholar

About the article

Received: 2014-02-08

Accepted: 2015-07-29

Published Online: 2015-11-18

Citation Information: Paladyn, Journal of Behavioral Robotics, Volume 6, Issue 1, ISSN (Online) 2081-4836, DOI: https://doi.org/10.1515/pjbr-2015-0011.

Export Citation

© 2015 Matthew Luciw et al.. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. BY-NC-ND 3.0

Comments (0)

Please log in or register to comment.
Log in