Jump to ContentJump to Main Navigation
Show Summary Details
In This Section

Paladyn, Journal of Behavioral Robotics

Editor-in-Chief: Schöner, Gregor

1 Issue per year

Open Access
See all formats and pricing
In This Section

Parsing of action sequences: A neural dynamics approach

David Lobato
  • Vision Lab (LARSyS), FCT, University of the Algarve, Gambelas Campus, 8000 Faro, Portugal
/ Yulia Sandamirskaya
  • Institut für Neuroinformatik, Ruhr-Universität Bochum, 44780 Bochum, Germany
/ Mathis Richter
  • Institut für Neuroinformatik, Ruhr-Universität Bochum, 44780 Bochum, Germany
/ Gregor Schöner
  • Institut für Neuroinformatik, Ruhr-Universität Bochum, 44780 Bochum, Germany
Published Online: 2015-05-15 | DOI: https://doi.org/10.1515/pjbr-2015-0008


Parsing of action sequences is the process of segmenting observed behavior into individual actions. In robotics, this process is critical for imitation learning from observation and for representing an observed behavior in a form that may be communicated to a human. In this paper, we develop a model for action parsing, based on our understanding of principles of grounded cognitive processes, such as perceptual decision making, behavioral organization, and memory formation.We present a neural-dynamic architecture, in which action sequences are parsed using a mathematical and conceptual framework for embodied cognition—the Dynamic Field Theory. In this framework, we introduce a novel mechanism, which allows us to detect and memorize actions that are extended in time and are parametrized by the target object of an action. The core properties of the architecture are demonstrated in a set of simple, proof-of-concept experiments.

Keywords : action parsing; sequence learning; elementary behaviors; dynamic neural fields


  • [1] Gutemberg Guerra-Filho and Yiannis Aloimonos. The syntax of human actions and interactions. Journal of Neurolinguistics, 25(5):500–514, 2012. [Crossref] [Web of Science]

  • [2] Yiannis Aloimonos. Sensory grammars for sensor networks. Journal of Ambient Intelligence and Smart Environments, 1(1):15–21, January 2009. [Web of Science]

  • [3] Florentin Wörgötter, Eren Erdal Aksoy, Norbert Krüger, Justus Piater, Ales Ude, and Minija Tamosiunaite. A simple ontology of manipulation actions based on hand-object relations. IEEE Transactions on Autonomous Mental Development, 5(2):117– 134, june 2013.

  • [4] Georg Layher, Martin A Giese, and Heiko Neumann. Learning representations of animated motion sequences—A neural model. Topics in Cognitive Science, 6(1):170–182, January 2014.

  • [5] Gregor Schöner. Dynamical systems approaches to cognition. In Ron Sun, editor, Cambridge Handbook of Computational Cognitive Modeling, pages 101–126, Cambridge, UK, 2008. Cambridge University Press.

  • [6] Yulia Sandamirskaya. Dynamic neural fields as a step towards cognitive neuromorphic architectures. Frontiers in Neuroscience, 7:276, 2013. [Web of Science]

  • [7] Yulia Sandamirskaya and Gregor Schöner. An embodied account of serial order: How instabilities drive sequence generation. Neural Networks, 23(10):1164–1179, December 2010. [Crossref] [Web of Science]

  • [8] Yulia Sandamirskaya and Gregor Schöner. Serial order in an acting system: a multidimensional dynamic neural fields implementation. In 9th IEEE International Conference on Development and Learning (ICDL 2010), 2010.

  • [9] Mathis Richter, Yulia Sandamirskaya, and Gregor Schöner. A robotic architecture for action selection and behavioral organization inspired by human cognition. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2012), pages 2457–2464, 2012.

  • [10] Yulia Sandamirskaya, Stephan K.U. Zibner, Sebastian Schneegans, and Gregor Schöner. Using dynamic field theory to extend the embodiment stance toward higher cognition. New Ideas in Psychology, 31(3):322 – 339, 2013. [Web of Science] [Crossref]

  • [11] Shun-ichi Amari. Dynamics of pattern formation in lateralinhibition type neural fields. Biological Cybernetics, 27:77–87, 1977. [Crossref]

  • [12] Hugh R Wilson and Jack D Cowan. A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue. Kybernetik, 13(2):55–80, 1973. [Crossref]

  • [13] Stephen Grossberg. Nonlinear neural networks: Principles, mechanisms, and architectures. Neural Networks, 1(1):17–61, 1988. [Crossref]

  • [14] Stephan K U Zibner, Christian Faubel, Ioannis Iossifidis, and Gregor Schöner. Dynamic Neural Fields as Building Blocks for a Cortex-Inspired Architecture of Robotic Scene Representation. IEEE Transactions on AutonomousMental Development, 3(1):74– 91, 2011. [Web of Science]

  • [15] Y Sandamirskaya and T Storck. Neural-dynamic architecture for looking: Shift from visual to motor target representation for memory saccade. In IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL EPIROB 2014), 2014.

  • [16] Radu Bogdan Rusu and Steve Cousins. 3D is here: Point Cloud Library (PCL). IEEE International Conference on Robotics and Automation (ICRA 2011), pages 1–4, May 2011.

  • [17] Martin A. Fischler and Robert C. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM, 24(6):381–395, June 1981. [Crossref]

  • [18] Michael Berger, Christian Faubel, Joseph Norman, Howard Hock, and Gregor Schöner. The Counter-Change Model of Motion Perception : An Account Based on Dynamic Field Theory The Dynamic Field Model of Counter-Change Motion. Lecture Notes in Computer Science, 7552:579–586, 2012.

  • [19] Rudolph Emil Kalman. A new approach to linear filtering and prediction problems. Transactions of the American Society of Mechanical Engineers - Journal of Basic Engineering, 82(1):35– 45, 1960. [Crossref]

  • [20] HaroldWKuhn. The hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1-2):83–97, 1955. [Crossref]

  • [21] James Munkres. Algorithms for the assignment and transportation problems. Journal of the Society for Industrial & Applied Mathematics, 5(1):32–38, 1957.

  • [22] Christian Faubel and Gregor Schöner. Learning to recognize objects on the fly: a neurally based dynamic field approach. Neural Networks, 21(4):562–576, 2008. [Web of Science] [Crossref]

  • [23] Yulia Sandamirskaya, Mathis Richter, and Gregor Schöner. A neural-dynamic architecture for behavioral organization of an embodied agent. In IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL EPIROB 2011), pages 1–7, 2011.

  • [24] Boris Duran and Yulia Sandamirskaya. Neural dynamics of hierarchically organized sequences: A robotic implementation. In 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012), pages 357–362, 2012.

  • [25] Kyuhwa Lee and Yiannis Demiris. Towards incremental learning of task-dependent action sequences using probabilistic parsing. IEEE International Conference of Development and Learning (ICDL), 2011.

  • [26] Falk Fleischer, Vittorio Caggiano, Peter Thier, and Martin a Giese. Physiologically inspired model for the visual recognition of transitive hand actions. The Journal of neuroscience : the official journal of the Society for Neuroscience, 33(15):6563–80, April 2013. [Crossref] [Web of Science]

  • [27] Sohrob Kazerounian, Matthew Luciw, Mathis Richter, and Yulia Sandamirskaya. Autonomous reinforcement of behavioral sequences in neural dynamics. In International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2013.

  • [28] Brenna D. Argall, Sonia Chernova, Manuela Veloso, and Brett Browning. A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57(5):469–483, May 2009. [Web of Science] [Crossref]

  • [29] Stefan Schaal. Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences, 3(6):233–242, 1999.

About the article

Received: 2014-03-31

Accepted: 2015-01-09

Published Online: 2015-05-15

Citation Information: Paladyn, Journal of Behavioral Robotics, ISSN (Online) 2081-4836, DOI: https://doi.org/10.1515/pjbr-2015-0008. Export Citation

© 2015 David Lobato et al.. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. (CC BY-NC-ND 3.0)

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

Nadia Rasheed and Shamsudin H. M. Amin
Computational Intelligence and Neuroscience, 2016, Volume 2016, Page 1
Yulia Sandamirskaya and Mikhail Burtsev
Biologically Inspired Cognitive Architectures, 2015

Comments (0)

Please log in or register to comment.
Log in