Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Paladyn, Journal of Behavioral Robotics

Editor-in-Chief: Schöner, Gregor

Covered by SCOPUS

CiteScore 2018: 2.17

SCImago Journal Rank (SJR) 2018: 0.336
Source Normalized Impact per Paper (SNIP) 2018: 1.707

ICV 2018: 120.52

Open Access
See all formats and pricing
More options …

Parsing of action sequences: A neural dynamics approach

David Lobato / Yulia Sandamirskaya / Mathis Richter / Gregor Schöner
Published Online: 2015-05-15 | DOI: https://doi.org/10.1515/pjbr-2015-0008


Parsing of action sequences is the process of segmenting observed behavior into individual actions. In robotics, this process is critical for imitation learning from observation and for representing an observed behavior in a form that may be communicated to a human. In this paper, we develop a model for action parsing, based on our understanding of principles of grounded cognitive processes, such as perceptual decision making, behavioral organization, and memory formation.We present a neural-dynamic architecture, in which action sequences are parsed using a mathematical and conceptual framework for embodied cognition—the Dynamic Field Theory. In this framework, we introduce a novel mechanism, which allows us to detect and memorize actions that are extended in time and are parametrized by the target object of an action. The core properties of the architecture are demonstrated in a set of simple, proof-of-concept experiments.

Keywords : action parsing; sequence learning; elementary behaviors; dynamic neural fields


  • [1] Gutemberg Guerra-Filho and Yiannis Aloimonos. The syntax of human actions and interactions. Journal of Neurolinguistics, 25(5):500–514, 2012. CrossrefWeb of ScienceGoogle Scholar

  • [2] Yiannis Aloimonos. Sensory grammars for sensor networks. Journal of Ambient Intelligence and Smart Environments, 1(1):15–21, January 2009. Web of ScienceGoogle Scholar

  • [3] Florentin Wörgötter, Eren Erdal Aksoy, Norbert Krüger, Justus Piater, Ales Ude, and Minija Tamosiunaite. A simple ontology of manipulation actions based on hand-object relations. IEEE Transactions on Autonomous Mental Development, 5(2):117– 134, june 2013. Google Scholar

  • [4] Georg Layher, Martin A Giese, and Heiko Neumann. Learning representations of animated motion sequences—A neural model. Topics in Cognitive Science, 6(1):170–182, January 2014. Google Scholar

  • [5] Gregor Schöner. Dynamical systems approaches to cognition. In Ron Sun, editor, Cambridge Handbook of Computational Cognitive Modeling, pages 101–126, Cambridge, UK, 2008. Cambridge University Press. Google Scholar

  • [6] Yulia Sandamirskaya. Dynamic neural fields as a step towards cognitive neuromorphic architectures. Frontiers in Neuroscience, 7:276, 2013. Web of ScienceGoogle Scholar

  • [7] Yulia Sandamirskaya and Gregor Schöner. An embodied account of serial order: How instabilities drive sequence generation. Neural Networks, 23(10):1164–1179, December 2010. CrossrefWeb of ScienceGoogle Scholar

  • [8] Yulia Sandamirskaya and Gregor Schöner. Serial order in an acting system: a multidimensional dynamic neural fields implementation. In 9th IEEE International Conference on Development and Learning (ICDL 2010), 2010. Google Scholar

  • [9] Mathis Richter, Yulia Sandamirskaya, and Gregor Schöner. A robotic architecture for action selection and behavioral organization inspired by human cognition. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2012), pages 2457–2464, 2012. Google Scholar

  • [10] Yulia Sandamirskaya, Stephan K.U. Zibner, Sebastian Schneegans, and Gregor Schöner. Using dynamic field theory to extend the embodiment stance toward higher cognition. New Ideas in Psychology, 31(3):322 – 339, 2013. Web of ScienceCrossrefGoogle Scholar

  • [11] Shun-ichi Amari. Dynamics of pattern formation in lateralinhibition type neural fields. Biological Cybernetics, 27:77–87, 1977. CrossrefGoogle Scholar

  • [12] Hugh R Wilson and Jack D Cowan. A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue. Kybernetik, 13(2):55–80, 1973. CrossrefGoogle Scholar

  • [13] Stephen Grossberg. Nonlinear neural networks: Principles, mechanisms, and architectures. Neural Networks, 1(1):17–61, 1988. CrossrefGoogle Scholar

  • [14] Stephan K U Zibner, Christian Faubel, Ioannis Iossifidis, and Gregor Schöner. Dynamic Neural Fields as Building Blocks for a Cortex-Inspired Architecture of Robotic Scene Representation. IEEE Transactions on AutonomousMental Development, 3(1):74– 91, 2011. Web of ScienceGoogle Scholar

  • [15] Y Sandamirskaya and T Storck. Neural-dynamic architecture for looking: Shift from visual to motor target representation for memory saccade. In IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL EPIROB 2014), 2014. Google Scholar

  • [16] Radu Bogdan Rusu and Steve Cousins. 3D is here: Point Cloud Library (PCL). IEEE International Conference on Robotics and Automation (ICRA 2011), pages 1–4, May 2011. Google Scholar

  • [17] Martin A. Fischler and Robert C. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM, 24(6):381–395, June 1981. CrossrefGoogle Scholar

  • [18] Michael Berger, Christian Faubel, Joseph Norman, Howard Hock, and Gregor Schöner. The Counter-Change Model of Motion Perception : An Account Based on Dynamic Field Theory The Dynamic Field Model of Counter-Change Motion. Lecture Notes in Computer Science, 7552:579–586, 2012. Google Scholar

  • [19] Rudolph Emil Kalman. A new approach to linear filtering and prediction problems. Transactions of the American Society of Mechanical Engineers - Journal of Basic Engineering, 82(1):35– 45, 1960. CrossrefGoogle Scholar

  • [20] HaroldWKuhn. The hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1-2):83–97, 1955. CrossrefGoogle Scholar

  • [21] James Munkres. Algorithms for the assignment and transportation problems. Journal of the Society for Industrial & Applied Mathematics, 5(1):32–38, 1957. Google Scholar

  • [22] Christian Faubel and Gregor Schöner. Learning to recognize objects on the fly: a neurally based dynamic field approach. Neural Networks, 21(4):562–576, 2008. Web of ScienceCrossrefGoogle Scholar

  • [23] Yulia Sandamirskaya, Mathis Richter, and Gregor Schöner. A neural-dynamic architecture for behavioral organization of an embodied agent. In IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL EPIROB 2011), pages 1–7, 2011. Google Scholar

  • [24] Boris Duran and Yulia Sandamirskaya. Neural dynamics of hierarchically organized sequences: A robotic implementation. In 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012), pages 357–362, 2012. Google Scholar

  • [25] Kyuhwa Lee and Yiannis Demiris. Towards incremental learning of task-dependent action sequences using probabilistic parsing. IEEE International Conference of Development and Learning (ICDL), 2011. Google Scholar

  • [26] Falk Fleischer, Vittorio Caggiano, Peter Thier, and Martin a Giese. Physiologically inspired model for the visual recognition of transitive hand actions. The Journal of neuroscience : the official journal of the Society for Neuroscience, 33(15):6563–80, April 2013. CrossrefWeb of ScienceGoogle Scholar

  • [27] Sohrob Kazerounian, Matthew Luciw, Mathis Richter, and Yulia Sandamirskaya. Autonomous reinforcement of behavioral sequences in neural dynamics. In International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2013. Google Scholar

  • [28] Brenna D. Argall, Sonia Chernova, Manuela Veloso, and Brett Browning. A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57(5):469–483, May 2009. Web of ScienceCrossrefGoogle Scholar

  • [29] Stefan Schaal. Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences, 3(6):233–242, 1999. Google Scholar

About the article

Received: 2014-03-31

Accepted: 2015-01-09

Published Online: 2015-05-15

Citation Information: Paladyn, Journal of Behavioral Robotics, Volume 6, Issue 1, ISSN (Online) 2081-4836, DOI: https://doi.org/10.1515/pjbr-2015-0008.

Export Citation

© 2015 David Lobato et al.. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. BY-NC-ND 3.0

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

Mathis Richter, Jonas Lins, and Gregor Schöner
Topics in Cognitive Science, 2017, Volume 9, Number 1, Page 35
Oliver Lomp, Mathis Richter, Stephan K. U. Zibner, and Gregor Schöner
Frontiers in Neurorobotics, 2016, Volume 10
Nadia Rasheed and Shamsudin H. M. Amin
Computational Intelligence and Neuroscience, 2016, Volume 2016, Page 1
Yulia Sandamirskaya and Mikhail Burtsev
Biologically Inspired Cognitive Architectures, 2015, Volume 13, Page 91

Comments (0)

Please log in or register to comment.
Log in