Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Journal of Artificial Intelligence and Soft Computing Research

The Journal of Polish Neural Network Society, the University of Social Sciences in Lodz & Czestochowa University of Technology

4 Issues per year

Open Access
Online
ISSN
2083-2567
See all formats and pricing
More options …

Using Particle Swarm Optimization to Accurately Identify Syntactic Phrases in Free Text

George Tambouratzis
  • Dept. of Machine Translation, Institute for Language and Speech Processing / Athena Research Centre, 6 Artemidos & Epidavrou Str., Paradissos Amaroussiou, 151 25, Greece
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2017-11-01 | DOI: https://doi.org/10.1515/jaiscr-2018-0004

Abstract

The present article reviews the application of Particle Swarm Optimization (PSO) algorithms to optimize a phrasing model, which splits any text into linguistically-motivated phrases. In terms of its functionality, this phrasing model is equivalent to a shallow parser. The phrasing model combines attractive and repulsive forces between neighbouring words in a sentence to determine which segmentation points are required. The extrapolation of phrases in the specific application is aimed towards the automatic translation of unconstrained text from a source language to a target language via a phrase-based system, and thus the phrasing needs to be accurate and consistent to the training data.

Experimental results indicate that PSO is effective in optimising the weights of the proposed parser system, using two different variants, namely sPSO and AdPSO. These variants result in statistically significant improvements over earlier phrasing results. An analysis of the experimental results leads to a proposed modification in the PSO algorithm, to prevent the swarm from stagnation, by improving the handling of the velocity component of particles. This modification results in more effective training sequences where the search for new solutions is extended in comparison to the basic PSO algorithm. As a consequence, further improvements are achieved in the accuracy of the phrasing module.

Keywords: parsing of natural language; machine translation; syntactically-derived phrasing; particle swarm optimization (PSO); parameter optimization; Adaptive PSO (AdPSO)

References

  • [1] Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, Robert L. Mercer, The Mathematics of Statistical Machine Translation: Parameter Estimation. Computational Linguistics, Vol. 19, No.2, pp. 263-311, 1993.Google Scholar

  • [2] Philipp Koehn, Statistical Machine Translation. Cambridge University Press, Cambridge, 2010.Google Scholar

  • [3] Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, Yoshua Bengio, On the properties of neural machine translation: Encoder–decoder approaches. Proceedings of Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, Doha, Qatar, 2014.Google Scholar

  • [4] George Tambouratzis, Sokratis Sofianopoulos Marina Vassiliou, Language-independent hybrid MT with PRESEMT. Proceedings of 2nd HYTRA Workshop, held within the ACL-2013 Conference, Sofia, Bulgaria, pp. 123-130, 2013.Google Scholar

  • [5] James Kennedy, Russel C. Eberhart, Particle Swarm Optimization. Proceedings of the IEEE International Conference on Neural Networks, Perth, Australia, November, pp. 1942-1947, 1995.Google Scholar

  • [6] John Lafferty, Andrew McCallum, Fernando C.N. Pereira, Conditional Random Fields: Probabilistic Models for Segmenting and Labelling Sequence Data. Proceedings of ICML Conference, Williamstown, USA, pp. 282-289, 2001.Google Scholar

  • [7] Fei Sha, Fernando C.N. Pereira, Shallow Parsing with Conditional Random Fields. Proceedings of HLT-NAACL Conference, pp. 213-220, 2003.Google Scholar

  • [8] J.R. Finkel, A. Kleeman and C.D. Manning, Efficient, Feature-Based, Conditional Random Field Parsing. Proceedings of ACL-2008 Meeting, Columbus, Ohio, USA, pp. 959-967, 2008.Google Scholar

  • [9] Yoshimasa Tsuruoka, J. Tsujii, Sophia Ananiadou, Fast Full Parsing by Linear-Chain Conditional Random Fields. Proceedings of the 12th EACL Conference, Athens, Greece, pp. 790–798, 2009.Google Scholar

  • [10] Greg Durrett, Dan Klein, Neural CRF Parsing. Proceedings of the 53rd ACL Meeting, Beijing, China, pp. 302-312, 2015.Google Scholar

  • [11] George Tambouratzis, Conditional Random Fields versus template-matching in MT phrasing tasks involving sparse training data. Pattern Recognition Letters, Vol. 53, pp. 44-52, 2015.Web of ScienceGoogle Scholar

  • [12] Elva J.H. Robinson, Francis L.W. Ratnieks, M. Holcombe, An agent-based model to investigate the roles of attractive and repellent pheromones in ant decision making during foraging. Journal of Theoretical Biology, Vol. 255, pp. 250-258, 2008.Web of ScienceGoogle Scholar

  • [13] Changsheng Zhang, Jigui Sun, An Alternate two phases particle swarm optimization algorithm for flow shop scheduling problem. Expert Systems with Applications, Vol. 36, pp. 5162-5167, 2009.CrossrefGoogle Scholar

  • [14] S.X. Yu, J. Shi, Segmentation with Pairwise Attraction and Repulsion. Proceedings of ICCV-2001 Conference, Vancouver, Canada, pp. 52–58, 2001.Google Scholar

  • [15] Karlheinz Stber, Petra Wagner, David Stall, Jens Helbig, Matthias Thomae, Jens Blauert, Wolfgang Hess, H. Mangold, Speech Synthesis by Multilevel Selection and Concatenation of Units from Large Speech Corpora. Verbmobil: Foundations of Speech-to-Speech Translation, Symbolic Computation, Springer, Berlin, pp. 519–537, 2000.Google Scholar

  • [16] George Tambouratzis, Applying PSO to Natural Language Processing Tasks: Optimizing the Identification of Syntactic Phrases. Proceedings of CEC-2016 Conference, Vancouver, Canada, pp. 1831-1838, 2016.Google Scholar

  • [17] Rui Mendes, James Kennedy, Jose Neves, The Fully Informed Particle Swarm: Simpler, Maybe Better. IEEE Transactions on Evolutionary Computation, 8 (3), 204-210, 2004.CrossrefGoogle Scholar

  • [18] Moayed Daneshyari, Gary G. Yen, Cultural-Based Multiobjective Particle Swarm Optimization. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, Vol. 41, No. 2, pp.553-567, 2011.Google Scholar

  • [19] Xin Chen, Yangmin Li, A Modified PSO Structure Resulting in High Exploration Ability with Convergence Guaranteed. IEEE Transactions on Systems, Man & Cybernetics – Part B: Cybernetics, Vol. 37, No. 5, pp. 1271-1289, 2007.Google Scholar

  • [20] Jens Gimmler, Thomas Sttzle, Thomas E. Exner, Hybrid Particle Swarm Optimization: An examination of the influence of iterative improvement algorithms on its behavior. Proceedings of ANTS-2006 Workshop; Brussels, Belgium, 4-7 September, Lecture Notes in Computer Science, Vol. 4150, pp. 436-443, Springer, 2006.Google Scholar

  • [21] Richard P. Brent, Algorithms for Minimization without Derivatives, Englewood Cliffs, NJ, Prentice-Hall, Chapter 5, 1973.Google Scholar

  • [22] William H. Press, Saul A. Teukolsky, W.T. Vetterling, Brian P. Flannery, Numerical Recipes in Fortran 77: The Art of Scientific Computing. New York, U.S.A., Cambridge University Press, Chapter 7, 2002.Google Scholar

  • [23] James Kennedy, Rui Mendes, Neighborhood Topologies in Fully Informed and Best-of-Neighborhood Particle Swarms. IEEE Transactions on Systems, Man & Cybernetics – Part C: Applications and Reviews, Vol. 36, No. 4, pp. 515-519, 1996.Google Scholar

  • [24] Zhi-Hui Zhan, Jun Zhang, Yun Li, Henry Shu-Hung Chung, Adaptive Particle Swarm Optimization. IEEE Transactions on Systems, Man & Cybernetics – Part B: Cybernetics, Vol. 39, No.6, pp. 1362-1381, 2009.Google Scholar

  • [25] Helmut Schmid, Probabilistic part-of-speech tagging using decision trees. Proceedings of International Conference on New Methods in Language Processing, Manchester, UK. 1994.Google Scholar

  • [26] Helmut Schmid, Improvements in part-of speech tagging with an application to german. Proceedings of the ACL SIGDAT-Workshop. Dublin, Ireland, Association for Computing Machinery, 1995.Google Scholar

  • [27] Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu, BLEU: A Method for Automatic Evaluation of Machine Translation. 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, USA, pp. 311-318, 2002.Google Scholar

  • [28] NIST, 2002. Automatic Evaluation of Machine Translation Quality Using n-gram Co-occurrences Statistics.Google Scholar

  • [29] Yuhui Shi, Russel C. Eberhart, Parameter selection in particle swarm optimization, In: Proceedings of the 7th International Conference on Evolutionary Programming, New York, U.S.A., pp. 591-600, 1998.Google Scholar

  • [30] Maurice Clerc, James Kennedy, The particle swarm - explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation, vol. 6, no. 1, pp. 58-73, 2002.Google Scholar

  • [31] Jos Garca-Nieto, Enrique Alba, Restart particle swarm optimization with velocity modulation: a scalability test. Soft Computing, Vol.15, pp. 2221-2232, 2010.Web of ScienceGoogle Scholar

  • [32] George I. Evers, Mounir Ben Ghalia, Regrouping particle swarm optimization: A new global optimization algorithm with improved performance consistency across benchmarks. IEEE International Conference on Systems, Man and Cybernetics, San Antonio, Texas, 11-14 October, pp. 3901-3908, 2009.Google Scholar

  • [33] Sabine Helwig, Frank Neumann, Rolf Wanka, Particle swarm optimization with velocity adaptation. International Conference on Adaptive and Intelligent Systems (ICAIS’09), 24 September, pp. 146-151, 2009.Google Scholar

  • [34] Xin Chen, Yangmin Li, A Modified PSO Structure Resulting in High Exploration Ability with Convergence Guaranteed. IEEE Transactions on Systems, Man & Cybernetics – Part B: Cybernetics, Vol. 37, No. 5, pp. 1271-1289, 2007.Google Scholar

  • [35] Lili Liu, Shengxiang Yang, Dingway Wang, Particle Swarm Optimization with Composite Particles in Dynamic Environments. IEEE Transactions on Systems, Man, And Cybernetics—Part B: Cybernetics, Vol. 40, No. 6, pp. 1634-1648, 2010.CrossrefGoogle Scholar

About the article

Received: 2017-01-17

Accepted: 2017-03-29

Published Online: 2017-11-01

Published in Print: 2018-01-01


Citation Information: Journal of Artificial Intelligence and Soft Computing Research, Volume 8, Issue 1, Pages 63–77, ISSN (Online) 2083-2567, DOI: https://doi.org/10.1515/jaiscr-2018-0004.

Export Citation

© 2018 George Tambouratzis, published by De Gruyter Open. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. BY-NC-ND 4.0

Comments (0)

Please log in or register to comment.
Log in