Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Journal of Artificial Intelligence and Soft Computing Research

The Journal of Polish Neural Network Society, the University of Social Sciences in Lodz & Czestochowa University of Technology

4 Issues per year

Open Access
Online
ISSN
2083-2567
See all formats and pricing
More options …

An English Neural Network that Learns Texts, Finds Hidden Knowledge, and Answers Questions

Yuanzhi Ke
  • Corresponding author
  • Graduate School of Science and Technology, Keio University, 3-14-1, Hiyoshi, Kohokuku, Yokohama, Japan
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Masafumi Hagiwara
  • Graduate School of Science and Technology, Keio University, 3-14-1, Hiyoshi, Kohokuku, Yokohama, Japan
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2017-05-03 | DOI: https://doi.org/10.1515/jaiscr-2017-0016

Abstract

In this paper, a novel neural network is proposed, which can automatically learn and recall contents from texts, and answer questions about the contents in either a large corpus or a short piece of text. The proposed neural network combines parse trees, semantic networks, and inference models. It contains layers corresponding to sentences, clauses, phrases, words and synonym sets. The neurons in the phrase-layer and the word-layer are labeled with their part-of-speeches and their semantic roles. The proposed neural network is automatically organized to represent the contents in a given text. Its carefully designed structure and algorithms make it able to take advantage of the labels and neurons of synonym sets to build the relationship between the sentences about similar things. The experiments show that the proposed neural network with the labels and the synonym sets has the better performance than the others that do not have the labels or the synonym sets while the other parts and the algorithms are the same. The proposed neural network also shows its ability to tolerate noise, to answer factoid questions, and to solve single-choice questions in an exercise book for non-native English learners in the experiments.

Keywords: natural language processing; neural network; question answering; natural language understanding

References

  • [1] E. Brill, A simple rule-based part of speech tagger, In Proceedings of the Workshop on Speech and Natural Language, HLT’91, pp. 112-116, Association for Computational Linguistics, Stroudsburg, PA, USA, 1992Google Scholar

  • [2] C. D. Manning and H. Sch¨utze, Foundations of statistical natural language processing, MIT press, 1999Google Scholar

  • [3] C. E. Shannon, A mathematical theory of communication, SIGMOBILE Mob. Comput. Commun. Rev., vol. 5, no. 1, pp. 3-55, 2001Google Scholar

  • [4] N. Chomsky, Three models for the description of language, Information Theory, IRE Transactions on, vol. 2, no. 3, pp.113-124, 1956Google Scholar

  • [5] W. A. Gale, K. W. Church, and D. Yarowsky, Work on statistical methods for word sense disambiguation, In Working Notes of the AAAI Fall Symposium on Probabilistic Approaches to Natural Language, vol. 54, p. 60. 1992Google Scholar

  • [6] J. Kupiec, Robust part-of-speech tagging using a hidden markov model, Computer Speech & Language, vol. 6, no. 3, pp. 225 - 242, 1992Google Scholar

  • [7] H. Schmid, Probabilistic part-of-speech tagging using decision trees, in Proceedings of the international conference on new methods in language processing, vol. 12, pp. 44-49. Citeseer, 1994Google Scholar

  • [8] A. Ratnaparkhi et al, A maximum entropy model for part-of-speech tagging, in Proceedings of the conference on empirical methods in natural language processing, vol. 1, pp. 133-142. Philadelphia, USA, 1996Google Scholar

  • [9] P. F. Brown, V. J. D. Pietra, S. A. D. Pietra, and R. L. Mercer, The mathematics of statistical machine translation: Parameter estimation, Computational linguistics, vol. 19, no. 2, pp. 263-311, 1993Google Scholar

  • [10] H. Hotta, M. Kittaka, and M. Hagiwara,Word vectorization using relations among words for neural network, IEEJ Transactions on Electronics, Information and Systems, vol. 130, pp. 75-82, 2010Google Scholar

  • [11] G. Tsatsaronis, I. Varlamis, and M. Vazirgiannis, Text relatedness based on a word thesaurus, Journal of Artificial Intelligence Research, vol. 37, no. 1, pp. 1-40, 2010Google Scholar

  • [12] G. Tsatsaronis, I. Varlamis, and M. Vazirgiannis, Word sense disambiguation with semantic networks, In Text, Speech and Dialogue, pp. 219-226. Springer, 2008Google Scholar

  • [13] R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa, Natural language processing (almost) from scratch, The Journal of Machine Learning Research, vol. 12, pp. 2493-2537, 2011Google Scholar

  • [14] T. Sagara and M. Hagiwara, Natural language neural network and its application to questionanswering system, Neurocomputing, vol. 142, pp. 201 - 208, 2014Google Scholar

  • [15] L. Dong, F. Wei, M. Zhou, and K. Xu, Question answering over Freebase with multi-column convolutional neural networks, in Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp. 260-269. Association for Computational Linguistics, Beijing, China, July 2015Google Scholar

  • [16] D. Bahdanau, K. Cho, and Y. Bengio, Neural machine translation by jointly learning to align and translate, CoRR, vol. abs/1409.0473, 2014Google Scholar

  • [17] J. E. Hummel and K. J. Holyoak, Distributed representations of structure: A theory of analogical access and mapping, Psychological Review, vol. 104, no. 3, p. 427, 1997Google Scholar

  • [18] J. E. Hummel and K. J. Holyoak, A symbolicconnectionist theory of relational inference and generalization, Psychological review, vol. 110, no. 2, p. 220, 2003Google Scholar

  • [19] J. E. Hummel and K. J. Holyoak, Relational reasoning in a neurally plausible cognitive architecture an overview of the LISA project, Current Directions in Psychological Science, vol. 14, no. 3, pp. 153-157, 2005Google Scholar

  • [20] M. Saito and M. Hagiwara, Natural language processing neural network for analogical inference, In The 2010 International Joint Conference on Neural Networks, pp.1-7, 2010Google Scholar

  • [21] T. Kudo and H. Kazawa, Web Japanese N-gram version 1, Gengo Shigen Kyokai, vol. 14, 2007Google Scholar

  • [22] M. Fukuda, S. Nobesawa, and I. Tahara, Knowledge representation for query-answer, In Forum on Information Technology, vol. 3, pp. 233-236, Information Processing Society of Japan, 2004Google Scholar

  • [23] S. Ikehara, M. Miyazaki, S. Shirai, A. Yokoo, H. Nakaiwa, K. Ogura, Y. Ooyama, and Y. Hayashi, GoiTaikei-A Japanese Lexicon, Iwanami Shoten, 1997Google Scholar

  • [24] T. Kudo, K. Yamamoto, and Y. Matsumoto, Applying conditional random fields to Japanese morphological analysis, in Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, vol. 2004, pp. 230-237. 2004Google Scholar

  • [25] G. A. Miller, WordNet: a lexical database for english, Communications of the ACM, vol. 38, no. 11, pp. 39-41, 1995Google Scholar

  • [26] G. Miller and C. Fellbaum, WordNet: An electronic lexical database, 1998Google Scholar

  • [27] M. P. Marcus, M. A. Marcinkiewicz, and B. Santorini, Building a large annotated corpus of english: The Penn Treebank, Comput. Linguist., vol. 19, no. 2, pp. 313-330, June 1993Google Scholar

  • [28] M. Marcus, G. Kim, M. A. Marcinkiewicz, R. MacIntyre, A. Bies, M. Ferguson, K. Katz, and B. Schasberger, The Penn Treebank: Annotating predicate argument structure, in Proceedings of the Workshop on Human Language Technology, HLT ’94, pp. 114-119, Association for Computational Linguistics, Stroudsburg, PA, USA, 1994Google Scholar

  • [29] P. K. Martha and M. Palmer, From Treebank to Propbank, in Proceedings of the International Conference on Language Resources and Evaluation 2002, Las Palmas, Canary Islands, Spain, 2002Google Scholar

  • [30] P. Kingsbury, M. Palmer, and M. Marcus, Adding semantic annotation to the penn treebank, in Proceedings of the Human Language Technology Conference, pp. 252-256, Citeseer, 2002Google Scholar

  • [31] M. Palmer, D. Gildea, and P. Kingsbury, The proposition bank: An annotated corpus of semantic roles, Comput. Linguist., vol. 31, no. 1, pp. 71-106, March 2005Google Scholar

  • [32] P. E.Woodford, The test of english for international communication (TOEIC), 1980Web of ScienceGoogle Scholar

  • [33] National Institute of Standards and Technology, NIST TREC Document Database: Disk 4, Retrieved June 25, 2016, from http://www.nist.gov/srd/nistsd22.cfmGoogle Scholar

  • [34] National Institute of Standards and Technology, NIST TREC Document Database: Disk 5, Retrieved June 25, 2016, from http://www.nist.gov/srd/nistsd23.cfmGoogle Scholar

  • [35] E. M. Voorhees et al. The TREC-8 question answering track report, in Proceedings of the 8th Text Retreval Conference, vol. 99, pp.77-82. NIST, Gaithersburg, MD, 1999Google Scholar

  • [36] L. Loungheed, Longman preparation series for the new TOEIC test: More practice tests, 2006Google Scholar

  • [37] T. S. Committee et al., TOEIC program data & analysis, 2014Google Scholar

  • [38] E. T. Service, Examinee handbook listening & reading, 2008Google Scholar

About the article

Received: 2016-05-18

Accepted: 2016-09-14

Published Online: 2017-05-03

Published in Print: 2017-10-01


Citation Information: Journal of Artificial Intelligence and Soft Computing Research, Volume 7, Issue 4, Pages 229–242, ISSN (Online) 2083-2567, DOI: https://doi.org/10.1515/jaiscr-2017-0016.

Export Citation

© 2017. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. BY-NC-ND 4.0

Comments (0)

Please log in or register to comment.
Log in