Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Peace Economics, Peace Science and Public Policy

Editor-in-Chief: Caruso, Raul

Ed. by Bove, Vincenzo / Kibris, Arzu / Sekeris, Petros

CiteScore 2018: 0.44

SCImago Journal Rank (SJR) 2018: 0.281
Source Normalized Impact per Paper (SNIP) 2018: 0.320

See all formats and pricing
More options …
Volume 24, Issue 4


Volume 17 (2011)

Volume 4 (1996)

Volume 3 (1995)

Volume 2 (1994)

Volume 1 (1993)

Beyond a Bag of Words: Using PULSAR to Extract Judgments on Specific Human Rights at Scale

Baekkwan Park
  • University of Pittsburgh, Department of Political Science, Pittsburgh, United States of America
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Michael Colaresi
  • Corresponding author
  • University of Pittsburgh, Department of Political Science, Pittsburgh, United States of America
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Kevin Greene
  • University of Pittsburgh, Department of Political Science, Pittsburgh, United States of America
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2018-10-20 | DOI: https://doi.org/10.1515/peps-2018-0030


Sentiment, judgments and expressed positions are crucial concepts across international relations and the social sciences more generally. Yet, contemporary quantitative research has conventionally avoided the most direct and nuanced source of this information: political and social texts. In contrast, qualitative research has long relied on the patterns in texts to understand detailed trends in public opinion, social issues, the terms of international alliances, and the positions of politicians. Yet, qualitative human reading does not scale to the accelerating mass of digital information available currently. Researchers are in need of automated tools that can extract meaningful opinions and judgments from texts. Thus, there is an emerging opportunity to marry the model-based, inferential focus of quantitative methodology, as exemplified by ideal point models, with high resolution, qualitative interpretations of language and positions. We suggest that using alternatives to simple bag of words (BOW) representations and re-focusing on aspect-sentiment representations of text will aid researchers in systematically extracting people’s judgments and what is being judged at scale. The experimental results below show that our approach which automates the extraction of aspect and sentiment MWE pairs, outperforms BOW in classification tasks, while providing more interpretable parameters. By connecting expressed sentiment and the aspects being judged, PULSAR (Parsing Unstructured Language into Sentiment-Aspect Representations) also has deep implications for understanding the underlying dimensionality of issue positions and ideal points estimated with text. Our approach to parsing text into aspects-sentiment expressions recovers both expressive phrases (akin to categorical votes), as well as the aspects that are being judged (akin to bills). Thus, PULSAR or future systems like it, open up new avenues for the systematic analysis of high-dimensional opinions and judgments at scale within existing ideal point models.

Keywords: text analysis; Human Rights; Natural Language Processing


  • Bespalov, D., Bai, B., Qi, Y., & Shokoufandeh, A. (2011). Sentiment classification based on supervised latent n-gram analysis. Proceedings of the 20th ACM international conference on Information and knowledge management – CIKM ’11. URL: http://dx.doi.org/10.1145/2063576.2063635

  • Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of Machine Learning Research, 3, 993–1022.Google Scholar

  • Brandt, P. T., Freeman, J. R., & Schrodt, P. A. (2014). Evaluating forecasts of political conflict dynamics. International Journal of Forecasting, 30(4), 944–962.CrossrefWeb of ScienceGoogle Scholar

  • Calzolari, N., Fillmore, C. J., Grishman, R., Ide, N., Lenci, A., MacLeod C., & Zampolli, A. (2002). Towards best practice for multiword expressions in computational lexicons. In LREC.Google Scholar

  • Cingranelli, D. L., Richards, D. L., & Clay, K. C. (2014). The CIRI human rights dataset. v.2014.04.14.Google Scholar

  • Colaresi, M., & Mahmood, Z. (2017). Do the robot: Lessons from machine learning to improve conflict forecasting. Journal of Peace Research, 54(2), 193–214.CrossrefWeb of ScienceGoogle Scholar

  • Feldman, R. (2013). Techniques and applications for sentiment analysis. Communications of the ACM, 56(4), 82–89. URL: http://dx.doi.org/10.1145/2436256.2436274CrossrefWeb of Science

  • Gibney, M., Cornett, L., Wood, R., Haschke, P., & Arnon, D. (2015). The political terror scale 1976–2015. Date Retrieved, from the Political Terror Scale website: http://www.politicalterrorscale.org.

  • Handler, A., Denny, M., Wallach, H., & O’Connor, B. (2016). Bag of what? Simple noun phrase extraction for text analysis. In Proceedings of the First Workshop on NLP and Computational Social Science. pp. 114–124.Google Scholar

  • Ho, D. E., & Quinn, K. M. (2008). Measuring explicit political positions of media. Quarterly Journal of Political Science, 3(4), 353–377.CrossrefWeb of ScienceGoogle Scholar

  • Justeson, J. S., & Katz, S. M. (1995). Technical terminology: some linguistic properties and an algorithm for identification in text. Natural Language Engineering, 1(1), 9–27.Google Scholar

  • Laver, M., Benoit, K., & Garry, J. (2003). Extracting policy positions from political texts using words as data. American Political Science Review, 97(1), 311–331.Google Scholar

  • Liu, B. (2015). Sentiment Analysis: Mining Opinions, Sentiments and Emotions. New York: Cambridge University Press.Google Scholar

  • Liu, Q., Gao, Z., Liu, B., & Zhang, Y. (2013). A logic programming approach to aspect extraction in opinion mining. In Web Intelligence (WI) and Intelligent Agent Technologies (IAT), 2013 IEEE/WIC/ACM International Joint Conferences. Vol. 1 IEEE pp. 276–283.Google Scholar

  • Liu, Q., Gao, Z., Liu, B., & Zhang, Y. (2015). Automated rule selection for aspect extraction in opinion mining. In International Joint Conference on Artificial Intelligence (IJCAI).Google Scholar

  • Lowe, W. (2013). There’s (basically) only one way to do it. Available at SSRN.Google Scholar

  • Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S. J., & McClosky, D. (2014). The stanford CoreNLP natural language processing toolkit. In Association for Computational Linguistics (ACL) System Demonstrations. pp. 55–60. URL: http://www.aclweb.org/anthology/P/P14/P14-5010

  • Monroe, B. L., & Maeda, K. (2004). Talk’s cheap: Text-based ideal point estimation. In presented to the Political Methodology Society. Palo Alto, CA.Google Scholar

  • Monroe, B. L., Colaresi, M. P., & Quinn, K. M. (2008). ‘Fightin’ words: Lexical feature selection and evaluation for identifying the content of political conflict. Political Analysis, 16(4), 372–403.CrossrefWeb of ScienceGoogle Scholar

  • Pang, B., & Lee, L. (2008). Opinion mining and sentiment analysis. Foundations and Trends in Informational Retrieval, 2(1-2), 1–135.CrossrefGoogle Scholar

  • Pang, B., Lee, L., & Vaithyanathan, S. (2002). Thumbs up? Proceedings of the ACL-02 conference on Empirical methods in natural language processing – EMNLP ’02. URL: http://dx.doi.org/10.3115/1118693.1118704

  • Qiu, G., Liu, B., Bu, J., Chen, C. (2011). Opinion word expansion and target extraction through double propagation. Computational Linguistics, 37(1), 9–27.CrossrefWeb of ScienceGoogle Scholar

  • Quinn, K. M., Monroe, B. L., Colaresi, M., Crespin, M., & Radev, D. R. (2010). How to analyze political attention with minimal assumptions and costs. American Journal of Political Science, 54(1), 209–228.Web of ScienceCrossrefGoogle Scholar

  • Sag, I. A., Baldwin, T., Bond, F., Copestake, A., & Flickinger, D. (2002). Multiword expressions: a pain in the neck for NLP. In International Conference on Intelligent Text Processing and Computational Linguistics. Springer pp. 1–15.Google Scholar

  • Schrodt, P. A., Beieler, J., & Idris, M. (2014). Three’sa charm?: Open event data coding with el: Diablo, Petrarch, and the open event data alliance. In ISA Annual Convention.Google Scholar

  • Slapin, J. B., & Proksch, S.-O. (2008). A scaling model for estimating time-series party positions from texts. American Journal of Political Science, 52(3), 705–722.CrossrefWeb of ScienceGoogle Scholar

  • Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013). Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing. pp. 1631–1642.Google Scholar

  • Socher, R., Huval, B., Manning, C. D., & Ng, A. Y. (2012). Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning. Association for Computational Linguistics pp. 1201–1211.Google Scholar

  • Socher, R., Lin, C. C., Manning, C., & Ng, A. Y. (2011). Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 28th international conference on machine learning (ICML-11). pp. 129–136.Google Scholar

  • Taylor, A., Marcus, M., & Santorini, B. (2003). The Penn treebank: an overview. In A. Abeillé (Ed.), Treebanks (pp. 5–22). Dordrecht: Springer.Google Scholar

  • Toutanova, K., Klein, D., Manning, C. D., & Singer, Y. (2003). Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1. Association for Computational Linguistics pp. 173–180.Google Scholar

  • Wallach, H. M. (2006). Topic modeling. Proceedings of the 23rd international conference on Machine learning – ICML ’06. URL: http://dx.doi.org/10.1145/1143844.1143967

  • Wu, Y., Zhang, Q., Huang, X., & Wu, L. (2009). Phrase dependency parsing for opinion mining. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing. Vol. 3 Association for Computational Linguistics pp. 1533–1541.Google Scholar

  • Yessenalina, A., & Cardie, C. (2011). Compositional matrix-space models for sentiment analysis. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. EMNLP ’11 Stroudsburg, PA, USA: Association for Computational Linguistics pp. 172–182. URL: http://dl.acm.org/citation.cfm?id=2145432.2145452

About the article

Published Online: 2018-10-20

Citation Information: Peace Economics, Peace Science and Public Policy, Volume 24, Issue 4, 20180030, ISSN (Online) 1554-8597, DOI: https://doi.org/10.1515/peps-2018-0030.

Export Citation

©2018 Walter de Gruyter GmbH, Berlin/Boston.Get Permission

Comments (0)

Please log in or register to comment.
Log in