Jump to ContentJump to Main Navigation
Show Summary Details
More options …

The Prague Bulletin of Mathematical Linguistics

The Journal of Charles University

2 Issues per year

Open Access
See all formats and pricing
More options …

The Machine Translation Leaderboard

Post Matt / Lopez Adam
Published Online: 2014-09-11 | DOI: https://doi.org/10.2478/pralin-2014-0012


Much of an instructor's time is spent on the management and grading of homework. We present the Machine Translation Leaderboard, a platform for managing, displaying, and automatically grading homework assignments. It runs on Google App Engine, which provides hosting and user management services. Among its many features are the ability to easily define new assignments, manage submission histories, maintain a development / test set distinction, and display a leaderboard. An entirely new class can be set up in minutes with minimal configuration. It comes pre-packaged with five assignments used in a graduate course on machine translation.


  • Bojar, Ondřej, Christian Buck, Chris Callison-Burch, Christian Federmann, Barry Haddow, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia. Findings of the 2013 workshop on statistical machine translation. In Proc. of WMT, 2013.Google Scholar

  • Callison-Burch, Chris, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Spe-cia. Findings of the 2012 workshop on statistical machine translation. In Proc. of WMT, 2012.Google Scholar

  • DeNero, John and Dan Klein. The complexity of phrase alignment problems. In Proc. of ACL, 2008.Google Scholar

  • Hajič, Jan, Jarmila Panevová, Eva Hajičová, Petr Sgall, Petr Pajas, Jan Štčpánek, Jiří Havelka, Marie Mikulová, Zdenčk Žabokrtský, and Magda Ševčíková Razímová. Prague Dependency Treebank 2.0. LDC2006T01, Linguistic Data Consortium, Philadelphia, PA, USA, ISBN 1-58563-370-4, Jul 2006, 2006. URL http://ufal.mff.cuni.cz/pdt2.0/.Google Scholar

  • Koehn, Philipp, Franz J. Och, and Daniel Marcu. Statistical phrase-based translation. In Proc. of NAACL, 2003.Google Scholar

  • Lopez, Adam, Matt Post, Chris Callison-Burch, Jonathan Weese, Juri Ganitkevitch, Narges Ahmidi, Olivia Buzek, Leah Hanson, Beenish Jamil, Matthias Lee, Ya-Ting Lin, Henry Pao, Fatima Rivera, Leili Shahriyari, Debu Sinha, Adam Teichert, Stephen Wampler, Michael Weinberger, Daguang Xu, Lin Yang, and Shang Zhao. Learning to translate with products of novices: a suite of open-ended challenge problems for teaching MT. Transactions of the Association for Computational Linguistics, (1):165–178, 2013.Google Scholar

  • Mihalcea, Rada and Ted Pedersen. An evaluation exercise for word alignment. In Proc. on Workshop on Building and Using Parallel Texts, 2003.Google Scholar

  • Minkov, Einat, Kristina Toutanova, and Hisami Suzuki. Generating complex morphology for machine translation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, volume 45, pages 128–135, 2007.Google Scholar

  • Och, Franz Josef and Hermann Ney. Improved statistical alignment models. In Proc. of ACL, 2000.Google Scholar

About the article

*Matt Post Human Language Technology Center of Excellence Johns Hopkins University 810 Wyman Park Drive Baltimore, MD 21211

Published Online: 2014-09-11

Citation Information: The Prague Bulletin of Mathematical Linguistics, ISSN (Online) 1804-0462, DOI: https://doi.org/10.2478/pralin-2014-0012.

Export Citation

© 2014 Post Matt and Lopez Adam. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. BY-NC-ND 3.0

Comments (0)

Please log in or register to comment.
Log in