Published by De Gruyter

Volume 3 Issue 2

June 2012

Issue of Statistics, Politics and Policy

Submit manuscript

Contents
Journal Overview

Article

June 27, 2012

The Value Deming's Ideas Can Add to Educational Evaluation

Sharon L. Lohr

Abstract

Value-added models are being implemented in many states in an attempt to measure the contributions of individual teachers and schools toward students' learning. Scores from these models are increasingly used for high-stakes purposes such as selecting teachers to receive merit pay increases, determining which teachers are to be fired, closing schools, and allocating money to programs. The statistician W. Edwards Deming wrote extensively about improving quality in education and the damage caused by performance rankings. In this paper, Deming's writings are related to uses of value-added models in the United States. Some of the models used for value-added assessment are reviewed and assessed. A simulation study based on Deming's red bead experiment shows the effects of strategies teachers and schools might adopt to raise their value-added scores without changing their teaching

June 27, 2012

Monitoring Progress Towards Millennium Development Goal 4: A Call for Improved Validation of Under-Five Mortality Rate Estimates

Leontine Alkema, Maria B. Wong, Pei Rong Seah

Abstract

Background: Millennium Development Goal 4 calls for a reduction in the under-5 mortality rate (U5MR) by two-thirds between 1990 and 2015. Due to issues with data quality and quantity, there is uncertainty about how much progress has been made in many countries, and conflicting estimates of the overall levels and rate of decline have been reported. Consequently, there is a clear and urgent need to validate estimates, but consensus guidelines for model validation are lacking. Methods: We propose a validation method, based on the question: If current U5MR estimation methodology had been used in the past, using all available data available at that time, how accurate would the U5MR estimates and uncertainty intervals have been?Findings: Validation of the estimates and uncertainty intervals reported in 2010 by the Institute for Health Metrics and Evaluation showed that estimates of both the past absolute levels and the rate of decline in under-5 mortality between 1990 and 2000 would have changed significantly between 2004 and 2010 for many countries, with the inclusion of additional data. For high-mortality countries in particular, over 40% of the updated estimates for the U5MR in 2000 fell outside previously constructed 95% uncertainty intervals, and the absolute change in the estimated rate of decline was more than 1% for about three out of five countries.Interpretation: In light of evaluating Millennium Development Goal 4 in 2015, we hope that extensive model validation exercises will become standard practice for validating the accuracy of under-5 estimates and uncertainty intervals. Ultimately the goal of including appropriate validation methods is to improve the reliability of both estimates and uncertainty intervals, which will allow for a more fully informed assessment of both the absolute levels of, and rates of change in, the under-5 mortality rate.

July 13, 2012

A New Method for Deriving Global Estimates of Maternal Mortality

John R. Wilmoth, Nobuko Mizoguchi, Mikkel Z. Oestergaard, Lale Say, Colin D. Mathers, Sarah Zureick-Brown, Mie Inoue, Doris Chou

Abstract

Maternal mortality is widely regarded as a key indicator of population health and of social and economic development. Its levels and trends are monitored closely by the United Nations and others, inspired in part by the UN's Millennium Development Goals (MDGs), which call for a three-fourths reduction in the maternal mortality ratio between 1990 and 2015. Unfortunately, the empirical basis for such monitoring remains quite weak, requiring the use of statistical models to obtain estimates for most countries.In this paper we describe a new method for estimating global levels and trends in maternal mortality. For countries lacking adequate data for direct calculation of estimates, we employed a parametric model that separates maternal deaths related to HIV/AIDS from all others. For maternal deaths unrelated to HIV/AIDS, the model consists of a hierarchical linear regression with three predictors and variable intercepts for both countries and regions. The uncertainty of estimates was assessed by simulating the estimation process, accounting for variability both in the data and in other model inputs.The method was used to obtain the most recent set of UN estimates, published in September 2010. Here, we provide a concise description and explanation of the approach, including a new analysis of the components of variability reflected in the uncertainty intervals.Final estimates provide evidence of a more rapid decline in the global maternal mortality ratio than suggested by previous work, including another study published in April 2010. We compare findings from the two recent studies and discuss topics for further research to help resolve differences.

July 25, 2012

Teacher Effect Estimates and Decision Rules for Establishing Student-Teacher Linkages: What are the Implications for High-Stakes Personnel Policies in an Urban School District?

Bing Han, Daniel McCaffrey, Matthew G. Springer, Michael Gottfried

Abstract

Both research and practice of value-added models (VAM) have been growing in recent years due to the widespread effort to quantify teacher effectiveness. The existing VAM literature has not yet tested the sensitivity of value-added estimates to the rules that define which students contribute to each teacher’s value-added estimate. Student-teacher linkages are often a complex network due to various transfers, students taking multiple courses in the same subject, and students receiving special education or other “pull-out” services. Complex linkages are often considered as among the main threats to the validity of VAM. In this paper we conducted a case study to examine the sensitivity of VAM to the alternative link definitions. We examined three popular VAM approaches and applied alternative rules for linking students to teachers with each method. We found no overall sensitivity of estimated teacher effects to the linking rules. Even though more teachers had value-added estimates under more inclusive rules, for a teacher with estimates under all three rules, the correlation among pairs of estimates created using different linking rules was always above 0.95 and generally above 0.98 for each VAM approach. The value-added estimates of a small number of teachers were affected by highly different link definitions and these tended to be teachers with small numbers of students. Restricting the minimal sample size for calculating individual teachers’ value-added largely reduced the sensitivity in link definition.

August 3, 2012

Comprehension of Graphs and Tables Depend on the Task: Empirical Evidence from Two Web-Based Studies

Matthias Schonlau, Ellen Peters

Abstract

Graphs and tables are an effective means of communication. However, relatively little experimental work exists examining differences between various formats in how well people understand provided information. We conducted two web-based experiments with a large, diverse sample to explore the effects of display format on respondents’ comprehension. We found that comprehension depended on task. Graphs were better for estimating differences; however, tables were better when estimating equality and sums. We found 3D display formats reduced comprehension of pie charts but not of bar charts. Although pie charts never assisted comprehension, they often did not significantly impair comprehension either. Comprehension based on a 3-way table was as good as that for clustered bar charts but was worse for divided bar charts. Information can be conveyed graphically even with 3-way tables, but the choice of display format needs to be sensitive to the task at hand.

About this journal

Objective
Statistics, Politics, and Policy studies the ways that statistical analysis drives public policy decisions, and publishes significant research on the application of statistical ideas to problems that relate to policy implementation.

The increasing amount and complexity of available data is constantly creating new challenges for statistical thinking in policy problems. While many academic statisticians tend to share among themselves their latest methods and models, less attention has been paid to the usefulness of those statistical methods and models to inform public policy decisions, and what statistical approaches might be most effective in designing how policies are implemented. In the policy sphere, statistical methods are sometimes taken as a given, with less attention to all the variations, assumptions, and effects of different methods in differing contexts. But it is in the policy sphere that statistical debates can have the great value and impact, and the intersection of statistics and public policy is a fertile ground for statistical research and analysis to address important policy issues that may have widespread ramifications.

As an electronic journal, Statistics, Politics, and Policy will use a mix of voices and approaches to reach a broad audience. The journal aims to open avenues of communication between statisticians and policy makers on questions that pique the interest of the public. The primary objective of the journal will be to highlight the use of innovative statistical methodology in order to elucidate and resolve important public policy issues.

Statistics, Politics, and Policy will publish applied research articles that explore the implications of statistical thinking and methods applied to public policy issues. The journal will also publish engaging commentary pieces and innovative policy ideas on the public issues of the day where statistics plays, or ought to play, a role.

Papers for the commentary and ideas section are short, ideally 600-2000 words, and are intended to be of general interest and readability. That is, they should contain deeper analysis than is found on the Op-Ed page of The Wall Street Journal or The New York Times, but to be of comparable interest the Statistics, Politics, and Policy readership.

Topics

Applied statistical research
Policy issues
Educational testing and policy
Energy and environmental policy
Demography
Climate change
Public finance
History and review of statistical ideas applied to public policy controversies
Taxation and business policy
Justice, crime and forensic analyses
Health policy including health care finance
Drug development
Approval and monitoring
Politics
Sociological policy analyses
Statistical methodology including study design and causal inference, and survey methods

Article formats
Research articles, commentaries