Skip to content
Publicly Available Published by De Gruyter April 13, 2021

Debate Reaction Ideal Points: Political Ideology Measurement Using Real-Time Reaction Data

  • Daniel Argyle , Lisa P. Argyle ORCID logo , Vlad Eidelman ORCID logo and Philip Resnik


Ideal point models have become a powerful tool for defining and measuring the ideology of many kinds of political actors, including legislators, judges, campaign donors, and members of the general public. We extend the application of ideal point models to the public using a novel data source: real-time reactions to statements by candidates in the 2012 presidential debates. Using these reactions as inputs to an ideal point model, we estimate individual-level ideology and evaluate the quality of the measure. Debate reaction ideal points provide a method for estimating a continuous, individual-level measure of ideology that avoids survey response biases, provides better estimates for moderates and the politically unengaged, and reflects the content of salient political discourse relevant to viewers’ attitudes and vote choices. As expected, we find that debate reaction ideal points are more extreme among respondents who strongly identify with a political party, but retain substantial within-party variation. Ideal points are also more extreme among respondents who are more politically interested. Using topical subsets of the debate statements, we find that ideal points in the sample are more moderate for foreign policy than for economic or domestic policy.

In representative democratic systems, public opinion is inextricably linked to representation. In theory, voters choose elected officials to implement a vision of government that in some way reflects their needs or desires, whether these choices are made based on partisan identity attachments (Achen and Bartels 2016) or policy positions (Costa 2020; Fowler 2020). Political scientists have spent decades trying to systematically summarize the interconnected constellations of people’s views on issues, their policy preferences, and their core values—in other words, their “ideology.”[1] To achieve this, researchers have developed a variety of tools for measuring the ideology of both elite political actors and everyday citizens, which can then be used in research to describe and explain the origins of different ideologies, how ideology relates to the demands people make on government, and how ideology and representation change over time. The consensus is that, at least among politicians, ideology is increasingly concentrated around extreme “conservative” and “liberal” positions—what Converse (1964) called ideological constraint—with few moderates remaining. However, there is significant debate about whether the highly visible ideological polarization among political actors extends to the public (see Barber and McCarty 2016 for a review).

The definition and measurement of ideology is not a trivial matter. The use of different measures leads scholars to reach radically different conclusions about the state and trajectory of public opinion in the United States. Some scholars argue that the ideological center is disappearing from American politics, and that this trend extends beyond political actors to other moderately informed and involved citizens (Abramowitz 2010; Hare and Poole 2015; Pew Research Center 2017). Other scholars maintain that although parties and ideologies are increasingly “sorted,” ideological polarization—meaning shifts to the extremes in the set of preferred policy outcomes—has not increased among the public (Bafumi and Herron 2010; Broockman 2016; Fiorina, Abrams, and Pope 2005; Hill and Tausanovitch 2015). Scholars who advocate a sorting explanation claim that increased alignment between party identification and policy positions is not accompanied by more extreme policy attitudes (Fiorina and Abrams 2009, 2015; Hill and Tausanovitch 2015). Still others note that increasing personal identification with political parties or ideological labels as social groups creates affective polarization (Iyengar, Sood, and Lelkes 2012; Mason 2018a, 2018b), and leads partisan attachments to drive policy views, not the other way around (Barber and Pope 2019). Despite strong and consistent evidence for greater alignment between partisanship and self-reported liberal/conservative ideological labels, debate remains about whether the underlying distribution of policy preferences has become more polarized, or even whether there exist widespread policy-relevant ideological commitments at all (Kalmoe 2020; Kinder and Kalmoe 2017).

We contribute to the growing field of quantitative ideology measurement by proposing a novel measurement strategy for individual-level citizen ideology, the debate reaction ideal point. Ideal points are widely used to estimate ideology of political actors, but are less common among members of the public due to data constraints. The debate reaction ideal point we propose uses relatively low-cost and scalable collection of real-time response data to generate ideal point estimates for members of the public. These ideal points can also be estimated based on subsets of the reactions, to generate ideal point measures for separate policy domains.

This measurement approach applies ideal-point estimation techniques to data derived from real-time positive, negative, and non-reactions to statements made during campaign debates, specifically the 2012 presidential and vice-presidential debates. We conceptualize positive and negative reactions as comparable to “votes” in a legislative setting—where ideal point models are commonly used to estimate ideology—and use them to estimate an ideal point for each debate watcher. The debate content provides a politically meaningful and widely viewed set of policy-relevant statements, and reactions to the statements reflect individual opinions in a format that is readily amenable to ideal point analysis.

Because ideology is an abstract psychological and attitudinal concept, there is no unequivocally perfect measure. This means there is not a ground truth comparison point to use when validating new methods or measures. Therefore, we examine the validity of the measure by evaluating whether it conforms to expectations derived from the extensive literature on public opinion and political ideology. We find the ideal point measurement correlates in expected ways with demographics, vote choice, and self-reported ideology.

We argue that debate reaction ideal points provide a number of advantages over existing approaches, which make them a valuable addition to the available research toolkit. We validate the ideal points through comparison with self-reported measures of ideology, partisanship, and political engagement, as well as aggregate vote totals. When limited to topical subsets of debate statements, we find that the estimated 2012 debate reaction ideal points are more moderate on issues of foreign policy than domestic or economic policy.

1 Obstacles to Measuring Ideology in the Public

Existing research on ideology in the mass public uses both self-reported attitudinal (Ansolabehere, Rodden, and Snyder 2008; Barber and Pope 2018) and observed behavioral (Barberá 2015; Bonica 2019) measures, at the individual (Jessee 2016) and aggregate (Abramowitz and Saunders 2008; Tausanovich and Warshaw 2013) levels. There are several challenges to the uses of these measures, and the different approaches result in different conclusions about the distribution of ideology in the public over time (cf. Abramowitz 2010; Fiorina and Abrams 2009). Therefore, it is important to think carefully about both the practical and theoretical implications of various measurement strategies.

For decades, the most common and straightforward measure of ideology in the United States has been to ask people to describe themselves on a scale ranging from “extremely conservative” to “extremely liberal.” This single-item self-reported ideology (alternatively called symbolic ideology or ideological identification) demonstrates high within-subject reliability over time and consistency with party identification or other aspects of individual identity (Kinder and Kalmoe 2017). In a thorough examination, Kinder and Kalmoe (2017) conclude that ideological self-identification only holds modest predictive power with regards to presidential voting, even less for down-ballot races, and very little constraining relationship with policy attitudes (Ellis and Stimson 2012; Kinder and Kalmoe 2017; Malka and Lelkes 2010). At best, direct self-identification of ideology can be understood as a general summary judgment of a range of policy- and identity-based considerations (Kinder and Kalmoe 2017), which reflect a variety of social cues beyond politics and policy (Ellis and Stimson 2012).

Because self-identification of ideology lacks the necessary consistent and constraining relationship with policy views, some researchers have instead turned to measuring ideology through a combination of multiple survey questions asking about specific issue positions. Averaging across multiple measures provides estimates that reveal more structure and stability in individual preferences than using single-item survey questions (Ansolabehere, Rodden, and Snyder 2008). Moving beyond averages, scholars use a variety of methods to generate a single estimate of ideology from a range of policy issue questions. For example, Abramowitz (2010), Abramowitz and Saunders (2008), and the Pew Research Center (2017) use a count of policy responses that are consistent with commonly understood “liberal” and “conservative” ideological positions. Baldassarri and Gelman (2008) use correlations between survey items on the ANES. Still others use factor analysis or item response theory to extract latent ideology from the set of policy responses (Hill and Tausanovitch 2015; Jacobson 2012; Tausanovitch and Warshaw 2013; Treier and Hillygus 2009; Ura and Ellis 2012).

However, survey responses have a number of weaknesses for measurement of political attitudes generally that are not clearly overcome by asking more policy questions or combining them using more complex algorithms. First, public opinion research has a long history of demonstrating that individuals are often unable or unwilling to accurately report their political opinions in a meaningful or consistent way (stemming from Converse 1964). Indeed, even small variations in question wording, order, and response options can produce sizeable variance in the attitudes people report on surveys (e.g. Smith 1989), with substantial implications for the perceived level of polarization (Fiorina and Abrams 2009). Survey responses based on policy information are impacted by dual concerns of framing and information. As researchers provide additional information to contextualize policy debates for less informed respondents, they risk influencing the responses through priming or framing processes (Smith 1989).

Second, declining response rates further exacerbate the potential for disconnect between the willing survey taker and the general public (Schill 2017). As response rates decline, a growing divide between politically interested survey takers and the general public may lead to less representative survey data. Additionally, as a specific manifestation of survey response error, many uninterested or uninformed survey respondents may pick moderate response options to avoid the cognitive burden of figuring out their position. It is therefore difficult to distinguish people who genuinely have no opinion from those who are ideological moderates (Fiorina and Abrams 2008; Kinder and Kalmoe 2017).

Third, measurement error can also be introduced by researchers through the process of creating and administering the survey. Due to space and cost constraints inherent in surveys, researchers must make selective decisions about the particular set of policy issue questions to include on the survey (Schill 2017), which can influence the resulting ideology measurements. Researcher priorities may not reflect the most salient political issues at the time when the respondent takes the survey. Furthermore, the connection of abstract ideologies to specific issue positions is not itself stable over time (Lewis 2019). Even survey-based measurements that rely on a well-established standard set of questions may not be reliable over time as the meaning of policy alternatives shift relative the current political climate and context.

Another strategy for measuring ideological preferences relies on observed political outcomes, such as election returns (Abramowitz 2010), vote totals, or candidate approval ratings (Kimball and Gross 2007) rather than self-reported policy opinions. These measures have the advantage of minimizing error due to the willingness or ability of respondents to provide accurate answers, either because they are entirely behavioral or because they have a lower cognitive and informational burden. Furthermore, they are deeply embedded in the reality of the contemporaneous political climate. However, using observed choices (such as votes) when the set of discrete choice options has itself grown more polarized may create the appearance of increased ideological consistency or extremity even when the underlying opinions have not changed (Fiorina and Abrams 2009; Fiorina, Abrams, and Pope 2008; Prior 2013). These measures also conflate ideological views with partisan choices (Fiorina, Abrams, and Pope 2008).

More recently, scholars have begun to mine available non-survey behavioral data to estimate ideology. Barberà (2015) uses the known ideology of political elites on Twitter to infer the ideology of the non-elite Twitter users who follow their accounts. Bonica (2019) uses political donation patterns to estimate the ideology of individual campaign donors. These approaches have the many advantages of being behavioral, contextual, real-world data from which to measure ideology, and are not influenced by researcher survey design choices or reliant on what people are willing or able to respond about their own preferences. However, they are necessarily limited to relatively narrow segments of the general population whose level of political interest and preferences are substantially different from the general public (for campaign donors, see Broockman and Malhotra 2020; for Twitter users, see Pew Research Center 2019).

While each of these approaches to measuring ideology have significant strengths, they also each face notable limitations. Because there is no ground truth measure for ideology against which to compare these various approaches and determine which is most accurate, researchers must rely on a varied toolkit of approaches and carefully consider which is most suited to a particular research approach. We next introduce our addition to this toolkit—debate reaction ideal points—and discuss how this approach mitigates many of the weaknesses of previous measures, along with consideration of its limitations.

2 Ideal Point Modeling Using Real-Time Presidential Debate Reactions

Real-time response data are gaining prominence as an alternative to survey methods in public opinion research (Schill 2017). Communication scholars have used real-time reactions in both laboratory and real-world settings to evaluate viewer reactions to the content of media material for nearly a century (Schill 2017). Political scientists have applied such techniques to political media, including advertisements (Iyengar, Jackman, and Hahn 2017; Kaid 2009; Maier, Hampe, and Jahn 2016), debates (Boydstun et al. 2014b; Hughes and Bucy 2017; Jarman 2005; Jasperson, Gollins, and Walls 2017; Schill and Kirk 2014), and multi-person panels (Waldvogel 2020). Real-time reaction approaches build on psychological theories of information processing to capture attitudes at the moment of exposure, capitalize on high frequency of measurement to provide additional statistical leverage, and, because of their breadth, can “capture the richness of individuals’ views” (Schill 2017, p. 19).

Here, we argue that viewer reactions to political debates tell us not only about the effectiveness of various aspects of the message, but also something about the viewers themselves. As viewers react positively to some messages and negatively to others over the course of the debate, we can model their political predispositions.

Presidential debates are a unique, information-rich political event, and real-time reactions to debates are well-suited to understanding viewers’ political attitudes. Debates provide an exceptional opportunity for ideal-point estimation because candidates in debates cover a wide range of policy topics using language and style designed to appeal to the lay voter (Benoit 2013). The real-time element of the debate reaction is particularly important, since post-debate coverage has a sizeable demonstrated impact on public opinion surrounding presidential debates (Fridkin et al. 2008).

Audience reactions to statements in a debate reflect, for the most part, reactions to salient policy topics that can have an impact on vote choice (Benoit 2013). In the 2012 presidential debates, 76% of statements were about policy rather than either candidate’s personal character (Benoit 2013), and the policy topics were driven more by public opinion salience than what the candidates might select based on their relative issue strengths (Boydstun, Glazier, and Pietryka 2013). Although the policy discussion in debates is sometimes criticized for lacking complexity (Rowland 2013), there is evidence that watching debates increases both issue knowledge and the importance that viewers place on policy positions in evaluating candidates (Benoit, Mckinney, and Holbert 2001, 2003). Debate content is a part of the actual information used by debate-watchers to make their voting decisions (Benoit 2013; Benoit, Hansen, and Verser 2003; Blais and Perrella 2008).[2] Thus, ideology measurements based on reactions of prospective voters to real-time statements in live debates represent a snapshot of political judgments as they would actually occur in the course of a developing political campaign, rather than being influenced by survey design or researcher biases.

Ideal point models are commonly used to provide an objective, data-driven, and scalable characterization of the ideological positions of political actors (Laver 2014). While these models were originally developed to generate ideal point estimates of elected officials (McCarty, Poole, and Rosenthal 1997; Shor and McCarty 2011), more recent scholars have applied them to members of the general public by using their survey responses (Bafumi and Herron 2010; Hill and Tausanovitch 2015; Jessee 2012, 2016; Tausanovitch and Warshaw 2013), donations to candidates (Bonica 2019), or decisions of who to follow on Twitter (Barberá 2015). More finely articulated computational models of legislative behavior have allowed for the generation of issue-specific ideal points (Gerrish and Blei 2012; Lauderdale and Clark 2014) and accounted for text-based framing (Nguyen et al. 2015). Recently, text-based ideal points have been estimated using only language inputs without additional behavioral signals (Vafa, Naidu, and Blei 2020). Along similar lines, we posit that reacting positively or negatively to a political statement is a choice amenable to ideal-point analysis. To that end, we use real-time responses to statements made in the 2012 presidential and vice-presidential debates collected by an online response app (Boydstun et al. 2014b; Resnik et al. 2017).

Ideal point modeling using real-time debate reactions overcomes many of the challenges faced by existing public ideology measurement strategies.

First, the topics of the debates are wide-ranging, allowing for a far more robust ideology measure than is available through analysis of a single vote choice or a limited set of issue positions on a survey. Additionally, these topics are directly tied to the salient issues in the political context and campaign cycle (Boydstun, Glazier, and Pietryka 2013), and so reflect the political reality as it exists at the moment of measurement. Rather than survey measures decided in advance by the researcher and subject to misinterpretation or shifting meaning over time, the debate moderator sets the agenda and the candidates themselves determine the salient aspects of the policy questions through their responses (Nguyen, Boyd-Graber, and Resnik 2012). It is true that the topics covered in a debate are not comprehensive, meaning debate data may not provide a comprehensive ideology measure based on the full range of the political landscape or subsets of issues that are not prominent on the agenda during the debate. However, debate reaction ideal points can be considered a snapshot of ideology that reflects the contemporaneous national political context.

Second, reaction data avoid some of the pitfalls of self-reported ideological or issue-position data, including social desirability biases or satisficing in survey responses. Real-time response data for live viewing are demonstrated to have high reliability and validity, while avoiding biases of survey responses (Waldvogel 2020). Rather than complex questions with many response options, the reactions involve simple choices in response to policy discussion presented in an accessible visual and auditory format directly from the candidates. Where surveys must decide between offering explanatory information and potentially influencing responses, or not providing background and increasing uninformed responses, debate candidates often present the relevant background information and counterarguments, informing voters as they make their cases.

Third, ideal point estimates are an established way of understanding ideology and polarization for political actors. Although the ideal points we estimate are not on a directly comparable policy space or an identical task to politicians considering proposed legislation or judicial cases, using an ideal point estimation model provides an important comparability in method to the estimates of ideology commonly used for elites. Additionally, ideal point estimation generates an ideology measure that is more meaningful and stable than alternatives based only on one or a few survey questions (Ansolabehere, Rodden, and Snyder 2008).

Fourth, respondents can choose which statements to respond to during the debates. Patterns of response and non-response provide meaningful information about a debate-watcher’s prioritization of different policy issues (Boydstun et al. 2014b). We use a Bayesian ordinal logit implementation of the ideal-point model, which, unlike the typical binary logit-based model, allows us to include information about both the choice to respond and the direction of response in the ideal point model.

Fifth, ideology estimation and conceptualization for moderates is particularly difficult using existing measures (Ahler and Broockman 2018), in part because people who are uninterested lack the necessary information to form a meaningful survey response (Fiorina and Abrams 2009), or are unlikely to be included in behavioral samples (i.e. campaign donors, political Twitter users). Debates are helpfully geared towards prime time viewers (Benoit 2013) and ideal points can be estimated for anyone who can be incentivized to watch and react. Additionally, while still subject to selection and non-response biases, real-time debate reaction data have potential to incorporate more uninterested or uninformed respondents than traditional survey methods (Waldvogel and Metz 2020).

Sixth, although the partisanship of the candidates is a highly salient cue in the debates, respondents can provide a mixture of many responses to statements by both candidates, instead of making a single binary choice between them (see the Online Appendix for the distribution of responses to in-party and out-party candidates). This helps mitigate the appearance of polarization that can arise when the measurement is based on a single binary choice between two increasingly polarized alternatives.

Finally, the availability of debate transcripts allows for the inclusion of textual information into the estimation and interpretation of the ideal points. Specifically, the researcher is able to remove statements that are likely to elicit non-policy reactions (e.g. insults or one-liners), or limit the ideology estimations to particular subsets of policy considerations. Additionally, the Bayesian ideal point estimation strategy produces additional information about the statements themselves that can be used to qualitatively evaluate the performance of the estimation procedure.

3 Data and Methods

During the October 2012 U.S. presidential and vice presidential debates, 6560 college students were recruited to watch and react to the debates using a browser-based mobile application (Resnik et al. 2017). Subjects were recruited by their instructors and received course credit for participation (Boydstun et al. 2014a). This method of recruitment generated a large and diverse respondent pool, with sample demographics that are highly comparable to a variety of national metrics and a sizeable number of respondents in rare demographic categories, such as African-American Republicans (Boydstun et al. 2014a). The sample as a whole is younger, somewhat more religious, and slightly more liberal than the general public (53–59% planned to vote for Obama). College students are distinct from the general population in important ways, including the noted instability of political ideology during the impressionable years (Krosnick and Alwin 1989). While the nature of the sample prevents us from making inferences about the distribution of ideology in the general population, the large sample size and nationwide recruitment provide ample variation to establish the relationships needed for validation of the method. Additional information about the recruitment process and sample demographics is available in the Online Appendix.

Notably, the process motivated respondents to participate in debate watching who might not have otherwise chosen to do so for their own purposes, which increases our ability to estimate ideology for people with a broad range of political interest and engagement. Note that Waldvogel and Metz (2020) also report broad reach with real-time debate response data in Germany. Real-time debate reaction data, then, provide an important potential for reaching otherwise uninterested or disengaged segments of the population.

Prior to each debate, participants completed a brief pre-debate survey that included demographic and political attitude questions. During the debate, the user could register a reaction by tapping a target (one of the candidates or the moderator) to indicate to whom they were reacting, followed by tapping one of Agree, Disagree, Spin, or Dodge to record a reaction. These responses were considered self-explanatory and participants were not given additional definitions.

Using time-stamped transcripts, we identify distinct statements made by the candidates and moderator during the four debates. We use the timestamps of the debate viewers’ reactions to link reactions to specific debate statements, and consider the reaction to be a “vote” in favor of or against that statement.[3] Agree reactions are considered a positive reaction to the statement, and coded as 1. Disagree, Spin, and Dodge reactions are considered a negative reaction to the statement, and coded as −1.[4] Statements to which a respondent did not react are coded as 0. Consistent with best practices in ideal point estimation, we also drop statements that have unanimous or near-unanimous response patterns (more than 99% of viewers have the same code). We also only include respondents who reacted to at least 10 statements. This results in 5119 debate watchers, reacting to 523 debate statements, with a median of 30 reactions per person. We pool the data for all four debates into a single dataset to estimate the ideal points.

We estimate a Bayesian ordinal logit parameterization of an ideal point model, based on respondents’ patterns of positive, negative, and no reactions to the debate statements.[5] Using an ordinal logit allows for explicit modeling of the decision not to respond as a separate choice from agreeing or disagreeing. Practically speaking, this increases the information available to estimate each individual’s ideal point, resulting in estimates that are more precise than with a standard logit implementation.[6]

4 Results

A kernel density plot of the estimated ideal points for all respondents is presented in Figure 1. The ideal points are normalized to have a mean of zero and variance of one. The distribution is notably bimodal, where there are peaks on both the liberal (negative) and conservative (positive) sides of the mean.[7] The majority of responses fall within one standard deviation of the mean indicating a substantial amount of moderation. The liberal peak is somewhat larger than the conservative peak, reflecting the somewhat higher number of self-described Democratic respondents in our sample (see Boydstun et al. 2014a). However, as we will demonstrate, there is substantial within-party variation in the ideal point estimates indicating that the measure is not merely a proxy for partisanship.

Figure 1: 
Kernel density plot of estimated ideal points.
Kernel density plots of debate reaction ideal points estimated using a Bayesian ordinal logit model. Normalized to mean = 0, standard deviation = 1. Negative values represent more liberal ideologies.
Figure 1:

Kernel density plot of estimated ideal points.

Kernel density plots of debate reaction ideal points estimated using a Bayesian ordinal logit model. Normalized to mean = 0, standard deviation = 1. Negative values represent more liberal ideologies.

Next, we validate these ideal points by performing a series of comparisons between the estimated ideal points and other metrics from the pre-debate survey, for which we expect a higher (partisanship, political interest) or lower (self-reported ideology) level of correlation. There is some difficulty inherent in validation of the measure, as there is no ground truth measurement of ideology with which to compare our metric. Therefore, the tests we conduct are based on the estimated ideal points relating in expected ways with other measures based on what we know from the existing literature on ideology.

The first test is whether the estimated debate reaction ideal points correlate with self-reported ideology. As previously discussed, self-reported ideology typically has modest correlation with policy-preference based measures, but it is not a strongly predictive of vote choice or consistently indicative of policy preferences across topic areas. The pre-debate survey asked respondents to report their ideology on a 101-point scale with zero being the most liberal and 100 being the most conservative. Figure 2 displays a scatterplot of the correlation between the ideal point estimate (y axis) and self-reported ideology measure (x axis). The Pearson’s correlation between self-reported ideology and estimated ideal points is 0.69.

Figure 2: 
Scatter plot correlation of ideal points with self-reported ideology.
Self-reported ideology ranges from 0 (most liberal) to 100 (most conservative). Observations are also distinguished by self-reported partisanship.
Figure 2:

Scatter plot correlation of ideal points with self-reported ideology.

Self-reported ideology ranges from 0 (most liberal) to 100 (most conservative). Observations are also distinguished by self-reported partisanship.

However, prior scholars have noted that the overall correlation between ideology and partisan or ideological-identity measures can be deceptive because it leverages the distinction between liberal and conservative categories but may have limited predictive value within party (see Bonica 2019; Hill and Huber 2017), and increasingly stark cross-party differences can mask important intra-party divisions (Nguyen et al. 2015). Our measure is no exception; restricted to subsamples by party, the correlation between self-reported and estimated ideology for Democrats is 0.28 and for Republicans is 0.38. As Kinder and Kalmoe (2017) argue, we expect this is because strength of ideological self-identification likely reflects a stronger identity-based attachment to the ideological label as a social identity attachment, rather than indicating a more extreme set of policy preferences. Modest correlation with ideological self-identification and, in particular, low correlation within parties, conforms to the expected pattern of relationships between self-reported ideology and ideology estimated from a set of policy positions.

We also note that the self-reported ideology distribution in Figure 2 visibly demonstrates one of the drawbacks of single-item survey-based measures, with clear clustering at the values of 0, 50, and 100. The estimated ideal-points, by contrast, are more continuous in nature.

Ideology has become increasingly aligned with party identification in recent years, so we next examine the distribution of ideal points by respondents’ self-reported party identification. We expect respondents who identify as closest to the Republican or Democratic parties to have the most extreme ideologies, with respondents who lean towards the Democratic or Republican parties closer to the middle, and independents to be the most centrist and also have the highest variance. Figure 3 shows a ridge plot of the kernel density of ideal points, broken down by party affiliation. As expected, those with stronger attachments to the Republican (Democratic) party have a more extreme median ideal point, 1.17 (−0.65) than those with weaker attachments to the Republican (Democratic) party, 0.89 (−0.42).

Figure 3: 
Distribution of ideal points by party affiliation.
Kernel density plots of the ideal points in Figure 1, separated by respondents’ self-reported party identification (right) and campaign interest (left). Vertical dashed lines indicate median values for each subgroup. Party ID is measured using a single question self-reported five-category scale.
Figure 3:

Distribution of ideal points by party affiliation.

Kernel density plots of the ideal points in Figure 1, separated by respondents’ self-reported party identification (right) and campaign interest (left). Vertical dashed lines indicate median values for each subgroup. Party ID is measured using a single question self-reported five-category scale.

Another common test of ideology measures is their predictive power regarding vote choice. Because we do not have actual individual-level post-election vote choice, and because individual evaluations of candidates immediately prior to the debate may be too closely connected to the debate reactions measure to provide a meaningful test, we test the correlation between ideal points and vote choice at an aggregate level. Figure 4 displays the correlation between debate-watchers’ average ideal points and actual 2012 presidential vote totals, by state. As expected, states that voted for Obama at higher rates also have more liberal mean ideal point scores. With one exception, states that had less than 50% voting for Obama have a mean ideal point greater than zero, indicating a more conservative average ideology. The Pearson’s correlation (omitting DC as a potential outlier) between these two measures is −0.74.

Figure 4: 
Scatterplot correlation between average ideal point and state vote totals.
Mean ideal point for all respondents in the state on the X axis; negative values are more liberal. State-level percent of vote received by Obama on the Y axis. States with fewer than 50 respondents are omitted from the plot. Because DC is a clear outlier, it is included on the graph but is not included in the correlation calculations.
Figure 4:

Scatterplot correlation between average ideal point and state vote totals.

Mean ideal point for all respondents in the state on the X axis; negative values are more liberal. State-level percent of vote received by Obama on the Y axis. States with fewer than 50 respondents are omitted from the plot. Because DC is a clear outlier, it is included on the graph but is not included in the correlation calculations.

Next, we expect that respondents who are more politically engaged will have more extreme responses (Barber and Pope 2018; Jennings 1992). Figure 5 shows ideal point distributions by quartiles of a survey question asking respondents to evaluate their level of self-reported campaign interest (on a 0–100 scale), with dashed lines indicating the median of Democratic, Independent, and Republican subgroups, respectively. The lowest two quartiles look very similar, with some separation between the parties and more respondents in the moderate space between the two peaks (Q1 Democratic median = −0.47, Republican median = 0.93). For the highest two quartiles, the median value for each party becomes more extreme, and the trough between parties becomes more pronounced, indicating more ideological division among those who have high levels of political interest (Q4 Democratic median = −0.75, Republican median = 1.22).

Figure 5: 
Distribution of ideal points by political interest.
Kernel density plots of the ideal points in Figure 1, separated by respondents’ campaign interest. Dashed vertical lines indicate median values for Democratic, Independent, and Republican subgroups, respectively. Interest measured using a self-reported thermometer rating, divided into quartiles. Q1: <50, Q2: 50–67, Q3: 67–88, Q4: >88.
Figure 5:

Distribution of ideal points by political interest.

Kernel density plots of the ideal points in Figure 1, separated by respondents’ campaign interest. Dashed vertical lines indicate median values for Democratic, Independent, and Republican subgroups, respectively. Interest measured using a self-reported thermometer rating, divided into quartiles. Q1: <50, Q2: 50–67, Q3: 67–88, Q4: >88.

Further demographic correlations by race, gender, and age, are available in the Online Appendix.

An additional advantage of debate reaction ideal point estimation is the ability to disaggregate ideal point estimation by policy subtopics of the debate statements, which allows for estimation of policy preferences for specific policy areas. This is beneficial for both practical and theoretical purposes.

Practically speaking, it is possible that estimating ideal points using the full range of policy and non-policy statements in the debate could overestimate the number of ideological moderates due to averaging across policy domains. In this case, people with debate reaction ideal points near zero may not be those who prefer traditionally moderate or centrist policies, but rather those who have a strong liberal position in some issue areas and a strong conservative position in other areas (Ahler and Broockman 2018). On the other hand, it could underestimate the number of ideological moderates if respondents react strongly or more frequently to partisan, non-policy cues.

Theoretically, one of the challenges in conceptualizing and estimating ideology is that people rarely have ideologically principled logic for holding particular policy beliefs that remain consistent across a variety of issue areas (Converse 1964; Kinder and Kalmoe 2017). While a lengthy survey could plumb the depths of opinion on one or more issue areas, that is a cognitively intensive task that stretches the limits of respondents willingness and ability to respond. By contrast, the nature of the high-frequency real-time reaction to debate statements allows us to estimate a separate ideology measure for different policy areas with lower demand on the respondent (Boydstun et al. 2014b). The estimation of separate ideal points by topic area allows for exploration of ideological principles that are more limited in scope. While such topic-specific estimation has been conducted for political actors (Gerrish and Blei 2012; Lauderdale and Clark 2014), there is little work extending the practice to the general public.

To account for the practical possibilities of bias due to non-policy statements in the debate, and as proof of concept for the single-topic ideal points, we estimate three separate ideal points based on issue area. We categorized a single primary topic for each debate statement using the Policy Agendas coding framework, and aggregated into three subsets of policy statements: Domestic Policy, Economy, and Foreign Policy (coding details in the Online Appendix). Figure 6 displays the distribution of ideal points based only on the statements made in each topic area.

Figure 6: 
Topic specific ideal point distributions.
Kernel density plots of the ideal points estimated using only the subtopics of policy statements. Domestic: 2636 respondents, 148 statements; Economy: 2726 respondents, 125 statements; Foreign: 2000 respondents, 178 statements.
Figure 6:

Topic specific ideal point distributions.

Kernel density plots of the ideal points estimated using only the subtopics of policy statements. Domestic: 2636 respondents, 148 statements; Economy: 2726 respondents, 125 statements; Foreign: 2000 respondents, 178 statements.

The distribution of domestic and economic policy-specific ideal points is very similar to the overall distribution, indicating that concerns about bias from non-policy statements are minimal. The distribution is noticeably less polarized when estimated using only statements about foreign policy, a result reinforced by a “dip” test for bimodality (Hartigan 1985). We are able to reject the null hypothesis of unimodality for both Domestic and Economic policy (p-value < 0.001 in both cases). We fail to reject a null of unimodality in the Foreign case (p-value 0.10). This is consistent with prior conclusions that although foreign policy and defense attitudes are increasingly aligned with party positions, sorting on foreign policy attitudes occurred later and to a lesser extent than other issues (Fiorina and Abrams 2008).

5 Popularity and Polarity of Debate Statements

As a further validation of the model specification, we examine the results of two additional parameters that are estimated in the Bayesian ideal point model: popularity (discrimination parameter) and polarity (difficulty parameter). Rather than describing the people who react to the statements, these metrics use the aggregate patterns of reactions to the statements to characterize features of the debate statements themselves. Popularity indicates how likely all respondents are to agree with the statement, and is higher if respondents from across the ideal point space agree with the statement. Polarity represents how strongly a statement splits respondents, and therefore represents the conservative or liberal extremity of a speaker’s statement. Statements that a range of respondents agree with or disagree with will have lower polarity; statements that are primarily agreed with by respondents on one end of the policy space and disagreed with by respondents on the other end of the policy space will have high polarity scores. These metrics provide an opportunity to examine the content of the statements, and ensure that the model output conforms to qualitative expectations about the kinds of statements that would be more (or less) popular or distinguish between liberal and conservative voters.

Table 1 shows the average polarity and popularity for the statements made by each speaker, across all debates. As expected, the Republican ticket has a more positive (conservative) polarity and the Democratic ticket has a more negative (liberal) polarity. On both tickets, the presidential candidates made statements that were, on average, more ideologically extreme than the vice-presidential candidates. Notably, the moderator had an average polarity very close to zero, and approximately equidistant from the two slates of candidates, demonstrating the moderators competently managed the debates in an unbiased manner.

Table 1:

Average polarity and popularity of statements by speaker.

Obama Biden Moderator Ryan Romney
Polarity −2.685 −2.183 −0.106 2.207 2.542
Popularity 1.668 1.343 0.709 0.14 0.241

Obama had the highest average popularity, while Ryan was lowest. While this is expected due to the somewhat more liberal nature of the sample, it also conforms to expectations based on public perceptions and media discourse regarding the candidates’ debate performances.[8] The full text of the 10 highest and lowest popularity and polarity statements are in the Online Appendix.

6 Conclusion

We have introduced a new measure of individual ideology that uses real-time reactions to political debates as the basis for ideal point estimation. Debate reaction ideal points have the advantage of providing robust, continuous scales that yield more information than traditional limited scales of self-reported ideology, and provide the flexibility to estimate based on a content subset. Using reactions to a live debate, the estimation is grounded in real-world policy content and issue presentation that directly reflects the current political context. Being real time, it reduces potential biases introduced by conscious reflection, satisficing, or question wording. Additionally, respondents can choose which statements to respond to and which to ignore, so the method can cover the broad range of issues in a debate, avoid researcher influence about what survey questions to ask, and distinguish lack of interest from moderate positions.

We find that more partisan and more interested study participants are more ideologically extreme. Unless they intentionally account for how to gather meaningful attitudinal data from less engaged citizens, estimates of public ideology are likely to overstate levels of polarization in the mass public, precisely because such practices remove an ideologically moderate segment of the American population. Estimation strategies, such as the one presented here, that provide a meaningful way to measure ideology for the least engaged are essential for accurate portrayal of ideology and polarization in the American public.

An important contribution of this method is the ability to use subsets of the debate statements to estimate ideal points within separate policy areas. This allows for a more robust understanding of how ideology varies across issue areas in the United States. As expected based on public opinion research, we find that attitudes about foreign policy issues are less polarized than attitudes about domestic and economic policy.

The biggest challenge facing widespread use of debate reaction ideal points is data collection. While our own available data limit the present study to the 2012 election, real-time reactions data are increasingly common and facilitated by innovations in virtual technology (Maier, Hampe, and Jahn 2016; Waldvogel 2020; Waldvogel and Metz 2020). Future research should consider ways to harness real-time reactions already conducted by a variety of news media outlets, to link real-time reaction data with other sources of public opinion information, or to integrate real-time reaction measures into surveys or other platforms. While we believe there are benefits to response data collected synchronously from debates in real time, before pundits have a chance to influence viewers’ interpretations (Fridkin et al. 2008) or political priorities shift (Boydstun, Glazier, and Pietryka 2013), future researchers may find applications for response data based on debate recordings or other stimuli that can be provided on-demand through survey platforms or in a lab. The ideal point estimation method described here is flexible enough for application to any reaction data, although the nature of the stimulus material may change the substantive meaning of the ideal points.

Additionally, linking real-time reactions with other estimates of public ideology (e.g. Twitter or campaign donations) would provide useful insights into the validity of these approaches. Such innovations promise to enhance the estimation of public ideology. Repetition of debate reaction measurement over time would be particularly valuable, as it would allow for examination of changes in ideology over time and across respondent characteristics. Continued use of this estimation strategy may provide valuable insight into the nature and extent of political polarization in the mass public.

Corresponding author: Lisa P. Argyle, Department of Political Science, Brigham Young University, 745 KMBL, Provo, UT, USA, E-mail:


Abramowitz, A. I. 2010. The Disappearing Center. New Haven, CT: Yale University Press.Search in Google Scholar

Abramowitz, A. I., and K. L. Saunders. 2008. “Is Polarization a Myth?” The Journal of Politics 70: 542–55, in Google Scholar

Achen, C. H., and L. M. Bartels. 2016. Democracy for Realists: Why Elections do not Produce Responsive Government. Princeton, NJ: Princeton University Press.10.1515/9781400882731Search in Google Scholar

Ahler, D. J., and D. E. Broockman. 2018. “The Delegate Paradox: Why Polarized Politicians Can Represent Citizens Best.” The Journal of Politics 80: 1117–33, in Google Scholar

Ansolabehere, S., J. Rodden, and J. M. Snyder. 2008. “The Strength of Issues: Using Multiple Measures to Gauge Preference Stability, Ideological Constraint, and Issue Voting.” American Political Science Review 102: 215–32. Cambridge University Press, in Google Scholar

Bafumi, J., and M. C. Herron. 2010. “Leapfrog Representation and Extremism: A Study of American Voters and Their Members in Congress.” American Political Science Review 104: 519–42, in Google Scholar

Baldassarri, D., and A. Gelman. 2008. “Partisans without Constraint: Political Polarization and Trends in American Public Opinion.” American Journal of Sociology 114: 408–54, in Google Scholar

Barber, M., and N. McCarty. 2016. “Causes and Consequences of Polarization.” In Political Negotiation: A Handbook, edited by J. Mansbridge, and C. J. Martin, 37–90. Washington, D.C.: Brookings Institution Press.10.1017/CBO9781316091906.002Search in Google Scholar

Barber, M., and J. C. Pope. 2018. “Who is Ideological? Measuring Ideological Consistency in the American Public.” The Forum 16: 92–122, in Google Scholar

Barber, M., and J. C. Pope. 2019. “Does Party Trump Ideology? Disentangling Party and Ideology in America.” American Political Science Review 113: 38–54, in Google Scholar

Barberá, P. 2015. “Birds of the Same Feather Tweet Together: Bayesian Ideal Point Estimation Using Twitter Data.” Political Analysis 23: 76–91, in Google Scholar

Benoit, W. L. 2013. Political Election Debates: Informing Voters about Policy and Character. New York: Lexington Books.Search in Google Scholar

Benoit, W. L., G. J. Hansen, and R. M. Verser. 2003. “A Meta-Analysis of the Effects of Viewing U.S. Presidential Debates.” Communication Monographs 70: 335–50, in Google Scholar

Benoit, W. L., M. S. Mckinney, and R. L. Holbert. 2001. “Beyond Learning and Persona: Extending the Scope of Presidential Debate Effects.” Communication Monographs 68: 259–73, in Google Scholar

Blais, A., and A. M. L. Perrella. 2008. “Systemic Effects of Televised Candidates’ Debates.” International Journal of Press/Politics 13: 451–64, in Google Scholar

Bonica, A. 2019. “Are Donation-Based Measures of Ideology Valid Predictors of Individual-Level Policy Preferences?” The Journal of Politics 81: 327–33, in Google Scholar

Boydstun, A. E., J. T. Feezell, R. A. Glazier, T. P. Jurka, and M. T. Pietryka. 2014a. “Colleague Crowdsourcing: A Method for Fostering National Student Engagement and Large-N Data Collection.” PS: Political Science & Politics 47: 829–34, in Google Scholar

Boydstun, A. E., R. A. Glazier, and M. T. Pietryka. 2013. “Playing to the Crowd: Agenda Control in Presidential Debates.” Political Communication 30: 254–77, in Google Scholar

Boydstun, A. E., R. A. Glazier, M. T. Pietryka, and P. Resnik. 2014b. “Real-Time Reactions to a 2012 Presidential Debate.” Public Opinion Quarterly 78: 330–43, in Google Scholar

Broockman, D. E. 2016. “Approaches to Studying Policy Representation.” Legislative Studies Quarterly 41: 181–215, in Google Scholar

Broockman, D., and N. Malhotra. 2020. “What Do Partisan Donors Want?” Public Opinion Quarterly 84: 104–18, in Google Scholar

Carpenter, B., A. Gelman, M. D. Hoffman, D. Lee, B. Goodrich, M. Betancourt, M. Brubaker, J. Guo, P. Li, and A. Riddell. 2017. “Stan: A Probabilistic Programming Language.” Journal of Statistical Software 76 (1): 1–32, in Google Scholar

Converse, P. E. 1964. “The Nature of Belief Systems in Mass Publics.” In Ideology and Discontent, edited by D. E. Apter, 206–61. New York: Free Press.10.4324/9780203505984-10Search in Google Scholar

Costa, Mia. 2020. “Ideology, Not Affect: What Americans Want from Political Representation.” American Journal of Political Science 1–17, in Google Scholar

Ellis, C., and J. A. Stimson. 2012. Ideology in America. New York: Cambridge University Press.10.1017/CBO9781139094009Search in Google Scholar

Fiorina, M. P., and S. J. Abrams. 2008. “Political Polarization in the American Public.” Annual Review of Political Science 11: 563–88, in Google Scholar

Fiorina, M. P., and S. J. Abrams. 2009. Disconnect: The Breakdown of Representation in American Politics. Norman, OK: University of Oklahoma Press.Search in Google Scholar

Fiorina, M. P., and S. J. Abrams. 2015. “Americans Are Not Polarized, Just Better Sorted.” In Political Polarization in American Politics, 41–7. New York: Bloomsbury Academic.Search in Google Scholar

Fiorina, M. P., S. J. Abrams, and J. C. Pope. 2005. Culture War? The Myth of a Polarized Electorate. New York: Pearson.Search in Google Scholar

Fiorina, M. P., S. A. Abrams, and J. C. Pope. 2008. “Polarization in the American Public: Misconceptions and Misreadings.” The Journal of Politics 70: 556–60, in Google Scholar

Fowler, A. 2020. “Partisan Intoxication or Policy Voting?” Quarterly Journal of Political Science 15: 141–79, in Google Scholar

Fridkin, K. L., P. J. Kenney, S. A. Gershon, and G. S. Woodall. 2008. “Spinning Debates: The Impact of the News Media’s Coverage of the Final 2004 Presidential Debate.” International Journal of Press/Politics 13: 29–51, in Google Scholar

Gerrish, S. M., and D. M. Blei. 2012. “How They Vote: Issue-Adjusted Models of Legislative Behavior.” Advances in Neural Information Processing Systems 25: 2753–61.Search in Google Scholar

Hare, C., and K. T. Poole. 2015. “How Politically Moderate Are Americans? Less than it Seems.” In Political Polarization in American Politics, edited by D. J. Hopkins, and J. Sides, 32–40. New York: Bloomsbury Academic.Search in Google Scholar

Hartigan, P. M. 1985. “Algorithm AS 217: Computation of the Dip Statistic to Test for Unimodality.” Applied Statistics 34: 320, in Google Scholar

Hill, S. J., and C. Tausanovitch. 2015. “A Disconnect in Representation? Comparison of Trends in Congressional and Public Polarization.” The Journal of Politics 77: 1058–75, in Google Scholar

Hill, S. J., and G. A. Huber. 2017. “Representativeness and Motivations of the Contemporary Donorate: Results from Merged Survey and Administrative Records.” Political Behavior 39: 3–29, in Google Scholar

Hughes, S. R., and E. P. Bucy. 2017. “Moments of Partisan Divergence in Presidential Debates: Indicators of Verbal and Nonverbal Influence.” In Political Communication in Real Time: Theoretical and Applied Research Approaches, edited by D. Schill, R. Kirk, and A. E. Jasperson, 249–73. New York: Routledge.Search in Google Scholar

Iyengar, S., G. Sood, and Y. Lelkes. 2012. “Affect, Not Ideology: A Social Identity Perspective on Polarization.” Public Opinion Quarterly 76: 405–31, in Google Scholar

Iyengar, S., S. Jackman, and K. Hahn. 2017. “Polarization in Less than Thirty Seconds: Continuous Monitoring of Voter Response to Campaign Advertising.” In Political Communication in Real Time: Theoretical and Applied Research Approaches, edited by D. Schill, R. Kirk, and A. E. Jasperson, 171–95. New York: Routledge.Search in Google Scholar

Jacobson, G. C. 2012. “The Electoral Origins of Polarized Politics.” American Behavioral Scientist 56: 1612–30, in Google Scholar

Jarman, J. W. 2005. “Political Affiliation and Presidential Debates: A Real-Time Analysis of the Effect of the Arguments Used in the Presidential Debates.” American Behavioral Scientist 49: 229–42, in Google Scholar

Jasperson, A. E., J. Gollins, and D. Walls. 2017. “Polarization in the 2012 Presidential Debates: A Moment-to-Moment, Dynamic Analysis of Audience Reactions in Ohio and Florida.” In Political Communication in Real Time: Theoretical and Applied Research Approaches, edited by D. Schill, R. Kirk, and A. E. Jasperson, 196–224. New York: Routledge.Search in Google Scholar

Jennings, M. K. 1992. “Ideological Thinking Among Mass Publics and Political Elites.” Public Opinion Quarterly 56: 419, in Google Scholar

Jessee, S. 2012. Ideology and Spatial Voting in American Elections. New York: Cambridge University Press.10.1017/CBO9781139198714Search in Google Scholar

Jessee, S. 2016. “(How) Can We Estimate the Ideology of Citizens and Political Elites on the Same Scale?” American Journal of Political Science 60: 1108–24, in Google Scholar

Kaid, L. L. 2009. “Immediate Responses to Political Television Spots in the US Elections: Registering Responses to Advertising Content.” In Real-Time Response Measurement in the Social Sciences: Methodological Perspectives and Applications, edited by J. Maier M. Maier, M. Maurer, C. Reinemann and V. Meyer, 137–53. Frankfurt: Peter Lang.Search in Google Scholar

Kalmoe, N. P. 2020. “Uses and Abuses of Ideology in Political Psychology.” Political Psychology 41: 771–93, in Google Scholar

Kimball, D. C., and C. A. Gross. 2007. “The Growing Polarization of American Voters.” In The State of the Parties, edited by J. C. Green. Boulder, CA: Rowman & LIttlefield.Search in Google Scholar

Kinder, D. R., and N. P. Kalmoe. 2017. Neither Liberal nor Conservative: Ideological Innocence in the American Public. Chicago: University of Chicago Press.Search in Google Scholar

Krosnick, J. A., and D. F. Alwin. 1989. “Aging and Susceptibility to Attitude Change.” Journal of Personality and Social Psychology 57 (3): 416–25, in Google Scholar

Lauderdale, B. E., and T. S. Clark. 2014. “Scaling Politically Meaningful Dimensions Using Texts and Votes.” American Journal of Political Science 58: 754–71, in Google Scholar

Laver, M. 2014. “Measuring Policy Positions in Political Space.” Annual Review of Political Science 17: 207–23. Annual Reviews Inc., in Google Scholar

Lewis, V. 2019. Ideas of Power: The Politics of American Party Ideology Development. New York: Cambridge University Press.10.1017/9781108568852Search in Google Scholar

Maier, J., J. F. Hampe, and N. Jahn. 2016. “Breaking Out of the Lab: Measuring Real-Time Responses to Televised Political Content in Real-World Settings.” Public Opinion Quarterly 80: 542–53, in Google Scholar

Malka, A., and Y. Lelkes. 2010. “More than Ideology: Conservative-Liberal Identity and Receptivity to Political Cues.” Social Justice Research 23: 156–88, in Google Scholar

Mason, L. 2018a. Uncivil Agreement: How Politics Became Our Identity. Chicago: University of Chicago Press.10.7208/chicago/9780226524689.001.0001Search in Google Scholar

Mason, L. 2018b. “Ideologues Without Issues: The Polarizing Consequences of Ideological Identities.” Public Opinion Quarterly 82: 280–301, in Google Scholar

McCarty, N., K. T. Poole, and H. Rosenthal. 1997. Income Redistribution and the Realignment of American Politics. Washington, D.C.: AEI Press.Search in Google Scholar

Nguyen, V.-A., J. Boyd-Graber, and P. Resnik. 2012. “SITS: A Hierarchical Nonparametric Model Using Speaker Identity for Topic Segmentation in Multiparty Conversations.” In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, 78–87.Search in Google Scholar

Nguyen, V.-A., J. Boyd-Graber, P. Resnik, and K. Miler. 2015. “Tea Party in the House: A Hierarchical Ideal Point Topic Model and its Application to Republican Legislators in the 112 th Congress.” In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, 1438–48.10.3115/v1/P15-1139Search in Google Scholar

Pew Research Center. 2017. The Partisan Divide on Political Values Grows Even Wider. Also available at in Google Scholar

Pew Research Center. 2019. How Twitter Users Compare to the General Public. Also available at in Google Scholar

Prior, M. 2013. “Media and Political Polarization.” Annual Review of Political Science 16: 101–27, in Google Scholar

Resnik, P., A. E. Boydstun, R. A. Glazier, and M. T. Pietryka. 2017. “Scalable Multidimensional Response Measurement Using a Mobile Platform.” In Political Communication in Real Time: Theoretical and Applied Research Approaches, edited by D. Schill, R. Kirk, and A. E. Jasperson. New York: Routledge.Search in Google Scholar

Riddell, A., A. Hartikainen, and M. Carter. 2021. “Pystan (3.0.0).” PyPI, in Google Scholar

Rowland, R. C. 2013. “The First 2012 Presidential Campaign Debate: The Decline of Reason in Presidential Debates.” Communication Studies 64: 528–47, in Google Scholar

Schill, D. 2017. “The History, Reliability, Validity, and Utility of Real Time Response.” In Political Communication in Real Time: Theoretical and Applied Research Approaches, edited by D. Schill, R. Kirk, and A. E. Jasperson, 3–28. New York: Routledge.Search in Google Scholar

Schill, D., and R. Kirk. 2014. “Courting the Swing Voter: ‘Real Time’ Insights into the 2008 and 2012 U.S. Presidential Debates.” American Behavioral Scientist 58: 536–55, in Google Scholar

Shor, B., and N. M. McCarty. 2011. “The Ideological Mapping of American Legislatures.” American Political Science Review 105: 530–51, in Google Scholar

Smith, E. R. A. N. 1989. The Unchanging American Voter. Berkeley, CA: University of California Press.Search in Google Scholar

Tausanovitch, C., and C. Warshaw. 2013. “Measuring Constituent Policy Preferences in Congress, State Legislatures, and Cities.” The Journal of Politics 75: 330–42, in Google Scholar

Treier, S., and D. S. Hillygus. 2009. “The Nature of Political Ideology in the Contemporary Electorate.” Public Opinion Quarterly 73: 679–703, in Google Scholar

Ura, J. D., and C. R. Ellis. 2012. “Partisan Moods: Polarization and the Dynamics of Mass Party Preferences.” The Journal of Politics 74: 277–91, in Google Scholar

Vafa, K., S. Naidu, and D. Blei. 2020. “Text-Based Ideal Points.”In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (July), 5345–57.10.18653/v1/2020.acl-main.475Search in Google Scholar

Waldvogel, T. 2020. “Applying Virtualized Real-Time Response Measurement on TV-Discussions with Multi-Person Panels.” Statistics, Politics, and Policy 11: 23–58, in Google Scholar

Waldvogel, T., and T. Metz. 2020. “Measuring Real-Time Response in Real-Life Settings.” International Journal of Public Opinion Research 32 (4): 659–75.10.1093/ijpor/edz050Search in Google Scholar

Warner, B. R., M. S. McKinney, J. C. Bramlett, F. J. Jennings, and M. E. Funk. 2020. “Reconsidering Partisanship as a Constraint on the Persuasive Effects of Debates.” Communication Monographs 87: 137–57, in Google Scholar

Supplementary Material

The online version of this article offers supplementary material (

Received: 2020-10-27
Accepted: 2021-03-16
Published Online: 2021-04-13
Published in Print: 2021-06-25

© 2021 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 1.4.2023 from
Scroll to top button