Title:      Estimating Party Policy Positions with Uncertainty
Based on Manifesto Codings

Authors:    Kenneth Benoit, Michael Laver, Slava Mikhaylov

Entrydate:  2007-08-21 07:29:27

Keywords:   Comparative Manifesto  Project, Mapping party
positions, party policy, error estimates,  measurement error

Abstract:   Spatial models of party competition are central to
modern political science. Before we can elaborate such models
empirically, we need reliable and valid measurements of agents'
positions on salient policy dimensions. The primary empirical
times series of estimated party positions in many countries
derives from the content analysis of party manifestos by the
Comparative Manifesto Project (CMP). Despite widespread use of
the CMP data, and despite the fact that estimates in these data
arise from documents coded once, and once only, by a single
human researcher, the level of error in the CMP estimates has
never been estimated or even fully characterized. This greatly
undermines the value of the CMP dataset as a scientific
resource. It is in many ways remarkable that so much has been
published in the best professional journals using data that
almost certainly has substantial, but completely
uncharacterized, error. We remedy this situation. We outline the
process of generating CMP document codings and positional
estimates. Error in this process arises, not only from the
obvious source of coder unreliability, but also from fundamental
variability in the stochastic process by which latent party
positions are translated into observable manifesto texts. Using
the quasi-sentence codings from the CMP project, we reproduce
the error-generating process by simulating coder unreliability
and bootstrapping analyses of coded quasi-sentences to reproduce
both forms of error. Using our estimates of these errors, we
suggest and demonstrate ways to correct otherwise biased
inferences derived from statistical analyses of the CMP data.

http://polmeth.wustl.edu/retrieve.php?id=718