Title:      Misunderstandings among Experimentalists and
Observationalists about Causal Inference

Authors:    Kosuke Imai, Gary King, Elizabeth Stuart

Entrydate:  2007-09-16 08:43:42

Keywords:   matching, blocking, causal inference, experimental
design, observational studies, average treatment effects,
covariate balance, field experiments, survey experiments

Abstract:   We attempt to clarify, and suggest how to avoid,
several serious misunderstandings about and fallacies of causal
inference in experimental and observational research.  These
issues concern some of the most basic advantages and
disadvantages of each basic research design.  Problems include
improper use of hypothesis tests for covariate balance between
the treated and control groups, and the consequences of using
randomization, blocking before randomization, and matching after
treatment assignment to achieve covariate balance.  Applied
researchers in a wide range of scientific disciplines seem to
fall prey to one or more of these fallacies, and as a result
make suboptimal design or analysis choices.  To clarify these
points, we derive a new four-part decomposition of the key
estimation errors in making causal inferences. We then show how
this decomposition can help scholars from different experimental
and observational research traditions better understand each
other's inferential problems and attempted solutions.

(This paper is forthcoming in the Journal of the Royal
Statistical Society, but we have some time for revisions and
would value any comments anyone might have.  This is a revised
and much more general version of an earlier paper, "The Balance
Test Fallacy in Causal Inference".)

http://polmeth.wustl.edu/retrieve.php?id=720

**********************************************************
             Political Methodology E-Mail List
   Editors: Melanie Goodrich, <[log in to unmask]>
            Delia Bailey, <[log in to unmask]>
**********************************************************
        Send messages to [log in to unmask]
  To join the list, cancel your subscription, or modify
           your subscription settings visit:

          http://polmeth.wustl.edu/polmeth.php

**********************************************************