Title:      Do Observational Methods Produce Reliable Results?
The Use of Matching in Estimating the Treatment Effect of Class
Size Reduction

Authors:    Adrienne  Hosek

Entrydate:  2008-07-09 06:31:45

Keywords:   

Abstract:   Several studies have tested the accuracy and
validity of observational research methods to evaluate what
estimation techniques . Randomized experiments are the gold
standard of research design. When conducted correctly, such
studies produce an unbiased estimate of the treatment effect for
the experimental sample. Unfortunately, randomized experiments
are rarely performed in the social sciences, largely due to
insufficient resources.  When a randomized experiment is not an
option, social scientists turn to observational research methods
to study the effects of a given treatment.  Several previous
studies have looked at the validity of using observational
techniques to determine whether the reliably provide an accurate
and consistent measure of a known treatment effect. In this
paper, we re-eximine the work of Hollister and Wilde (2007),
which did not systematically recover the experimental benchmark
through propensity score analysis using data from an
experimental study on class size reduction. They concluded that
observational methods performed poorly based on these results.
We find that they did not develop an appropriate test and thus
the inability to achieve the experimental benchmark should not
reflect flaws in the methodological approach, but rather stem
from problems in test design. 

http://polmeth.wustl.edu/retrieve.php?id=797

**********************************************************
             Political Methodology E-Mail List
   Editors: Melanie Goodrich, <[log in to unmask]>
            Delia Bailey, <[log in to unmask]>
**********************************************************
        Send messages to [log in to unmask]
  To join the list, cancel your subscription, or modify
           your subscription settings visit:

          http://polmeth.wustl.edu/polmeth.php

**********************************************************