Dear Colleagues:
I am conducting an evaluation of a nation-wide (non-US) program in which individuals were treated, mostly but not always one time, and on a voluntary (self-selected) basis.  Based on past experience, I know that treated individuals are likely to be less poor and less socially-isolated than the average person, with a few other less-pronounced demographic differences.  The evaluation is to be a one-shot, post-only design comparing a random sample of treated individuals with a control group.  The treatment sample will be drawn from lists kept by the organizations that implemented the program. 

I have a question concerning the best sampling method for obtaining the control group.

In past evaluations, I have matched on certain demographic characteristics by using a random walk procedure in each treated individual's neighborhood to find a person with similar demographics who was not treated.  I have sometimes used propensity-score matching in conjunction with this procedure when doing the statistical analyses.  Would it be just as good, better or worse to obtain a control group with a national random sample and simply use PSM in the analyses?  This would seem to give a better control group in terms of generalizability, and also would seem to correspond to the advice given in some textbook treatments of PSM in terms of running the first stage logistic regression to generate the probability of treatment.  It would also be cheaper to implement. On the other hand, the "manual" matching sampling method is intuitively straightforward, appeals to the non-statisticians who are funding the study, and also guarantees balance on place of
 residence and other specific demographics I specify in the matching procedure.

Any insights or advice about the merits and drawbacks of these approaches would be very much appreciated.

Best,

Steve Finkel
Department of Political Science
University of Pittsburgh

**********************************************************
             Political Methodology E-Mail List
   Editors: Melanie Goodrich, <[log in to unmask]>
            Delia Bailey, <[log in to unmask]>
**********************************************************
        Send messages to [log in to unmask]
  To join the list, cancel your subscription, or modify
           your subscription settings visit:

          http://polmeth.wustl.edu/polmeth.php

**********************************************************