Title:      Bayesian statistical decision theory and a critical
test for substantive significance

Authors:    Justin Esarey

Entrydate:  2009-09-09 21:13:28

Keywords:   inference, t-test, substantive significance,
Bayesian

Abstract:   I introduce a new critical test statistic, c*, that
uses Bayesian statistical decision theory to help an analyst
determine whether quantitative evidence supports the existence
of a substantively meaningful relationship. Bayesian statistical
decision theory takes a rational choice perspective toward
evidence, allowing researchers to ask whether it makes sense to
believe in the existence of a statistical relationship given how
they value the consequences of correct and incorrect decisions.
If a relationship of size c* is not important enough to
influence future research and policy advice, then the evidence
does not support the existence of a substantively significant
effect. A replication of findings from the American Journal of
Political Science and Journal of Politics illustrates that
statistical significance at conventional levels is neither
necessary nor sufficient to accept a hypothesis of substantive
significance using c*. I also make software packages available
for Stata and R that allow political scientists to easily use c*
for inference in their own research.

http://polmeth.wustl.edu/retrieve.php?id=935

**********************************************************
             Political Methodology E-Mail List
   Editors: Xun Pang        <[log in to unmask]>
            Jon C. Rogowski <[log in to unmask]>
**********************************************************
        Send messages to [log in to unmask]
  To join the list, cancel your subscription, or modify
           your subscription settings visit:

          http://polmeth.wustl.edu/polmeth.php

**********************************************************