Subject: | |
From: | |
Reply To: | |
Date: | Fri, 23 Dec 2005 08:55:35 -0600 |
Content-Type: | text/plain |
Parts/Attachments: |
|
|
title: The difference between ``significant'' and ``not significant'' is not itself statistically significant
authors: Andrew Gelman, Hal Stern
entrydate: 2005-12-23 08:25:49
keywords: multilevel modeling, multiple comparisons,
replication, statistical significance
abstract: A common error in statistical analyses is to summarize comparisons by declarations of statistical significance or non-significance. There are a number of difficulties with this approach. First is the oft-cited dictum that statistical significance is not the same as practical significance. Another difficulty is that this dichotomization into significant and non-significant results encourages the dismissal of observed differences in favor of the usually less interesting null hypothesis of no difference.
Here, we focus on a less commonly noted problem, namely that changes in statistical significance are not themselves significant. By this, we are not merely making the commonplace observation that any particular threshold is arbitrary---for example, only a small change is required to move an estimate from a 5.1% significance level to 4.9%, thus moving it into statistical significance. Rather, we are pointing out that even large changes in significance levels can correspond to small, non-significant changes in the underlying variables. We illustrate with a theoretical and an applied example.
http://polmeth.wustl.edu/retrieve.php?id=573
**********************************************************
Political Methodology E-Mail List
Editor: Karen Long Jusko <[log in to unmask]>
**********************************************************
Send messages to [log in to unmask]
To join the list, cancel your subscription, or modify
your subscription settings visit:
http://polmeth.wustl.edu/polmeth.php
**********************************************************
|
|
|