I think the crucial difference here is p-hat versus p. If you want to
know whether the effect is "significant" at any specified point on the
logit curve, that is simply the same thing as asking whether the
coefficient is "significant" (because, for fixed p, the only thing that
is random in beta-hat_j*p*(1-p) is beta-hat_j). If you want to know
whether the effect is "significant" for a particular constellation of
explanatory variables, then the variance of p-hat enters as well, and
that depends on the entire covariance matrix of the parameter estimates.

Larry


-----Original Message-----
From: Political Methodology Society [mailto:[log in to unmask]] On
Behalf Of Franzese, Robert
Sent: Tuesday, April 17, 2007 10:28 AM
To: [log in to unmask]
Subject: [POLMETH] Significance of parameter vs. of effects in
non-linear-additive models, like logit/probit

Fellow PolMethers,

I write with (yet another) simple-but-very-good-and(-perhaps)-deep
question that the students in my methods class this term raised. They
noticed, in the course of a problem set, that a coefficient in a logit
estimation could be statistically distinguishable from zero while
marginal or first-difference effects of the associated variable are not,
or vice versa the former may be insignificant but the latter significant
(in some or all ranges of other variable values). As we all know, in a
logit model,

d(p-hat)/dX_j = beta_j*(phat)*(1-phat)

So, the marginal effect is zero if beta_j is zero. So, for years, I'd
simply been teaching that one quick way to assess the zero or non-zero
effect of a variable was to look at our old familiar friend, estimated
coefficient divided by its standard error, emphasizing however that the
significance/certainty of estimated effects must be assessed using the
estimated variance of the above formula. That seemed to satisfy everyone
before, but, as I said, this is a very deep-thinking cohort.

The question they raised is what to make of the results when these two
signals about the significance of some explanatory variable(s) conflict.
I could assemble my own thoughts on the conundrum, but I thought I'd
check with this audience to see if there are any extant good discussions
of the matter and/or what you all think.

Here are my thoughts so far on the matter:
My first thought was: Are these close calls? The tests that beta_j=0 and
that beta_j*phat*(1-phat)=0 might be asymptotically equivalent. In that
case, the differences amount simply, mostly, to a reminder not to place
too much emphasis on any knife edge at .05 or anywhere else. On the
other hand, perhaps they are not even asymptotically equivalent, since
the covariances of the coefficient estimates enter in the latter but not
the former. In that case, just as in a time-serial context that we
discussed some in this same class, where b/(1-rho) might be
distinguishable from zero when b is not, or vice versa, we might find
that a particular coefficient is distinguishable from zero whereas its
effect under various configurations of other coefficient estimates and
variable values is not or vice versa. What to make of this in the
logit/probit context seems more complicated than in the time-serial one,
though. (In the time-serial case, I do not find it particularly
difficult to comprehend how we might be more certain that the long-run
impact of some variable is non-zero whereas we are uncertain that that
the instantaneous/contemporaneous impact is so, or vice versa.)
Essentially, the difference in this case seems to be more on the order
of being able to distinguish from zero one parameter in an inseparable
formula involving multiple parameters vs. being able to distinguish the
latter whole AND substantively meaningful formula from zero, and I guess
I would lean toward putting more weight on the latter result.

Thoughts? Suggested readings? (I seem to recall a recent piece, maybe to
the PolMeth WPA, on linear interactions in logit/probit that emphasized
this distinction, e.g.)

Thanks!
Rob

************************************************************************
*
Robert (Rob) J. Franzese, Jr.                  US Mail:   (ISR Room
4246)
Assoc. Prof. Political Science                              P.O. Box
1248
The University of Michigan                       Ann Arbor, MI
48106-1248
Research Assoc. Prof.                        TeleComm:
[log in to unmask]
Center for Political Studies                        734-936-1850
(office)
Institute for Social Research,                         734-764-3341
(fax)
426 Thompson St., Room 4246
http://www-personal.umich.edu/~franzese
------------------------------------------------------------------------
-
Interim Director, Center for Political Studies,
                  Institute for Social Research,
                  University of Michigan, Ann Arbor: Sep 2006-May 2007
************************************************************************
*