The subsequent article by Drukker and Wiggins (2004) (cf. McCullough
and Vinod's (2004)) does not alter McCullough and Vinod's original
point regarding the poor quality of statistical software for even
estimating a *probit*. Although, in the data example, the Hessian can
be made well behaved after a rescaling, software programs, such as
Gauss, were reporting solutions under a scaling where there should be
none given the limitations of computer numerical precision, and the
software packages failed to report the problem. (note: the software
was not doing an internal rescaling which is what good software should
do with these problems). Since the well scaled solution isn't very
different from Shachar and Nalebuff's (1999) original solution, their
substantive finding survives, but the issue remains that software was
claiming to have converged even though the the Hessian was
ill-conditioned. And, in general, the well scaled solution need not be
near the poorly scaled one. Rescaling has long been known to be an
important issue in optimization and indeed even for inverting (X'X)-1.
See, for example, Gill, Murray and Wright (1981).
And if this issue is too subtle for you to worry about regarding
software packages, recall the infamous WinBUGS syntax interpreter bug
which parsed -(b-c) as -b-c. Screwing up the basic properties of
arithmetic: http://www.cognigencorp.com/nonmem/nm/99jun042004.html
Jas.
=======================================
Jasjeet S. Sekhon
Associate Professor
Travers Department of Political Science
Survey Research Center
UC Berkeley
http://sekhon.berkeley.edu/
V: 510-642-9974 F: 617-507-5524
=======================================
Robert W. Walker writes:
> Just to clarify Jas's comment about McCullough and Vinod (2003), the
> followups in the AER show that McCullough and Vinod were largely
> incorrect [see the replies by Wiggins and Drukker (2004) and
> McCullough and Vinod (2004)], though there is much to be said for the
> spirit of their paper. In the end, the solution indeed exists with
> some rescaling and the solution is quite close to the result
> originally reported by Shachar and Nalebuff. My suspicion is that
> they were unlucky, there are plenty of false numerical solutions out
> there; they just found one that appeared false and was not.
>
> Their conclusion [in 2004 and other papers] concerns a replication
> policy and the need for both code and data. Whether one is
> inherently skeptical or otherwise of the programming talents of
> social scientists is a non-issue; we have no way of verifying this
> conclusion without evidence and we seem to have decided, as a
> discipline, that we would just rather not know. The most pernicious
> of falsehoods survive under the guise of science without the critical
> attributes that make them scientific -- reproducibility.
>
> Best,
> RWW
>
> Robert W. Walker
> Assistant Professor
> Department of Political Science
> Program in Applied Statistics and Computation
> Washington University in Saint Louis
> Campus Box 1063
> One Brookings Drive
> Saint Louis, Missouri 63130-3899
> rww at wustl.edu
> http://rww.wustl.edu
>
> **********************************************************
> Political Methodology E-Mail List
> Editor: Karen Long Jusko <[log in to unmask]>
> **********************************************************
> Send messages to [log in to unmask]
> To join the list, cancel your subscription, or modify
> your subscription settings visit:
>
> http://polmeth.wustl.edu/polmeth.php
>
> **********************************************************
**********************************************************
Political Methodology E-Mail List
Editor: Karen Long Jusko <[log in to unmask]>
**********************************************************
Send messages to [log in to unmask]
To join the list, cancel your subscription, or modify
your subscription settings visit:
http://polmeth.wustl.edu/polmeth.php
**********************************************************
|
