Neal and Jake raise a good point regarding the quality of code written
by most social scientists. But the quality of professional
statistical software also leaves much to be desired. For examples of
serious quality issues see the work of B. D. McCullough including his
2003 AER article with Vinod. In this article, a large number of
software packages give a solution when none exists because of
quasi-complete separation in a probit regression. The probit
likelihood under the usual identifying assumptions is globally concave
hence it is considered an easy optimization problem. And
quasi-complete separation is easy to check for with analytical first
and second derivatives or with a good implementation of a modern
numerical derivative algorithm. If these packages fail with probit,
forget about more complicated problems. "Verifying the Solution from a
Nonlinear Solver: A Case Study," American Economic Review 93(3),
873-892, 2003.
And there was the infamous WinBUGS syntax interpreter bug which parsed
-(b-c) as -b-c. Screwing up the basic properties of arithmetic---cool!
See, http://www.cognigencorp.com/nonmem/nm/99jun042004.html
Of course, I trust code from the R core team or SAS far more than that
from a random person, political scientists or not. But show me the
source and the results of rigorous regression tests----no, not that
kind of regression, see
http://en.wikipedia.org/wiki/Regression_testing.
Cheers,
Jas.
=======================================
Jasjeet S. Sekhon
Associate Professor
Travers Department of Political Science
Survey Research Center
UC Berkeley
http://sekhon.berkeley.edu/
V: 510-642-9974 F: 617-507-5524
=======================================
Jake Bowers writes:
> Hi All,
>
> Thanks to all for the insightful and interesting comments about R,
> Stata, SPSS as well as about my off topic question!
>
> I had to post one more time, just to articulate one important
> criticism of encouraging wide adoption of lower-level, high-level
> languages (like C and Fortran): quality control. As Neal Beck put it:
>
> " "do you really want to now trust our colleagues to not only do good
> work but also to program correctly?" - how many gauss/r complicated
> maxlik/simulation resuls do you really believe?"
>
> Although as Jas and Phil note, the durability and power of these
> languages are not in doubt, Neal's point deserves mention. Heck, I
> know I have plenty of worry in my heart about and endlessly test my
> own programs.
>
> Best
>
> Jake
>
>
> Jake Bowers
> currently:
> Robert Wood Johnson Health Policy Scholar, '05-'07
> Institute for Quantitative Social Science
> Harvard University
>
> on leave:
> Assistant Professor of Political Science
> Faculty Associate in the Center for Political Studies, ISR
> University of Michigan
>
> http://www.umich.edu/~jwbowers
>
**********************************************************
Political Methodology E-Mail List
Editor: Karen Long Jusko <[log in to unmask]>
**********************************************************
Send messages to [log in to unmask]
To join the list, cancel your subscription, or modify
your subscription settings visit:
http://polmeth.wustl.edu/polmeth.php
**********************************************************
|
