title:         How to Test for Interaction in Models with Binary Dependent Variables
authors:       William Berry, Jacqueline Rubin
entrydate:     2007-01-25 17:31:44
keywords:      logit,binary dependent variable,interaction,Monte Carlo analysis
abstract:      Political scientists often use logit to test a hypothesis that two variables, X and Z, interact in influencing the probability that a binary dependent variable (BDV), Y (scored 0 or 1), equals 1.  The standard practice is to (i) estimate a model including X, Z and their product (XZ) as independent variables, (ii) determine whether the coefficient estimate for the product term is statistically significant in the predicted direction, and if it is, (iii) evaluate the hypothesis by using the coefficient estimates to compute estimates of how the effect of one of the independent variables on Pr(Y=1) varies with the value of the other.  In reality, however, logit is unlikely to capture the exact functional form of the true data generating process (DGP).  Thus, the effectiveness of the standard practice depends on whether it can overcome the likely model misspecification and yield estimates of the effects of variables on Pr(Y=1) that are close to their true values, even when the true DGP does not take the exact form of a logit equation.  We use Monte Carlo methods to assess the effectiveness of the practice: we construct numerous data sets with known �true� relationships among X, Z and Pr(Y=1) � some of which involve strong interaction [i.e., the effect of X on Pr(Y=1) varies substantially with Z], and others in which effects are additive [i.e., the effect of X does not vary with Z], but none of which is exactly a logit specification � and see whether the practice successfully detects the true extent of interaction.  We find that determining whether the coefficient estimate for the product term is statistically significant often yields a deceptive conclusion about whether there is interaction in influencing Pr(Y=1); indeed even the sign of this coefficient can be misleading.  For this reason, the standard practice for testing a hypothesis that X and Z interact in influencing Pr(Y=1) should be abandoned.  Instead, all attention should be focused on estimates of the how the effec
t of X on Pr(Y=1) varies with Z.  Our good news is that when the true relationship is interactive, such estimates derived from a logit model including X, Z and their product are very likely to yield a correct inference about whether interaction is present even when the true functional form is not logit.  Thus, when there is strong a priori theory that interaction is present, a logit model including a product term seems to yield satisfactory estimates of the extent of the interaction.  But our Monte Carlo analysis shows that the inclusion of a product term in a logit model tends to make true relationships in which no interaction is present appear strongly interactive.  For this reason, when a priori theory is weak, there are situations in which it is better to exclude the product term.  Empirical measures of model performance � e.g., ROC scores � can help researchers decide whether it is best to include a product term.

http://polmeth.wustl.edu/retrieve.php?id=685