Confounder Selection in Environmental Epidemiology: Assessment of Health Effects of Prenatal Mercury Exposure

Purpose

The purpose of the study is to compare different approaches to the identification of confounders needed for analyzing observational data. Whereas standard analysis usually is conducted as if the confounders were known a priori, selection uncertainty also must be taken into account.

Methods

Confounders were selected by using backward elimination (BE), change in estimate (CIE) method, Akaike information criterion, Bayesian information criterion (BIC), and an empirical approach using a priori information. A modified ridge regression estimator, which shrinks effects of confounders toward zero, also was considered. For each criterion, uncertainty in the estimated exposure effect was assessed by using bootstrap simulations for which confounders were selected in each sample. These methods were illustrated by using data for mercury neurotoxicity in Faroe Islands children. Point estimates and standard errors of mercury effects on confounder-sensitive neurobehavioral outcomes were calculated for each selection procedure.

Results

The full model and the empirical a priori model showed approximately the same precision, and these methods were (slightly) inferior to only modified ridge regression. Lower precisions were obtained by using BE with a low cutoff level, BIC, and CIE.

Conclusions

Standard analysis ignores model selection uncertainty and is likely to yield overoptimistic inferences. Thus, the traditional BE procedure with p = 5% should be avoided. If data-dependent procedures are required for confounder identification, we recommend that inferences be based on bootstrap statistics to describe the selection process.

Introduction

In observational studies, exposure values are not assigned randomly to study subjects. Therefore, exposed and unexposed subjects are likely to differ on a number of variables. If some of these variables are affecting the outcome, the crude relation between exposure and outcome may give a distorted (confounded) reflection of the causal exposure effect. Control of confounding factors has been one of the central issues in epidemiologic research, and adjustment routinely is achieved by stratification or by applying some sort of multiple regression analysis.

The important question now is how the investigator decides which of the potential confounders to control for and which to ignore. Often prior knowledge about population relations is weak, and the data are used in the confounder identification process. Unfortunately, no standard procedure is fully satisfactory. One approach (backward deletion) is based on stepwise testing of effects of potential confounders on the outcome, whereas another (change in estimate [CIE]) removes potential confounders as long as the exposure effect does not change too much. Despite frequent use of such automated techniques, very little formal knowledge is available about the impact of the selection process on subsequent analysis of the exposure effect. Results from simulation studies of the simple situation in which only one potential confounder is present seemed to favor the CIE method over methods based on significance testing 1, 2, and other simulation studies indicated that forward selection procedures are of limited value in epidemiology (3). Results from the related problem of “best subset selection” suggest that precision is overestimated if inference is based on a model selected by using stepwise significance testing 4, 5. Although there is widespread awareness of this, the selection process is almost always ignored in the final analysis, and inferences are made as if the selected model was given a priori. Breiman (6) described this routine procedure as a “quiet scandal.”

In this report, we compare different strategies for confounder selection by using data from an epidemiologic study performed in the Faroe Islands to investigate adverse health effects of prenatal mercury exposure. Methylmercury is a common contaminant in seafood and freshwater fish. Although adverse effects were shown unequivocally in poisoning incidents, implications of lower level exposures in fish-eating populations have been controversial (7). The original analysis of Faroese data showed adverse effects of prenatal mercury exposure on childhood cognitive development (8), whereas a study carried out in the Seychelles did not report significant effects (9). In 1998, the White House therefore arranged a workshop to assess the quality of the main mercury studies. It was concluded that the Faroese study had chosen an appropriate approach to confounder identification and adjustment (10). However, further analysis was outlined, including adjustment for new potential confounders. Because of the emphasis on residual confounding and the public health implications, these variables are included in advanced analyses presented next. The mercury effect is estimated by using conventional confounder selection criteria, as well as the method originally used by the Faroese study group (8).

Furthermore, adjusted precision estimates, which take the confounder selection process into account, are calculated by using the bootstrap method.