Flawed MIREC fluoride and intelligence quotient publications: A failed attempt to undermine community water fluoridation.

3.1 Measurement of exposure: fluoride

For a foetus to be exposed to fluoride, fluoride must reach the foetus via the bloodstream. Because foetal blood samples are not available, measuring fluoride exposure of the foetus is necessarily indirect. The researchers presumed that maternal intakes and blood levels of fluoride correspondingly affected the foetus. The MIREC database used three methods to attempt to determine fluoride exposure of the pregnant woman as a proxy measure of foetal exposure: (1) Maternal exposure through community water fluoridation; (2) Analysis of self-reports by the pregnant woman of what the woman drank; and (3) Measurement of fluoride in three untimed spot urine samples from the pregnant woman.

The first measure is uncontroversial; community water fluoridation was a practice in three of the six cities that were studied—Hamilton, Toronto and Halifax, and not in Vancouver, Kingston and Montreal. The concentrations of fluoride in the water are publicly available information. This is a measure of fluoride exposure experienced by the whole community. But these authors wished to undertake individual intake analyses to provide greater precision in determining effects, so they analysed two other exposure measures.

The second exposure measure—self-reports of intake—is weak. The self-reports provide estimates of how many ‘cups’ of tap water and water-based beverages were usually consumed daily, during the pregnancy at trimesters one and three. Yet, neither the MIREC beverage questionnaire nor the estimates of fluoride intake from those data has been validated, as stated by some of these authors themselves.⁵ Such estimates are subject to recall error. Self-reports of usual energy intake by dietary questionnaire are unreliable.¹⁴ Reports of fluid intake based on people’s estimates of what they usually drink are similarly uncertain.

The third fluoride measurement—measuring fluoride in maternal urine—is an attempted proxy measure of maternal blood levels. The MIREC database contains only spot urine samples from the pregnant women collected in each trimester of the pregnancy, at delivery and in the early postnatal period (up to 10 weeks). (These ‘spot’ samples had no limits or requirements on time of day, or recent intake of fluoride.) The researchers detail how the fluoride levels in the urine samples were chemically assayed, with checks for validity. However, there are at least two problems with this third measure of chronic (i.e. duration of pregnancy) fluoride exposure to the foetus.

This measurement ‘enhancement’ is, in fact, minimal. Pooling three samples reduces the random variation of single measurements, but does not make the measurement ‘more sensitive’, which would imply being able to detect smaller quantities. (Perhaps, the authors mean ‘consistent’, i.e. with lower variation). But even averaging three spot urine samples of the pregnant woman cannot reliably tell us the woman’s chronic fluoride levels or intake. Further, by excluding women who did not supply three samples, the authors have limited the study population even further, with the possibility of selection bias (e.g. those who provided three urine samples might be more consistent in other behaviours compared to those who did not). That bias might confound any relationships with child IQ.

In their 2016 article, Thomas et al.²⁰ report that the correlation coefficients do not show a correlation between the amount of fluoride in a pregnant woman’s urine and in her blood. They assessed correlation coefficients relating fluoride levels in the woman’s urine (creatinine-adjusted) and her blood plasma at two different stages of pregnancy. Among the participants, the correlations were 0.29 and -0.24 during early and late pregnancy respectively. These values are below 0.5—the recommended standard to indicate ‘moderate’ effects.²¹ Furthermore, a multiple regression analysis did not show an association between the fluoride biomarkers of the pregnant woman’s urine and blood plasma. In other words, the 2016 Thomas article describes low and variable, sometimes negative correlation of creatine-adjusted maternal urine fluoride with blood plasma fluoride, at least in Mexico City where fluoride supplementation occurs through fluoridated salt rather than water. It is unclear why the authors assert the opposite of what Thomas et al. state.

To summarize concerns about measuring foetal exposure to fluoride using spot maternal urine samples, there is a consensus in the scientific community that a spot maternal urine sample is not a valid biomarker of an individual’s chronic exposure to fluoride.²² Spot urine tests are not valid to quantify fluoride exposure in a pregnant woman or her foetus, no matter how precisely the researchers measured fluoride in the spot urine samples. Because the researchers report accessing only spot maternal urine tests in the MIREC database to assess foetal exposure (three in total) and there is low correlation of fluoride levels in urine and blood plasma, users of that database simply cannot reliably measure the pregnant woman’s or the foetus’ exposure to fluoride.

3.2 Measurement of the outcome: IQ of children

The MIREC data claim to report the IQ of children using the Wechsler Preschool and Primary Scale of Intelligence, Third Edition, (WPPSI-III). But the IQ testing of the children in the MIREC sample is unreliable.

To measure the IQ of children in a comparable way, one must use standardized test administration. This testing approach is important especially because of the young age of the children—aged 3 to 4 years at the time of the IQ test. (A small subset of the children were aged 5–6 years when tested.) IQ testing of young children is notoriously difficult because the test must be administered and scored by an interviewing examiner, unlike the more straightforward written ‘tests’ that older children can complete on paper. Moreover, assessing such young children challenges even experienced examiners.²³ Little is published about sex differences in attention in preschool children, but they are found in slightly older children, so sex differences might affect test scoring.²⁴ In addition and irrespective of the age of the research subject, major sources of errors can occur in IQ testing. When different people administer the test, they introduce the possibility of differences in administration, scoring and interpretation across the population being studied. These issues of variability are well known. The National Institute of Environmental Health Sciences report, Evaluating Features and Application of Neurodevelopmental Tests in Epidemiological Studies, provides examples of factors that could present problems and introduce bias in the test results.²⁵ As Styck and Walsh report, ‘examiner errors occur frequently and impact index and FSIQ [full scale IQ] scores’, concluding that ‘current estimates for the standard error of measurement of popular IQ tests may not adequately capture the variance due to the examiner’.²⁶

So, for young children, it is vital to assess IQ using a consistent and reliable approach: Ideally, the same test administrator, with quality checks. Yet, the MIREC IQ data were created by a ‘single staff person from each study site’ according to Etzel et al.²⁷; in other words, there were different MIREC IQ assessors in each of six cities. If examiners differed in the way in which they coded the children’s behaviours into IQ, then this difference could lead to an apparent effect of fluoridation even if none existed. Thus, systematic variations among the six examiners might be a confounding variable. The examiners’ backgrounds are described only as being ‘research assistants’, not formally trained psychometricians. According to Etzel et al., ‘Study staff from each participating study site completed a 3-day training session that was led by a PhD-level psychologist (E.O.) and focused on specialized training of these assessment tools’.²⁷ According to some MIREC fluoride-IQ researchers,¹⁷ regular site visits were made to observe these examiners, and test protocols were double scored. However, none of the MIREC fluoride-IQ articles describes the training process. A doctoral degree does not, of itself, provide the specialized training necessary to administer the WPPSI-III and it is not stated whether the psychologist who trained the administrators was an accredited, formally trained administrator of the WPPSI-III, as is desirable for test validity. Even the best testers will differ within themselves (if they retested the same subject) and between one another. Their accuracy may ‘drift’ over time. Analysis of accuracy would enable us to understand the variation, so we can understand how much difference in test score is meaningful. Therefore, validation of the IQ test scores would normally be expected in a research study. But none of MIREC fluoride-IQ articles present quality assurance statistics of the results, comparisons over time or among the study centres. For all these reasons, the MIREC IQ data do not permit anyone to know whether comparisons of IQ in fluoridated and non-fluoridated cities are meaningful.

Such failure to assure that the IQ assessments are valid is concerning. It appears that either the MIREC fluoride-IQ researchers are oblivious to the issues that cause variability in test results, or they have chosen not to address these issues—either in their articles or in their response to expressed concerns.^{28, 29} The issues of variability and therefore potential bias created by using different assessors of IQ in each city make it difficult to accept any claim of a small IQ decrement in children associated with fluoride exposure.

Indeed, if foetal fluoride exposure at fluoridation levels really did affect IQ, then the mean IQ of children in fluoridated cities should be systematically different from that in non-fluoridated cities. In the total sample of children, the mean IQ scores were 108.07 (SD = 13.31) and 108.21 (SD = 13.72) in non-fluoridated and fluoridated communities respectively. Yet, the children’s IQ scores varied by 4–8 points among the six cities regardless of the fluoridation status.^{28, 30} This anomaly suggests that differences among cities in measurement or other factors such as maternal IQ are at least as large or larger than the effects of water fluoridation.

In the MIREC Green et al. article,⁵ the authors reported in a table that average IQ was the same in fluoridated and unfluoridated cities but did not discuss the null finding. They engaged in post hoc analysis claiming differences between boys and girls, reporting this secondary analysis as the main effect. They allege that there is a 4-point difference between boys and girls, but given the measurement variation in scores for IQ testing, 4 points is not a large or even medium-sized effect.³¹ Indeed, according to Ferguson²¹ an IQ difference of over six points is required for the ‘recommended minimum effect size representing a “practically” significant effect for social science data’.

Figure 3A in Green et al.⁵ shows the relationship between the (disputed) estimate of maternal fluoride concentration and IQ for boys and girls. The figure reveals a very wide dispersion of fluoride levels, with most children being exposed to low levels of fluoride. The scores of the 10 or so boys exposed to the highest values of fluoride will have an undue influence on the regression coefficient. In other words, the presence of a few boys who were exposed to high levels of fluoride will lead to the relationship between fluoride concentration and IQ being overestimated for the group as a whole.

The Green et al.⁵ article claims an association between fluoride exposure and IQ and purports to find only a 4-point difference in boys but not in girls. The random variance and bias due to variation in test administrators can account for far more than that difference.

By reporting an IQ point difference that was assessed using different IQ test administrators in different cities without careful validation, that is less than a third of the standard deviation of the test, and that varies independently of the city’s fluoridation status, the MIREC database does not offer high-quality, reliable data about the children’s IQ. Therefore, finding a decrease of four points in an IQ score only in boys (but not in girls) for every 1 mg/L increase in alleged fluoride concentration in urine is almost certainly a spurious finding.^32–34

Measurement error has often been incorrectly considered to result only in a ‘bias toward the null’. That is, errors in measurement make it less likely that one can observe a true association. However, in other situations, measurement error can result in a spurious over-estimate of the true association.³⁵ That may be the case here. Other biases, such as measurement errors of outcome and covariates, confounding and selection bias, are also present in these MIREC fluoride-IQ articles.

3.3 Lack of an appropriate conceptual model to assess causation

Addressing the complex question of the causal effect of fluoride on children’s IQ requires models that incorporate knowledge and understanding of the causal mechanisms.³⁶ The MIREC fluoride-IQ articles apparently disregard several prenatal and postnatal factors that may influence children’s IQ, such as maternal IQ, paternal IQ, gestational age, birth weight, breastfeeding, disease infections, traumatic events, exposure to environmental pollutants and social factors (e.g. attendance at daycare and kindergarten, and other social interactions). Conducting data analysis without an operational model that translates abstract concepts into measurable variables can oversimplify a complex problem.³⁷

A conceptual model such as a directed acyclic graph can guide variable selection as well as indicate scenarios that might give rise to unanticipated ‘backdoor pathways’.³⁸ In other MIREC articles, some of the MIREC fluoride-IQ authors used a simple version of a directed acyclic graph,^{39, 40} suggesting that the MIREC fluoride-IQ authors are aware of the value of the organizational tool but chose not to use it in all the articles they created.

Another major issue that raises questions about the validity of this set of articles is the absence of an operational model. The MIREC fluoride-IQ authors are not explicit about the question they are asking and the assumptions guiding their data analysis. Their analytical strategy relies on the selection of confounders based on p-values, a method widely acknowledged as not providing a reliable measure of evidence regarding a model or hypothesis.⁴¹ This modelling strategy aligns more with prediction models rather than a causal inference model that specifies the role of each variable, including exposures, mediators, effect modifiers and confounders, which specification is necessary to address a causal question.

As mentioned, the Green et al.⁵ MIREC fluoride-IQ article reported in a table that the main overall effect was null. The question arises as to why the authors emphasize their post hoc analysis that claims to show a 4-point reduction for boys. This isolated secondary finding lacks empirical corroboration, which is necessary for it to be regarded as causative. The authors themselves rightly note that such a finding ‘requires more investigation’.

Several post-publication letters with respect to the Green et al. article specifically^{32, 34} and reviews of the MIREC fluoride-IQ articles generally^{33, 42–44} raise concerns regarding the overinterpretation of the findings resulting from their analytical approach. These concerns include inappropriate regression modelling based on a convenience sample of clusters of individuals and correlated observations, and inadequate attention to influential outlying observations. The MIREC fluoride-IQ articles in Table 1 show improper focus on subgroup analysis, and the failure to adjust significance levels given that the authors engaged in multiple testing. This ‘data-dredging’ approach ensures that at least some of the p-values will be ‘significant’ by chance. Such approaches might assist in exploratory analyses, but need replication before being accepted as evidence of an association – let alone cited as proof of causation, as some of the MIREC fluoride-IQ authors assert.^{45, 46}

3.4 Other studies

Other published studies have addressed the same question: whether fluoridation affects foetal and child IQ. One replication study in Spain, which also used spot maternal urine as a proxy for foetal fluoride exposure, but different standard IQ tests, showed opposite effects: Fluoride led to a slight increase in boys’ cognition, but not girls’.⁴⁷ Long-term cohort studies with larger samples and with IQ measurements at older ages, in New Zealand⁴⁸ and of military conscripts in Sweden⁴⁹ have shown no effect of fluoridation on cognitive development. A national cohort study of Australian children reported no impact of exposure to water fluoridation throughout life on measures of school-age executive functioning, emotional and behavioural development.⁵⁰

Our critique is important because, among other reasons, members of the MIREC fluoride-IQ research group have self-cited their MIREC-based fluoride-IQ studies. For example, a recent report¹¹ of an alleged association between fluoride and cognition combined Danish data with data from two other studies, one of which was the MIREC study. The Danish data alone showed no association between fluoride and cognitive performance and, as our analysis has shown, the MIREC data cannot be used to claim an association between maternal fluoride exposure and diminished IQ. Consequently, this recent article¹¹ is also invalid.