The following is the National Academy of Sciences’ review of the draft NTP Monograph on the Systematic Review of Fluoride Exposure and Neurodevelopmental and Cognitive Health Effects
Excerpts
HUMAN EVIDENCE
NTP based its conclusion in the monograph primarily on human evidence. NTP considered the human evidence to be “relatively robust” and evaluated the association of fluoride exposure and neurodevelopmental and cognitive effects as reported in 82 publications. Although it evaluated all publications, its confidence in its conclusion is primarily based on the studies that were rated as having a lower risk of bias; NTP concluded that studies rated with a higher risk of bias did not affect its confidence in its hazard conclusion. The committee had substantive concerns regarding NTP’s evaluation of the human evidence as noted below.
The strategy used for the literature search indicated that NTP used FAN as a source to identify relevant literature. The process by which FAN identified and selected studies is unclear, and that uncertainty raises the question of whether the process could have led to a biased selection of studies. Such a concern raises the need for a formal evaluation of any potential bias that might have been introduced into the literature-search process. Another issue with the literature is that it appears that multiple publications are based on a single study and thus should not be considered independent studies. That lack of independence needs to be addressed in evaluating the findings and conclusions.
Several issues in the evaluation of risk of bias of individual studies were identified.
First, there appeared to be inconsistent application of the risk-of-bias criteria across studies, perhaps stemming from differences in the approaches presented in the protocol and monograph.
Second, the committee identified many cases in which NTP’s evaluation of confounding was insufficient, difficult to understand, or applied inconsistently across studies. NTP should develop clear criteria that are defined in the protocol to identify critical confounders and, if these are not consistently applied to individual studies, explain why some potential confounders are considered to be of greater importance in some studies and not others. NTP should also address critical aspects of confounding, such as magnitude and directionality.
Third, NTP noted the possibility of exposure misclassification in several cases but did not discuss its likely magnitude and direction and did not discuss it in the context of whether a given study reported an association. The failure to address exposure misclassification thoroughly and consistently raises the question of whether NTP’s evaluations were sufficient and supported its conclusion.
Fourth, it is imperative to protect examiners from information about exposure that could bias their administration and interpretation of outcome assessments, especially when they are assessing cognition or other neurobehavioral outcomes in human studies. Several studies reviewed by NTP did include information on techniques of blinding of examiners, but many did not. Because failure to blind examiners might result in a high risk of bias of study results and conclusions, NTP should consider this aspect more carefully when assessing the risk of bias of human studies.
Fifth, NTP in some cases classified studies as having a low risk of bias when the measure of the neurodevelopmental and cognitive outcome was seriously flawed. Given the importance of that outcome in determining whether fluoride is hazardous, its proper measurement should be considered more carefully.
Finally, the committee is concerned that the studies included in the systematic review did not undergo rigorous statistical review. That flaw is problematic because some of the studies identified as having low risk of bias did not adequately account for the hierarchical structure of their data or had errors in their summary statistics—faults that compromised their internal validity.
The committee also identified several issues with the analysis, summary, and presentation of the data. A key conclusion of the monograph is that the results of the epidemiologic studies consistently show a positive association. Although the desire to provide a simple summary of a complex array of evidence is understandable, doing so requires comparing studies that have similar parameters, and this was not done in the monograph. In fact, the studies that are reviewed in the monograph used various measures of fluoride exposure and analytic techniques and evaluated neurodevelopmental and cognitive outcomes at different developmental times. The committee recognizes that drawing conclusions always requires aggregating or summarizing data that have some degree of heterogeneity among other considerations, but the monograph should juxtapose results across broadly comparable studies and use that information to provide a text summary of the patterns observed. If comparing “like to like” results yields consistent results across all measures, ages, exposure sources, statistical approaches, and exposure ranges, taking random error into account, that will indeed warrant a statement that results consistently show adverse effects. However, the monograph does not provide the evidence in a manner that leads to that conclusion. The committee notes that NTP did not conduct a meta-analysis. Given that meta-analysis is a useful tool for aggregating and summarizing data and analyzing comparable studies, the committee strongly recommends that NTP reconsider its decision not to perform one.
Lastly, the discussion section of the monograph provides an informal assessment of the evidence with regard to exposure and concludes that adverse health effects are observed largely in association with exposures above those associated with water fluoridation. The basis of that conclusion is not apparent and seems to contradict the earlier assertion that nearly all the studies are positive, including ones that evaluated groups exposed to lower concentrations. More important, as noted above, this discussion gives a false impression that NTP conducted a formal dose–response assessment. NTP should be clear that the monograph cannot be used to assess what concentrations of fluoride are safe.
NTP CONCLUSION
The monograph “concludes that fluoride is presumed to be a cognitive neurodevelopmental hazard to humans. This conclusion is based on a consistent pattern of findings in human studies across several different populations showing that higher fluoride exposure is associated with decreased IQ or other cognitive impairments in children.” The committee was tasked with assessing whether NTP satisfactorily supports its conclusion. Given the issues raised by the committee regarding the analysis of various aspects of some studies and the analysis, summary, and presentation of the data in the monograph, the committee does not find that NTP has adequately supported its conclusion. That finding does not mean that the conclusion is incorrect; rather, further analysis or reanalysis as noted in the present report is needed to support conclusions in the monograph.
ANIMAL EVIDENCE
The monograph presents a systematic review of animal studies of fluoride exposure related to learning and memory that were published from 2015 to August 20, 2019, as an update to the NTP systematic review published in 2016. Examination of the animal studies published since 2015 led NTP to conclude that the animal data are inadequate to support conclusions on human cognitive effects.
The committee has serious concerns about the risk-of-bias evaluations of the animal literature and whether they identified important threats to internal validity that are specific to neurobehavioral outcomes in animal tests. The guidance in the protocol touched on some of the threats, but insufficient details appear to have been provided to ensure a rigorous, consistent evaluation of neurobehavioral studies.
Specifically, the committee had concerns about the risk-of-bias evaluations for attrition, outcome assessment, and statistical analyses. It also found one element—maternal, fetal, and pup toxicity—that did not appear to have been adequately captured in the risk-of-bias criteria. Although severe postnatal toxicity was mentioned in risk-of-bias evaluations of some studies, it is unclear whether maternal, fetal, and pup toxicity was routinely assessed for all studies. Such effects can seriously confound interpretation of neurodevelopmental effects. Overall, the committee found that some studies cited in the monograph had severe methodologic shortcomings that could warrant exclusion from the body of evidence.
NTP justifies its conclusion that the animal evidence is inadequate on the grounds that it is not possible to separate effects from effects on locomotor activity. Although locomotor activity can affect learning and memory outcomes, it has been demonstrated many times that the presumed influence of locomotor activity on learning and memory does not occur. Thus, the committee does not agree with NTP’s rationale for dismissing the animal evidence and finds that it is a mistake to dismiss studies of learning and memory because of minor, brief locomotor-activity changes or when other assessments can rule out confounding locomotor effects in cognitive assessments.
Given the serious concerns raised by the committee in the present report, NTP will need to decide whether it should reanalyze the animal evidence. The committee cautions, however, that given the poor quality of the animal studies that it reviewed, revising the systematic review to address the concerns highlighted might not affect the ultimate finding that the animal evidence is inadequate to inform conclusions about fluoride exposure and neurodevelopmental and cognitive effects in humans.
Literature Search
In the monograph, NTP clearly displayed the results of the literature search and screening process in a PRISMA flow diagram, a widely accepted framework for reporting a screening process and the ultimate number of included studies. The committee, however, had some concerns regarding NTP’s literature search strategy. One of the sources used to identify articles for the systematic review was the Fluoride Action Network (FAN). The committee acknowledges FAN’s efforts in providing several studies that appear to be relevant for the review. However, the process by which FAN identified and selected studies is not clear. FAN identified a number of studies published in Chinese language journals—some of which are not in PubMed or other commonly used databases—and translated them into English. That process might have led to a biased selection of studies and raises the question of whether it is possible that there
are a number of other articles in the Chinese literature that FAN did not translate and about which NTP is unaware. NTP should evaluate the potential for any bias that it might have introduced into the literature search process. Possible ways of doing so could include conducting its own searches of the Chinese or other non–English-language literature and conducting subgroup analyses of study quality and results based on the resource used to identify the study (for example, PubMed vs non-PubMed articles). As an initial step in such evaluations, NTP should consider providing empirical information on the pathway by which each of the references was identified. That information would also improve understanding of the sources that NTP used for evidence integration and the conclusions drawn in the monograph. The committee emphasizes that its comments regarding FAN are aimed only at evaluating bias; they are not intended to discourage stakeholder input into the systematic-review process, and the committee acknowledges and encourages the important contributions of FAN and other stakeholder organizations in this process.
Acknowledgments
The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.
We thank the following for their review of this report:
Ana Navas-Acien, Columbia University
David Bellinger, Harvard Medical School
Weihsueh Chiu, Texas A&M University
David Dorman, North Carolina State University
Mary Gilbert, US Environmental Protection Agency
Jayanth Kumar, California State Dental Association
Karen Robinson, Johns Hopkins University
Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations of this report, nor did they see the final draft before its release. The review of the report was overseen by Martin Philbert, University of Michigan, and Jonathan Samet, Colorado School of Public Heath, who were responsible for making certain that an independent examination of the report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content rests entirely with the authoring committee and the National Academies.
The committee gratefully acknowledges the staff of the National Institute of Environmental Health Sciences for their presentations to the committee during open sessions.
Print 
PDF