Critique of the review of ‘Water fluoridation for the prevention of dental caries’ published by the Cochrane Collaboration in 2024.

Community water fluoridation (CWF) is one of the most comprehensively studied public health measures, with a body of evidence spanning more than seven decades.^1,2,3 Globally, CWF has been associated with significant reductions in dental caries, particularly among children and socioeconomically disadvantaged groups.^1,2 Numerous population-level evaluations have been made across time, geography, and varying oral health environments. Such evaluations have demonstrated the preventive impact and contribution that CWF has in reducing oral health inequalities. However, assessing the full public health value of CWF demands methods that reflect the nature of the intervention. Such research methods must be long-term, population-wide, and embedded within dynamic real-world systems.

Against this backdrop, systematic reviews of randomised controlled trials (RCTs) such as those conducted by the Cochrane Collaboration have considerable influence on evidence, policy and public perception.^4,5,6 However, the method chosen to assess CWF can either clarify or obscure the true weight of the evidence. The most recent Cochrane Review,⁷ published in 2024, falls into the latter category. While the review presents itself as a rigorous and impartial synthesis of the evidence, the design and underlying assumptions render the methodology unsuited to evaluating a population-based public health intervention such as CWF.

The retention of water fluoridation as a public health measure has been much debated recently in the USA, with the States of Florida and Utah ceasing fluoridation. An example of the misuse of the Cochrane 2024 review can be found in the public hearing submissions to the Pinellas County Commission,⁸ Florida: ‘Cochrane again found no clear evidence of benefit from fluoridation’. This is the background for this critique as we argue that evaluating the evidence using new advances in evaluation demonstrate the true benefits of CWF.

The core flaw is conceptual: the review applies evidence hierarchies that privilege RCTs and narrowly defined inclusion criteria, in a context where RCTs are rarely feasible or ethical, and where natural experiments and high-quality observational studies are the norm. This approach effectively excludes a substantial and policy-relevant portion of the evidence base, leading to an inevitable conclusion of ‘insufficient evidence’. This reflects the constraints of the method more than the reality of the intervention.

This overarching limitation gives rise to a cascade of secondary flaws in the methods used which are conceptual and interpretative. These include the misapplication of bias assessment tools designed for clinical interventions, inconsistent treatment of older versus newer studies, and a failure to contextualise findings within the broader corpus of international research and public health practice. Comparison with the 2015 Cochrane Review⁹ shows that the 2024 update contains two newer studies by Blinkhorn¹⁰ and Goodwin¹¹ (Blinkhorn et al. was included in the 2015 review as ‘unpublished’). In the update review, the ROBINS-I tool is applied to try to control for biases resulting from the nonrandomised nature of much of the evidence, the fundamental framing of the review remains unchanged from the 2015 iteration, and indeed from the ‘York Review’ 25 years ago.¹² As a result, the same biases persist.

This critique addresses both the macro-level methodological misfit of applying a narrow clinical trials model to a complex public health intervention, and the micro-level inconsistencies and omissions within the conduct and reporting of the 2024 review. We argue that these are not separate concerns but are intrinsically linked: the choice of an inappropriate evidentiary lens inevitably affects what is included, what is excluded, and how the results are interpreted. Of further concern is the quality of one of the new studies¹¹ which has itself been extensively criticised.¹³

Our intention is to provide a constructive, evidence-informed critique that helps clarify both the limitations of the 2024 Cochrane Review and the broader lessons for evaluating population-level interventions. In doing so, we hope to promote a more appropriate and scientifically robust approach to synthesising the evidence on community water fluoridation, consistent with best practice in public health evaluation.

The 2024 Cochrane update⁷ states its objectives as follows.

• To evaluate the effects of initiation or cessation of CWF programmes for the prevention of dental caries
• To evaluate the association of water fluoridation (artificial or natural) with dental fluorosis.

Before engaging with the detail of the review, we must revisit a fundamental question: how should the effectiveness of water fluoridation be reviewed? While the main consideration is water fluoridation here, there are other vehicles of delivery such as milk and salt where the same principles of evaluation apply.

Cochrane’s Oral Health Group continues to use highly restrictive inclusion criteria which may be appropriate when evaluating tightly controlled clinical interventions, but which are fundamentally unsuited to assessing population-level public health measures. As noted in the 2016 critique¹⁴ of the 2015 Cochrane Review,⁹ ‘The American Academy of Pediatrics commented that the Cochrane Review of community water fluoridation had excluded 97% of the available evidence’. Such a restrictive use of a hierarchy of study designs, as a measure of validity, is considered by Rothman to be a ‘persistent research misconception’.¹⁵ This misconception should be avoided when investigating important public health programmes such as CWF. There is a call for a pluralistic approach in epidemiology,¹⁶ which is also important for evaluation of public health programmes.

The rigidity of the methods used rests on a longstanding privileging of RCTs, with low risk of bias and high internal validity, as the highest standard of evidence. RCTs, while invaluable in many domains of clinical research, are not a realistic standard for evaluating CWF. The random allocation of entire communities to fluoridation or control is not logistically feasible, ethically acceptable, or politically practical. Moreover, populations are not static: people move in and out of fluoridated areas, baseline risk varies across time and geography. While blinding is difficult, it has been achieved in both Hardwick’s Cheshire study¹⁷ and with radiographs in studies such as the Tiel-Culemburg evaluation.¹⁸ The Cochrane Review excluded a large number of relevant studies that impaired its ability to evaluate the effectiveness of CWF and inform policies.

There has been significant progress in the causal inference approach using observational study designs.^19,20 This progress has proved its role in public health research.²¹ The dental research circle has called for a better understanding and use of the approach in evaluating dental public health programmes.^22,23 An appropriate application of the causal inference approach using observational data can contribute quality scientific evidence of the effectiveness of population-based programmes.²⁴ Such observational studies can capture large-scale, real-world effects over time and across diverse populations, helping to avoid the lack of external validity of experimental studies, such as RCTs. An evaluation of the effectiveness of CWF would benefit from a combination of evidence from experimental and observational studies. The scientific evidence from such a combination can also better inform policy development and implementation of CWF.

Another concern related to the Cochrane Review is its exclusion of research on the evaluation of ongoing CWF programmes. With many CWF programmes in existence for many years, particularly in USA, Australia and Ireland, continuing evaluation is important to evaluate the effectiveness, contribution, and efficiency of public health programmes.^25,26 This is not easy when coverage is 100% as in the case of Singapore. However, the Cochrane Review automatically excluded those studies through its restrictive objective (initiation of fluoridation). Hence, the review ended up with a limited number of scientific studies which restricts its usefulness to inform policies and practices.

Recognising these realities, most research into fluoridation has necessarily been observational, often cross-sectional or ecological in nature. Such studies capture large-scale, real-world effects over time and across diverse populations. While these designs are more vulnerable to bias and confounding than RCTs, they remain entirely appropriate and necessary for evaluating public health interventions. Moreover, they often represent the only feasible means of understanding long-term, population-level outcomes.

By continuing to apply restrictive inclusion criteria, the 2024 Cochrane Review necessarily excluded a vast array of relevant studies, especially those conducted after fluoridation’s introduction in the mid-20^th century. Early studies that were pivotal in establishing the case for fluoridation were largely completed before 1975, and many do not meet today’s more exacting risk of bias standards. Consequently, Cochrane continues to include only a small number of studies, mostly focused on children, with no recent data on adults or the elderly.

This represents a fundamental mismatch between the methodological tool and the question it seeks to answer. Public health policy must be informed by the best available evidence and not just the most narrowly defined. A broader approach that incorporates the full range of relevant observational research is both scientifically valid and policy relevant.

Cessation of water fluoridation

The critique¹⁴ of the 2015 Cochrane Review⁹ included a lengthy discussion on methods used for evaluating the effect of cessation of CWF programmes, concluding that the highly restrictive inclusion criteria were inappropriate. Methods used in the 2024 Cochrane Review⁷ were unchanged resulting in the inclusion of just one recent study.¹¹ In brief, the inclusion criteria required a ‘positive’ control community (where fluoridation had continued) with clinical examinations both before and after cessation. As pointed out previously¹⁴ termination of CWF programmes usually happens due to regional or national decisions, such as in the case of Wick²⁷ and Stranraer²⁸ in this country, denying the possibility of a positive control. In Calgary, Canada, the effect of water fluoridation cessation was measured;²⁹ the resulting increase in caries experience was deemed unacceptable and a decision taken to re-instate CWF.³⁰ This indicates the importance of measuring the effect of terminating a public health programme.

Reducing inequalities in dental health

The Cochrane Review has not fully investigated the potential effect of CWF on reducing inequalities in dental health as it was not included in their search criteria. Several observational and cross-sectional studies suggest that CWF can have a greater impact in more deprived groups, who usually have higher caries levels.^31,32,33

By excluding these types of studies, the review ends up missing an important part of the picture. Reducing health inequalities is a key public health goal^34,35 and it would have been helpful if the review had looked at socioeconomic status as a subgroup analysis or in its interpretation of its findings. If fluoridation reduces average caries levels, that tells us nothing about whether it also reduces inequalities, a crucial question for policy.

This flaw in the 2024 review is not due to a lack of available evidence, but again a by-product of how the review was conducted. By using strict inclusion criteria that focus on highly controlled study designs, the review excluded much of the observational research where inequalities were examined. Many of the studies that explore socioeconomic variation in caries outcomes are cross-sectional or ecological in design, often using large population datasets and stratifying results by area-level deprivation or other social indicators. These ecological studies have reported greater reductions in caries among more deprived groups, suggesting that water fluoridation may help reduce oral health inequalities.^32,36,37 Clearly these studies might not meet Cochrane’s preferred design for clinical reviews, but they are still relevant when looking at public health outcomes. This reflects the broader issue in public health, where an overreliance on RCTs can lead to the exclusion of important real-world evidence.³⁸

Due to these methodological choices, the review could not meaningfully assess whether the benefits of fluoridation are distributed equally, or whether they are greater in the groups that have greater need. This limitation should have been acknowledged more clearly, especially since equity is one of the main policy arguments for introducing or maintaining fluoridation schemes.

This has been explicitly acknowledged by the UK Chief Medical Officers, who have stated that water fluoridation is an effective public health intervention for improving dental health equality across the UK.³⁹ This was also reflected in Public Health England’s report on oral health inequalities, which recommended water fluoridation as a population-level intervention with the potential to reduce disparities, particularly among more deprived communities.³³

A more flexible approach to study inclusion, or perhaps a structured narrative synthesis of socio-economic status-related findings, would have allowed this issue to be explored. Although this may not have been part of the formal objectives of the review, in its current form, the review is not able to answer one of the most important public health questions: whether fluoridation contributes to reducing dental health inequalities. Cochrane’s conclusion that the evidence on inequalities is limited is, in large part, a consequence of the methodological decisions that excluded the very studies addressing this issue.

Absence of data on adults and the older population

The limited objectives and strict inclusion criteria employed by Cochrane once again result in a notable absence of data on adults and older populations. Such groups are known to benefit from lifelong exposure to fluoridated water.^40,41,42 This absence has significant implications for understanding the full public health value of CWF.

While the ideal study design for such populations may not meet Cochrane’s standard criteria (requiring pre-fluoridation caries data, which is quite impractical for evaluations in adults), lower-grade evidence (e.g., well-designed observational studies) must be considered to ensure a complete assessment. The exclusive focus on studies involving children, particularly historic ones, leads to an artificially narrow evidence base and potentially misinforms policy decisions intended for whole populations.

Fluorosis: reanalysis without new evidence

No new studies on dental fluorosis were included in the 2024 Cochrane Review⁷ and analysis of the fluorosis data was unchanged. No acknowledging or addressing the specific criticisms raised in the 2016 critique is included.¹⁴

A significant concern is the inclusion of studies involving fluoride levels as high as 5 ppm to examine dose-response. Such concentrations bear little resemblance to regulated artificial fluoridation schemes, which are tightly controlled and typically maintained at or below 1ppm. Optimal levels vary from 0.5 in Singapore to 1ppm in the UK. The inclusion of these high-exposure studies risks inflating estimates of fluorosis prevalence and distorting conclusions relevant to CWF policy.

Moreover, the review fails to apply a consistent comparative framework to fluorosis data. While caries effects were calculated by comparing caries levels in fluoridated versus non-fluoridated areas, no such comparison was made for fluorosis. The background prevalence of fluorosis in low-fluoride (<0.2 ppm) communities is not zero, and the failure to subtract this baseline from prevalence figures in fluoridated areas leads to an overestimation of fluoridation-attributable fluorosis. This omission is particularly problematic given that other causes, for example amelogenesis imperfecta, can mimic fluorosis clinically. A variety of indices were used to record ‘dental fluorosis’: in 8 of the 90 studies, the developmental defects of enamel index was used which records all enamel defects whether considered to be ‘fluorosis’ or not.⁴³

Another limitation lies in the geographic diversity of the fluorosis studies included. Climate has a direct effect on dental fluorosis.⁴⁴ Early studies suggested that water consumption was related to temperature.^45,46 Since then, a country, or region within a large country, will determine the optimal concentration of fluoride for CWF taking temperature into account; as recommended by the World Health Organization.⁴⁷ For example, the UK, with a temperate climate, opts for 1.0 ppm, while Singapore, with a tropical climate, opts for 0.5 ppm. The Cochrane Review did not consider this issue.

Many of their included studies were conducted in hot climates, where water intake is higher and the optimal fluoride concentration lower. Without stratifying data by climate or baseline exposure, the pooled prevalence figures are not applicable to temperate climates such as the UK or Ireland, inflating estimates of fluorosis. Out of the 90 studies included in the Cochrane Review, 11 were conducted in India, six in China and four in Mexico. Ten studies were conducted in Africa and 22 in Asia. 25 studies were conducted in Europe, leaving 65 studies conducted outside Europe. Dental fluorosis studies in China and India need careful interpretation as, apart from higher temperature compared with Europe, use of ‘brown coal’ for cooking in China is a known source of fluoride intake, and nutritional status in India influences the occurrence of fluorosis. Thus, if the intention is to assess the likely impact of CWF on the occurrence of dental fluorosis in the UK, it would be very misleading to accept as relevant Cochrane’s conclusions.

Eleven of the 90 studies were conducted in the UK; these should be examined if the intention is to determine the impact of CWF in the UK, rather than rely on the conclusions of Cochrane.

Finally, there remains a lack of clarity over the threshold at which fluorosis is of aesthetic concern. Several indices were used in the review and fluorosis recorded in children. Post-eruptive changes such as abrasion and erosion may mitigate the visible impact of fluorosis over time: such factors were not addressed.⁴⁸

The CATFISH study

The CATFISH study is the only recent UK-based study included in the review.¹¹ CATFISH has drawn criticism due to complications arising from the early cessation of the fluoridation scheme under study.¹³ Additionally, the CATFISH study like Cochrane relies heavily on absolute differences in caries outcomes, such as prevalence and mean dmft scores. However, absolute differences in dmft are strongly influenced by background caries levels and participant age. A more informative and standardised metric is the percent caries reduction (PCR) or prevented fraction (PF), which allows better comparison across populations with differing baseline risks. PCR and PF have been the standard way of expressing the effectiveness of fluoride-containing toothpastes⁴⁹ and fluoride varnish.⁴

It is unclear whether Cochrane will continue to use absolute differences in its future reviews (e.g., of fluoride toothpaste), or whether it will revert to PCR. Consistency across reviews is important for comparison, transparency, and utility.

Despite its methodological limitations, the 2024 Cochrane Review⁷ includes some welcome expansions:

• Cochrane undertakes a comprehensive review of single-time-point cross-sectional studies. While these still fall outside the inclusion criteria for the main analysis, the authors acknowledge that these studies can help interpret broader trends. They note that more recent studies tend to show smaller, but still present, reductions in caries
• A brief commentary on environmental sustainability is included, with support of previous conclusions that CWF is a sustainable⁵⁰ public health measure. The environmental analysis is limited (based on a single study)⁵⁰ but suggests that CWF is more sustainable than individual toothbrushing programmes though upfront infrastructure costs remain a consideration.

The review also attempts to address previous criticisms, such as the question of the ‘halo effect’ which is the diffusion of fluoridation benefits beyond scheme boundaries via food and drink products. However, Cochrane finds little evidence to support this consequence. While the authors acknowledge the concept, their analysis remains constrained by a lack of contemporary studies capable of robustly testing this possible effect.

Lastly, the authors conclude that fluoridation ‘may lead to a slightly greater reduction in deft’ (decayed, extracted, and filled teeth in primary dentition). The choice of language, ‘slightly greater’, is vague and is not accompanied by numerical estimates such as PCR or PF. This imprecision undermines the utility of the finding for policymakers, who require clearer quantification to inform decisions.

Reviews by the Cochrane Collaboration have been influential. It is important, therefore, that methods used are fit for purpose. This critique of the 2024 Cochrane Review of water fluoridation⁷ demonstrates several shortcomings which render it unfit for purpose and was little different from that published in 2015,⁹ itself subject to strong criticism.¹⁴ A major shortcoming was restrictive inclusion criteria, particularly the omission of current statistical methods, such as causal inference, for evaluating effect in longitudinal and cross-sectional observational studies resulting in a missed opportunity to include a broader range of published evidence of clinical effectiveness. The section on water fluoridation and dental fluorosis was unchanged from that published in 2015,⁹ using methods previously criticised,¹⁴ and is unsuitable guidance particularly for those in temperate climates. Decision-makers in public health would be wise to consider the broader evidence base, particularly studies relevant to local needs.