Water fluoridation and the quality of information available online

Introduction

Water fluoridation was first introduced into local drinking water supplies after it was found that those living in Colorado Springs, an area with naturally high water fluoride concentrations, were extremely resistant to dental caries¹. The dramatic reduction in the prevalence and severity of dental caries following the introduction of water fluoridation led the Centers for Disease Control and Prevention (CDC) to proclaim it as one of the major public health initiatives of the 20th century². Many studies have been carried out demonstrating the effectiveness of community water fluoridation (CWF), with both children and adults living in fluoridated areas exhibiting a lower prevalence of dental caries compared with their counterparts living in non-fluoridated or low-level fluoridation areas3., 4.. Like many public health initiatives, CWF has attracted controversy from a seemingly small, yet vocal, section of the population who claim, among other unsubstantiated health risks, that CWF is an unethical socialised medicine5., 6.. This is particularly concerning as, for the most part, decisions regarding the implementation of CWF are made by local governments who may be more likely to be influenced by these vocal opponents, often local residents. This is evident from the cessation of CWF in communities where it had been previously established, as well as the hesitancy to introduce CWF in Europe1., 5.. Therefore, it is important that when the topic of CWF is being debated in local communities, the population is able to access accurate, unbiased information, enabling them to make an informed decision. In today’s digital age, many people seek out this information on the Internet.

With over 3 billion people worldwide now online⁷, the Internet has transformed the way people approach their health care. While physicians remain the most trusted source of information, they have been superseded by the Internet as the primary source of information⁸. This has two major drawbacks: first, anyone can post material on the Internet without review, which raises concerns regarding credibility⁹. Second, discrepancies can arise between what the author has written and the ability of the intended audience to comprehend and interpret it correctly. The large volume of people able to access the Internet for health information highlights the importance of having both credible and effectively presented online information. While previous research has focussed on the quality of online information as a whole, or on particular conditions10., 11., little work has been carried out on specific public health issues, such as CWF. This study attempts to address this gap in the literature by assessing the information available to individuals who search online for water-fluoridation information, with emphasis on credibility and quality of websites.

Methods

The most commonly used search engines were identified via an online Search Engine List that ranks sites via an average of their Alexa Global Traffic Rank and their US Traffic Rank¹². The search engines used were: Google.com.au; bing.com; yahoo.com; Ask.com; and aol.com. Additional search criteria, such as region restrictions, were not implemented as this study was attempting to simulate a global search. Before starting the searches, and between entries of each search term, the browser history of the computer being used was cleared. Additionally, search engines with login capabilities, such as Google, were not logged into in an attempt to minimise ‘filter bubble’¹³ – the effect of algorithms based on individual user search histories.

All searches were performed on 30 January, 2016. The final list of terms (see Table 1), were entered consecutively into each of the search engines. The search results were limited to the top 10 results returned for each search term as research has shown that people rarely search further than the top 10 results¹⁴. The results of each search were transferred into a Microsoft Excel spreadsheet in the order that they were returned by the search engines. News articles were excluded, as this was not an analysis of media portrayal of water fluoridation. Furthermore, sponsored sites (paid advertisements) were not included as people performing focussed searches avoid advertised material¹⁵.

Web pages were categorised as pro-fluoridation, neutral and anti-fluoridation. Anti-fluoridation pages were defined as those pages that built an argument against water fluoridation. Similarly, pro-fluoridation pages were defined as those that built an argument in favour of water fluoridation. Alternatively, pages that presented all views evenly, or without an overt position, were deemed neutral.

The web pages were assessed under the domains of credibility, quality and health literacy. In the case of the credibility criteria the site as a whole was explored to find the relevant details. Conversely, the quality and health-literacy criteria were limited to the page generated from the search. This was because of the fact that searchers rarely look past the first page¹⁶; however, the decision was made to explore the whole site for credibility as often these criteria are found on dedicated pages within the site and ignoring these may have resulted in an underestimation of the credibility of the websites.

To examine the suitability and clarity of the scoring criteria, an independent rater was enlisted to score a random selection of 20 web pages and a measure of inter-rater reliability was used. The scores given by the independent rater and primary rater of this study were compared via a Bland-Altman plot in which the mean score of both raters is plotted against the difference in the two scores to demonstrate reliability of the scoring system.

Results

Quantity

Overall, 500 web pages were returned via the search strategy, with 407 duplicates removed. A further 38 web pages were excluded from the scoring process for the following reasons: contained insufficient information pertaining to water fluoridation; focussed on the chemical nature of the compounds used in fluoridation; addressed bottled water; or presented information as personal opinion (for example, blogs about water fluoridation in the user’s local area; see Figure 1 for a flow diagram). A number of websites returned multiple pages in the search process, resulting in a greater representation in the search results (see Table 3). The most commonly occurring websites were: Wikipedia; Fluoride Alert; About (now Verywell); the U.S. Center for Disease Control and Prevention; and Queensland Health.

1. Download : Download full-size image

Figure 1.. Search strategy flow diagram.

Table 3. List of websites that returned multiple web pages about water fluoridation following the search strategy

Website title	Website URL	No. of pages returned by search
Wikipedia	https://en.wikipedia.org	4
Fluoride Action Network	http://fluoridealert.org	3
About	http://www.about.com/	3
Centre for Disease Control and Prevention	http://www.cdc.gov	3
Queensland Health	https://www.health.qld.gov.au/	3

Quality of scoring system

Bland-Altman plot analysis revealed that when scores differed between the independent rater and primary rater they only differed by a maximum of 4.5 out of a possible difference of 100 (see Figure 2). A difference of 4.5 can be attributed to a difference between the marker’s scores in a single criterion. Across the web pages analysed, the criterion in which differences occurred varied; however, the most common was in the presentation of the most pertinent information in the top half of the page.

1. Download : Download full-size image

Figure 2.. Bland-Altman plot showing the distribution of agreement between two raters of 55 web pages about water fluoridation.

Quality of information

The quality of information assessed using the GACVS and CDC guidelines varied greatly between websites. The median score across all web pages assessed was 80/100, with a maximum score of 96 (www.cancer.org/cancer/cancercauses/othercarcinogens/athome/water-fluoridation-and-cancer-risk) and a minimum score of 22.2 (www.angelfire.com/az/sthurston/fluoride.html).

Grouping web pages according to their domain (.com, .gov and .org) and ranking them according to score revealed a clear trend (see Figure 3). Overall, .gov websites scored the highest; however, their score fell below the median in the final two sites, ranking with scores ranging from 95 to 69. The score of two .gov sites fell below the median because they failed to indicate to the user when they were moving to pages not maintained by the original website (responsible partnering) and to provide insightful figures with clear explanations. Additionally, .org websites outscored .com sites, with scores ranging from 96 to 39 and 95 to 22, respectively.

1. Download : Download full-size image

Figure 3.. Fifty-five web pages about water fluoridation, grouped according to domain and ranked by score.

Furthermore, when grouped according to stance and examined at the levels of credibility, content quality and health literacy, further trends appeared (see Figure 4). Overall, all three stances corresponded well to standards required to support health literacy, with average scores of 0.91, 0.90 and 0.81 for pro, neutral and anti-fluoridation web pages, respectively. The most common shortcomings regarding health-literacy standards were failing to provide the most pertinent information in the upper half of the web page, as well as irrelevant, or not clearly relevant, figures. Similarly, neutral and pro-fluoridation web pages showed strong credibility with mean scores of 0.80 and 0.85, although these web pages most commonly did not disclose any potential conflicts of interest or sources of funding for the site. By contrast, anti-fluoridation web pages had a lower mean credibility score of 0.62. These pages often did not have clear qualifications, provide contact details or exhibit responsible partnering (informing the user they were moving to pages not maintained by the original website). Regardless of their stance on fluoridation, most pages performed poorly in terms of quality of content. The majority of sites provided a moderate amount of superficial information, with little information regarding authors and lacked references to scientific articles. In cases where author’s names were provided, their qualifications were unclear. Similarly, when references to scientific articles were provided they were not accompanied by hypertext links, making even freely available articles harder to access for the general public.

1. Download : Download full-size image

Figure 4.. Mean credibility, content quality and health-literacy scores of 55 web pages about water fluoridation, grouped according to fluoridation stance (anti, neutral or pro).

Discussion

To the best of our knowledge this is the first published assessment of the quality of online information regarding water fluoridation. We found that .com and anti-fluoridation websites scored lower in credibility, quality and health literacy.

It is a long-standing view that anti-fluoridation lobbyists make up a relatively small, yet vocal, component of the fluoridation debate⁵. In this study, anti-fluoridation web pages were a minority (20%) and yet produce considerable numbers of attention-grabbing claims (albeit unsubstantiated, through failing to reference reputable scientific articles to support their assertions).

There was a reasonable prevalence of neutral web pages (29%) returned by the search, which presented pros and cons of water fluoridation supported in the literature, such as its dental-health benefit and the possibility of fluorosis. Critical consumers of health information frequently seek information they perceive as unbiased, allowing them to come to their own conclusions²¹. We also found that the majority of pro-fluoridation web pages came from sources such as government websites, which were more likely to present a balanced view of water fluoridation.

Geolocation, that is the location of the person conducting the search, is an issue regarding web searches as it has a significant impact on the results generated. The effect of geolocation can clearly be seen through this study as, despite clearing the browser history before each search and between search terms, a large proportion of sites returned in this study originated within Australia. One study has found that geolocation generates the greatest variability in terms of search results compared with any other factor²². Specifically, it was shown that the majority of web pages returned tended to reflect the pre-existing beliefs of the particular region based on previous Internet traffic and omit conflicting information that the algorithm deems unnecessary, generating a ‘filter bubble’. This enhances human confirmation bias in which people tend only to read information that conforms to their already established beliefs²³. In this case, the search engine is influencing the information provided via location and previous search terms. This is particularly important when considering potentially divisive public health issues, such as water fluoridation, as it would result in searchers in regions where the community is strongly opposed to water fluoridation being more exposed to anti-fluoridation sites, preventing them from being able to make an unbiased and well-informed decision. We recommend that this information forms the basis of a set of guidelines for health-related web searches to assist the general public to perform these searches. These guidelines would need to be fully evaluated to contribute to a limited evidence base on how to enhance online health literacy²⁴. More broadly, public health and pro-fluoridation communication also needs to provide compelling information via a range of sources to counterbalance anti-fluoride information in communities where this activity is prominent.

A common misconception held by individuals performing health-related searches is that the order in which search engine results appear is an indication of their quality or relevance¹⁴. However, the order of web page appearance is, in fact, generated by an algorithm that takes into account the number of times a page is linked to the search term, with pages that have more links to them appearing higher in the search results. While this algorithm is highly effective when performing consumer searches in which the most popular pages are desirable, it produces a false perception of quality in health-related searches.

In terms of quality, while the majority of pro-fluoridation pages presented accurate information, it was often not referenced or credited to a qualified author. Additionally, they did not disclose affiliations that could produce a conflict of interest or sources of funding for the website. All of these factors are key to establishing a sense of transparency and credibility of the health information and are likely to increase levels of trust. We recommend that those developing online information about water fluoridation credit information to appropriate authors, disclose conflict of interests and state sources of funding, as this will improve their overall quality according to international criteria.

This study has some limitations. First, the dynamic nature of the Internet means that there is constant addition, removal or updating of information. Our sample captured a snapshot from January 2016 using a search conducted with a clear browser history. The degree to which any individual’s search results would differ has been shown, in other research, to account for an 11.7% variation, with one study showing less variation in higher-ranked hits22., 25.. Additionally, the two compounds most commonly used to fluoridate water supplies – hydrofluorosilicic acid and silicofluoride – were not used as search terms. As neither of these terms appeared in the Google Trends ‘related search’ function when searching for commonly associated terms, they are unlikely to have greatly impacted the results of this study. Furthermore, the search captured a predominance of Australian servers as a result of the search being performed in Australia. Nonetheless, this study does provide a diverse snapshot of the stance and quality of information generally available regarding water fluoridation and includes international sites. Finally, our quality scoring system was based on a World Health Organization (WHO) system for rating vaccine safety rather than a pre-established and validated scoring system. Our tool was fit-for-purpose as it was a global tool that had been applied to vaccination, a thematically similar topic with respect to its public health importance, safety claims and controversy. Furthermore, the tool included many of the same domains described in the review of Zhang et al. ²⁶, across currency, credibility and readability. However, our design elements were viewed through a health literacy framework.

In conclusion, members of the public performing a search regarding water fluoridation are presented with a moderate amount of superficial information that is easy to understand but is poorly referenced. The sites available were, for the most part, credible and presented information that was easy to comprehend and remember. Our findings suggest that when developing resources regarding water fluoridation, greater emphasis on providing well-accredited, transparent web pages that are open in terms of potential conflicts of interest as well as funding, is required. Furthermore, we recommend that communicators give clearly explained, relevant figures that aid in consolidating, or further explaining, the information presented in the text. Finally, regarding future use, the criteria used to assess the web pages in this study were by no means specific to water fluoridation and therefore provide a basis for further studies where they can be used directly, or as a scaffold to be tailored to specific research questions.