Residents living in fluorosis areas generally experienced long-term exposure to excessive fluoride in drinking water. The adverse effects of high fluoride levels on the nervous system have been studied; however, the effect of fluoride exposure on cognitive functions of elderly people in fluorosis areas is rarely reported. This study was aimed to find out the potential risk factors of cognitive impairment among elderly people who lived in fluorosis areas of China. A total of 511 subjects, aged 60 years or above, were investigated in fluorosis areas of Heilongjiang Province, Inner Mongolia Autonomous Region, Qinghai Province, and Xinjiang Uygur Autonomous Region. The Mini-Mental State Examination (MMSE) was used to examine cognitive functions of the study subjects. Based on the MMSE scores, the study subjects were divided into normal group and cognitive impairment group that consisted of mild, moderate, and severe groups. Multivariable logistic regression showed that a higher risk of cognitive impairment was associated with increased age and decreased education levels. Multiple linear regression analysis revealed that MMSE scores were negatively associated with serum homocysteine (Hcy) levels. However, both urinary fluoride and serum Hcy levels in the normal group were not the lowest among the four groups. Spearman’s correlation analysis showed that urinary fluoride levels were positively correlated with serum Hcy (r s = 0.209, P < 0.01). Our study suggests that people with cognitive impairment in fluorosis areas have elevated serum Hcy levels, which was positively correlated with urinary fluoride concentrations. A certain low dose of fluoride intake may play a potential protective rather than harmful role in cognitive functions; however, high fluoride exposure is a potential risk factor for cognitive impairment.