Fluoride Action Network


In a century where toxicology and chemical risk assessment are embracing alternative methods to animal testing, there is an opportunity to understand the causal factors of neurodevelopmental disorders such as learning and memory disabilities in children, as a foundation to predict adverse effects. New testing paradigms, along with the advances in probabilistic modelling, can help with the formulation of mechanistically-driven hypotheses on how exposure to environmental chemicals could potentially lead to developmental neurotoxicity (DNT). This investigation aimed to develop a Bayesian hierarchical model of a simplified AOP network for DNT. The model predicted the probability that a compound induces each of three selected common key events (CKEs) of the simplified AOP network and the adverse outcome (AO) of DNT, taking into account correlations and causal relations informed by the key event relationships (KERs). A dataset of 88 compounds representing pharmaceuticals, industrial chemicals and pesticides was compiled including physicochemical properties as well as in silico and in vitro information. The Bayesian model was able to predict DNT potential with an accuracy of 76%, classifying the compounds into low, medium or high probability classes. The modelling workflow achieved three further goals: it dealt with missing values; accommodated unbalanced and correlated data; and followed the structure of a directed acyclic graph (DAG) to simulate the simplified AOP network. Overall, the model demonstrated the utility of Bayesian hierarchical modelling for the development of quantitative AOP (qAOP) models and for informing the use of new approach methodologies (NAMs) in chemical risk assessment.

Graphical abstract

The conceptual framework used to develop the Bayesian hierarchical model. The model comprised three common key events shown in green selected based on topology analysis and expert knowledge to describe developmental neurotoxicity. These served as a basis to define the structure of the quantitative model. Appropriate data were collected to fit the model with the aim of predicting the probability of whether a compound triggers the biological path.

An external file that holds a picture, illustration, etc. Object name is ga1.jpg


Keywords: Developmental Neurotoxicity, Bayesian hierarchical model, Adverse Outcome Pathway, Common Key Event, New Approach Methodology
Abbreviations: ADMET, Absorption, distribution, metabolism, excretion, and toxicity; AO, Adverse outcome; AOP, Adverse outcome pathway; BBB, Blood-brain-barrier; BDNF, Brain-derived neurotrophic factor; CAS RN, Chemical Abstracts Service Registry Number; CI, Credible interval CKE, Common key event; CNS, Central nervous system; CRA, Chemical risk assessment; DAG, Directed acyclic graph; DNT, Developmental neurotoxicity; DTXSID, The US EPA Comptox Chemical Dashboard substance identifier; EC, Effective concentration; HDI, Highest density interval; IATA, Integrated Approaches to Testing and Assessment; KE, Key event; KER, Key event relationship; LDH, Lactate dehydrogenase; MCMC, Markov chain Monte Carlo; MIE, Molecular initiating event; NAM, New approach methodology; OECD, Organisation for Economic Cooperation and Development; PBK, Physiologically-based kinetic; P-gp, P-glycoprotein; qAOP, Quantitative adverse outcome pathway; QSAR, Quantitative structure-activity relationship; SMILES, Simplified molecular input line entry system


1. Introduction

Neurodevelopmental disorders such as impairment of learning and memory, and cognitive dysfunction, are of serious concern due to the health risks and consequences on the developing brain resulting from exposure to exogenous chemicals [1], [2], [3]. The assessment of developmental neurotoxicity (DNT) is not a mandatory requirement in the European Union or the United States of America and is not routinely conducted. However, it may be undertaken when data from developmental and/or reproductive toxicity studies on adult animals indicate a possible concern for neurotoxicity [4]. When testing is carried out, it is based on available DNT testing guidelines using in vivo methods [5], [6], [7]. These animal tests are a starting point in deciphering complex endpoints such as DNT. However, with the limitations of in vivo testing for DNT [8], there is an opportunity to consider new approach methodologies (NAMs), such as a battery of in vitro test methods, omics technologies and in silico models as a viable alternative. Whilst NAMs for DNT are not yet standardised or required by regulatory authorities, they can provide valuable mechanistic insights regarding potential developmental neurotoxicants [4], [9], [10], [11].

Frameworks are required to organise information from NAMs to predict complex toxicological endpoints, including neurotoxicity and DNT. A promising approach to organise DNT information and subsequently use the information to develop a predictive model is provided by the adverse outcome pathway (AOP) concept [12]. An AOP represents a formal description of a series of events from the molecular initiating event(s) (MIE(s)), key events (KEs) at corresponding molecular, cellular, and tissue levels to adverse outcomes (AOs) at organ, organism and population levels [12]. Recent progress in the development of qualitative and quantitative AOPs underlines their utility to design appropriate experiments and computational simulations [13], [14]. However, as DNT is a complex process with multiple molecular and cellular paths, no single AOP is able to fully explain this complex event. Thus, a network of AOPs allows for a better depiction of the overall mechanistic understanding of DNT than a single AOP. To illustrate this point, given the limited resources available to quantify biological paths of DNT, identification of common key events (CKEs) that intersect the individual paths can assist in the design of testing strategies and in the computational modelling of data-rich biological events [15]. Such CKEs are characterised by high connectivity located, are centrally within the network of AOPs and are essential to link multiple linear AOPs, i.e., MIE(s) to the AO(s) [16], [17].

Quantitative AOPs (qAOPs) and qAOP networks are increasingly being modelled using probabilistic methods [18], [19]. Bayesian modelling is an approach that fits, and makes inferences from, data using Bayes’ theorem for variables. Bayes’ theorem is a mathematical formula that transforms the prior, i.e., our knowledge about the data before seeing the data, into posterior distributions based on the evidence provided [20], [21]. An advantage of this approach, besides the ease of computing predictions and associated uncertainties relating to variables of interest, lies in the reproducibility of the predictions; once the prior beliefs are defined, similar values will be obtained each time the model is run unless new evidence is added [20], [22].

Given the limited availability and heterogeneity of DNT information, i.e., in vivo and in vitro data, and the potential for Bayesian machine learning to investigate chemical-induced DNT, the aim of this investigation was to quantify a simplified, reduced version of the AOP network for neurotoxicity developed by Spînu et al. [23]. The main objective was to predict the probability that a compound induces individual CKEs, in addition to predicting the probability of inducing the AO. The analysis took into account potential correlations and causal relations informed by the key event relationships (KERs) and additional information, such as physicochemical, in silico and in vitro data. This investigation also aimed to explore whether Bayesian hierarchical modelling is fit-for-purpose in chemical risk assessments informed by qAOP models.

2. Materials and methods

2.1. Simplification of the AOP network

The AOP network for neurotoxicity developed and analysed by Spînu et al. [23] served as a foundation to establish the graphical structure of the quantitative model. Specifically, three CKEs were selected to describe a biological path for DNT representative of the AOP network. These CKEs were amongst those identified by the topology analysis reported by Spînu et al. [23]. The choice of CKEs was, however, also made based on expert judgement to avoid non-specific CKEs such as cell injury/death as well as on data availability that would allow quantification. The expert decisions were made during a workshop entitled “e-Resources to Revolutionise Toxicology: Linking Data to Decisions”, held at the Lorentz Center (Leiden, The Netherlands) in October 2019 [24]. The biological path chosen describes the reduction of brain-derived neurotrophic factor (BDNF) that leads to a decrease of synaptogenesis and a decrease of neural network formation and function involved in DNT (Fig. S1). Additionally, this biological path is supported by the description of the AOP ID 13 (https://aopwiki.org/aops/13; [25]) and assessed for its availability in the Organisation for Economic Cooperation and Development (OECD) AOP-Wiki Knowledge Base in Table S1. Importantly, the AOP network reported by Spînu et al. [23] was not a directed acyclic graph (DAG), i.e., a graph with no cyclic paths (no loops). However, the simplified version used in this investigation represented a DAG to allow for the formulation of a causal hypothesis and methodological approach. Such a combination of expert judgement and topology analysis can provide a foundation to establish quantitative models.

2.2. Data description

Two in vitro studies [26], [27] that tested compounds for their DNT potential were chosen as primary data sources. The compounds tested from the two studies were merged into a single list (e.g., nine pairs of compounds, tested as different forms of salts, were combined) and aligned with information referring to the compound name, Chemical Abstracts Service Registry Number (CAS RN), simplified molecular input line entry system (SMILES) string and the US EPA Comptox Chemical Dashboard substance identifier (DTXSID). In total, 88 compounds served as a starting point to collect additional information to improve the modelling. The list contained different types of compounds: pharmaceuticals, pesticides and industrial chemicals. The two in vitro studies differed in the concentration ranges used, type of viability assay applied, plating densities of the cells, calculation of the effective concentration (EC), and the exposure period (five days vs 12 days). Synaptogenesis was measured by a battery of high content imaging and microplate reader-based assays, while the viability was based on the number of cells per field or determined in sister plates using a luminescent assay [27]. Neural network activity was measured on day 12 by the microelectrode array recordings and the viability by two assays, the total lactate dehydrogenase (LDH) release and Alamar blue assay [26]. Also, valproate was tested in two concentration ranges in both studies: lower and higher. The range of higher concentrations showed response/activity in both studies and, thus, the ECx values of the range of higher concentrations tested, and associated responses, for valproate were retained. The difference in the concentration range and time of effect shows that impacts on synaptogenesis are observed at lower levels than impacts on neural network formation. This concentration–response concordance fits the expected pattern for a causal relationship between the two key events in an AOP [28].

The compounds were assigned as positive or negative for DNT induction based on the in vivo studies summarised and evaluated in a literature review by Mundy et al. [29]. In the latter paper, the authors applied rigorous selection criteria to 408 chemicals, concluding that 97 showed evidence of developmental neurotoxicity. Of these 97 DNT-positive chemicals, 21 had evidence in human studies. The compounds were assigned positive/negative for the reduction of BDNF based on a literature review performed to evaluate the peer-reviewed publications that studied the impact of compounds on this effect (Supplementary material, Table S2). Given that at present there is no standardised protocol for assessing the inhibition of BDNF available and thus, no existing datasets, the main selection criteria of the peer-reviewed publications included any available in vitro and/or in vivo studies that showed a decreased or increased level of BDNF for each chemical included in the test set. Compounds were further classified as active/inactive for the CKEs relating to the decrease of synaptogenesis and neural network formation; this was based on the selectivity and specificity identified from the corresponding in vitro studies taking into account cell viability and the results of toxicity assays.

Other data collected included the calculated logarithm of the distribution coefficient (LogD, pH = 7.4), a measure of lipophilicity for ionizable compounds; blood–brain-barrier (BBB) permeability; and the capability to bind to the P-glycoprotein (P-gp) transporter, i.e., if the compound acts as a P-gp inhibitor or substrate, or is (non-)active against P-gp. Predictions for BBB permeability and P-gp interactions were made using in silico models based on curated SMILES [30], [31], [32]. BBB permeability is essential for understanding whether a compound crosses into, and has the possibility to act on, the central nervous system (CNS) [33]. P-gp is a transmembrane protein belonging to the ATP-binding cassette family of transporters (ABC-transporters), highly associated with the absorption, distribution, metabolism, excretion, and toxicity (ADMET) properties of compounds. P-gp may contribute to a decrease in toxicity by eliminating the compound from cells and preventing its intracellular accumulation [30].

The final data set contained missing information given by the in silico models for e.g., inorganic compounds, and there were compounds with no available evidence in the literature for the reduction of BDNF. In addition, inactive compounds in any of the in vitro assays were treated as missing information. Eight compounds contained complete details for this biological path. The final data set is provided in Table S3 in the supplementary material. The types of data utilised for modelling DNT are illustrated in Fig. 1, while further details are provided in Table 1.

An external file that holds a picture, illustration, etc. Object name is gr1.jpg

Types of information collected for model development exemplified for bisphenol A. This figure illustrates how different streams of data can be integrated for causal predictions to complement the information on key events. The full data set is provided in Table S3 in the supplementary material. The ECx values were extracted from the corresponding in vitro studies as published.

Table 1

Types of data and their sources collected for the development of the Bayesian hierarchical model. The table describes all variables, i.e., predictors and outcomes defined as features included in the model for the type of data and performance where applicable. See also Tables S1-S2 in the supplementary material.

Feature Description/Relevance Data Type Performance Source
Chemical Name The names used to define the compounds tested in both in vitro studies. Not Applicable Not Applicable [26], [27]
CAS RN Chemical Abstracts Service Registry Number associated with the tested compounds used to identify and track them during the modelling. Not Applicable Not Applicable [26], [27]
DNT Classification Each compound was classified as either positive, known or potential inducing DNT/negative, safe or without evidence for inducing DNT based on in vivo studies. Binary, i.e., positive (i.e., associated with DNT) or negative Not Applicable [26], [27], [29]
LogD The logarithm distribution coefficient calculated based on the compounds’ SMILES strings. Continuous, unitless values Not Applicable ChemSpider database [34]
BBB Each compound was classified for its capability to permeate the blood–brain-barrier (BBB) based on curated SMILES. Predicting BBB permeability means indicating whether compounds pass across the BBB. Compounds that cross the BBB have the potential to be CNS-active, whereas compounds that do not cross are expected to be CNS-inactive. Binary, i.e., positive (BBB permeable) and negative The in silico model available in admetSAR v2.0 has the area under the curve (AUC) with a range from 0.625 to 0.99. Literature review
Online BBB Predictor v.0.9
admetSAR v.2.0 [32], [35]
Cbrain/Cblood In vivo blood–brain-barrier penetration represented as BB = [Brain]/[Blood], where [Brain] and [Blood] are the steady-state concentration of radiolabelled compounds in the brain and peripheral blood. High absorption to CNS had a value of more than 2.0, medium absorption: 2.0–0.1, and low absorption: less than 0.1. The predictions are based on in vivo data on rats. Continuous, unitless values The QSAR model of Ma et al. [31] had R = 0.955 with s = 0.232, used by PreADMET v.2.0 to model the predictions. PreADMET v.2.0 [31], [36]
P-glycoprotein Status Each compound was classified based on curated SMILES as a substrate or not, inhibitor or not, active or inactive for P-glycoprotein (P-gp) transporter using an in silico model. Binary, i.e., yes or no for a compound acting as a substrate or an inhibitor, or activity against P-gp The non-error rate and the average precision was 0.70 for the external validation set. [30]
Feature Description/Relevance Data Type Performance Source
Reduction of BDNF Each compound was classified as either positive, inducing the reduction of BDNF levels, or negative based on a literature search for in vivo studies, e.g., in rats and mice, and/or in vitro studies, e.g., human neuroblastoma SH-SY5Y cell line, embryonic mouse hypothalamus cell line. Binary, i.e., positive (evidence showing alterations of BDNF), or negative Not Applicable Literature review of historical and peer-reviewed studies that evaluated compounds for their BDNF reduction potential
Decrease of Synaptogenesis Selectivity and potency of a chemical were kept as classified in the reference based on results for viability and effective concentrations in rat primary cortical cells. Categorical, i.e., inducing or not alterations of synaptogenesis The battery assay had a sensitivity of 87% and a specificity of 71%. [27]
Synaptogenesis Viability The amount of ATP present in each well was calculated to assess compounds for their viability in rat primary cortical cells. Continuous Not Applicable [27]
Synaptogenesis Activity, EC30 (?M) 30% change compared to control expressed as an effective concentration EC30 (?M) for puncta per total dendrite length (the most sensitive endpoint) measured in rat primary cortical cells for five days using an imaging assay. Continuous Not Applicable [27]
Decrease of Neural Network Formation Selectivity and potency of a chemical were kept the way the reference classified a compound based on results for viability and effective concentrations in rat primary cortical cells. Categorical, i.e., inducing or not alterations of neural network formation The model had a mean accuracy of 80.2%. [26]
Neural Network Formation Viability Total lactate dehydrogenase (LDH) release upon cell lysis and Alamar blue assay were used to assess compounds for their viability in rat primary cortical cells. Continuous Not Applicable [26]
Neural Network Formation Activity, EC50min and EC50max (?M) 50% change compared to control expressed as an effective concentration EC50 (?M) with minimum and maximum values of all 17 parameters measured in rat primary cortical cells over 12 days using microelectrode array (MEA) recordings. Continuous Not Applicable [26]

3. Results

The exploratory data analysis allowed for a better understanding of the data collected and utilised for the Bayesian analysis (Supplementary material, Figs. S2–S5). The statistical parameters of the model did not show the presence of any divergent transitions to approximate the posterior distributions (Supplementary materials, Figs. S6-S7 and Table S10). The estimates of the hyperpriors and priors are summarised in the supplementary material, Fig. S8.

The posterior distributions, (i.e., likelihood, posterior densities) and posterior predictive distributions were obtained from the Bayesian analysis. The posterior distributions represent the evidence provided by the data combined with the prior that incorporates our knowledge before analysing the data. The posterior probability distributions for each CKE, including the AO, are summarised in Fig. S9. The Bayesian CI was broader for compounds with missing information. Thus, the Bayesian CI quantifies the uncertainty given by the sources of variability and missingness. For instance, fluoxetine with information across all levels had a 95% HDI of 0.83–1.0 with a mean probability of 0.94 for the induction of DNT (Fig. 3). In contrast, sodium fluoride with missing data for the CKEs had a 95% HDI of 0.23–0.99 with a mean probability of 0.65 for the induction of DNT (Fig. 3). Glyphosate, a known negative DNT, had a very low 95% HDI with a maximum of 0.55 with a mean of 0.22 for the induction of DNT (Fig. 3).

An external file that holds a picture, illustration, etc. Object name is gr3.jpg

Visualisations of the predictions for three compounds: fluoxetine, sodium fluoride and glyphosate to show the relationships between the compounds and CKEs and the AO following the structure of the simplified biological path for DNT. The shaded blue distribution represents the predicted severity of the CKEs and the AO as well as the uncertainty in the prediction for the 95% highest density interval (HDI). For a complete overview of the results of all compounds, the reader is referred to Fig. S9 in the supplementary material. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

The posterior predictive distributions are given by the binary classification of compounds that were analysed against the likelihood. The binary classification was informed by the literature review for the CKE reduction of BDNF and by the in vitro studies for the other two CKEs and expressed as a Bernoulli distribution to describe the outcome in a Boolean way, as explained in the Materials and methods section. Two thresholds derived from the results of the predicted posterior distributions were used to classify the compounds for the low, medium or high probability of inducing a CKE and the AO. Such a classification might be used for screening and prioritisation purposes. All compounds with a probability between 0.59 and 0.60 for posterior predictive distributions were classified overall as having a medium probability of inducing a reduction of BDNF (Supplementary material, Fig. S10). This might be a result of the imputation method chosen to deal with the missing information. The results for the decrease of synaptogenesis showed a probability between 0.36 and 0.76 and as such medium and high levels of inducing this CKE (Supplementary material, Fig. S11). For the decrease of neural network formation, the posterior predictive probability was between 0.33 and 0.77 for all compounds that were classified as having low, medium and high levels of probability to induce this CKE (Supplementary material, Fig. S12). An overview of the distribution of the predicted posterior probabilities is shown in Fig. 4.A. The predictions for the AO, which is of most interest for e.g., regulatory decision making, helped to classify the compounds into the three levels (Fig. 4.B). Relatively few compounds had a low level of probability (5% of compounds), and most of the compounds had a high level of probability of inducing DNT (57% of compounds). This unbalanced classification is explained by the initial list of compounds for which 74% of compounds were known to be positive for DNT in vivo (in humans or animals). The full results for the prediction of CKEs and the AO are provided in Tables S6–S9 in the supplementary material.

An external file that holds a picture, illustration, etc. Object name is gr4.jpg

The results of the posterior predictive probabilities. A. The distribution of the posterior probabilities of the three CKEs and the AO for positive and negative compounds, and compounds with a level of missingness for the CKE reduction of BDNF. It shows how the mean of the predicted probability of compounds was clustered and distributed based on the two thresholds to describe a low, medium and high probability for the induction of the corresponding CKE and AO. A detailed graphical representation is shown in Supplementary material, Figs. S10–S12. B. Predicted probabilities of compounds for the induction of developmental neurotoxicity. The predicted probabilities are colour-coded based on two thresholds estimated from the results set to group the compounds for their low, medium and high probability. Compounds were listed in the order of increasing probability.

The sensitivity analysis underlined that weakly-informative priors did not have a significant influence on the model, which is instead data-driven (Table S11). The overall performance of the model is summarised in Table S12 with an accuracy of 76% for the prediction of DNT. There were several misclassifications due to a number of reasons including the quality of the data used for modelling and the level of the missing information associated with the CKEs. For example, a posterior predictive probability of 0.71 was obtained for loperamide hydrochloride, which was misclassified with a high level of probability for inducing DNT. The potential reasons for these misclassifications may be in part because of (1) the in silico model used for the prediction of P-gp, which classified the compound as an inhibitor, substrate and active, (2) the absence of data for the CKE of reduction of BDNF, (3) the in vitro assays that identified it as active, (4) the chosen threshold for classification. However, the large CI associated with the predicted mean value of 0.36–0.99 emphasises the caution required in making decisions for this compound.


Edited by Dr. Daniel Dietrich


Appendix ASupplementary data to this article can be found online at https://doi.org/10.1016/j.comtox.2021.100206.

Appendix A. Supplementary data

The following are the Supplementary data to this article:

Supplementary data 1:

Supplementary data 2:

Supplementary data 3:

Supplementary data 4:

Supplementary data 5:

Supplementary data 6:


1. Bennett D., et al. Project TENDR: Targeting environmental neuro-developmental risks. The TENDR Consensus Statement. Environ. Health Perspect. 2016;124(7):A118–A122. [Europe PMC free article] [Abstract] [Google Scholar]
2. Grandjean P., Landrigan P.J. Developmental neurotoxicity of industrial chemicals. Lancet. 2006;368(9553):2167–2178. [Abstract] [Google Scholar]
3. Crofton K.M., Mundy W.R., Shafer T.J. Developmental neurotoxicity testing: a path forward. Congenit Anom (Kyoto) 2012;52(3):140–146. [Abstract] [Google Scholar]
4. Bal-Price A., et al. Strategies to improve the regulatory assessment of developmental neurotoxicity (DNT) using in vitro methods. Toxicol. Appl. Pharmacol. 2018;354:7–18. [Europe PMC free article] [Abstract] [Google Scholar]
5. OECD . OECD Publishing; Paris: 2018. [Google Scholar]
6. OECD . OECD Publishing; Paris: 2007. [Google Scholar]
7. USEPA, Health effects guidelines OPPTS 870.6300 Developmental Neurotoxicity Study, 1998.
8. Paparella M., Bennekou S.H., Bal-Price A. An analysis of the limitations and uncertainties of in vivo developmental neurotoxicity testing and assessment to identify the potential for alternative approaches. Reprod. Toxicol. 2020;96:327–336. [Abstract] [Google Scholar]
9. Fritsche E., et al. Consensus statement on the need for innovation, transition and implementation of developmental neurotoxicity (DNT) testing for regulatory purposes. Toxicol. Appl. Pharmacol. 2018;354:3–6. [Europe PMC free article] [Abstract] [Google Scholar]
10. Bal-Price A., et al. International STakeholder NETwork (ISTNET): creating a developmental neurotoxicity (DNT) testing road map for regulatory purposes. Arch. Toxicol. 2015;89(2):269–287. [Europe PMC free article] [Abstract] [Google Scholar]
11. Sachana M., et al. International regulatory and scientific effort for improved developmental neurotoxicity testing. Toxicol. Sci. 2019;167(1):45–57. [Abstract] [Google Scholar]
12. Ankley G.T., et al. Adverse outcome pathways: a conceptual framework to support ecotoxicology research and risk assessment. Environ. Toxicol. Chem. 2010;29(3):730–741. [Abstract] [Google Scholar]
13. Mahony C., et al. New ideas for non-animal approaches to predict repeated-dose systemic toxicity: Report from an EPAA Blue Sky Workshop. Regul. Toxicol. Pharm. 2020;114 [Abstract] [Google Scholar]
14. Spînu N., et al. Quantitative adverse outcome pathway (qAOP) models for toxicity prediction. Arch. Toxicol. 2020;94(5):1497–1510. [Europe PMC free article] [Abstract] [Google Scholar]
15. Pistollato F., et al. Assessment of developmental neurotoxicity induced by chemical mixtures using an adverse outcome pathway concept. Environ. Health. 2020;19(1):23. [Europe PMC free article] [Abstract] [Google Scholar]
16. Knapen D., et al. Adverse outcome pathway networks I: Development and applications. Environ. Toxicol. Chem. 2018;37(6):1723–1733. [Europe PMC free article] [Abstract] [Google Scholar]
17. Villeneuve D.L., et al. Adverse outcome pathway networks II: network analytics. Environ. Toxicol. Chem. 2018;37(6):1734–1748. [Europe PMC free article] [Abstract] [Google Scholar]
18. Perkins E.J., et al. Chemical hazard prediction and hypothesis testing using quantitative adverse outcome pathways. ALTEX. 2019;36(1):91–102. [Abstract] [Google Scholar]
19. Perkins E.J., et al. Building and applying quantitative adverse outcome pathway models for chemical hazard and risk assessment. Environ. Toxicol. Chem. 2019;38(9):1850–1865. [Europe PMC free article] [Abstract] [Google Scholar]
20. Gelman A., et al. third ed. Taylor & Francis; 2013. [Google Scholar]
21. van de Schoot R., et al. Bayesian statistics and modelling. Nat. Rev. Methods Primers. 2021;1(1):1. [Google Scholar]
22. McElreath R. Chapman and Hall/CRC; New York: 2016. [Google Scholar]
23. Spînu N., et al. Development and analysis of an adverse outcome pathway network for human neurotoxicity. Arch. Toxicol. 2019;93(10):2759–2772. [Abstract] [Google Scholar]
24. Paini A, et al., Towards a qAOP framework for predictive toxicology – linking data to decisions (in press), 2021.
25. Sachana M., Munn S., Bal-Price A. OECD Series on Adverse Outcome Pathways, No. 5. OECD Publishing; Paris: 2016. [Europe PMC free article] [Abstract] [Google Scholar]
26. Frank C.L., et al. From the Cover: Developmental neurotoxicants disrupt activity in cortical networks on microelectrode arrays: results of screening 86 compounds during neural network formation. Toxicol. Sci. 2017;160(1):121–135. [Abstract] [Google Scholar]
27. Harrill J.A., et al. Testing for developmental neurotoxicity using a battery of in vitro assays for key cellular events in neurodevelopment. Toxicol. Appl. Pharmacol. 2018;354:24–39. [Abstract] [Google Scholar]
28. Becker R.A., et al. Increasing scientific confidence in adverse outcome pathways: application of tailored Bradford-Hill considerations for evaluating weight of evidence. Regul. Toxicol. Pharm. 2015;72(3):514–537. [Abstract] [Google Scholar]
29. Mundy W.R., et al. Expanding the test set: chemicals with potential to disrupt mammalian brain development. Neurotoxicol. Teratol. 2015;52:25–35. [Abstract] [Google Scholar]
30. Mora Lagares L., Minovski N., Novic M. Multiclass classifier for P-glycoprotein substrates, inhibitors, and non-active compounds. Molecules. 2019;24(10) [Europe PMC free article] [Abstract] [Google Scholar]
31. Ma X.L., Chen C., Yang J. Predictive model of blood-brain barrier penetration of organic compounds. Acta Pharmacol. Sin. 2005;26(4):500–512. [Abstract] [Google Scholar]
32. Yang H., et al. admetSAR 2.0: web-service for prediction and optimization of chemical ADMET properties. Bioinformatics. 2019;35(6):1067–1069. [Abstract] [Google Scholar]
33. Kaplan L., Chow B.W., Gu C. Neuronal regulation of the blood-brain barrier and neurovascular coupling. Nat. Rev. Neurosci. 2020;21(8):416–432. [Abstract] [Google Scholar]
34. Williams A.J. Collaborative Computational Technologies for Biomedical Research. 2011. pp. 363–386. [Google Scholar]
35. Liu H., et al. AlzPlatform: an Alzheimer’s disease domain-specific chemogenomics knowledgebase for polypharmacology and target identification research. J. Chem. Inf. Model. 2014;54(4):1050–1060. [Europe PMC free article] [Abstract] [Google Scholar]
36. PreADMET. ADME Prediction v.2.0. 25.10.2020]; Available from: <https://preadmet.bmdrc.kr/adme-prediction/>.
37. Gelman A. Multilevel (hierarchical) modeling: what it can and cannot do. Technometrics. 2006;48(3):432–435. [Google Scholar]
38. Salvatier J., Wiecki T., Fonnesbeck C. Probabilistic programming in Python using PyMC3. PeerJ Comput. Sci. 2016 2:e55. [Google Scholar]
39. Vehtari A., Gelman A., Gabry J. Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Stat. Comput. 2017;27(5):1413–1432. [Google Scholar]
40. Brier G.W. Verification of Forecasts expressed in terms of probability. Mon. Weather Rev. 1950;78(1):1–3. [Google Scholar]
41. Gelman A., Rubin D.B. Inference from iterative simulation using multiple sequences. Statist. Sci. 1992;7(4):457–472. [Google Scholar]
42. Bois F.Y., Jamei M., Clewell H.J. PBPK modelling of inter-individual variability in the pharmacokinetics of environmental chemicals. Toxicology. 2010;278(3):256–267. [Abstract] [Google Scholar]
43. Bois F.Y., et al. Population toxicokinetics of benzene. Environ. Health Perspect. 1996;104(suppl 6):1405–1411. [Europe PMC free article] [Abstract] [Google Scholar]
44. Zgheib E., et al. Application of three approaches for quantitative AOP development to renal toxicity. Comput. Toxicol. 2019;11:1–13. [Google Scholar]
45. Lazic S.E., Semenova E., Williams D.P. Determining organ weight toxicity with Bayesian causal models: Improving on the analysis of relative organ weights. Sci. Rep. 2020;10(1):6625. [Europe PMC free article] [Abstract] [Google Scholar]
46. Semenova E., et al. A Bayesian neural network for toxicity prediction. Comput. Toxicol. 2020;16 [Google Scholar]
47. Jannicke Moe S., et al. Quantification of an adverse outcome pathway network by Bayesian regression and Bayesian network modeling. Integr Environ Assess Manag. 2020 [Europe PMC free article] [Abstract] [Google Scholar]
48. Burgoon L.D., et al. Predicting the probability that a chemical causes steatosis using adverse outcome pathway Bayesian networks (AOPBNs) Risk Anal. 2020;40(3):512–523. [Europe PMC free article] [Abstract] [Google Scholar]
49. Jaworska J.S., et al. Bayesian integrated testing strategy (ITS) for skin sensitization potency assessment: a decision support system for quantitative weight of evidence and adaptive testing strategy. Arch. Toxicol. 2015;89(12):2355–2383. [Abstract] [Google Scholar]
50. Behl M., et al. Screening for developmental neurotoxicity at the National Toxicology Program: the future Is here. Toxicol. Sci. 2019;167(1):6–14. [Europe PMC free article] [Abstract] [Google Scholar]
51. Coecke S., et al. Guidance on good cell culture practice. a report of the second ECVAM task force on good cell culture practice. Altern. Lab. Anim. 2005;33(3):261–287. [Abstract] [Google Scholar]
52. Pamies D., et al. Advanced Good Cell Culture Practice for human primary, stem cell-derived and organoid models as well as microphysiological systems. ALTEX. 2018;35(3):353–378. [Abstract] [Google Scholar]
53. Hartung T., et al. Toward good in vitro reporting standards. ALTEX. 2019;36(1):3–17. [Abstract] [Google Scholar]
54. Masjosthusmann S., et al. Establishment of an a priori protocol for the implementation and interpretation of an in-vitro testing battery for the assessment of developmental neurotoxicity. EFSA Support. Publ. 2020;17(10):1938E. [Google Scholar]
55. Gelman A., Hill J., Vehtari A. Cambridge University Press; 2020. [Google Scholar]
56. OECD, Users’ handbook supplement to the guidance document for developing and assessing adverse outcome pathways. OECD Series on adverse outcome pathways No 1. Series on testing & assessment No. 233. (ENV/JM/MONO(2016)12), 2018; Available from: 10.1787/5jlv1m9d1g32-en. [CrossRef]
57. OECD, Revised guidance document on developing and assessing adverse outcome pathways (ENV/JM/MONO(2013)6), 2017; Available from: <https://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=env/jm/mono(2013)6&doclanguage=en>.
58. Amrhein V., Greenland S., McShane B. Scientists rise up against statistical significance. Nature. 2019;567(7748):305–307. [Abstract] [Google Scholar]
59. Avila A.M., et al. An FDA/CDER perspective on nonclinical testing strategies: classical toxicology approaches and new approach methodologies (NAMs) Regul. Toxicol. Pharm. 2020;114 [Abstract] [Google Scholar]
60. Bal-Price A., et al. Recommendation on test readiness criteria for new approach methods in toxicology: exemplified for developmental neurotoxicity. ALTEX. 2018;35(3):306–352. [Europe PMC free article] [Abstract] [Google Scholar]
61. Worth A.P., Patlewicz G. In: Validation of Alternative Methods for Toxicity Testing. Eskes C., Whelan M., editors. Springer International Publishing; 2016. pp. 317–342. [Google Scholar]
62. Depaoli S., Winter S.D., Visser M. The Importance of prior sensitivity analysis in Bayesian statistics: demonstrations using an interactive Shiny app. Front. Psychol. 2020;11(3271) [Europe PMC free article] [Abstract] [Google Scholar]
63. Kruschke J.K. Bayesian analysis reporting guidelines. Nat. Hum. Behav. 2021 [Europe PMC free article] [Abstract] [Google Scholar]
64. OECD, Overview of Concepts and Available Guidance related to Integrated Approaches to Testing and Assessment (IATA). Series on Testing and Assessment, No. 329. OECD Publishing, Paris. ENV/JM/MONO(2020)25, 2020; Available from: <http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=ENV/JM/MONO(2020)25&docLanguage=en>.