In the last decade there is an increasing interest in using in silico tools for potential risk assessment of newly released chemicals due to the large number of chemicals enter the market yearly and the big uncertainty on their possible hazardous effects. Different tools and methods based on machine learning techniques already exist and were used in a wide range of applications starting from quantitative structure-property relationships and expanding into predictive toxicology. There is a lot of historical data accumulated across multiple databases which is publicly available and can be used with novel machine learning methods. Unfortunately, due to different datasets, metrics and validation strategies, the significant gaps remain in both the quantity and quality of data available coupled with optimal predictive methods. This work is an attempt to develop a multitask system which can serve as searchable curated collections of multiple chemical datasets and ready to use novel machine learning methods solely built using open source frameworks and libraries. We have implemented a set of self-tuned, using grid search and k-fold validation, traditional machine learning methods (shallow methods) such as Naïve Bayes, k-Nearest Neighbors, Random Forest, Boosted Decision Trees, Regularized Logistic Regression, and Support Vector Machines base on open source Scikitlearn (http://scikit-learn.org/stable/). The novel Deep Neural Networks models of different complexity have been also implemented using Keras (https://keras.io/), a deep learning open library, and a Tensorflow (www.tensorflow.org) as a backend. The machine learning models were trained and evaluated to predict measures of toxicity from the physical characteristics of the structure of chemicals using the same datasets as in the Toxicity Estimation Software Tool (https://www.epa.gov/chemical-research/toxicity-estimation-software-tool-test). The Deep Learning models showed very good performance evaluation characteristics and were found to be useful in predicting of toxicological and physicochemical parameter endpoints. The results of this work support an optimistic view that some of current obstacles in cheminformatics can be overcome by using Deep Learning methods.

1. Machine learning methods for chemical
properties and toxicity-based endpoints
prediction using open source
libraries
Valery Tkachenko, Alexandru Korotcov , Rick Zakharov,
Boris Sattarov, Artem Mitrofanov
SCIENCE DATA SOFTWARE, LLC, Rockville, Maryland, United States

2. Data, Information, Knowledge, Wisdom

3. Molecule representation
RDKit: Descriptors, Fingerprints; Autoencoder FP, CNN

4. AI and Machine Learning methods in OSDR for QSAR, QSPR, and material SPR
Shallow Machine Learning (SML) methods:
• Naive Bayes, k Nearest Neighbors, Linear Logistic Regression, AdaBoost Decision Tree, Random Forest, Elastic Net regression
(L1 and L2 regularizes), Support Vector Machine, XGBoost
• Clustering, Classification and Regression models
• Open source based advanced ML libraries such as scikit-learn (CPU for training and prediction) for training, tuning
(automated optimization), and validating all SML models
Deep Neural Networks (DNN) methods:
• Different complexity or feed-forward DNN (up to 6 hidden layers)
• Encoder-decoder NN for feature selection and clustering
• Long-Shot Term Memory (LSTM) NN for inverse QSAR (generate SMILES of molecules with desired properties)
• Convolutional Graph NN (Graph CNN)
• Clustering, Classification, Regression and Generative models, models optimization and tuning
• Open source Keras and Tensorflow (GPU training and CPU or GPU for prediction)

5. OPERA
• Original train and test sets (20-25% for test)
• 5x k-fold cross validation
• DNN with 3 hidden layers of 1024 nodes
• MinMax scaling of feature vectors (separately for train and test)
• MACCS, ECFP(4,6), FCFC(4,6), Avalon FP, RDkit descriptors set (DESC)
and their combinations
Details

6. OPERA
Dataset Size,
molecules
Test R^2,
ref
R^2,
DESC
R^2,
our FP
Octanol-water Partition Coefficient 14208 20% 0.86 0.8969 0.9158
Melting Point 8656 25% 0.73 0.8066 0.8173
Water solubility 2010 20% 0.86 0.9154 0.9311
Vapor Pressure 2716 25% 0.92 0.9443 0.9443
Boiling Point 5426 25% 0.93 0.9636 0.9636
Fish Biotransformation Half-life 541 25% 0.73 0.7683 0.8936
Biological Half-life 150 25% 0.75 0.8316 0.8316
Bioconcentration Factor 608 25% 0.83 0.7435 0.7754
Soil Adsorption Coefficient 735 25% 0.71 0.7774 0.7853

7. TEST

8. OpenPHACTS
Dataset R^2
Refractivity Index 0.9844
Boiling point 0.9296
Density 0.9031
Dipole moment 0.9951
PSA 0.9833
Octanol-water partition 0.8742

9. Datasets used for evaluating multiple computational methods
for activity chemical properties prediction
Model
Datasets used and
references
Cutoff for active
Number of molecules
and ratio
solubility Huuskonen J. J Chem Inf
Comput Sci 2000
Log solubility = −5 1144 active, 155 inactive,
ratio 7.38
probe-like Litterman N. et al. J Chem Inf
Model 2014
described in reference 253 active, 69 inactive,
ratio 3.67
hERG Wang S. et al. Mol Pharm 2012 described in reference 373 active, 433 inactive,
ratio 0.86
KCNQ1 PubChem BioAssay: AID 2642
98
using actives assigned in PubChem 301,737 active, 3878 inactive,
ratio 77.81
Bubonic plague
(Yersina pestis)
PubChem single-point screen
BioAssay: AID 898
active when inhibition ≥50% 223 active, 139,710 inactive,
ratio 0.0016
Chagas disease
(Typanosoma cruzi)
Pubchem BioAssay: AID 2044 with EC50 <1 μM, >10-fold
difference in cytotoxicity as active
1692 active, 2363 inactive,
ratio 0.72
TB (Mycobacterium
tuberculosis)
in vitro bioactivity and
cytotoxicity data from MLSMR,
CB2, kinase, and ARRA
datasets
Mtb activity and acceptable Vero
cell cytotoxicity selectivity index =
(MIC or IC90)/CC50 ≥10
1434 active, 5789 inactive,
ratio 0.25
malaria (Plasmodium
falciparum)
CDD Public datasets (MMV, St.
Jude, Novartis, and TCAMS)
3D7 EC50 <10 nM 175 active, 19,604 inactive,
ratio 0.0089
Clark et al. J Chem Inf Model 2015

10. Solubility dataset: polar plots of the model evaluation metrics
BNB - Bernoulli Naive Bayes, LLR - Logistic linear regression, ABDT - AdaBoost Decision Trees, RF - Random Forest,
SVM - Support Vector Machines, DNN-N (N is number of hidden layers).

11. BNB - Bernoulli Naive Bayes, LLR - Logistic linear regression, ABDT - AdaBoost Decision Trees, RF - Random Forest,
SVM - Support Vector Machines, DNN-N (N is number of hidden layers).
Chagas disease dataset: polar plots of the model evaluation
metrics

12. AUC for all tested datasets (FCFP6, 1024)

13. F1-scores for all tested datasets (FCFP6, 1024)
F1-score BNB LLR ABDT RF SVM DNN-2 DNN-3 DNN-4 DNN-5
solubility train 0.942 0.963 0.960 0.956 0.954 0.992 0.992 0.992 0.992
solubility test 0.909 0.945 0.946 0.945 0.940 0.959 0.961 0.961 0.961
probe-like train 0.931 0.900 0.967 0.967 0.961 1.000 1.000 1.000 1.000
probe-like test 0.830 0.804 0.841 0.811 0.852 0.860 0.870 0.870 0.870
hERG train 0.854 0.841 0.956 0.825 0.885 1.000 1.000 1.000 1.000
hERG test 0.798 0.798 0.715 0.780 0.784 0.776 0.784 0.784 0.792
KCNQ train 0.796 0.865 0.819 0.833 0.856 0.999 1.000 1.000 1.000
KCNQ test 0.794 0.858 0.816 0.825 0.851 0.991 0.992 0.993 0.993
Bubonic plague train 0.078 0.095 0.107 0.114 0.150 0.771 0.873 0.932 0.962
Bubonic plague test 0.042 0.065 0.048 0.061 0.071 0.191 0.225 0.233 0.235
Chagas disease train 0.692 0.727 0.743 0.661 0.815 0.999 0.999 0.999 0.999
Chagas disease test 0.618 0.652 0.645 0.608 0.676 0.676 0.692 0.678 0.683
Tuberculosis train 0.430 0.452 0.460 0.445 0.500 0.970 0.970 0.970 0.970
Tuberculosis test 0.385 0.390 0.401 0.409 0.417 0.357 0.345 0.326 0.315
Malaria train 0.394 0.361 0.191 0.518 0.426 0.881 0.927 0.946 0.956
Malaria test 0.323 0.325 0.185 0.455 0.373 0.674 0.643 0.625 0.658

15. Open Data Science Platform
D
a
t
a
Data Lake
Social
Media
Electronic
Notebooks
Databases
Sensor Med
Dev
IoT
Curated
Repository
Models
Curation &
Integration
Validation
Decision
Support
Analysis &
Modeling
Open Data Science Platform
Mining
USERS
Model-driven experimental studies

16. Open Data Science Platform

17. Chemical processing
● Support for chemical
formats
● Chemistry validation
and standardization
● Automatic processing
and visualization

18. Built-in Machine Learning
● Automated ML
pipeline
● Pre-built ML
modules
● Comparison
between different
ML algorithms
● NB, NN, RF, SVM, LR
● DNN

21. Sharing

22. Prediction

23. In progress - materials

24. Technologies
● Mix of technologies connected
through microservices architecture
● Open source toolkits and libraries
with permissive licenses
● NoSQL Databases
● Containerization
● Leading practices in CI/CD
● Automated testing, rapid
development

25. Summary
• A Machine Learning toolkit with simple user interface have been
developed for the Open Science Data Repository software.
• Two major pipelines are implemented: Classic Machine learning methods
(Bernoulli Naive Bayes, Linear Logistic Regression, AdaBoost Decision Tree,
Random Forest, Support Vector Machine), and Deep Neural Networks.
• Multiple models’ performance evaluation metrics, such as ROC, AUC, F1
score, Accuracy, Cohen’s kappa, and Matthews correlation coefficient
were implemented.

26. Summary
• All model were evaluated using relevant to pharmaceutical research
include absorption, distribution, metabolism, excretion and toxicity
(ADME/Tox) properties, as well as activity against pathogens and drug
discovery datasets.
• DNN learning models were found to be very good in predicting activities
and can outperform most of the CML models. The models were applied to
real world drug discovery task like assessing solubility, and exhibited very
good prediction performances.
• Open-source toolkits based libraries show at least at good performance as
commercial ones.

27. Thank you!
On Web:
scidatasoft.com
Slides:
https://www.slideshare.net/valerytkachenko16
Contact us:
info@scidatasoft.com