This document describes research using an ensemble of heterogeneous flexible neural trees (FNTs) to predict the dissolution profiles of poly(lactic-co-glycolic acid) (PLGA) micro- and nanoparticles. The researchers trained multiple FNT regression models on different feature subsets and parameter settings. They then combined the models using an ensemble approach to improve predictive accuracy. Their best model achieved a root mean square error of 11.541 on test data, an improvement over other methods. Feature selection identified the most influential factors on PLGA dissolution. The ensemble of diverse FNT models and feature selection led to more accurate PLGA dissolution profile predictions.

1. Ensemble of Heterogeneous Flexible Neural
Tree for the approximation and feature-
selection of poly (lactic-co-glycolic acid)
micro- and nanoparticle
Varun Kumar Ojha1, Ajith Abraham1, Vaclav Snasel1
1IT4Innovations, VŠB Technical University of Ostrava, Ostrava, Czech
Republic
{varun.kumar.ojha, ajith.abraham, vaclav.snasel}@vsb.cz
Second International Afro-European Conference for Industrial Advancement
September 9-11, 2015, Villejuif, France

2. Problem
• Problem: Prediction of the dissolution prole of Poly Lactic-co-Glycolic
Acid (PLGA) micro- and nanoparticles.
• Motivation: PLGA micro-particles are important diluents in the
• formulation of drugs in the dosage form.
• It act as an excipient in drug formation.
• It helps dissolution of the drugs, thus increases absorbability
• and solubility of drugs.
• It helps in pharmaceutical manufacturing process by improving
• APIs powder's flow ability and non-stickiness.

3. Approach
• Critical Issue: PLGA dissolution prediction is a complex problem as
there are several potential factors influencing dissolution of PLGA
protein particles. Collecting all such influencing factors leads to three
hundred input features in dataset.
• Background: Szlek et al.1 in their article offered a dataset with three
hundred input features divided into four groups, namely protein
descriptor, plasticizer, formulation characteristics, and emulsifier
collected from various literature.
• Goal: Dimensionality reduction using feature selection and finding a
suitable prediction model.
1Szlek, J., Paclawski, A., Lau, R., Jachowicz, R., Mendyk, A.: Heuristic modeling of macromolecule release from PLGA microspheres.
International journal of nanomedicine 8 (2013) 4601.

4. The PLGA dataset description
# Group name # features Importance
1 Protein descriptors 85 Describes the type of molecules and proteins used
2 Formulation characteristics 17 Describe the molecular properties such as molecular
weight, particle size, etc.
3 Plasticizer 98 Describe the properties such as fluidity of the material
used
4 Emulsifier 99 Describe the properties of stabilizing/increase the
pharmaceutical product life
5 Time in days 1 Time taken to dissolve
6 % of molecules dissolved 1 PLGA micro-nanoparticle dissolution-rate

5. Methodology
[Feature Selection and Regression Model Training]
(Training of regression models in order to discover input output relationship)
↓
[Ensemble of regression models]
↓
(To exploit the goodness of all trained regression models instead of relaying on single best)
↓
[Selected features - Regression models/Ensemble model]

6. Flexible Neural Tree (FNT)
𝑅𝑀𝑆𝐸 =
1
𝑁 𝑖=1
𝑁
𝑦𝑖 − 𝑦𝑖
2 ,
Objective: For a dataset with 𝑛 many independent
variables 𝑋 and a dependent variable 𝑌, an
approximation model tries finds relationship between
them. Moreover, it tries to find unknown parameter
𝜃 such that root mean square error (RMSE) between
models’ output 𝑌 and actual output 𝑌 be zero. we
may write RMSE as”
where 𝑁 is number of examples.

7. Flexible Neural Tree
• Analogy with Neural Network
• Function Node: Resembles the Active Nodes.
• Leaf Node: Indicates the Input Nodes
• Edge: Indicates the Synaptic Weights
• Root Node: Indicates Output Node.
• Structure Optimization: Finding an optimal or near-optimal neural
tree is formulated as a product of evolution. For that purpose a
Genetic Programming may be used.
• Parameter Optimization: Particle Swarm Optimization (PSO), Artificial
Bee Colony etc. may be used for the parameter optimization.
• Input Feature Selection: Leaf Nodes represent input features that may
be selected randomly
7

8. Metaheuristics
• To find a solution to a problem using certain rules or
mechanism that may be inspired by the nature.
• The operators of metaheuristics
• Transition: Searching for the solutions (exploration and
exploitation).
• Evaluation: Evaluating the objective function.
• Determination: Deciding the search directions.
• Verifying Goal: Convergence
8

9. Evolutionary Algorithms
• Evolutionary Algorithms
• Genetic population based meta-heuristic algorithm that finds optimal solution using
the dynamics of evolutionary process. Basically uses genetic operates such as
• Selection
• Cross-over
• Mutation.
• Genetic Programming(GP)
• Introduced by John Koza, 1992
• The basic concept of GP is to evolve a program instead of bit-string
• i.e. the Genetic operators are directly applied on the Phenotype rather than
on the Genotype.
• It search for an optimum tree structure (Phenotype) in a program space.
9

10. Cross-Over Operator
10
√ +
+ √

11. Mutation Operator
• Mutation at a single leaf
node.
• Mutation at all leaf nodes
• Mutation by punning a sub-
tree and replace by
randomly generated-Sub-
tree
• Mutation by growing a
tree/appending a randomly
generated sub-tree
11
3
*
*

12. Metaheuristics for Parameter optimization
• Deferential Evaluation (Storn and Price, 1995) Evolutionary Algorithm
based optimization algorithm [Operators – Selection and Crossover ].
• Swarm Based Metaheuristics
• Particle Swarm Optimization (Eberhart and Kennedy, 1995) is a
population based meta-heuristic algorithm imitates the mechanisms
of the foraging behavior of swarms. Depends of velocity and position
update of the particles in a swarm.
• Artificial Bee Colony (Karaboga, 2005) is a meta-heuristic algorithm
inspired by foraging behavior of honey bee swarm. Depends of food
position that is updated by the artificial bees in an iterative fashion.
12

13. Ensemble
• A collective decision with consensus of many member is better
than the decision of an Individual
• Two components of Ensemble
• Construction of diverse and accurate models
• Training models with different sets of data (Bagging)
• Training models with different set of input features (Random Sub-space)
• Training models with different set of parameters
• Combining the models using a combination rules
• Non-trainable
• Trainable
13

14. Ensemble of FNTS
• Making Use of Final Population
• Diversity:
• Models in the final population can have different input features.
• Models in the final population can have different structure.
• Models in the final population can have different active nodes.
• Combination of FNTs
• Regression Problem: Mean of Output, Weighted Mean (Rank based or Trainable
based)
• Classification Problem: Majority Voting, Weighed Majority Voting (Rank based or
Trainable based)
14

15. Experiment Design and Parameter Set-Up
# Parameter Name Parameter Utility Values
1 Tree Height Maximum number of level of FNT 5
2 Tree Arity Maximum number of siblings a node. 10
3 Tree Node Type Indicates the type of activation at nodes. Gaussian
4 GP Population Number of candidates in genetic population. 30
5 Mutation probability The probability that a candidate will be mutated in a genetic
programing. 0.4
6 Crossover probability The probability that candidates will take part in crossover
operation. 0.5
7 elitism The probability that a candidate will propagate to next
generation as it is. 0.1
8 Tournament Size It indicate the size of the pool used for the selection of the
candidates. 15
9 MH Algorithm Population The initial size of the swarm (population). 50
10 MH Algorithm Node Range It defines the search space of the transfer-function arguments. [0,1]
11 MH Algorithm Edge Range It defines the search space for the edges (weights) of tree. [-1.0,1.0]
13 Maximum Structure Iteration Maximum number of generation of genetic programing 100000
14 Maximum Parameter Iteration Maximum number of evaluation of parameter optimization 10000

16. Results
Exp.
Training Test
Ensemble weights
RMSE 𝑟 RMSE 𝑟
1 12.885 0.908 12.741 0.909 0.355686
2 12.907 0.908 13.248 0.903 0.319278
3 13.855 0.897 13.776 0.897 0.259922
4 14.599 0.881 14.374 0.884 0.085882
Ensemble - - 11.541 0.928 -

17. Comparison
# RMSE Features Model Literature
1 12.885 15 FNT Current Work
2 13.34 15 REP Tree [24] Ojha et al. [12]
3 14.3 17 MLP [16] Szlęk et al. [3]
4 14.88 15 GP Regression [23] Ojha et al. [12]
5 15.2 15 MLP [16] Ojha et al. [12]
6 15.4 11 MLP [16] Szlęk et al. [3]

18. Feature Selection
# Feature Index Abbreviation Probability
1 Time Days 299 TD 0.94
2 Prod method 100 PM 0.83
3 PVA conc. inner phase 88 PVA 0.78
4 Ring atom count 110 RAC 0.61
5 Heteroaliphatic ring count 23 HIRC 0.50
6 Aliphatic bond count 104 ABC 0.44
7 Diss. add 98 DA 0.44
8 pH 11 msdon 195 PH11MD 0.39
9 pH 12 msacc 181 PH12MC 0.39
10 Ring count 23 RC 0.39
11 a(yy) 119 AYY 0.28
12 Chain bond count 213 CBC 0.28
13 Diss. add conc. 99 DAC 0.28
14 Fragment count 133 FC 0.28
15 Aromatic ring count 24 ARC 0.22

19. Conclusion
• The aim of PLGA dissolution-rate prediction experiment was to find
the significant variables that governs the prediction rate and to create
a model for realizing the PLGA prediction profile.
• Our current experiment provides an insight of the PLGA dissolution
rate prediction. We have discovered a list of most significant features
by computing their probability of selection using our model (higher
the probability of selection, higher the significance on prediction).
• We achieved high accuracy using FNT model.
• Ensemble of distinct predictors (models) helped in improving
achieved accuracy.

20. Thank You!
varun.kumar.ojha@vsb.cz