Use of Definitive Screening Designs to Optimize an Analytical Method

Applications of Designed Experiments in the Development of an
Analytical Method for Glycoprofiling
Eliza Yeung, Ph.D.
R&D Strategic Projects Manager/
PC Study Director
Cytovance Biologics, Inc
800 Research Parkway, Ste 200
Oklahoma City, OK, USA
eyeung@cytovance.com
Philip J. Ramsey, Ph.D.
Department of Math. and Stat.
North Haven Group, consultancy
University of New Hampshire
Durham, NH, USA
Cell: 1-315-3518
philip.ramsey@unh.edu

Glycoproteins are the largest group of biologically-derived drugs.
ICH Q6B guideline requires extensive physiochemical characterization
of biopharmaceuticals including inherent structural heterogeneity due
to glycosylation (post-translationally modified) and lot-to-lot
consistency is required.
 Carbohydrate content.
 Carbohydrate chain structure.
 Oligosaccharide pattern
(antennary profile).
 Glycosylation site.
Currently, there is a lack of a universal analysis technique for
glycosylation analysis.
(C) 2015 Cytovance 2
High-Mannose Complex-types
biantennary triantennary tetrantennary
Common types of N-glycans. UPS General Chapter <1084>.

The goal of the research is to develop a robust and cost effective
method to characterize glycoproteins.
High Performance Anion Exchange Chromatography with pulsed
amperometic detection (HPAE-PAD) was selected as a potential
method for characterization.
The goal is to express glycan
peaks as a function of an
in-house glucose ladder (GU).
An experimental design approach
was selected to characterize
and optimize (make robust)
the HPAE-PAD method.
0 10 20 30 40 50
-10
0
10
20
30
40
50
Glucose Ladder
GU=1
GU=2
GU=3
GU=4
GU=5
GU=6
GU=7
GU=8
GU=9
GU=10
GU=11
Time (min)
Response

The approach uses the GU ladder as a reference to identify glycoform
peaks from an actual human antibody sample.
Glycoform
glycan
G-Unit
A 3.59
B 3.89
C 4.23
D 4.42
E 9.17
F 10.8
0 10 20 30 40 50
-10.0
-7.5
-5.0
-2.5
0.0
2.5
5.0
7.5
10.0
human IgG
Glucose ladder
GU=1
GU=2
GU=3
GU=4
GU=5
GU=6
GU=7
GU=8
GU=9
GU=10
GU=11
Neutral
A
B
C
D
E F
Charged
Time (min)
Response
Neutrall Charged

Sources of variation in glycoprofiling and analytical target profile.
Analytical Target Profile (ATP)
• A good separation of neutral & charge glycoform glycans in a single run
• Consistent glycan peak assignment

Test biomolecule
glycans released
by PNGase F
digestion of a
glycoprotein
HPAE-PAD
glycoprofiling of
test biomolecule
glycans
(each glycan is
expressed as
GU, glucose
unit)
Define responses
of interested and
factors for DOE
(e.g. resolution
between targeted
GUs)
Use the JMP
statistical
software to
design the
experiment
(DOE)
Use glucose
ladder for DOE
study
Use JMP to select
models for each
response
Use JMP to
optimize all
responses of
interested

A total of 8 runs using a human IgG sample were performed to
demonstrate the stability or run to run robustness of the HPAE-PAD
method using the Glucose Ladder – some of the runs were
performed on separate days.
Below is a table of the results and one can see that the identified
glycoform peaks in the IgG sample consistently eluted with respect
to the GU peaks.
Glycoform G-Unit Std Dev %CV
A 3.59 0.07 1.9%
B 3.89 0.09 2.4%
C 4.23 0.08 1.9%
D 4.42 0.08 1.7%
E 9.17 0.09 0.9%
F 10.8 0.12 1.1%

Five factors were selected to manipulate in the experiment.
* Gradient_01, _02 and _03 are % A (500 mM NaOAc) increases over 12 min, 12 min and 18
min respectively and at constant initial % B (200 mM NaOH,10 mM NaOAc). The values are
expressed as mM NaOAc per min.
Factor (level) -1 0 1
Initial %NaOAc (% A) 0 10 20
Initial %NaOH (% B) 30 40 50
Gradient_01
(mM NaOAc /min)
0.415 1.25 2.085
Gradient_02
(mM NaOAc /min)
1.25 2.085 2.915
Gradient_03
(mM NaOAc /min)
4.72 5.555 6.39

Seven responses were chosen to optimize in the experiment.
Response Description Optimization
RT_G03 Retention Time Target ~ 8.5 min
Resol_G03 Resolution G03-G04 Maximize
USP Tailing USP Tailing G04 Monitor (0.8-1.2)

There are many choices for the experimental design and it was expected
that nonlinear and interaction effects would occur.
Below is a table of commonly used types of designs.
The number of runs displayed assume 3 replicate center points for the
DSD and Fractional Factorial and 2 for the CCD.
No. of
Factors
Definitive Screening
Design
Screening Design
e.g. Fractional Factorial
Response Surface
e.g. Central
Composite Design
4 12 11 26
5 16 19 28
6 16 19 or 35 46
7 20 19 or 35 80
8 20 19 or 35 82

A new class of screening designs were developed by Jones and
Nachtsheim (2011a, 2011b) referred to as Definitive Screening
Designs (DSD); the designs have been enhanced by Xiao (2012).
 For K factors DSDs require 2K+1 runs if K is even and 2K+3 if K
is odd (to ensure main effect orthogonality).
 All factors are run at three levels in a factorial arrangement.
 Main effects are orthogonal and free of aliasing (partial or full)
with quadratic effects and two-way interaction effects.
 No quadratic or two-way interaction effect is fully aliased with
another quadratic or two-way interaction effect.
 It is possible to estimate every term of a full quadratic model, but
not in a single model.

Although the DSD is thought of as a screening design it can just as
easily thought of as a very efficient response surface design.
Traditional response surface designs, such as the Central Composite
Design, are necessarily large because they support the estimation
of the full quadratic model
Full quadratic: a model that contains all main effects, all quadratic
effects, and all possible two-factor interactions.
For K factors the number of terms N in the full quadratic model is:
In practice, effect sparsity almost always holds, so it is not necessary
to estimate all N terms; in fact often <50% are actually important.
  2 1
2
K K
N
 


It was decided to perform both the Definitive Screening Design
(DSD) and the larger Central Composite Design in parallel to
determine how well the DSD performed compared to the much
larger CCD.
Our focus in this talk is primarily on the DSD, but the CCD results
will be used as a basis of comparison.
For a set of p potential experimental effects there are potentially 2P
possible models to consider – a rather daunting challenge.
For K = 5 factors there are 20 possible experimental effects in the
full quadratic model resulting in a total of 220 = 1,048,576 possible
models of varying size in terms of included effects.
Fortunately only a subset of the 20 potential effects are likely to be
important, so our model sizes are considerably smaller than 20.

In building predictive models we have two competing issues:
 Under-fitting the model resulting in biased or inaccurate
prediction;
 Over-fitting the model resulting in inflated prediction error.
Although the classic approach to the under- and over-fitting problem
is to find a single, best compromise model, this is not necessarily an
optimal strategy.
In no way can one consider a single model to be correct; a
correct model only exists in a simulation study.
Modern computing power and statistical algorithms available allow us
to look for models, or a combination of models, that best predict
the behavior of the physical system.

Two widely accepted measures of fit for a model are:
AICc = bias corrected Akaike Information Criterion;
BIC = Bayesian Information Criterion;
Both criteria punish under- and over-fitting, but in a different way.
So, they may not agree on the best model(s) – they often do not
(see Burnham and Anderson, 2002).
There is not agreement in the statistical community as to whether
AICc or BIC criterion is preferred; it almost surely depends on the
application.
For both the AICc and BIC smaller values indicate better
predictive models.

A major focus in contemporary statistics and science is research into
model selection strategies with no shortage of ideas and opinions.
We will focus on a straightforward approach referred to as All
Possible Models where we compute all possible models up to a
specified size.
As an example, we fit all one factor models, all two factors, etc.
Given the DSD is supersaturated for the full quadratic model, that is
the number of possible effects p > n, the largest possible model is
determined by the number of experimental settings in the DSD.
The strategy we use is to define a full quadratic model and then fit all
possible models up to and including a specified size.
The fitted models are then sorted by AICc and BIC.

Instead of AICc and BIC we use a form of them with nice theoretical
properties for model selection.
It is best to rank models based on AICc differences computed as
where AICcmin is the smallest AICc among the candidate models.
It is also best to work with BIC differences in ranking models where
the differences are computed as
Typically models where  or B exceed a range of 2 – 4 are excluded
from further consideration – this is not a hard and fast rule.
We will use this strategy on the HPAE-PAD experiment.
mini iAICc AICc  
mini iB BIC BIC 

Below is a view of the DSD trials and experimental results for the
response.
The analyses are preformed with the JMP statistical software.

Below is a plot of the delta AICc and BIC results that help us identify
potential candidate models; response is RT_03.
AICc suggests
models with
about p =5.
BIC suggesting
models with
around p = 7
to 10.
As expected the
two criteria do
not agree as to
best models.

Using All Possible Models we fit a number of models for each
response and select the best models in terms of fit and prediction
error.
Below is a typical model for RT_03

Optimized
Settings.
DSD runs.

Using function decomposition methods from applied math we can
assess the relative importance of the input factors.
This is a unique capability in JMP 12.

A similar modeling and optimization process was performed using the
data from the much larger CCD experiment.
Below is a comparison table of the recommended optimum settings for
the inputs for the two designs.
There is close agreement on the three most important inputs.
Input DSD Optimum CCD Optimum
Initial %Ac 1.20 1.55
Initial %NaOH 50.00 50.00
Gradient_01 1.23 0.42
Gradient_02 1.25 2.92
Gradient_03 4.87 4.72

Below are two Actual vs Predicted plots based on the DSD-based
models predicting the CCD responses, which is used as a validation
data set.

The predicted responses are also very close for the two designs. One
can see that the predicted responses at the optimum settings are
close.
Response
DSD
Optimum
CCD
Optimum
2*(Std. Errors)
RT_03 8.50 8.50 0.60
Resol_03 8.38 8.68 2.40
Resol_04 8.80 9.97 1.20
Resol_05 8.37 7.73 1.00
Resol_09 4.69 4.14 1.20
Resol_10 3.78 3.54 0.66

Robust analytical methods are required by QbD for GMP in
Biopharmaceuticals and in general for sound research results.
Definitive Screening Designs are a cost effective type of experimental
design that can be used to characterize and optimize analytical
methods or many physical phenomena.
In this presentation we have shown that the DSD performed as well as
the CCD in optimizing the HPAE-PAD despite having
approximately half the total number of experimental trials.
Note: When substantial amounts of observational data are available,
the methods shown in this talk can often still be used.

The following are couple of additional events that you might be
interested in attending.
Session C2 Friday Morning. Application of Definitive Screening
Design (DSD) to the icIEF Assay Development of Antibodies and
Therapeutic Proteins. Dr. Srividya Suryanarayana (R & D Services
Cytovance Biologics, United States).
Session C2 Friday 13:50 – 17:00. Workshop: Modern Design and
Analysis of Experiments for Biological Applications Using the
JMP® Statistical Software. Dr. Philip J. Ramsey (University of New
Hampshire, United States)

Use of Definitive Screening Designs to Optimize an Analytical Method

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Use of Definitive Screening Designs to Optimize an Analytical Method

Similar to Use of Definitive Screening Designs to Optimize an Analytical Method (20)

Recently uploaded

Recently uploaded (20)

Use of Definitive Screening Designs to Optimize an Analytical Method