Multivariate analysis techniques allow researchers to analyze multiple variables simultaneously. Some key techniques include multiple regression, discriminant analysis, multivariate analysis of variance, factor analysis, cluster analysis, multidimensional scaling, and latent structure analysis. These techniques help reduce complex data into simpler representations and support various types of decision making.

1. Multivariate Analysis
Techniques
DR. KEERTI JAIN
NIIT UNIVERSITY NEEMRANA

2. Introduction
All statistical techniques which simultaneously analyse more than two variables on a sample of
observations can b categorized as multivariate techniques.
Multi variate analysis is collection of methods for analysing data in which a number of observations are
available for each object.
These techniques are being applied in many fields such as economics, sociology, psychology,
agriculture, anthropology, biology and medicine.
These techniques are used when the variables concerning research studies of these fields are supposed
to be correlated with each other and rigorous probabilistic models cannot be appropriately used.
Application of multivariate techniques in practice have been accelerated in modern times because of the
advent of high speed electronic computers.

3. Characteristics and Applications
Multivariate techniques are largely empirical and deal with the reality
The posses the ability to analysis complex data. Multivariate
techniques represent a collection of massive data in simplified way.
It also helps in various types of decision making.

4. Types of Multivariate Techniques
• Dependent Methods: They are used when data containing both dependent and
independent variables. This includes techniques like multiple regression
analysis, multiple discriminant analysis, multivariate analysis of variance and
canonical analysis.
• Interdependent Methods: They are used when data contains several variables
without dependency relationship. This includes techniques like factor analysis,
cluster analysis, multidimensional scaling or MDS and latent structure analysis

5. Classification of Multivariate Techniques

6. All
multivariate
Methods
Are some
variables
dependent?
Yes
Dependent
Methods
No
Interdependent
Methods

7. Dependent
Methods
How many
variables
dependent?
One
Is it
Metric?
Yes
Multiple
Regression
No
Multiple
discriminant
Analysis
Several
Are they
Metric?
Yes
Multivariate
Analysis of
Variance
No
Canonical
Analysis

8. Interdependent
Methods
Are input
metric?
Yes
Factor Analysis
Cluster
Analysis
Metric MDS
No
Non Metric MDS
Latent
Structure
analysis

9. Multiple Regression
• Multiple regression is an extension of simple linear regression.
• It is used when we want to predict the value of a variable based on the value of
two or more other variables.
• The variable we want to predict is called the dependent variable (or sometimes,
the outcome, target or criterion variable).
• The variables we are using to predict the value of the dependent variable are
called the independent variables (or sometimes, the predictor, explanatory or
regressor variables).

10. Example
• You could use multiple regression to understand whether exam performance can
be predicted based on revision time, test anxiety, lecture attendance and gender.
• Alternately, you could use multiple regression to understand whether daily
cigarette consumption can be predicted based on smoking duration, age when
started smoking, smoker type, income and gender.

11. Characteristics of Multiple Regression
• Multiple regression also allows you to determine the overall fit (variance
explained) of the model and the relative contribution of each of the predictors to
the total variance explained.
• For example, you might want to know how much of the variation in exam
performance can be explained by revision time, test anxiety, lecture attendance
and gender "as a whole", but also the "relative contribution" of each independent
variable in explaining the variance.

12. Multiple Discriminant Analysis (MDA)
• Multiple Discriminant Analysis (MDA) is a multivariate dimensionality reduction technique.
• It has been used to predict signals as diverse as neural memory traces and corporate failure.
• MDA is not directly used to perform classification.
• It merely supports classification by yielding a compressed signal amenable to classification.
• This technique reduces the differences between some variables so that they can be classified in
a set number of broad groups, which can then be compared to another variable.
• Multiple discriminant analysis is related to discriminant analysis, which helps classify a data
set by setting a rule or selecting a value that will provide the most meaningful separation.
• MDA has been used to reveal neural codes.

13. Example
• An analyst who wants to select securities based on values that measure volatility
and historical consistency might use multiple discriminant analysis to factor out
other variables such as price.

14. Characteristics Of MDA
• MDA is sensitive to outliers. These should be identified and treated accordingly.
• MDA is only suitable when evaluating the variables' ability
to linearly discriminate between any grouping.
• Highly correlated variables will contribute very similarly to an MDA solution
and may be redundant. Thus, variables that are uncorrelated are preferable.
• While unequal group sizes can be tolerated, very large differences in group sizes
can distort results, particularly if there are very few (< 20) objects per group.

15. • If MANOVA tests on a given set of explanatory variables are insignificant, MDA is
unlikely to be useful.
• When interpreting the coefficients of a discriminant function, carefully distinguish
between standardised and unstandardised coefficients.
• Heteroscedasticity (differing dispersion)is likely to lead to invalid significance tests.
• Across implementations, the absolute values of discriminant weights may vary due
to different scaling and standardisation approaches, but their relative proportions
should be the same.

16. Multivariate analysis of variance (MANOVA)
• Multivariate analysis of variance (MANOVA) is an extension of the univariate analysis of
variance (ANOVA).
• In an ANOVA, we examine for statistical differences on one continuous dependent variable by an
independent grouping variable.
• The MANOVA extends this analysis by taking into account multiple continuous dependent variables,
and bundles them together into a weighted linear combination or composite variable.
• The MANOVA will compare whether or not the newly created combination differs by the different
groups, or levels, of the independent variable.
• In this way, the MANOVA essentially tests whether or not the independent grouping variable
simultaneously explains a statistically significant amount of variance in the dependent variable.

17. Examples
• Do the various school assessments vary by grade level?
• Do the rates of graduation among certain state universities differ by degree
type?
• Which diseases are better treated, if at all, by either X drug or Y drug?

18. Canonical Correlation Analysis
• Canonical correlation analysis is used to identify and measure the associations
among two sets of variables.
• Canonical correlation is appropriate in the same situations where multiple
regression would be, but where, there are multiple intercorrelated outcome
variables.
• Canonical correlation analysis determines a set of canonical variates, orthogonal
linear combinations of the variables within each set that best explain the
variability both within and between sets.

19. Examples
• Example 1. A researcher has collected data on three psychological variables, four academic
variables (standardized test scores) and gender for 600 college freshman. She is interested in
how the set of psychological variables relates to the academic variables and gender. In
particular, the researcher is interested in how many dimensions (canonical variables) are
necessary to understand the association between the two sets of variables.
• Example 2. A researcher is interested in exploring associations among factors from two
multidimensional personality tests, the MMPI and the NEO. She is interested in what
dimensions are common between the tests and how much variance is shared between them.
She is specifically interested in finding whether the neuroticism dimension from the NEO can
account for a substantial amount of shared variance between the two tests.

20. Factor Analysis
• Factor analysis is a technique that is used to reduce a large number of variables into fewer
numbers of factors.
• This technique extracts maximum common variance from all variables and puts them into a
common score.
• As an index of all variables, we can use this score for further analysis.
• Factor analysis is part of general linear model (GLM)
• This method also assumes several assumptions: there is linear relationship, there is no
multicollinearity, it includes relevant variables into analysis, and there is true correlation
between variables and factors.
• Several methods are available, but principle component analysis is used most commonly.

21. Example
• Suppose a psychologist has the hypothesis that there are two kinds of intelligence, "verbal
intelligence" and "mathematical intelligence", neither of which is directly
observed. Evidence for the hypothesis is sought in the examination scores from each of 10
different academic fields of 1000 students. If each student is chosen randomly from a
large population, then each student's 10 scores are random variables. The psychologist's
hypothesis may say that for each of the 10 academic fields, the score averaged over the group
of all students who share some common pair of values for verbal and mathematical
"intelligences" is some constant times their level of verbal intelligence plus another constant
times their level of mathematical intelligence, i.e., it is a combination of those two "factors". The
numbers for a particular subject, by which the two kinds of intelligence are multiplied to obtain
the expected score, are posited by the hypothesis to be the same for all intelligence level pairs,
and are called "factor loading" for this subject.

22. Cluster Analysis
• It is a class of techniques that are used to classify objects or cases into relative
groups called clusters.
• Cluster analysis is also called classification analysis or numerical taxonomy.
• In cluster analysis, there is no prior information about the group or cluster
membership for any of the objects.
• Cluster analysis involves formulating a problem, selecting a distance measure,
selecting a clustering procedure, deciding the number of clusters, interpreting
the profile clusters and finally, assessing the validity of clustering.

23. Example
• Cluster Analysis has been used in marketing for various
purposes. Segmentation of consumers in cluster analysis is used on the basis of
benefits sought from the purchase of the product. It can be used to identify
homogeneous groups of buyers.

24. Multidimensional Scaling (MDS)
• Multidimensional scaling (MDS) can be considered to be an alternative to factor
analysis.
• In general, the goal of the analysis is to detect meaningful underlying
dimensions that allow the researcher to explain observed similarities or
dissimilarities (distances) between the investigated objects.
• In factor analysis, the similarities between objects (e.g., variables) are expressed
in the correlation matrix. With MDS, you can analyze any kind of similarity or
dissimilarity matrix, in addition to correlation matrices.

25. Example
• Suppose we take a matrix of distances between major US cities from a map. We then
analyze this matrix, specifying that we want to reproduce the distances based on two
dimensions. As a result of the MDS analysis, we would most likely obtain a two-
dimensional representation of the locations of the cities, that is, we would basically
obtain a two-dimensional map.
• In general then, MDS attempts to arrange "objects" (major cities in this example) in a
space with a particular number of dimensions (two-dimensional in this example) so
as to reproduce the observed distances. As a result, we can "explain" the distances in
terms of underlying dimensions; in our example, we could explain the distances in
terms of the two geographical dimensions: north/south and east/west.

26. Latent Structure Analysis
• Latent structure analysis is a general class of methods that involve manifest and latent variables that are continuous or
categorical.
• Manifest variables (directly measured or observed) are observed and are usually used as measures of the latent
variables.
• Latent variables are not observed and are the constructs of interest in a theory.
• When the latent variables are continuous, the models are known as structural equation models, which have been widely
used in a number of disciplines, such as psychology, education, biology, and medicine.
• When the latent variables are discrete, the models are known as latent class models, which have been widely used in
sociology and to a lesser extent in psychology.
• In both cases, the manifest variables can be treated as being either continuous or discrete (with ordered or un-ordered
categories).
• Recent developments include models that combine aspects of latent class analysis and structural equation modeling.