The document provides an overview of matrix algebra operations in R, including vectors, matrices, and their applications in psychological data analysis. It covers vector operations like addition, multiplication, and combining vectors into matrices. Matrix topics include addition, multiplication, finding the diagonal, identity matrices, and inversion. The document also demonstrates how these operations can be used for data manipulation tasks like calculating statistics, finding test reliability, and multiple correlation analyses.
Principal Component Analysis, or PCA, is a factual method that permits you to sum up the data contained in enormous information tables by methods for a littler arrangement of "synopsis files" that can be all the more handily envisioned and broke down.
Principal Component Analysis and ClusteringUsha Vijay
Identifying the borrower segments from the give bank data set which has 27000 rows and 77 variable using PROC PRINCOMP. variables, it is important to reduce the data set to a smaller set of variables to derive a feasible
conclusion. With the effect of multicollinearity two or more variables can share the same plane in the in dimensions. Each row of the data can
be envisioned as a 77 dimensional graph and when we project the data as orthonormal, it is expected that the certain characteristics of the
data based on the plots to cluster together as principal components. In order to identify these principal components. PROC PRINCOMP is
executed with all the variables except the constant variables(recoveries and collection fees) and we derive a plot of Eigen values of all the
principal components
Process of converting data set having vast dimensions into data set with lesser dimensions ensuring that it conveys similar information concisely.
Concept
R code
Principal component analysis - application in financeIgor Hlivka
Principal component analysis is a useful multivariate times series method to examine and study the drivers of the changes in the entire dataset. The main advantage of PCA is the reduction of dimensionality where the large sets of data get transformed into few principal factors that explain majority of variability in that group. PCA has found many applications in finance – both in risk and yield curve analytics
Principal Component Analysis, or PCA, is a factual method that permits you to sum up the data contained in enormous information tables by methods for a littler arrangement of "synopsis files" that can be all the more handily envisioned and broke down.
Principal Component Analysis and ClusteringUsha Vijay
Identifying the borrower segments from the give bank data set which has 27000 rows and 77 variable using PROC PRINCOMP. variables, it is important to reduce the data set to a smaller set of variables to derive a feasible
conclusion. With the effect of multicollinearity two or more variables can share the same plane in the in dimensions. Each row of the data can
be envisioned as a 77 dimensional graph and when we project the data as orthonormal, it is expected that the certain characteristics of the
data based on the plots to cluster together as principal components. In order to identify these principal components. PROC PRINCOMP is
executed with all the variables except the constant variables(recoveries and collection fees) and we derive a plot of Eigen values of all the
principal components
Process of converting data set having vast dimensions into data set with lesser dimensions ensuring that it conveys similar information concisely.
Concept
R code
Principal component analysis - application in financeIgor Hlivka
Principal component analysis is a useful multivariate times series method to examine and study the drivers of the changes in the entire dataset. The main advantage of PCA is the reduction of dimensionality where the large sets of data get transformed into few principal factors that explain majority of variability in that group. PCA has found many applications in finance – both in risk and yield curve analytics
Methods for feature/variable selection in Regression AnalysisRupak Roy
Definition of the types of algorithms to choose from Forward, Backward to Stepwise elimination method. Let me know if anything is required.Ping me at google #bobrupakroy
Grouping and Displaying Data to Convey Meaning: Tables & Graphs chapter_2 _fr...Prashant Borkar
This presentation is about Grouping and Displaying Data to Convey Meaning: Tables and Graphs
Contents were taken from Statistics for Management by Levin & Rubin.
Presentation includes,
How can we Arrange Data?
Raw Data
Arranging Data using Data Array & Frequency Distribution
Constructing a Frequency Distribution
Graphing Frequency Distributions
It also covers some solved examples of it.
Cluster analysis of classification is often called the 'non-supervised technique'.
It is a multivariate technique used to determine group membership for cases or variables.
Principal Component Analysis (PCA) and LDA PPT SlidesAbhishekKumar4995
Machine learning (ML) technique use for Dimension reduction, feature extraction and analyzing huge amount of data are Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) are easily and interactively explained with scatter plot graph , 2D and 3D projection of Principal components(PCs) for better understanding.
Machine Learning Tutorial Part - 2 | Machine Learning Tutorial For Beginners ...Simplilearn
This presentation on Machine Learning will help you understand what is clustering, K-Means clustering, flowchart to understand K-Means clustering along with demo showing clustering of cars into brands, what is logistic regression, logistic regression curve, sigmoid function and a demo on how to classify a tumor as malignant or benign based on its features. Machine Learning algorithms can help computers play chess, perform surgeries, and get smarter and more personal. K-Means & logistic regression are two widely used Machine learning algorithms which we are going to discuss in this video. Logistic Regression is used to estimate discrete values (usually binary values like 0/1) from a set of independent variables. It helps to predict the probability of an event by fitting data to a logit function. It is also called logit regression. K-means clustering is an unsupervised learning algorithm. In this case, you don't have labeled data unlike in supervised learning. You have a set of data that you want to group into and you want to put them into clusters, which means objects that are similar in nature and similar in characteristics need to be put together. This is what k-means clustering is all about. Now, let us get started and understand K-Means clustering & logistic regression in detail.
Below topics are explained in this Machine Learning tutorial part -2 :
1. Clustering
- What is clustering?
- K-Means clustering
- Flowchart to understand K-Means clustering
- Demo - Clustering of cars based on brands
2. Logistic regression
- What is logistic regression?
- Logistic regression curve & Sigmoid function
- Demo - Classify a tumor as malignant or benign based on features
About Simplilearn Machine Learning course:
A form of artificial intelligence, Machine Learning is revolutionizing the world of computing as well as all people’s digital interactions. Machine Learning powers such innovative automated technologies as recommendation engines, facial recognition, fraud protection and even self-driving cars.This Machine Learning course prepares engineers, data scientists and other professionals with knowledge and hands-on skills required for certification and job competency in Machine Learning.
We recommend this Machine Learning training course for the following professionals in particular:
1. Developers aspiring to be a data scientist or Machine Learning engineer
2. Information architects who want to gain expertise in Machine Learning algorithms
3. Analytics professionals who want to work in Machine Learning or artificial intelligence
4. Graduates looking to build a career in data science and Machine Learning
Learn more at: https://www.simplilearn.com/
Regularized Principal Component Analysis for Spatial DataWen-Ting Wang
In many atmospheric and earth sciences, it is of interest to identify dominant spatial patterns of variation based on data observed at p locations and n time points with the possibility that p > n. While principal component analysis (PCA) is commonly applied to find the dominant patterns, the eigenimages produced from PCA may exhibit patterns that are too noisy to be physically meaningful when p is large relative to n. To obtain more precise estimates of eigenimages, we propose a regularization approach incorporating smoothness and sparseness of eigenimages, while accounting for their orthogonality. Our method allows data taken at irregularly spaced or sparse locations. In addition, the resulting optimization problem can be solved using the alternating direction method of multipliers, which is easy to implement, and applicable to a large spatial dataset. Furthermore, the estimated eigenfunctions provide a natural basis for representing the underlying spatial process in a spatial random-effects model, from which spatial covariance function estimation and spatial prediction can be efficiently performed using a regularized fixed-rank kriging method. Finally, the effectiveness of the proposed method is demonstrated by several numerical examples.
As part of the GSP’s capacity development and improvement programme, FAO/GSP have organised a one week training in Izmir, Turkey. The main goal of the training was to increase the capacity of Turkey on digital soil mapping, new approaches on data collection, data processing and modelling of soil organic carbon. This 5 day training is titled ‘’Training on Digital Soil Organic Carbon Mapping’’ was held in IARTC - International Agricultural Research and Education Center in Menemen, Izmir on 20-25 August, 2017.
In MATLAB, a vector is created by assigning the elements of the vector to a variable. This can be done in several ways depending on the source of the information.
—Enter an explicit list of elements
—Load matrices from external data files
—Using built-in functions
—Using own functions in M-files
Methods for feature/variable selection in Regression AnalysisRupak Roy
Definition of the types of algorithms to choose from Forward, Backward to Stepwise elimination method. Let me know if anything is required.Ping me at google #bobrupakroy
Grouping and Displaying Data to Convey Meaning: Tables & Graphs chapter_2 _fr...Prashant Borkar
This presentation is about Grouping and Displaying Data to Convey Meaning: Tables and Graphs
Contents were taken from Statistics for Management by Levin & Rubin.
Presentation includes,
How can we Arrange Data?
Raw Data
Arranging Data using Data Array & Frequency Distribution
Constructing a Frequency Distribution
Graphing Frequency Distributions
It also covers some solved examples of it.
Cluster analysis of classification is often called the 'non-supervised technique'.
It is a multivariate technique used to determine group membership for cases or variables.
Principal Component Analysis (PCA) and LDA PPT SlidesAbhishekKumar4995
Machine learning (ML) technique use for Dimension reduction, feature extraction and analyzing huge amount of data are Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) are easily and interactively explained with scatter plot graph , 2D and 3D projection of Principal components(PCs) for better understanding.
Machine Learning Tutorial Part - 2 | Machine Learning Tutorial For Beginners ...Simplilearn
This presentation on Machine Learning will help you understand what is clustering, K-Means clustering, flowchart to understand K-Means clustering along with demo showing clustering of cars into brands, what is logistic regression, logistic regression curve, sigmoid function and a demo on how to classify a tumor as malignant or benign based on its features. Machine Learning algorithms can help computers play chess, perform surgeries, and get smarter and more personal. K-Means & logistic regression are two widely used Machine learning algorithms which we are going to discuss in this video. Logistic Regression is used to estimate discrete values (usually binary values like 0/1) from a set of independent variables. It helps to predict the probability of an event by fitting data to a logit function. It is also called logit regression. K-means clustering is an unsupervised learning algorithm. In this case, you don't have labeled data unlike in supervised learning. You have a set of data that you want to group into and you want to put them into clusters, which means objects that are similar in nature and similar in characteristics need to be put together. This is what k-means clustering is all about. Now, let us get started and understand K-Means clustering & logistic regression in detail.
Below topics are explained in this Machine Learning tutorial part -2 :
1. Clustering
- What is clustering?
- K-Means clustering
- Flowchart to understand K-Means clustering
- Demo - Clustering of cars based on brands
2. Logistic regression
- What is logistic regression?
- Logistic regression curve & Sigmoid function
- Demo - Classify a tumor as malignant or benign based on features
About Simplilearn Machine Learning course:
A form of artificial intelligence, Machine Learning is revolutionizing the world of computing as well as all people’s digital interactions. Machine Learning powers such innovative automated technologies as recommendation engines, facial recognition, fraud protection and even self-driving cars.This Machine Learning course prepares engineers, data scientists and other professionals with knowledge and hands-on skills required for certification and job competency in Machine Learning.
We recommend this Machine Learning training course for the following professionals in particular:
1. Developers aspiring to be a data scientist or Machine Learning engineer
2. Information architects who want to gain expertise in Machine Learning algorithms
3. Analytics professionals who want to work in Machine Learning or artificial intelligence
4. Graduates looking to build a career in data science and Machine Learning
Learn more at: https://www.simplilearn.com/
Regularized Principal Component Analysis for Spatial DataWen-Ting Wang
In many atmospheric and earth sciences, it is of interest to identify dominant spatial patterns of variation based on data observed at p locations and n time points with the possibility that p > n. While principal component analysis (PCA) is commonly applied to find the dominant patterns, the eigenimages produced from PCA may exhibit patterns that are too noisy to be physically meaningful when p is large relative to n. To obtain more precise estimates of eigenimages, we propose a regularization approach incorporating smoothness and sparseness of eigenimages, while accounting for their orthogonality. Our method allows data taken at irregularly spaced or sparse locations. In addition, the resulting optimization problem can be solved using the alternating direction method of multipliers, which is easy to implement, and applicable to a large spatial dataset. Furthermore, the estimated eigenfunctions provide a natural basis for representing the underlying spatial process in a spatial random-effects model, from which spatial covariance function estimation and spatial prediction can be efficiently performed using a regularized fixed-rank kriging method. Finally, the effectiveness of the proposed method is demonstrated by several numerical examples.
As part of the GSP’s capacity development and improvement programme, FAO/GSP have organised a one week training in Izmir, Turkey. The main goal of the training was to increase the capacity of Turkey on digital soil mapping, new approaches on data collection, data processing and modelling of soil organic carbon. This 5 day training is titled ‘’Training on Digital Soil Organic Carbon Mapping’’ was held in IARTC - International Agricultural Research and Education Center in Menemen, Izmir on 20-25 August, 2017.
In MATLAB, a vector is created by assigning the elements of the vector to a variable. This can be done in several ways depending on the source of the information.
—Enter an explicit list of elements
—Load matrices from external data files
—Using built-in functions
—Using own functions in M-files
As part of the GSP’s capacity development and improvement programme, FAO/GSP have organised a one week training in Izmir, Turkey. The main goal of the training was to increase the capacity of Turkey on digital soil mapping, new approaches on data collection, data processing and modelling of soil organic carbon. This 5 day training is titled ‘’Training on Digital Soil Organic Carbon Mapping’’ was held in IARTC - International Agricultural Research and Education Center in Menemen, Izmir on 20-25 August, 2017.
Aplicaciones de Espacios y Subespacios Vectoriales en la Carrera de MecatrónicaBRYANDAVIDCUBIACEDEO
Se da a conocer un poco sobre los espacios y subespacios vectoriales, además de distintas aplicaciones de los mismos en la mecatrónica y distintos ejercicios aplicando el método Wronskiano para determinar la linealidad de un conjunto de funciones.
Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)MdTanvirMahtab2
This presentation is about the working procedure of Shahjalal Fertilizer Company Limited (SFCL). A Govt. owned Company of Bangladesh Chemical Industries Corporation under Ministry of Industries.
Water scarcity is the lack of fresh water resources to meet the standard water demand. There are two type of water scarcity. One is physical. The other is economic water scarcity.
COLLEGE BUS MANAGEMENT SYSTEM PROJECT REPORT.pdfKamal Acharya
The College Bus Management system is completely developed by Visual Basic .NET Version. The application is connect with most secured database language MS SQL Server. The application is develop by using best combination of front-end and back-end languages. The application is totally design like flat user interface. This flat user interface is more attractive user interface in 2017. The application is gives more important to the system functionality. The application is to manage the student’s details, driver’s details, bus details, bus route details, bus fees details and more. The application has only one unit for admin. The admin can manage the entire application. The admin can login into the application by using username and password of the admin. The application is develop for big and small colleges. It is more user friendly for non-computer person. Even they can easily learn how to manage the application within hours. The application is more secure by the admin. The system will give an effective output for the VB.Net and SQL Server given as input to the system. The compiled java program given as input to the system, after scanning the program will generate different reports. The application generates the report for users. The admin can view and download the report of the data. The application deliver the excel format reports. Because, excel formatted reports is very easy to understand the income and expense of the college bus. This application is mainly develop for windows operating system users. In 2017, 73% of people enterprises are using windows operating system. So the application will easily install for all the windows operating system users. The application-developed size is very low. The application consumes very low space in disk. Therefore, the user can allocate very minimum local disk space for this application.
Automobile Management System Project Report.pdfKamal Acharya
The proposed project is developed to manage the automobile in the automobile dealer company. The main module in this project is login, automobile management, customer management, sales, complaints and reports. The first module is the login. The automobile showroom owner should login to the project for usage. The username and password are verified and if it is correct, next form opens. If the username and password are not correct, it shows the error message.
When a customer search for a automobile, if the automobile is available, they will be taken to a page that shows the details of the automobile including automobile name, automobile ID, quantity, price etc. “Automobile Management System” is useful for maintaining automobiles, customers effectively and hence helps for establishing good relation between customer and automobile organization. It contains various customized modules for effectively maintaining automobiles and stock information accurately and safely.
When the automobile is sold to the customer, stock will be reduced automatically. When a new purchase is made, stock will be increased automatically. While selecting automobiles for sale, the proposed software will automatically check for total number of available stock of that particular item, if the total stock of that particular item is less than 5, software will notify the user to purchase the particular item.
Also when the user tries to sale items which are not in stock, the system will prompt the user that the stock is not enough. Customers of this system can search for a automobile; can purchase a automobile easily by selecting fast. On the other hand the stock of automobiles can be maintained perfectly by the automobile shop manager overcoming the drawbacks of existing system.
Final project report on grocery store management system..pdfKamal Acharya
In today’s fast-changing business environment, it’s extremely important to be able to respond to client needs in the most effective and timely manner. If your customers wish to see your business online and have instant access to your products or services.
Online Grocery Store is an e-commerce website, which retails various grocery products. This project allows viewing various products available enables registered users to purchase desired products instantly using Paytm, UPI payment processor (Instant Pay) and also can place order by using Cash on Delivery (Pay Later) option. This project provides an easy access to Administrators and Managers to view orders placed using Pay Later and Instant Pay options.
In order to develop an e-commerce website, a number of Technologies must be studied and understood. These include multi-tiered architecture, server and client-side scripting techniques, implementation technologies, programming language (such as PHP, HTML, CSS, JavaScript) and MySQL relational databases. This is a project with the objective to develop a basic website where a consumer is provided with a shopping cart website and also to know about the technologies used to develop such a website.
This document will discuss each of the underlying technologies to create and implement an e- commerce website.
Quality defects in TMT Bars, Possible causes and Potential Solutions.PrashantGoswami42
Maintaining high-quality standards in the production of TMT bars is crucial for ensuring structural integrity in construction. Addressing common defects through careful monitoring, standardized processes, and advanced technology can significantly improve the quality of TMT bars. Continuous training and adherence to quality control measures will also play a pivotal role in minimizing these defects.
Hybrid optimization of pumped hydro system and solar- Engr. Abdul-Azeez.pdffxintegritypublishin
Advancements in technology unveil a myriad of electrical and electronic breakthroughs geared towards efficiently harnessing limited resources to meet human energy demands. The optimization of hybrid solar PV panels and pumped hydro energy supply systems plays a pivotal role in utilizing natural resources effectively. This initiative not only benefits humanity but also fosters environmental sustainability. The study investigated the design optimization of these hybrid systems, focusing on understanding solar radiation patterns, identifying geographical influences on solar radiation, formulating a mathematical model for system optimization, and determining the optimal configuration of PV panels and pumped hydro storage. Through a comparative analysis approach and eight weeks of data collection, the study addressed key research questions related to solar radiation patterns and optimal system design. The findings highlighted regions with heightened solar radiation levels, showcasing substantial potential for power generation and emphasizing the system's efficiency. Optimizing system design significantly boosted power generation, promoted renewable energy utilization, and enhanced energy storage capacity. The study underscored the benefits of optimizing hybrid solar PV panels and pumped hydro energy supply systems for sustainable energy usage. Optimizing the design of solar PV panels and pumped hydro energy supply systems as examined across diverse climatic conditions in a developing country, not only enhances power generation but also improves the integration of renewable energy sources and boosts energy storage capacities, particularly beneficial for less economically prosperous regions. Additionally, the study provides valuable insights for advancing energy research in economically viable areas. Recommendations included conducting site-specific assessments, utilizing advanced modeling tools, implementing regular maintenance protocols, and enhancing communication among system components.
Event Management System Vb Net Project Report.pdfKamal Acharya
In present era, the scopes of information technology growing with a very fast .We do not see any are untouched from this industry. The scope of information technology has become wider includes: Business and industry. Household Business, Communication, Education, Entertainment, Science, Medicine, Engineering, Distance Learning, Weather Forecasting. Carrier Searching and so on.
My project named “Event Management System” is software that store and maintained all events coordinated in college. It also helpful to print related reports. My project will help to record the events coordinated by faculties with their Name, Event subject, date & details in an efficient & effective ways.
In my system we have to make a system by which a user can record all events coordinated by a particular faculty. In our proposed system some more featured are added which differs it from the existing system such as security.
Forklift Classes Overview by Intella PartsIntella Parts
Discover the different forklift classes and their specific applications. Learn how to choose the right forklift for your needs to ensure safety, efficiency, and compliance in your operations.
For more technical information, visit our website https://intellaparts.com
About
Indigenized remote control interface card suitable for MAFI system CCR equipment. Compatible for IDM8000 CCR. Backplane mounted serial and TCP/Ethernet communication module for CCR remote access. IDM 8000 CCR remote control on serial and TCP protocol.
• Remote control: Parallel or serial interface.
• Compatible with MAFI CCR system.
• Compatible with IDM8000 CCR.
• Compatible with Backplane mount serial communication.
• Compatible with commercial and Defence aviation CCR system.
• Remote control system for accessing CCR and allied system over serial or TCP.
• Indigenized local Support/presence in India.
• Easy in configuration using DIP switches.
Technical Specifications
Indigenized remote control interface card suitable for MAFI system CCR equipment. Compatible for IDM8000 CCR. Backplane mounted serial and TCP/Ethernet communication module for CCR remote access. IDM 8000 CCR remote control on serial and TCP protocol.
Key Features
Indigenized remote control interface card suitable for MAFI system CCR equipment. Compatible for IDM8000 CCR. Backplane mounted serial and TCP/Ethernet communication module for CCR remote access. IDM 8000 CCR remote control on serial and TCP protocol.
• Remote control: Parallel or serial interface
• Compatible with MAFI CCR system
• Copatiable with IDM8000 CCR
• Compatible with Backplane mount serial communication.
• Compatible with commercial and Defence aviation CCR system.
• Remote control system for accessing CCR and allied system over serial or TCP.
• Indigenized local Support/presence in India.
Application
• Remote control: Parallel or serial interface.
• Compatible with MAFI CCR system.
• Compatible with IDM8000 CCR.
• Compatible with Backplane mount serial communication.
• Compatible with commercial and Defence aviation CCR system.
• Remote control system for accessing CCR and allied system over serial or TCP.
• Indigenized local Support/presence in India.
• Easy in configuration using DIP switches.
Democratizing Fuzzing at Scale by Abhishek Aryaabh.arya
Presented at NUS: Fuzzing and Software Security Summer School 2024
This keynote talks about the democratization of fuzzing at scale, highlighting the collaboration between open source communities, academia, and industry to advance the field of fuzzing. It delves into the history of fuzzing, the development of scalable fuzzing platforms, and the empowerment of community-driven research. The talk will further discuss recent advancements leveraging AI/ML and offer insights into the future evolution of the fuzzing landscape.
2. Appendix A
Matrix Algebra in R
Much of psychometrics in particular, and psychological data analysis in general consists of
operations on vectors and matrices. This appendix offers a quick review of matrix oper-
ations with a particular emphasis upon how to do matrix operations in R. For more in-
formation on how to use R, consult the short guide to R for psychologists (at http://
personality-project.org/r/r.guide.html) or the even shorter guide at personality-project.
org/r/r.205.tutorial.html.
A.1 Vectors
A vector is a one dimensional array of n numbers. Basic operations on a vector are addition
and subtraction. Multiplication is somewhat more complicated, for the order in which two
vectors are multiplied changes the result. That is AB 6= BA.
Consider V1 = the first 10 integers, and V2 = the next 10 integers:
> V1 <- as.vector(seq(1, 10))
[1] 1 2 3 4 5 6 7 8 9 10
> V2 <- as.vector(seq(11, 20))
[1] 11 12 13 14 15 16 17 18 19 20
We can add a constant to each element in a vector
> V4 <- V1 + 20
[1] 21 22 23 24 25 26 27 28 29 30
or we can add each element of the first vector to the corresponding element of the second
vector
> V3 <- V1 + V2
[1] 12 14 16 18 20 22 24 26 28 30
2
3. Strangely enough, a vector in R is dimensionless, but it has a length. If we want to multiply
two vectors, we first need to think of the vector either as row or as a column. A column
vector can be made into a row vector (and vice versa) by the transpose operation. While a
vector has no dimensions, the transpose of a vector is two dimensional! It is a matrix with
with 1 row and n columns. (Although the dim command will return no dimensions, in terms
of multiplication, a vector is a matrix of n rows and 1 column.)
Consider the following:
> dim(V1)
NULL
> length(V1)
[1] 10
> dim(t(V1))
[1] 1 10
> dim(t(t(V1)))
[1] 10 1
> TV <- t(V1)
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,] 1 2 3 4 5 6 7 8 9 10
> t(TV)
[,1]
[1,] 1
[2,] 2
[3,] 3
[4,] 4
[5,] 5
[6,] 6
[7,] 7
[8,] 8
[9,] 9
[10,] 10
A.1.1 Vector multiplication
Just as we can add a number to every element in a vector, so can we multiply a number
(a“scaler”) by every element in a vector.
> V2 <- 4 * V1
[1] 4 8 12 16 20 24 28 32 36 40
3
4. There are three types of multiplication of vectors in R. Simple multiplication (each term in
one vector is multiplied by its corresponding term in the other vector), as well as the inner
and outer products of two vectors.
Simple multiplication requires that each vector be of the same length. Using the V1 and V2
vectors from before, we can find the 10 products of their elements:
> V1
[1] 1 2 3 4 5 6 7 8 9 10
> V2
[1] 4 8 12 16 20 24 28 32 36 40
> V1 * V2
[1] 4 16 36 64 100 144 196 256 324 400
The “outer product” of a n * 1 element vector with a 1 * m element vector will result in a n
* m element matrix. (The dimension of the resulting product is the outer dimensions of the
two vectors in the multiplication). The vector multiply operator is %*%. In the following
equation, the subscripts refer to the dimensions of the variable.
nX1 ∗1 Ym =n (XY )m (A.1)
> V1 <- seq(1, 10)
> V2 <- seq(1, 4)
> V1
[1] 1 2 3 4 5 6 7 8 9 10
> V2
[1] 1 2 3 4
> outer.prod <- V1 %*% t(V2)
> outer.prod
[,1] [,2] [,3] [,4]
[1,] 1 2 3 4
[2,] 2 4 6 8
[3,] 3 6 9 12
[4,] 4 8 12 16
[5,] 5 10 15 20
[6,] 6 12 18 24
[7,] 7 14 21 28
[8,] 8 16 24 32
[9,] 9 18 27 36
[10,] 10 20 30 40
The outer product of the first ten integers is, of course, the multiplication table known to all
elementary school students:
4
5. > outer.prod <- V1 %*% t(V1)
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,] 1 2 3 4 5 6 7 8 9 10
[2,] 2 4 6 8 10 12 14 16 18 20
[3,] 3 6 9 12 15 18 21 24 27 30
[4,] 4 8 12 16 20 24 28 32 36 40
[5,] 5 10 15 20 25 30 35 40 45 50
[6,] 6 12 18 24 30 36 42 48 54 60
[7,] 7 14 21 28 35 42 49 56 63 70
[8,] 8 16 24 32 40 48 56 64 72 80
[9,] 9 18 27 36 45 54 63 72 81 90
[10,] 10 20 30 40 50 60 70 80 90 100
The “inner product” is perhaps a more useful operation, for it not only multiplies each corre-
sponding element of two vectors, but also sums the resulting product:
inner.product =
N
X
i=1
V 1i ∗ V 2i (A.2)
> V1 <- seq(1, 10)
> V2 <- seq(11, 20)
> V1
[1] 1 2 3 4 5 6 7 8 9 10
> V2
[1] 11 12 13 14 15 16 17 18 19 20
> in.prod <- t(V1) %*% V2
> in.prod
[,1]
[1,] 935
Note that the inner product of two vectors is of length =1 but is a matrix with 1 row and 1
column. (This is the dimension of the inner dimensions (1) of the two vectors.)
A.1.2 Simple statistics using vectors
Although there are built in functions in R to do most of our statistics, it is useful to understand
how these operations can be done using vector and matrix operations. Here we consider how
to find the mean of a vector, remove it from all the numbers, and then find the average
squared deviation from the mean (the variance).
Consider the mean of all numbers in a vector. To find this we just need to add up the numbers
(the inner product of the vector with a vector of 1’s) and then divide by n (multiply by the
scaler 1/n). First we create a vector of 1s by using the repeat operation. We then show three
different equations for the mean.V, all of which are equivalent.
5
6. > V <- V1
[1] 1 2 3 4 5 6 7 8 9 10
> one <- rep(1, length(V))
[1] 1 1 1 1 1 1 1 1 1 1
> sum.V <- t(one) %*% V
[,1]
[1,] 55
> mean.V <- sum.V * (1/length(V))
[,1]
[1,] 5.5
> mean.V <- t(one) %*% V * (1/length(V))
[,1]
[1,] 5.5
> mean.V <- t(one) %*% V/length(V)
[,1]
[1,] 5.5
The variance is the average squared deviation from the mean. To find the variance, we first
find deviation scores by subtracting the mean from each value of the vector. Then, to find
the sum of the squared deviations take the inner product of the result with itself. This Sum
of Squares becomes a variance if we divide by the degrees of freedom (n-1) to get an unbiased
estimate of the population variance). First we find the mean centered vector:
> V - mean.V
[1] -4.5 -3.5 -2.5 -1.5 -0.5 0.5 1.5 2.5 3.5 4.5
And then we find the variance as the mean square by taking the inner product:
> Var.V <- t(V - mean.V) %*% (V - mean.V) * (1/(length(V) - 1))
[,1]
[1,] 9.166667
Compare these results with the more typical scale, mean and var operations:
> scale(V, scale = FALSE)
[,1]
[1,] -4.5
[2,] -3.5
[3,] -2.5
[4,] -1.5
[5,] -0.5
[6,] 0.5
[7,] 1.5
6
7. [8,] 2.5
[9,] 3.5
[10,] 4.5
attr(,"scaled:center")
[1] 5.5
> mean(V)
[1] 5.5
> var(V)
[1] 9.166667
A.1.3 Combining vectors
We can form more complex data structures than vectors by combining the vectors, either by
columns (cbind) or by rows (rbind). The resulting data structure is a matrix.
> Xc <- cbind(V1, V2, V3)
V1 V2 V3
[1,] 1 11 12
[2,] 2 12 14
[3,] 3 13 16
[4,] 4 14 18
[5,] 5 15 20
[6,] 6 16 22
[7,] 7 17 24
[8,] 8 18 26
[9,] 9 19 28
[10,] 10 20 30
> Xr <- rbind(V1, V2, V3)
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
V1 1 2 3 4 5 6 7 8 9 10
V2 11 12 13 14 15 16 17 18 19 20
V3 12 14 16 18 20 22 24 26 28 30
> dim(Xc)
[1] 10 3
> dim(Xr)
[1] 3 10
7
8. A.2 Matrices
A matrix is just a two dimensional (rectangular) organization of numbers. It is a vector of
vectors. For data analysis, the typical data matrix is organized with columns representing
different variables and rows containing the responses of a particular subject. Thus, a 10 x
4 data matrix (10 rows, 4 columns) would contain the data of 10 subjects on 4 different
variables. Note that the matrix operation has taken the numbers 1 through 40 and organized
them column wise. That is, a matrix is just a way (and a very convenient one at that) of
organizing a vector.
R provides numeric row and column names (e.g., [1,] is the first row, [,4] is the fourth column,
but it is useful to label the rows and columns to make the rows (subjects) and columns
(variables) distinction more obvious. 1
> Xij <- matrix(seq(1:40), ncol = 4)
> rownames(Xij) <- paste("S", seq(1, dim(Xij)[1]), sep = "")
> colnames(Xij) <- paste("V", seq(1, dim(Xij)[2]), sep = "")
> Xij
V1 V2 V3 V4
S1 1 11 21 31
S2 2 12 22 32
S3 3 13 23 33
S4 4 14 24 34
S5 5 15 25 35
S6 6 16 26 36
S7 7 17 27 37
S8 8 18 28 38
S9 9 19 29 39
S10 10 20 30 40
Just as the transpose of a vector makes a column vector into a row vector, so does the
transpose of a matrix swap the rows for the columns. Note that now the subjects are columns
and the variables are the rows.
> t(Xij)
S1 S2 S3 S4 S5 S6 S7 S8 S9 S10
V1 1 2 3 4 5 6 7 8 9 10
V2 11 12 13 14 15 16 17 18 19 20
V3 21 22 23 24 25 26 27 28 29 30
V4 31 32 33 34 35 36 37 38 39 40
1
Although many think of matrices as developed in the 17th century, O’Conner and Robertson discuss the
history of matrix algebra back to the Babylonians (http://www-history.mcs.st-andrews.ac.uk/history/
HistTopics/Matrices_and_determinants.html)
8
9. A.2.1 Matrix addition
The previous matrix is rather uninteresting, in that all the columns are simple products of
the first column. A more typical matrix might be formed by sampling from the digits 0-9.
For the purpose of this demonstration, we will set the random number seed to a memorable
number so that it will yield the same answer each time.
> set.seed(42)
> Xij <- matrix(sample(seq(0, 9), 40, replace = TRUE), ncol = 4)
> rownames(Xij) <- paste("S", seq(1, dim(Xij)[1]), sep = "")
> colnames(Xij) <- paste("V", seq(1, dim(Xij)[2]), sep = "")
> print(Xij)
V1 V2 V3 V4
S1 9 4 9 7
S2 9 7 1 8
S3 2 9 9 3
S4 8 2 9 6
S5 6 4 0 0
S6 5 9 5 8
S7 7 9 3 0
S8 1 1 9 2
S9 6 4 4 9
S10 7 5 8 6
Just as we could with vectors, we can add, subtract, muliply or divide the matrix by a scaler
(a number with out a dimension).
> Xij + 4
V1 V2 V3 V4
S1 13 8 13 11
S2 13 11 5 12
S3 6 13 13 7
S4 12 6 13 10
S5 10 8 4 4
S6 9 13 9 12
S7 11 13 7 4
S8 5 5 13 6
S9 10 8 8 13
S10 11 9 12 10
> round((Xij + 4)/3, 2)
V1 V2 V3 V4
S1 4.33 2.67 4.33 3.67
S2 4.33 3.67 1.67 4.00
S3 2.00 4.33 4.33 2.33
S4 4.00 2.00 4.33 3.33
S5 3.33 2.67 1.33 1.33
9
10. S6 3.00 4.33 3.00 4.00
S7 3.67 4.33 2.33 1.33
S8 1.67 1.67 4.33 2.00
S9 3.33 2.67 2.67 4.33
S10 3.67 3.00 4.00 3.33
We can also multiply each row (or column, depending upon order) by a vector.
> V
[1] 1 2 3 4 5 6 7 8 9 10
> Xij * V
V1 V2 V3 V4
S1 9 4 9 7
S2 18 14 2 16
S3 6 27 27 9
S4 32 8 36 24
S5 30 20 0 0
S6 30 54 30 48
S7 49 63 21 0
S8 8 8 72 16
S9 54 36 36 81
S10 70 50 80 60
A.2.2 Matrix multiplication
Matrix multiplication is a combination of multiplication and addition. For a matrix mXn of
dimensions m*n and nYp of dimension n * p, the product, mXYp is a m * p matrix where each
element is the sum of the products of the rows of the first and the columns of the second.
That is, the matrix mXYp has elements xyij where each
xyij =
n
X
k=1
xik ∗ yjk (A.3)
Consider our matrix Xij with 10 rows of 4 columns. Call an individual element in this matrix
xij. We can find the sums for each column of the matrix by multiplying the matrix by our
“one” vector with Xij. That is, we can find
PN
i=1 Xij for the j columns, and then divide by the
number (n) of rows. (Note that we can get the same result by finding colMeans(Xij).
We can use the dim function to find out how many cases (the number of rows) or the number
of variables (number of columns). dim has two elements: dim(Xij)[1] = number of rows,
dim(Xij)[2] is the number of columns.
> dim(Xij)
[1] 10 4
> n <- dim(Xij)[1]
10
11. [1] 10
> one <- rep(1, n)
[1] 1 1 1 1 1 1 1 1 1 1
> X.means <- t(one) %*% Xij/n
V1 V2 V3 V4
[1,] 6 5.4 5.7 4.9
A built in function to find the means of the columns is colMeans. (See rowMeans for the
equivalent for rows.)
> colMeans(Xij)
V1 V2 V3 V4
6.0 5.4 5.7 4.9
Variances and covariances are measures of dispersion around the mean. We find these by first
subtracting the means from all the observations. This means centered matrix is the original
matrix minus a matrix of means. To make them have the same dimensions we premultiply
the means vector by a vector of ones and subtract this from the data matrix.
> X.diff <- Xij - one %*% X.means
V1 V2 V3 V4
S1 3 -1.4 3.3 2.1
S2 3 1.6 -4.7 3.1
S3 -4 3.6 3.3 -1.9
S4 2 -3.4 3.3 1.1
S5 0 -1.4 -5.7 -4.9
S6 -1 3.6 -0.7 3.1
S7 1 3.6 -2.7 -4.9
S8 -5 -4.4 3.3 -2.9
S9 0 -1.4 -1.7 4.1
S10 1 -0.4 2.3 1.1
To find the variance/covariance matrix, we can first find the the inner product of the means
centered matrix X.diff = Xij - X.means t(Xij-X.means) with itself and divide by n-1. We
can compare this result to the result of the cov function (the normal way to find covari-
ances).
> X.cov <- t(X.diff) %*% X.diff/(n - 1)
> round(X.cov, 2)
V1 V2 V3 V4
V1 7.33 0.11 -3.00 3.67
V2 0.11 8.71 -3.20 -0.18
V3 -3.00 -3.20 12.68 1.63
V4 3.67 -0.18 1.63 11.43
> round(cov(Xij), 2)
11
12. V1 V2 V3 V4
V1 7.33 0.11 -3.00 3.67
V2 0.11 8.71 -3.20 -0.18
V3 -3.00 -3.20 12.68 1.63
V4 3.67 -0.18 1.63 11.43
A.2.3 Finding and using the diagonal
Some operations need to find just the diagonal. For instance, the diagonal of the matrix X.cov
(found above) contains the variances of the items. To extract just the diagonal, or create a
matrix with a particular diagonal we use the diag command. We can convert the covariance
matrix X.cov to a correlation matrix X.cor by pre and post multiplying the covariance matrix
with a diagonal matrix containing the reciprocal of the standard deviations (square roots
of the variances). Remember that the correlation, rxy, is merely the covariancexy/
p
VxVy.
Compare this to the standard command for finding correlations cor.
> round(X.cov, 2)
V1 V2 V3 V4
V1 7.33 0.11 -3.00 3.67
V2 0.11 8.71 -3.20 -0.18
V3 -3.00 -3.20 12.68 1.63
V4 3.67 -0.18 1.63 11.43
> round(diag(X.cov), 2)
V1 V2 V3 V4
7.33 8.71 12.68 11.43
> sdi <- diag(1/sqrt(diag(X.cov)))
> rownames(sdi) <- colnames(sdi) <- colnames(X.cov)
> round(sdi, 2)
V1 V2 V3 V4
V1 0.37 0.00 0.00 0.0
V2 0.00 0.34 0.00 0.0
V3 0.00 0.00 0.28 0.0
V4 0.00 0.00 0.00 0.3
> X.cor <- sdi %*% X.cov %*% sdi
> rownames(X.cor) <- colnames(X.cor) <- colnames(X.cov)
> round(X.cor, 2)
V1 V2 V3 V4
V1 1.00 0.01 -0.31 0.40
V2 0.01 1.00 -0.30 -0.02
V3 -0.31 -0.30 1.00 0.14
V4 0.40 -0.02 0.14 1.00
> round(cor(Xij), 2)
12
13. V1 V2 V3 V4
V1 1.00 0.01 -0.31 0.40
V2 0.01 1.00 -0.30 -0.02
V3 -0.31 -0.30 1.00 0.14
V4 0.40 -0.02 0.14 1.00
A.2.4 The Identity Matrix
The identity matrix is merely that matrix, which when multiplied by another matrix, yields
the other matrix. (The equivalent of 1 in normal arithmetic.) It is a diagonal matrix with 1
on the diagonal I <- diag(1,nrow=dim(X.cov)[1],ncol=dim(X.cov)[2])
A.2.5 Matrix Inversion
The inverse of a square matrix is the matrix equivalent of dividing by that matrix. That
is, either pre or post multiplying a matrix by its inverse yields the identity matrix. The
inverse is particularly important in multiple regression, for it allows is to solve for the beta
weights.
Given the equation
Y = bX + c (A.4)
we can solve for b by multiplying both sides of the equation by
X−1
orY X−1
= bXX−1
= b (A.5)
We can find the inverse by using the solve function. To show that XX−1 = X−1X = I, we
do the multiplication.
> X.inv <- solve(X.cov)
V1 V2 V3 V4
V1 0.19638636 0.01817060 0.06024476 -0.07130491
V2 0.01817060 0.12828756 0.03787166 -0.00924279
V3 0.06024476 0.03787166 0.10707738 -0.03402838
V4 -0.07130491 -0.00924279 -0.03402838 0.11504850
> round(X.cov %*% X.inv, 2)
V1 V2 V3 V4
V1 1 0 0 0
V2 0 1 0 0
V3 0 0 1 0
V4 0 0 0 1
> round(X.inv %*% X.cov, 2)
V1 V2 V3 V4
V1 1 0 0 0
13
14. V2 0 1 0 0
V3 0 0 1 0
V4 0 0 0 1
There are multiple ways of finding the matrix inverse, solve is just one of them.
A.3 Matrix operations for data manipulation
Using the basic matrix operations of addition and multiplication allow for easy manipulation
of data. In particular, finding subsets of data, scoring multiple scales for one set of items, or
finding correlations and reliabilities of composite scales are all operations that are easy to do
with matrix operations.
In the next example we consider 5 extraversion items for 200 subjects collected as part of the
Synthetic Aperture Personality Assessment project. The items are taken from the Interna-
tional Personality Item Pool (ipip.ori.org). The data are stored at the personality-project.org
web site and may be retrieved in R. Because the first column of the data matrix is the subject
identification number, we remove this before doing our calculations.
> datafilename = "http://personality-project.org/R/datasets/extraversion.items.txt"
> items = read.table(datafilename, header = TRUE)
> items <- items[, -1]
> dim(items)
[1] 200 5
We first use functions from the psych package to describe these data both numerically and
graphically. (The psych package may be downloaded from the personality-project.org web
page as a source file.)
> library(psych)
[1] "psych" "methods" "stats" "graphics" "grDevices" "utils"
"datasets" "base"
> describe(items)
var n mean sd median min max range se
q_262 1 200 3.07 1.49 3 1 6 5 0.11
q_1480 2 200 2.88 1.38 3 0 6 6 0.10
q_819 3 200 4.57 1.23 5 0 6 6 0.09
q_1180 4 200 3.29 1.49 4 0 6 6 0.11
q_1742 5 200 4.38 1.44 5 0 6 6 0.10
> pairs.panels(items)
NULL
We can form two composite scales, one made up of the first 3 items, the other made up of the
last 2 items. Note that the second (q1480) and fourth (q1180) are negatively correlated with
the remaining 3 items. This implies that we should reverse these items before scoring.
14
16. To form the composite scales, reverse the items, and find the covariances and then correlations
between the scales may be done by matrix operations on either the items or on the covariances
between the items. In either case, we want to define a “keys” matrix describing which items
to combine on which scale. The correlations are, of course, merely the covariances divided by
the square root of the variances.
A.3.1 Matrix operations on the raw data
> keys <- matrix(c(1, -1, 1, 0, 0, 0, 0, 0, -1, 1), ncol = 2)
> X <- as.matrix(items)
> X.ij <- X %*% keys
> n <- dim(X.ij)[1]
> one <- rep(1, dim(X.ij)[1])
> X.means <- t(one) %*% X.ij/n
> X.cov <- t(X.ij - one %*% X.means) %*% (X.ij - one %*% X.means)/(n - 1)
> round(X.cov, 2)
[,1] [,2]
[1,] 10.45 6.09
[2,] 6.09 6.37
> X.sd <- diag(1/sqrt(diag(X.cov)))
> X.cor <- t(X.sd) %*% X.cov %*% (X.sd)
> round(X.cor, 2)
[,1] [,2]
[1,] 1.00 0.75
[2,] 0.75 1.00
A.3.2 Matrix operations on the correlation matrix
> keys <- matrix(c(1, -1, 1, 0, 0, 0, 0, 0, -1, 1), ncol = 2)
> X.cor <- cor(X)
> round(X.cor, 2)
q_262 q_1480 q_819 q_1180 q_1742
q_262 1.00 -0.26 0.41 -0.51 0.48
q_1480 -0.26 1.00 -0.66 0.52 -0.47
q_819 0.41 -0.66 1.00 -0.41 0.65
q_1180 -0.51 0.52 -0.41 1.00 -0.49
q_1742 0.48 -0.47 0.65 -0.49 1.00
> X.cov <- t(keys) %*% X.cor %*% keys
> X.sd <- diag(1/sqrt(diag(X.cov)))
> X.cor <- t(X.sd) %*% X.cov %*% (X.sd)
> keys
16
17. [,1] [,2]
[1,] 1 0
[2,] -1 0
[3,] 1 0
[4,] 0 -1
[5,] 0 1
> round(X.cov, 2)
[,1] [,2]
[1,] 5.66 3.05
[2,] 3.05 2.97
> round(X.cor, 2)
[,1] [,2]
[1,] 1.00 0.74
[2,] 0.74 1.00
A.3.3 Using matrices to find test reliability
The reliability of a test may be thought of as the correlation of the test with a test just like
it. One conventional estimate of reliability, based upon the concepts from domain sampling
theory, is coefficient alpha (alpha). For a test with just one factor, α is an estimate of the
amount of the test variance due to that factor. However, if there are multiple factors in the
test, α neither estimates how much the variance of the test is due to one, general factor, nor
does it estimate the correlation of the test with another test just like it. (See Zinbarg et al.,
2005 for a discussion of alternative estimates of reliabiity.)
Given either a covariance or correlation matrix of items, α may be found by simple matrix
operations:
1) V = the correlation or covariance matrix
2) Let Vt = the sum of all the items in the correlation matrix for that scale.
3) Let n = number of items in the scale
3) alpha = (Vt - diag(V) )/Vt * n/(n-1)
To demonstrate the use of matrices to find coefficient α, consider the five items measuring
extraversion taken from the International Personality Item Pool. Two of the items need to
be weighted negatively (reverse scored).
Alpha may be found from either the correlation matrix (standardized alpha) or the covariance
matrix (raw alpha). In the case of standardized alpha, the diag(V) is the same as the number
of items. Using a key matrix, we can find the reliability of 3 different scales, the first is made
up of the first 3 items, the second of the last 2, and the third is made up of all the items.
> datafilename = "http://personality-project.org/R/datasets/extraversion.items.txt"
> items = read.table(datafilename, header = TRUE)
> items <- items[, -1]
17
18. > key <- matrix(c(1, -1, 1, 0, 0, 0, 0, 0, -1, 1, 1, -1, 1, -1, 1), ncol = 3)
> key
[,1] [,2] [,3]
[1,] 1 0 1
[2,] -1 0 -1
[3,] 1 0 1
[4,] 0 -1 -1
[5,] 0 1 1
> raw.r <- cor(items)
> V <- t(key) %*% raw.r %*% key
> rownames(V) <- colnames(V) <- c("V1-3", "V4-5", "V1-5")
> round(V, 2)
V1-3 V4-5 V1-5
V1-3 5.66 3.05 8.72
V4-5 3.05 2.97 6.03
V1-5 8.72 6.03 14.75
> n <- diag(t(key) %*% key)
> alpha <- (diag(V) - n)/(diag(V)) * (n/(n - 1))
> round(alpha, 2)
V1-3 V4-5 V1-5
0.71 0.66 0.83
A.4 Multiple correlation
Given a set of n predictors of a criterion variable, what is the optimal weighting of the n
predictors? This is, of course, the problem of multiple correlation or multiple regression.
Although we would normally use the linear model (lm) function to solve this problem, we
can also do it from the raw data or from a matrix of covariances or correlations by using
matrix operations and the solve function.
Consider the data set, X, created in section A.2.1. If we want to predict V4 as a function of
the first three variables, we can do so three different ways, using the raw data, using deviation
scores of the raw data, or with the correlation matrix of the data.
For simplicity, lets relable V4 to be Y and V1 ... V3 to be X1 ...X3 and then define X as the
first three columns and Y as the last column:
X1 X2 X3
S1 9 4 9
S2 9 7 1
S3 2 9 9
S4 8 2 9
S5 6 4 0
S6 5 9 5
18
19. S7 7 9 3
S8 1 1 9
S9 6 4 4
S10 7 5 8
S1 S2 S3 S4 S5 S6 S7 S8 S9 S10
7 8 3 6 0 8 0 2 9 6
A.4.1 Data level analyses
At the data level, we can work with the raw data matrix X, or convert these to deviation
scores (X.dev) by subtracting the means from all elements of X. At the raw data level we
have
mŶ1 =m Xnnβ1 +m 1 (A.6)
and we can solve for nβ1 by pre multiplying by nX0
m (thus making the matrix on the right side
of the equation into a square matrix so that we can multiply through by an inverse.)
nX0
mmŶ1 =n X0
mmXnnβ1 +m 1 (A.7)
and then solving for beta by pre multiplying both sides of the equation by (XX0)−1
β = (XX0
)−1
X0
Y (A.8)
These beta weights will be the weights with no intercept. Compare this solution to the one
using the lm function with the intercept removed:
beta - solve(t(X) %*% X) %*% (t(X) %*% Y)
round(beta, 2)
[,1]
X1 0.56
X2 0.03
X3 0.25
lm(Y ~ -1 + X)
Call:
lm(formula = Y ~ -1 + X)
Coefficients:
XX1 XX2 XX3
0.56002 0.03248 0.24723
If we want to find the intercept as well, we can add a column of 1’s to the X matrix.
one - rep(1, dim(X)[1])
X - cbind(one, X)
print(X)
19
20. one X1 X2 X3
S1 1 9 4 9
S2 1 9 7 1
S3 1 2 9 9
S4 1 8 2 9
S5 1 6 4 0
S6 1 5 9 5
S7 1 7 9 3
S8 1 1 1 9
S9 1 6 4 4
S10 1 7 5 8
beta - solve(t(X) %*% X) %*% (t(X) %*% Y)
round(beta, 2)
[,1]
one -0.94
X1 0.62
X2 0.08
X3 0.30
lm(Y ~ X)
Call:
lm(formula = Y ~ X)
Coefficients:
(Intercept) Xone XX1 XX2 XX3
-0.93843 NA 0.61978 0.08034 0.29577
We can do the same analysis with deviation scores. Let X.dev be a matrix of deviation scores,
then can write the equation
Ŷ = Xβ + (A.9)
and solve for
β = (X.devX.dev0
)−1
X.dev0
Y (A.10)
. (We don’t need to worry about the sample size here because n cancels out of the equa-
tion).
At the structure level, the covariance matrix = XX’/(n-1) and X’Y/(n-1) may be replaced by
correlation matrices by pre and post multiplying by a diagonal matrix of 1/sds) with rxy and
we then solve the equation
β = R−1
rxy (A.11)
Consider the set of 3 variables with intercorrelations (R)
x1 x2 x3
x1 1.00 0.56 0.48
x2 0.56 1.00 0.42
x3 0.48 0.42 1.00
20
21. and correlations of x with y ( rxy)
x1 x2 x3
y 0.4 0.35 0.3
From the correlation matrix, we can use the solve function to find the optimal beta weights.
R - matrix(c(1, 0.56, 0.48, 0.56, 1, 0.42, 0.48, 0.42, 1), ncol = 3)
rxy - matrix(c(0.4, 0.35, 0.3), ncol = 1)
colnames(R) - rownames(R) - c(x1, x2, x3)
rownames(rxy) - c(x1, x2, x3)
colnames(rxy) - y
beta - solve(R, rxy)
round(beta, 2)
y
x1 0.26
x2 0.16
x3 0.11
A.4.2 Non optimal weights and the goodness of fit
Although the beta weights are optimal given the data, it is well known (e.g., the robust
beauty of linear models by Robyn Dawes) that if the predictors are all adequate and in the
same direction, the error in prediction of Y is rather insensitive to the weights that are used.
This can be shown graphically by comparing varying the weights of x1 and x2 relative to
x3 and then finding the error in prediction. Note that the surface is relatively flat at its
minimum.
We show this for sevaral different values of the rxy and R matrices by first defining two
functions (f and g) and then applying these functions with different values of R and rxy. The
first, f, finds the multiple r for values of bx1/bx3and bx2/bx3 for any value or set of values given
by the second function. ranging from low to high and then find the error variance (1 − r2)
for each case.
f - function(x, y) {
+ xy - diag(c(x, y, 1))
+ c - rxy %*% xy
+ d - xy %*% R %*% xy
+ cd - sum(c)/sqrt(sum(d))
+ return(cd)
+ }
g - function(rxy, R, n = 60, low = -2, high = 4, ...) {
+ op - par(bg = white)
+ x - seq(low, high, length = n)
+ y - x
+ z - outer(x, y)
+ for (i in 1:n) {
21
22. + for (j in 1:n) {
+ r - f(x[i], y[j])
+ z[i, j] - 1 - r^2
+ }
+ }
+ persp(x, y, z, theta = 40, phi = 30, expand = 0.5, col = lightblue,
ltheta = 120, shade = 0.75,
+ ticktype = detailed, zlim = c(0.5, 1), xlab = x1/x3,
ylab = x2/x3, zlab = Error)
+ zmin - which.min(z)
+ ymin - trunc(zmin/n)
+ xmin - zmin - ymin * n
+ xval - x[xmin + 1]
+ yval - y[trunc(ymin) + 1]
+ title(paste(Error as function of relative weights min values at x1/x3 = ,
round(xval, 1),
+ x2/x3 = , round(yval, 1)))
+ }
R - matrix(c(1, 0.56, 0.48, 0.56, 1, 0.42, 0.48, 0.42, 1), ncol = 3)
rxy - matrix(c(0.4, 0.35, 0.3), nrow = 1)
colnames(R) - rownames(R) - c(x1, x2, x3)
colnames(rxy) - c(x1, x2, x3)
rownames(rxy) - y
Figure A.2 shows the residual error variance as a function of relative weights of bx1/bx3and
bx2/bx3 for a set of correlated predictors.
R
x1 x2 x3
x1 1.00 0.56 0.48
x2 0.56 1.00 0.42
x3 0.48 0.42 1.00
rxy
x1 x2 x3
y 0.4 0.35 0.3
g(R, rxy)
22
24. x1/x3
−2
−1
0
1
2
3
4
x
2
/
x
3
−2
−1
0
1
2
3
4
E
r
r
o
r
0.5
0.6
0.7
0.8
0.9
1.0
Error as function of relative weights min values at x1/x3 = 1.5 x2/x3 = 1.2
Figure A.3: With orthogonal predictors, the squared error surface is somewhat better defined,
although still large changes in beta weights do not lead to large changes in squared error.
With independent predictors, the response surface is somewhat steeper (Figure A.3), but still
the squared error function does not change vary rapidly as the beta weights change. This
phenomenon has been well discussed by Robyn Dawes and his colleagues.
x1 x2 x3
x1 1 0 0
x2 0 1 0
x3 0 0 1
x1 x2 x3
y 0.4 0.35 0.3
24