Computer assignment 1 of the data mining course. This assignment is about using weka on an external database.

1. Data mining
‘REGRESSION: CPU Performance’
Visualized data with WEKA
COMPUTER ASSIGNMENT 1
BARRY KOLLEE
10349863

2. Regression
|
CPU
performance
1. Do you think that ERP should be at least partially predictable from the input attributes?
Not in all cases. This is only possible if we’re able to see correlation between the two attributes that we
compare. In case both values correlate with each other we can state that we can predict certain values
from the input attribute.
2. Do any attributes exhibit significant correlations?
I’ve loaded up the delivered database file into WEKA. With visualising the data as a graph (which shows
the correlation between all attributes) I’m seeing the plotted graphs which is listed below. To see
correlation between all ‘dots’ it is necessary to see a linear pattern. The following correlated graphs
seems to correlate with ERP; respectively MYCT, MMIN and MMAX:
• Green MMAX, with MMAX plotted on the X-axis I see a pattern which is increasing slowly at
first and after words it increases rapidly. If we swap the y and x axis we see the opposite result.
It starts with increasing rapidly and after words it increases slowly.
• Blue MYCT, with MYCT plotted on my x-axis I see a pattern within the correlation between
ERP and MYCT. The pattern look like a (1/n) math graph where we start of with a high value.
When increasing the x-axis you see a direct decrease in the pattern which is going to the
‘zeropoint’ of the Y-axis. When increasing the x-axis even more we don’t see the slope
decreasing anymore. If we swap the x and y axis we see a similar pattern.
• Red MMIN, the pattern which I see within MMIN is similar to the one of MMAX.
2

3. Regression
|
CPU
performance
3. Now we have a feel for the data and we will try fitting a simple linear regression model to
the data. On the Classify tab, select Choose > functions > LinearRegression.
• Use the default options and click Start. This will use 10-fold cross-validation to fit the linear
regression model. Examine the results:
• Record the Root relative squared error and the Relative absolute error. The Relative squared
error is computed by dividing (normalizing) the sum of the squared prediction errors by the sum
of the prediction errors obtained by always predicting the mean. The Root relative squared
error is obtained by taking the square root of the Relative squared error. The Relative absolute
error is similar to the Relative squared error, but uses absolute values rather than squares.
Therefore, if we have a relative error of 100%, the learned model is no better than this very
dumb predictor.
When I perform the linear regression function onto the ERP attribute I’m getting the following
information about this attribute. The ‘Root relative squared error’ is given in red.
Scheme:weka.classifiers.functions.LinearRegression -S 0 -R 1.0E-8
Relation: cpu-weka.filters.unsupervised.attribute.Remove-R1
Instances: 209
Attributes: 7
MYCT
MMIN
MMAX
CACH
CHMIN
CHMAX
ERP
Test mode:10-fold cross-validation
=== Classifier model (full training set) ===
Linear Regression Model
ERP =
0.0661 * MYCT +
0.0142 * MMIN +
0.0066 * MMAX +
0.4871 * CACH +
1.1868 * CHMAX +
-66.5968
Time taken to build model: 0 seconds
=== Cross-validation ===
=== Summary ===
Correlation coefficient 0.928
Mean absolute error 35.4878
Root mean squared error 57.5296
Relative absolute error 40.4842 %
Root relative squared error 37.1725 %
Total Number of Instances 209
The Root relative squared error looks pretty high. That’s because we take all of the attributes into
account and we fit that into our calculation. You can also see that we take 5 attributes into account for
our scope of our linear regression model. Below you see the actual given function of y = ax + b which
represents our linear regression graph model. And we eventually have a scope of 0.928 if we take all
these attributes within our calculation. This calculation looks like:
y = a x + b
a = 0.0661 * MYCT + 0.0142 * MMIN + 0.0066 * MMAX + 0.4871 * CACH + 1.1868 * CHMAX
b = -66.5968
3

4. Regression
|
CPU
performance
I prospect that we can make a better fitting linear regression model if we only take the attributes into
account which correlates best with ERP which we gave in answer 2. If we want to achieve this we only
take MMIN and MMAX into account because it looks like that these attributes correlates best if we
stipulate the output which is given in answer 2. I made another linear regression model where I’ve only
used the MMIN and MMAX attribute, which is given below (Root relative error in red):
=== Run information ===
Scheme:weka.classifiers.functions.LinearRegression -S 0 -R 1.0E-8
Relation: cpu-weka.filters.unsupervised.attribute.Remove-R1-
weka.filters.unsupervised.attribute.Remove-R1,4-6
Instances: 209
Attributes: 3
MMIN
MMAX
ERP
Test mode:10-fold cross-validation
=== Classifier model (full training set) ===
Linear Regression Model
ERP =
0.0128 * MMIN +
0.0087 * MMAX +
-39.814
Time taken to build model: 0 seconds
=== Cross-validation ===
=== Summary ===
Correlation coefficient 0.9022
Mean absolute error 39.8811
Root mean squared error 66.584
Relative absolute error 45.4961 %
Root relative squared error 43.023 %
Total Number of Instances 209
My assumptions were actually wrong. I see that when only taking MMIN and MMAX into account the
correlation coefficient is lower and we’ve got a higher error rate; i.e. the Mean absolute error which is
higher. This value gives us the average of the difference that we find between the actual value and the
value of all the test cases. Also the value Root relative squared error has increased with ca. 6 %.
4. Did you expect such a performance given your earlier observations? Hint: We are fitting a
linear model.
Because we’re trying to fit a linear model we’re searching for the attributes which correlates best with
ERP. The performance boost is clearly visible if we look at the correlation coefficient. A rate of ca. 0.93
is really close to ‘1’ which is the best rate possible.
However the root relative squared error is pretty high. If we would like to get a better fitting linear
regression model we should only try to take attributes into account which correlates best with ERP. This
would result in a correlation coefficient closer to 1 and an error rate which is closer to 0%. However my
observation when only using MMIN and MMAX weren’t that hopeful. Perhaps that’s because these
errors are less seen if we include more attributes. The using of more attributes seems to decrease the
error rate.
On the other hand I would expect that including more attributes would be more error sensitive
4

5. Regression
|
CPU
performance
5. Above we deleted the vendor variable. However, we can use nominal attributes in
regression by converting them to numeric. The standard way of so doing is to replace the
nominal variable with a bunch of binary variables of the form "is_first_nominal_value,
is_second_nominal_value" and so on. Reload the unmodified data file cpu.arff.
• On the Preprocess tab select Choose > filters > unsupervised > attribute >
NominaltoBinary and click Apply. This replaces the vendor variable with 30 binary
variables and we now have 37 attributes (we started with 8).
Now train a linear regression model as in (4) and examine the results.
• Record the Relative absolute error and the Root relative squared error
Scheme:weka.classifiers.functions.LinearRegression -S 0 -R 1.0E-8
Relation: cpu-weka.filters.unsupervised.attribute.NominalToBinary-Rfirst-last
Instances: 209
Attributes: 37
vendor=adviser
vendor=amdahl
vendor=apollo
vendor=basf
vendor=bti
vendor=burroughs
vendor=c.r.d
vendor=cdc
vendor=cambex
vendor=dec
vendor=dg
vendor=formation
vendor=four-phase
vendor=gould
vendor=hp
vendor=harris
vendor=honeywell
vendor=ibm
vendor=ipl
vendor=magnuson
vendor=microdata
vendor=nas
vendor=ncr
vendor=nixdorf
vendor=perkin-elmer
vendor=prime
vendor=siemens
vendor=sperry
vendor=sratus
vendor=wang
MYCT
MMIN
MMAX
CACH
CHMIN
CHMAX
ERP
Test mode:10-fold cross-validation
=== Classifier model (full training set) ===
Linear Regression Model
ERP =
-132.1272 * vendor=adviser +
-34.3319 * vendor=burroughs +
-52.3128 * vendor=gould +
-35.8202 * vendor=honeywell +
-16.7597 * vendor=ibm +
-144.1856 * vendor=microdata +
-22.7172 * vendor=nas +
41.5185 * vendor=sperry +
0.0696 * MYCT +
0.0167 * MMIN +
0.0055 * MMAX +
0.6304 * CACH +
-1.5416 * CHMIN +
1.6106 * CHMAX +
-57.432
Time taken to build model: 0.02 seconds
=== Cross-validation ===
=== Summary ===
5

6. Regression
|
CPU
performance
Correlation coefficient 0.9252
Mean absolute error 35.9725
Root mean squared error 58.5821
Relative absolute error 41.0372 %
Root relative squared error 37.8525 %
Total Number of Instances 209
6. Compare the performance to the one we had previously. Did adding the binarized vendor
variable help?
The errors of the first linear model where:
Relative absolute error 40.4842 %
Root relative squared error 37.1725 %
The error rate of the latest linear regression model is:
Relative absolute error 41.0372 %
Root relative squared error 37.8525 %
It looks like that the error rate has only increased. I think that’s because we now take a lot more
attributes into account what makes our slope (the a in y=ax+b) more complex and error sensitive. I
predict that the error rate would be less higher of we would only take attributes into account which
correlates best with ERP.
6