Managerial Statistics II - Hardware or Software: What Leads to Higher Review Ratings?

Alberto Ciaraldi and Jameson Cook
Team 124
5 December 2015
Hardware or Software: What Leads to Higher Review Ratings?
1. Introduction
A. General Overview
We would like to know what aspects of a phone leads to a higher review rating from well
respected tech blogs. We have decided to analyze the software of a phone based on operating
system and the hardware of a phone based on hardware specifications of a phone’s internal
components, while considering every phone’s price at launch for their baseline model.
This information could be useful for business executives at phone manufacturing companies,
because consumers in the 21st century are increasingly using internet reviews by well trusted
sources to help determine their purchasing decisions. Our outcomes will provide insights to
which attributes of a phone lead to higher ratings online, that can in return lead to heightened
customer demand to increase sales and revenue. This issue is extremely interesting, because the
report investigates how a phone’s software affects ratings as well as hardware. We will also be
able to give specific recommendations for the hardware components that lead to higher ratings
most directly; therefore, we can tell executives what hardware to prioritize.
B. Relevant Sources
A comprehensive report, titled “Gigahertz, Megapixels, and Millimeters - Do Specs Matter At
All?” from Android Central, another influential tech blog, discusses the topic of whether
hardware features and specs really matter to the end user. However, the article is very qualitative
and does not use any hard data or analysis of that data to reach conclusions. We think our model
will show direct correlations between individual phone specifications and the way that the phone
is viewed by reputable tech websites. This will give us specific recommendations and ideas on
how to improve future phone ratings by changing certain internal smartphone components.
In an article called “The Phone Specs That Matter,” TechHive states which phone specifications
do matter to consumers, which we predict would also increase CNET ratings. Again it is only
qualitative, so our model using specific data will clearly defend or negate these assertions.
C. Our Project

We intend to contribute useful and statistically significant findings about the relationship
between the 100 most recently reviewed phones on CNET and their respective phone price,
hardware specifications, and operating system. Using the 100 most recently reviewed phones as
our data points adds a sense of consistency and timeliness to the information that may not have
been present if we’d researched phones that were too old, no longer in the marketplace, or
released at very different times.
Our project differs from other analyses on this question, because other analyses in this topic
focus solely on whether to care about specs at all, but do not include any type of analytical
research using data. There is a large amount of speculation and theories present on the web in
regards to this question, however they reside on the qualitative side. This analysis will create a
very specific, data-driven results that will illustrate what variables affect a phone’s rating.
D. Hypothesis
Similarly to TechHive’s article, we expect to see a positive correlation between a phone’s rating
on CNET and certain component specs, such as screen resolution, screen size, processing speed,
RAM, and camera megapixels. We also believe that a phone’s operating system will either
negatively or positively impact a phone’s rating significantly. We think that iOS phones will
have higher ratings, Windows Phone and BlackBerry OS phones will have lower ratings, and are
unsure what effect Android will have on ratings due to the large number and variety of Android
phones. Price is a very important factor in every consumer’s decision making process, so we
think that price will have the greatest impact on review scores.
2. Key Results
Our model gave us conclusive findings on the relationship that a phone’s price, hardware, and
software have on its’ review rating on CNET. One of the most interesting findings wasn’t just
the coefficients of these relationships, but the relationships that mattered and were statistically
significant.
Through backwards elimination to find the best fit model, we found that weight, RAM, battery
size, memory, thickness, and secondary camera megapixels do not have a statistically significant
relationship with CNET rating. In our best fit model, we found that, on average, choice of
operating system has a more significant correlation with review rating than any individual
hardware component; however, the hardware components combined have more of an influence
on rating than any particular operating system.
The hardware components that do have a significant correlation with rating are main camera
megapixels and screen resolution, with respect to their magnitude of correlation from highest to
lowest. Additionally, we found that price had a lower correlation with rating than camera and

software, but higher than screen resolution. We found that processor speed and screen size did
not have a statistically significant relationship with CNET rating.
Below is the individual relationship between CNET rating with camera and screen resolution
respectively.
3. Data
A. Data Sources
To get the dependent variable, the CNET review ranking, we went through the most recent 100
reviewed phones and recorded their rating manually.
To get the independent variables, the phone’s specs, we used GSM Arena to manually record the
hardware specifications for all of the 100 most recently reviewed phones on CNET. GSM Arena
provided us with a detailed report of all the different components that make up cellular phones.

B. Dependent Variable:
1) CNET Rating: The score that CNET gave the phone when reviewing it upon launch out
of a 1-10 scale.
C. Independent Variables:
1) Baseline Price ($): The initial price at launch for the baseline model of the phone.
2) Main Camera Megapixels (MP): The number of megapixels of the phone’s back camera.
3) Secondary Camera Megapixels (MP): The number of megapixels of the phone’s front
camera.
4) Screen Size (Inches): The diagonal length in inches of the phone’s screen.
5) Screen Resolution (Pixels per Inch): The amount of pixels within every square inch of the
phone’s display, known as PPI.
6) Processor Speed: The speed of the phone’s CPU taken by multiplying the number of
cores by the processing speed of each individual CPU core in GHz for the phone’s
baseline model.
7) RAM (GB): The amount of RAM the phone has in gigabytes.
8) Base Memory Storage Capacity (GB): The amount of gigabytes of internal memory
storage for the baseline model of the phone.
9) Battery Size (mAh): The size of the phone’s battery in Milliamp Hours.
10) Thickness (mm): The thickness of the phone in millimeters.
11) Weight (Grams): The weight of the phone in grams.
12) iOS: Indicator variable determining whether or not the phone’s operating system is
Apple’s proprietary OS iOS. We used iOS as our base case for our indicator variables.
13) Android: Indicator variable determining whether or not the phone’s operating system is
Google’s open source Android.
14) Windows Phone: Indicator variable determining whether or not the phone’s operating
system is Microsoft’s proprietary OS that can be licensed to third-party OEMs, such as
Nokia.
15) BlackBerry OS: Indicator variable determining whether or not the phone’s operating
system is Microsoft’s proprietary OS.
D. Potential Criticism & Supplemental Data:
The used data came from two separate, respected websites. For the phone’s specifications, all
data was credible, because GSM Arena is notorious for being the most accurate and qualified site
for hardware specifications on the Internet. CNET review ratings are credible because they are
one of the oldest running tech blogs that remains to be an industry leader with very high traffic.
Someone might say that review ratings are subjective, however that is the very nature of ratings,
so criticizing such a process would be irrelevant to the point that is being tried to be solved.

We had to record the data manually, which could have led to human error in the data.
Additionally, three people manually recorded the data, so someone could have been recording
the data differently than others. To prevent internal errors, we created a unified method for
recording the data and all looked over the websites together, to ensure that our data recording
was consistent overall. Furthermore, as a final measure, one person went through all of the data
to make sure that all data entries appeared consistent within each variable’s respective range, and
identified and fixed any possible errors.
Our model would be even more accurate if we could see how the phone is being received by the
public. We could analyze the amount of mentions on different social media platforms and the
general tone when the phone is mentioned in a post. Unfortunately, we do not have any access to
this data through Facebook, Twitter, or other social networks.
We would have liked to see how ratings affected phone sales to give better recommendations to
tech executives, however finding this data by ourselves would not have been consistent due to
differing phone release dates. Our MLR could have also been more accurate if we included the
release date of each phone to account for technological improvements that naturally occur over
time.
E. Descriptive Statistics and Scatter Plots
Below is the descriptive statistics for all independent variables. We will include the scatter plots
for all independent variables in the appendix, part A.

1. Baseline Price:
For baseline price, there is a positive correlation and linear relationship, with a coefficient of
0.0029, between our dependent variable, CNET rating, and the phones’ baseline price, on
average, holding all other independent variables equal.
All descriptive statistics seem accurate and the data looks internally consistent.
Some companies may charge higher prices due to aggressive and well functioning marketing
campaigns, while some companies may charge a low price in order to increase sales and market
share.
2. Main Camera Megapixels:
For main camera megapixels, there is a positive correlation and linear relationship, with a
coefficient of 0.1237, between our dependent variable, CNET rating, and the amount of
megapixels on the back camera of the phone, on average, holding all other independent variables
equal.

Some companies may segment towards different consumers that either care more or less about
having a high quality camera in their cell phone. Depending on their business strategy, this could
create extremes of a very high megapixel back camera as a core feature or a very low megapixel
back camera as an afterthought.
3. Secondary Camera Megapixels:
For baseline secondary camera megapixels, there is a positive correlation and linear relationship,
with a coefficient of 0.0917, between our independent variable, CNET rating, and the phones’
baseline secondary camera megapixels, on average, holding all other dependent variables equal.
Similar to the megapixels for back phone cameras, companies can either provide a front facing
camera with high megapixels to appeal to younger audiences that appreciate selfies. On the other
side, they can not include a front facing camera or provide a low megapixel camera to bring
down overall cost.
4. Screen Size:
For screen size, there is a positive correlation and linear relationship, with a coefficient of
0.8696, between our dependent variable, CNET rating, and the phone’s screen size, on average,
holding all other independent variables equal.
The outliers will come from companies’ business strategy as they can have bigger screen sizes
for pro and advanced users, while offering small screen sizes for younger or price-conscious
audiences.
5. Screen Resolution:
For the display’s pixels per inch, there is a positive correlation and linear relationship, with a
coefficient of 0.0058, between our dependent variable, CNET rating, and the phone's screen
resolution, on average, holding all other independent variables equal.
There are examples of outliers that have the sharpest resolution technology offers, but demand a
high price. There are others that have low screen resolution, in order to be a budget device that
will most likely receive a lower rating.

6. Processor Speed:
For the number of processing cores, there is a positive correlation and linear relationship, with a
coefficient of 0.0758, between our dependent variable, CNET rating, and the phone’s processor
speed, on average, holding all other independent variables equal.
Phones that have a high processor speed may be pricier and get better reviews. Phones that have
lower processor speed may have a lower price or could use an operating system, such as iOS,
that doesn’t require a high amount of processing power.
7. RAM:
For RAM, there is a positive correlation and linear relationship, with a coefficient of 0.5599,
between our independent variable, CNET rating, and the phones’ RAM, on average, holding all
other dependent variables equal.
Phones with a high RAM are going to be more expensive and thick, while phones that have
lower RAM are going to be cheaper to be sold as budget phones.
8. Base Memory Storage Capacity:
For base memory storage capacity, there is a positive correlation and linear relationship, with a
coefficient of 0.0353, between our independent variable, CNET rating, and the phone’s amount
of memory, on average, holding all other dependent variables equal.
Some phones give high amounts of memory to incentivize pro users, while others include little to
be a budget phone and focus on other specs.
9. Battery Size:
For battery size, there is a positive correlation and linear relationship, with a coefficient of
0.0006, between our independent variable, CNET rating, and the phone’s battery size, on
average, holding all other dependent variables equal.

Some companies focus on including big batteries in order to increase use time or allow the user
to do more complex functions on the phone. Other phones may sacrifice battery size in order to
increase other hardware or decrease thickness, weight, and price.
10. Thickness:
For thickness, there is a negative correlation and linear relationship, with a coefficient of -
.02469, between our independent variable, CNET rating, and the phone’s thickness, on average,
holding all other dependent variables equal.
There are outliers that are thick in order to include more advanced processors and memory or to
have a more durable build quality. Some phones are made intentionally as thin as possible to be
attractive to the consumers and increase marketing power.
11. Weight:
For weight, there is a positive correlation and linear relationship, with a coefficient of 0.0107,
between our independent variable, CNET rating, and the phone’s weight, on average, holding all
other dependent variables equal.
Phones that are very heavy include more hardware, have a better build quality, or have a bigger
screen. Phones that are light can be due to having a small amount of hardware or attempting to
be attractive to buyers.
12. iOS:
iOS, was our base case so we cannot determine its’ relationship with our dependent variable,
CNET rating, but only how it compares with the other indicator variables.
13. Android:
For operating system, there is a negative correlation and linear relationship, with a coefficient of
-1.355, between our dependent variable, CNET rating, and Android as the OS of the phone, on

We believe that this is because many companies that use Android change it to add new features
that cause bugs and a slow, confusing user experience.
14. Windows Phone:
-1.027, between our dependent variable, CNET rating, and Windows as the OS of the phone, on
We believe that this is because Windows is a less fast and easy OS to use than iOS.
15. BlackBerry OS:
-1.317, between our dependent variable, CNET rating, and BlackBerry OS as the OS of the
phone, on average, holding all other independent variables equal.
We believe that this is because BlackBerry has a much slower user experience than iOS that is
much more complex and confusing.
4. Modelling
A. Modelling Technique

We decided that the most efficient way to find the best fit model was to use backwards
elimination, with respect to standard error. We took our initial model and deleted the variable
with the highest p-value, re-ran the model, and checked if it had a lower standard error. We
continued to remove the variable with the highest p-value until we reached a new model that had
a higher standard error than the previous model, and then used the previous model as our best fit.
We created a multiple linear regression model finding the relationship between our independent
variable, which was the CNET rating score, and our dependent variables, which in the best fit
model follow below. To make the coefficients relevant between all variables, we multiplied each
variable's coefficient by its respective range:
CNET Ratings = 5.513
+ 0.963 (Price)
+ 1.076 (Camera)
+ 0.667 (Screen Size)
+ 0.959 (PPI)
– 0.768 (Speed)
– 1.355 (Android)
– 1.027 (Windows)
– 1.317 (BlackBerry)
Metrics for the model:
● R^2: 0.60
● Standard Error: 0.67
● Standard Error / Y Average: 0.09
● F-test: 4.75x10^15

● Independent Variable p-values:
○ Baseline price: 0.016
○ Main camera megapixels: 0.019
○ Screen size: 0.109
○ Screen resolution: 0.021
○ Processor speed: 0.146
○ Android: 0.001
○ Windows Phone: 0.023
○ BlackBerry OS: 0.015
This model is very statistically significant, because the standard error/Y average is 0.09, which is
far below our benchmark of 0.2. The independent variables do not perfectly describe all
variations in the dependent variable, CNET rating, because the R^2 is 0.60 which is good but not
great. The model passes the F-test with a value of 4.75x10^15. Unfortunately, 2 independent
variables, screen size and processor speed, are not statistically significant with p-values of 0.109
and 0.146 respectively. The other 6 independent variables are all statistically significant with p-
values below 0.05.
B. Other Potential Models and Data
We could also have used 3 other models that we discussed in class, but did not find them helpful
in improving our model or did not have the relevant data to make them useful. The first is a
second order model that raises an independent variable to the second power to account for non-
linear relationships between independent and dependent variables. We looked at the trendlines of
our residual scatter plots for each independent variable and believe that all have a linear and non-
quadratic relationship, so we believe this model would not have expanded our analysis of this
problem.
We could have used autocorrelation, such as time series AR(1) model, to check if there was
correlation between the errors of phones that were released at consecutive time periods. We did
not know how to deal with time-series related data when we manually recorded our data, so we
did not record the release dates of phones. Autocorrelation would help the most when trying to
determine the affects time has on the expectations for the hardware specifications should have
every year, but we think that our observations were reviewed in a close enough time period that
this did not effect our data significantly.
As discussed earlier, we would have loved to have and include data on mass social responses to
phones to observe how it related to or led to CNET ratings. It would have also been interesting to
see how tech blog ratings were related to phone sales. If we had used phone sales data, a Cobb-
Douglass model would have been appropriate to use for our analyzation, but we did not have
access to or use sales data. Other correlations between CNET ratings and phone differences were

mostly hardware and software quality, which we attempted to represent through phone operating
system, but were not able to include phone manufacturer.
C. Suspect Points
Using SPSS, we got the following output for the residuals of our model.

Outliers:
As you can see in the above outputs, there are 4 observations, at 24, 68, 71, and 93, that have
standardized residuals below -2 or above 2, with values of -2.5, 2.2, -3.7, and -2.2 respectively.
There are 5 observations, at 24, 68, 71, 93, and 94, that have studentized residuals below -2 or
above 2, with values of -2.7, 2.2, -3.7, -2.3, and 2.0 respectively. Observations that have
residuals below -2 or above 2 are considered outliers.
We believe that Observation 24, the Yota YotaPhone 2, is an outlier because it has an incredibly
high price and high processor speed for such a low rating, 6.0; we think this is due to poor
software integration. Observation 68, the HTC Desire 610, is an outlier because it has a very
high rating, 7.6, compared it’s extremely low component specifications in all areas; this is
because the phone is specifically made to be a budget phone, and simply delivers the best overall
package of good enough hardware accompanied with great software for a competitive price that
causes it to have favorable ratings. Observation 71, the Ascend Y550, is an outlier because it has
an incredibly low rating, at 3.7 which happened to be the lowest rating of all our observations,
even though it has decent hardware specifications; we believe this low rating is due to poor
integration of Android software. Observation 93, the Kyocera Hydro Icon, is an outlier because it
has a very low rating, 5.0, in relation to it’s medium level hardware specifications; again, we
think this is due to poor implementation of the Android software which hinders user experience
no matter how good hardware components are. Observation 94, the Nokia Lumia 635, is an
outlier, because it has a relatively high rating, 7.5, even though it has low hardware component

specs in screen resolution and processor speed; we think this is due to its’ cheap price, strong
build quality, and good implementation of the Windows Phone software.
Luckily, it is normal to have up to 5 observations that are outliers in a model that is to trying to
predict linear relationships with 95% accuracy out of a total of 100 observations. Therefore, we
did not do anything about the outliers and do not believe that they have affected our data
significantly. However, we will check each of these observations respective Cook’s D to see if
any of them asserted influence in the model, which would cause us to want to remove them.
Below is an included histogram plotting the frequency of standardized residuals within certain
buckets.
Leverage:
After comparing the data outputted by SPSS for leverage, we compared every observations
leverage with the model’s critical leverage value, (3*(k+1))/n, which was 0.27. We found 5
observations that have a leverage above 0.27. They are observations 9, 31, 39, 40, and 47 with
leverage values of 0.27, 0.37, 0.34, 0.27, and 0.38 respectively.
We believe that observation 9, the Apple iPhone 6s, has leverage because it has a relatively high
score, 8.9, while relatively very low values for important components, such as camera
megapixel, screen size, screen resolution, and processor speed; we think that it has a high score
due to its’ operating system, iOS. Observation 40, the Apple iPhone 6 plus, represents a similar
relationship between high score and low hardware specifications as stated above.
We believe that observation 31, the BlackBerry Classic, has leverage because it has a relatively
high score, 7.3, in comparison to its very poor hardware specifications, in camera megapixels,
screen size, screen resolution, and processor speed, for a phone of that score; BlackBerry has
made up for this through its operating system and hardware tailored towards its target audience’s
needs. Observation 47, the BlackBerry Leap, has leverage due to a similar problem as described

above; it has a relative high score, 6.6, in comparison to the specifications of its’ internal
components. Observation 39, the BlackBerry Passport, has leverage in the opposite way because
it has a very low score in comparison to its relatively high component specifications, in screen
resolution and processor speed and price; this happened because BlackBerry attempted to make a
better phone by adopting other companies’ product best practices instead of focusing on the
features their core users demanded.
Leverage alone is not a reason to remove an observation but we will check these observations’
respective Cook’s D to see if any of them asserted influence in the model, which would cause us
to want to remove them.
Cook’s Distance:
As you can see in the data above, there are no observations that have a Cook’s D value of above
0.5. All outliers and leverage points previously mentioned have a Cook’s D of less than 0.15, so
we do not believe that any observation points have affected our model greatly and will not
remove any points.
D. Collinearity
Below is our printed out correlation and variance inflationary factor (VIF) matrix tables for all
independent variables included in the final model.
As you can see, there is only one relationship between our independent variables that show signs
of multicollinearity. Variables may be collinear if they have correlations above 0.7 or below -0.7.
The correlation between Android and Windows Phone is -0.75, which is, as stated above, is
indicative of multicollinearity. This is completely normal, because there should be some
correlation between these two variables as they are indicator variables. Additionally, we decided
to ignore this present multicollinearity because it did not seem to have an effect on our model as
their respective p-values, t-stats, and coefficient signs all look correct.

Variance Inflationary Factors (VIF) are an even better way of identifying multicollinearity.
Relationships between variables will display signs of multicollinearity if their VIF is 4 or greater.
We got this VIF table by using the inverse Excel function on the correlation table. As in the
provided table above, the relationship between Android and Windows Phone have a VIF above
4. However, as stated above, we ignored this issue, because we did not see any massive impacts
that this multicollinearity had on our model.
E. Model Development Roadmap
To reach our final model, we used the backwards elimination model that we previously
described. In the appendix part B, there are printouts of the 8 MLR’s that we ran in order to get
to the final model. In the first model, we included all variables. By doing backwards elimination,
we removed weight, RAM, battery size, memory, thickness, and secondary camera megapixels
to get to the best fit model. When we removed processor speed, the standard error went up so we
put it back into the model, and that is our final model that is displayed above.
We then looked at all of the descriptive statistics and scatter plots for all our independent
variables and determined that we wanted to include all of them in order to get the most holistic
and representative view of the entire smartphone market.
We next looked at all observations’ respective residuals, identified all outliers, leverage points,
and influence points, and considered what to do with them. In the end, we thought all of them
were normal so we didn’t remove any individual observations.
We finally looked at the correlation between all independent variables to see if there was any
cases of multicollinearity. As discussed, we identified the multicollinear relationship between
Android and iOS, but concluded that it did not affect our model.
Summary
Through our research, we can conclude that hardware specs and software both have a very
significant impact on CNET ratings.
Many hardware components don’t have a relationship with CNET rating, but the ones that do
have a significant relationship with rating are parts of the core user experience of the phone.
Camera and screen resolution both have statistically significant and highly influential
relationship with CNET rating, because the camera and the screen of the phone are the two most
used pieces of hardware on a phone by its’ users. Reviewers highly value being able to take
quality pictures and having a very vibrant and clear display.

Interestingly, price had a strong positive relationship with CNET rating, which we believe is
explained by the fact that price represents overall value of a phone. If it is a higher priced phone,
than users will get a lot of quality out of that phone in hardware and software. It’s important to
price phones in a reasonable manner, but executives should focus on adding better camera and
screen components and spend time developing their operating system than pricing their phones
very competitively low.
Because processor speed and RAM both don’t have a statistically significant relationship with
CNET rating, it is critical to ensure users have a fast and fluid user experience through an
efficient and bug-free operating system.
If a phone was using Android, on average, the phone got a 1.355 lower review score than an iOS
phone due to a lower quality user experience for mass consumers and poor implementation by
phone manufacturers, which is usually caused by adding skins that then have bugs or slow the
phone down. To fix this, manufacturers should focus on creating as much of a bug free Android
experience for their phones by either using pure Android or only adding a few extra core features
that your target market clearly expresses a demanded need. If adding extra features, take the time
to ensure that they are bug-free and programmed well in an efficient manner in terms of speed.
If a phone was using BlackBerry, on average, the phone got a 1.317 lower review score than an
iOS phone due to a confusing, outdated, and buggy user experience. Only BlackBerry produces
Android phones, so they should focus on updating the operating system frequently by working
on improving overall OS efficiency every update and a few core apps every update, prioritizing
the most used first.
If a phone was using Windows Phone, on average, the phone got a 1.027 lower review score than
an iOS phone due to not having as many features as iOS that are user friendly, easy to
understand, and execute quickly. However, we think that Windows has the lowest negative
coefficient, because Microsoft has focused on improving efficiency of the OS with every update,
has a consistent user interface, and prioritizes most important core apps and features.
Additionally, Microsoft manufactures the majority of phones using Windows Phone so they are
able to integrate fast software with fast internal components.
In the end, we recommend that phone manufacturer executives should pay top dollar for the best
camera and screen displays on their phones that are present in the market and that they highly
prioritize making a phone’s operating system fast, fluid, and easy to understand over adding a
multitude of additional features.
Works Cited

Rubino, Daniel, Kevin Michaluk, Phil Nickinson, and Rene Ritchie. "Gigahertz,
Megapixels, and Millimeters - Do Specs Matter at All? - Talk Mobile." Android Central.
Mobile Nations, 15 July 2013. Web. 1 Nov. 2015.
Mies, Ginny. "The Phone Specs That Matter." TechHive. PC World, 7 Mar. 2014. Web.
1 Nov. 2015.
Appendix
A. Independent Variables Scatter Plots
1. Baseline Price:
2. Main Camera Megapixels:

3. Secondary Camera Megapixels:
4. Screen Size: