Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
R Programming
Guy Lebanon
September 22, 2015
Goals
I Understand when to use R and when not to use it
I Understand basic syntax and be able to write short programs
I Un...
R, Matlab, and Python
R is similar to Matlab and Python:
I They run inside an interactive shell or graphical user interfac...
R, Matlab, and Python
The three languages di↵er:
I R and Python are open-source and free. Matlab is not.
I It is easier to...
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
100
1000
10000
2002 2004 2006 2008 2010 2012 2014
Near exponential growth in con...
Running R
I Interactively:
I Type R in prompt (type q() to quit)
I R graphic application
I R-Studio
I Within Emacs
I Non-I...
R Language
I drops whitespace, semi-colons optional but are needed for
multiple commands in the same line
I comments: #
I ...
ls() # list variable names in workspace memory
# save all variables to a file
save.image(file = "R_workspace")
# save spec...
Scalars
Major scalar types: numeric, integer, logical, string, dates, and
factors (NA: not available)
a = 3.2; b = 3 # dou...
Factors can be ordered or unordered
# ordered factor
current.season = factor("summer",
levels = c("summer", "fall", "winte...
Vectors and Arrays
x = c(4, 3, 3, 4, 3, 1) # c for concatenate
length(x) # return length
2*x+1 # element-wise arithmetic
#...
q = rep(3.2, times = 10) # repeat value multiple times
w = seq(0, 1, by = 0.1) # values in [0,1] in 0.1 increments
w = seq...
Arrays are multidimensional generalization of vectors.
z = seq(1, 20,length.out = 20) # create a vector 1,2,..,20
x = arra...
Lists
Lists are ordered collections of possibly di↵erent types. Named
positions allow creating self-describing data.
L=lis...
Dataframes
Dataframe are ordered sequence of lists sharing the same
signature. A popular usecase is a table where rows cor...
Datasets
Example: Iris dataset (in datasets package)
names(iris) # lists the dimension (column) names
head(iris, 4) # show...
If-Else
a = 10; b = 5; c = 1
if (a < b) {
d = 1
} else if (a == b) {
d = 2
} else {
d = 3
}
print(d)
## [1] 3
AND: &&, OR:...
Loops
For, repeat, and while loops:
sm=0
# repeat for 100 iteration, with num taking values 1:100
for (num in seq(1, 100, ...
Functions
By default, arguments flow into the parameters according to their
order at the call site. Providing parameter nam...
# myPower(.,.) raises the first argument to the power of the
# second. The first argument is named bas and has default val...
Vectorized Code
Vectorized code runs much faster than loops due to R interpreter
overhead.
a = 1:10000000; res = 0
system....
External/Native API
Often, 10% percent of the code is responsible for 90% of
computing time. Implementing bottlenecks in C...
## [1] 13.34 17.48 21.21 24.71 28.03 31.24 34.34 37.37 40.33 43
## [1] 13.34 17.48 21.21 24.71 28.03 31.24 34.34 37.37 40....
Graphing Data with R
Guy Lebanon
September 22, 2015
Goals
I Learn how to use base graphics
I Learn how to use base ggplot2
I Understand basic graph types and when to use them...
Base Graphics
Base graphics syntax: plot function followed by helper functions
for annotating the graph.
plot(x = datafram...
GGPLOT2
Philosophy: (a) Grammar of graphics, (b) logical separation of
graphics and data, (c) concise and maintainable cod...
Option 2: Use the ggplot function. Pass dataframe, column
names through aes function. Compose function output with
additio...
Datasets
We will use the three datasets below.
I faithful: eruption time and waiting time to next eruption
(both in minute...
names(faithful)
## [1] "eruptions" "waiting"
names(mtcars)
## [1] "mpg" "cyl" "disp" "hp" "drat" "wt" "qsec" "vs"
## [11] ...
Strip Plot
Strip plots graph one-dimensional numeric data as points in a
two-dimensional space, with one coordinate corres...
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●...
I We conclude from the figure above that Old Faithful has two
typical eruption times — a long eruption time around 4.5
minu...
Histograms
Histograms graph one-dimensional numeric data by dividing the
range into bins and counting number of occurrence...
0
10
20
30
40
40 60 80 100
waiting
count
Waiting time to next eruption (min)
There are clearly two typical eruption times – one around 2
minutes and one around 4.5 minutes.
y values can be replaced w...
0
10
20
30
40
40 60 80 100
waiting
count
Waiting time to next eruption (min)
0
5
10
15
40 60 80
waiting
count
Waiting time to next eruption (min)
0
20
40
60
80
50 75 100
waiting
count
Waiting time to next eruption (min)
Line Plot
Line plot: a graph displaying a relation between x and y as a line
in a Cartesian coordinate system. The relatio...
S = sort.int(mpg$cty, index.return = T)
# x: city mpg
# ix: indices of sorted values of city mpg
plot(S$x, # plot sorted c...
0 50 100 150 200
101520253035
mpg
highway mpg
city mpg
Smoothed Histograms
Denoting n values by x(1), . . . , x(n), the smoothed histogram is the
following function fh : R ! R+
...
Four popular kernel choices are the tricube, triangular, uniform,
and Gaussian kernels, defined as Kh(r) = h 1K(r/h) where ...
h=1 h=2
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
gaussiantriang...
−2 0 2 4 6 8
0.000.100.200.30
x
f_h(x)
Smoothed histogram (h=1/6)
−2 0 2 4 6 8
0.000.050.100.150.200.25
x
f_h(x)
Smoothed histogram (h=1/3)
−2 0 2 4 6 8
0.050.100.15
x
f_h(x)
Smoothed histogram (h=1)
In ggplot2:
ggplot(faithful, aes(x = waiting, y = ..density..)) +
geom_histogram(alpha = 0.3) +
geom_density(size = 1.5, c...
0.00
0.02
0.04
40 60 80 100
waiting
density
Scatter Plot
A scatter plot graphs the relationships between two numeric
variables. It graphs each pair of variables as a ...
50 60 70 80 90
1.52.53.54.5
waiting times (min)
eruptiontime(min)
I We conclude from the two clusters in the scatter plot above
that there are two distinct cases: short eruptions and long
...
The relationship between two numeric variables and a categorical
variable can be graphed using a scatter plot where the ca...
●●
●
●
●
●
●
●
●
●
●
●
●
50 100 150 200 250 300
1015202530
mpg vs. hp by transmission
horsepower
milespergallon
●
automati...
We draw several conclusions from this graph.
I There is an inverse relationship between horsepower and mpg.
I For a given ...
Changing marker size in a scatter plot
qplot(x = wt,
y = mpg,
data = mtcars,
size = cyl,
main = "MPG vs. weight (x1000 lbs...
● ●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
10
15
20
25
30
35
2 3 4 5
wt
mpg
cyl
●
●
●
●
●
4
5
6
7
8
MPG ...
I When data is noisy, it is useful to add a smoothed line curve
to visualize median trends
I One technique to address this...
qplot(disp,
mpg,
data = mtcars,
main = "MPG vs Eng. Displacement") +
stat_smooth(method = "loess",
degree = 0,
span = 0.2,...
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
10
15
20
25
30
35
100 200 300 400
disp
mpg
MPG vs Eng. Displac...
Facets
I Facets are a way to display multiple graphs next to each other
in the same scale with shared axes.
I This is an e...
automatic manual
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
10
15
20
25
30
35
2 3 4 5 2 3 4 5
wt
mpg
MPG...
● ●
●
●
●●
●
● ●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
10
15
20
25
30
35
10
15
20
25
30
35
flatV−shape
2 3 4 5
wt
mp...
automatic manual
●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
10
15
20
25
30
35
10
15
20
25
30
35
flatV−sha...
I Manual transmission cars tend to have lower weights and be
more fuel e cient
I Cars with V-shape engines tend to weigh l...
ctyhwydispl
cty hwy displ
10
15
20
25
30
35
Corr:
0.956
Corr:
−0.799
20
30
40
● ●
●●
●●●●●
●●
●●●●●●
●
●
●
●
●●
●
●
●●●
●
...
Contour Plots
Contour plots graph relationship between three numeric variables:
z as a fuction of x, y. Steps: (a) create ...
−1.0
−0.5
0.0
0.5
1.0
−1.0 −0.5 0.0 0.5 1.0
x
y
Quantiles and Box-Plots
Box plots are an alternative to histograms that are usually more
“lossy” but emphasize quantiles a...
The box plot is composed of;
I box denoting the IQR,
I an inner line bisecting the box denoting the median,
I whiskers ext...
●●●
20 30 40
hwy
ggplot(mpg, aes(reorder(class, -hwy, median), hwy)) +
geom_boxplot() +
coord_flip() +
scale_x_discrete("class")
● ●● ●
●●
●
●● ●● ●●●●
●●● ●
compact
midsize
subcompact
2seater
minivan
suv
pickup
20 30 40
hwy
class
I The graph suggests the following fuel e ciency order among
vehicle classes: pickups, SUV, minivans, 2-seaters,
sub-compa...
QQ-Plots
I Quantile-quantile plots are useful for comparing two datasets,
one of which may be sampled from a certain distr...
I A straight line with slope 1 that passes through the origin
implies that the two datasets have identical quantiles, and
...
D = data.frame(samples = c(rnorm(200, 1, 1),
rnorm(200, 0, 1),
rnorm(200, 0, 2)))
D$parameter[1:200] = 'N(1,1)';
D$paramet...
0
10
20
30
40
50
0
10
20
30
40
50
0
10
20
30
40
50
N(0,1)N(0,2)N(1,1)
−4 0 4 8
samples
count
D = data.frame(samples = c(rnorm(200, 1, 1),
rnorm(200, 0, 1),
rnorm(200, 0, 2)));
D$parameter[1:200] = 'N(1,1)';
D$parame...
N(0,1) N(0,2) N(1,1)
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●...
x_grid = seq(-6, 6, length.out = 200)
R = data.frame(density = dnorm(x_grid, 0, 1))
R$tdensity = dt(x_grid, 1.5)
R$x = x_g...
0.0
0.1
0.2
0.3
0.4
−6 −3 0 3 6
x
density
N(0,1) (shaded) and t−distribution (1.5 dof)
x_grid = seq(-6, 6, length.out = 200)
R = data.frame(density = dnorm(x_grid, 0, 1))
R$samples = rnorm(200, 0, 1)
pm = list...
● ●
● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●...
Preprocessing Data
Guy Lebanon
September 30, 2015
Goals
I Learn how to handle missing data
I Learn how to handle outliers
I Learn when and how to transform data
I Learn sta...
Missing Data
Data may be missing for a variety of reasons.
I corrupted during its transfer or storage
I some instances in ...
Examples of Missing Data
I Recommendation systems recommend to users items from a
catalog based on historical user rating....
Missing Completely at Random
I If a variable (dataframe column) is as likely to be missing as
all other variables, we say ...
Missing at Random (MAR)
I MAR occurs when the probability that a variable is missing
depends only on the other information...
Handling Missing Data
Most methods are designed to work with fully observed data.
Below are some general ways to convert m...
Missing Data and R
I R represents missing data using the NA symbol.
I The function is.na returns a data structure having T...
The code below analyzes the dataframe movies in the ggplot2
package, which contains 24 attributes (genre, year, budget, us...
moviesNoNA = na.omit(movies)
qplot(rating, budget, data = moviesNoNA, size = I(1.2)) +
stat_smooth(color = "red", size = I...
moviesNoNA = na.omit(movies)
qplot(rating, votes, data = moviesNoNA, size = I(1.2))
●
●
●●
●
●● ●●
● ●
●
●
●
●
●
●
● ●
●
●...
I Number of votes (which can be used as a surrogate for
popularity) tend to increase as the average rating increase.
I Spr...
Outliers
Two types of outliers
I Corrupted values, for example, human errors during a manual
process of entering measureme...
library(Ecdat)
data(SP500, package = 'Ecdat')
qplot(r500,
main = "Histogram of log(P(t)/P(t-1)) for SP500 (1981-91)",
xlab...
qplot(seq(along = r500),
r500,
data = SP500,
geom = "line",
xlab = "trading days since January 1981",
ylab = "log returns"...
Robustness
Robustness describes a lack of sensitivity of data analysis
procedures to outliers.
I The mean of n numbers is ...
Dealing with Outliers
Truncating. Remove all values deemed as outliers.
Winsorization. Replace outliers with the most extr...
Removing Outliers
To remove outliers we need to first detect them.
I Values below the ↵ percentile or above the 100 ↵ perce...
originalData = rnorm(20)
originalData[1] = 1000
sortedData = sort(originalData)
originalData = originalData[3:18]
lowerLim...
library(robustHD)
originalData = c(1000, rnorm(10))
print(originalData[1:5])
## [1] 1000.0000 -0.6265 0.1836 -0.8356 1.595...
Data Transformations: Skewness and Power
Transformation
I In many cases, data is drawn from a highly-skewed
distribution t...
I Intuitively, the power transform maps x to x , up to
multiplication by a constant and addition of a constant.
I This map...
print(diamonds[1:10,1:8])
## carat cut color clarity depth table price x
## 1 0.23 Ideal E SI2 61.5 55 326 3.95
## 2 0.21 ...
diamondsSubset = diamonds[sample(dim(diamonds)[1], 1000),]
qplot(price, data = diamondsSubset)
0
50
100
150
200
0 5000 100...
qplot(log(price), size = I(1), data = diamondsSubset)
0
20
40
6 7 8 9 10
log(price)
count
I Power transformations are useful also for examining the
relationship between two or more data variables.
I The following...
qplot(log(price), size = I(1), data = diamondsSubset)
0
20
40
6 7 8 9 10
log(price)
count
qplot(carat,
price,
size = I(1),
data = diamondsSubset)
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
...
qplot(carat,
log(price),
size = I(1),
data = diamondsSubset)
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●...
qplot(log(carat),
price,
size = I(1),
data = diamondsSubset)
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●...
qplot(log(carat),
log(price),
size = I(1),
data = diamondsSubset)
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
...
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Data Analysis with R (combined slides)
Upcoming SlideShare
Loading in …5
×

Data Analysis with R (combined slides)

1,669 views

Published on

Data Analysis with R: R programming, graphing data with R, preprocessing data with R.

Published in: Technology

Data Analysis with R (combined slides)

  1. 1. R Programming Guy Lebanon September 22, 2015
  2. 2. Goals I Understand when to use R and when not to use it I Understand basic syntax and be able to write short programs I Understand scalability issues in R and di↵erent ways to resolve them I Prepare for the next module: visualizing data with R Module will be separated to 4 parts: (a) getting started, (b) data types, (c) control flow and functions, and (c) scalability and interfaces.
  3. 3. R, Matlab, and Python R is similar to Matlab and Python: I They run inside an interactive shell or graphical user interface I They emphasize storing and manipulating data as multidimensional arrays. I They include many general purpose and specialized packages (linear algebra, statistics, ML, etc.) I They are typically slower than C, C++, and Fortran (though vectorization can help) I They can interface with native C++ code for speeding up bottlenecks
  4. 4. R, Matlab, and Python The three languages di↵er: I R and Python are open-source and free. Matlab is not. I It is easier to contribute packages to R I R has a large group of motivated contributors who contribute high quality packages I R syntax is more suitable for statistics and data I R has better graphics capabilities I R is popular in statistics, biostatistics, and social sciences. Matlab is popular in engineering and applied math. Python is popular in web development and scripting.
  5. 5. ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 100 1000 10000 2002 2004 2006 2008 2010 2012 2014 Near exponential growth in contributed packages
  6. 6. Running R I Interactively: I Type R in prompt (type q() to quit) I R graphic application I R-Studio I Within Emacs I Non-Interactively: I call script from R: source("foo.R") I call script from shell: R CMD BATCH foo.R I call script from shell: Rscript foo.R I executable script, prefixed by #!/usr/bin/Rscript, followed by ./foo.R < inFile > outFile
  7. 7. R Language I drops whitespace, semi-colons optional but are needed for multiple commands in the same line I comments: # I case sensitive I functional and object oriented programming (a=b rephrased as ’=’(a,b)) I interpreted but with lazy evaluation I not strongly typed I help() displays help on a function, dataset, etc. a = 3.2 a = "a string"; b = 2 # no strong typing print(a) ## [1] "a string"
  8. 8. ls() # list variable names in workspace memory # save all variables to a file save.image(file = "R_workspace") # save specified variables save(new.var, legal.var.name, file = "R_workspace") # load variables saved in file load("R_workspace") help("load") install.packages("ggplot2") library(ggplot2) system("ls -al")
  9. 9. Scalars Major scalar types: numeric, integer, logical, string, dates, and factors (NA: not available) a = 3.2; b = 3 # double types c = as.integer(b) # cast to integer type d = TRUE e = as.numeric(d) # casting to numeric f = "this is a string" # string ls.str() # show variables and their types ## a: num 3.2 ## b: num 3 ## c: int 3 ## d: logi TRUE ## e: num 1 ## f: chr "this is a string"
  10. 10. Factors can be ordered or unordered # ordered factor current.season = factor("summer", levels = c("summer", "fall", "winter", "spring"), ordered = TRUE) # unordered factor my.eye.color = factor("brown", levels = c("brown", "blue", "green"), ordered = FALSE)
  11. 11. Vectors and Arrays x = c(4, 3, 3, 4, 3, 1) # c for concatenate length(x) # return length 2*x+1 # element-wise arithmetic # Boolean vector (default is FALSE) y = vector(mode = "logical", length = 4) # numeric vector (default is 0) z = vector(length = 3, mode = "numeric")
  12. 12. q = rep(3.2, times = 10) # repeat value multiple times w = seq(0, 1, by = 0.1) # values in [0,1] in 0.1 increments w = seq(0, 1, length.out = 11) # equally spaced values w <= 0.5 # boolean vector any(w <= 0.5) # is it true for some elements? all(w <= 0.5) # is it true for all elements? which(w <= 0.5) # for which elements is it true? w[w <= 0.5] # extracting from w entries for which w<=0.5 subset(w, w <= 0.5) # an alternative with the subset function w[w <= 0.5] = 0 # zero out all components <= 0.5
  13. 13. Arrays are multidimensional generalization of vectors. z = seq(1, 20,length.out = 20) # create a vector 1,2,..,20 x = array(data = z, dim = c(4, 5)) # create a 2-d array x[2,3] # refer to the second row and third column x[2,] # refer to the entire second row x[-1,] # all but the first row - same as x[c(2,3,4),] y = x[c(1,2),c(1,2)] # 2x2 top left sub-matrix 2 * y + 1 # element-wise operation y %*% y # matrix product (both arguments are matrices) x[1,] %*% x[1,] # inner product t(x) # matrix transpose outer(x[,1], x[,1]) # outer product rbind(x[1,], x[1,]) # vertical concatenation cbind(x[1,], x[1,]) # horizontal concatenation
  14. 14. Lists Lists are ordered collections of possibly di↵erent types. Named positions allow creating self-describing data. L=list(name = 'John', age = 55, no.children = 2, children.ages = c(15, 18)) names(L) # displays all position names L[[2]] # second element L[2] # list containing second element L$name # value in list corresponding to name L['name'] # same thing L$children.ages[2] # same as L[[4]][2]
  15. 15. Dataframes Dataframe are ordered sequence of lists sharing the same signature. A popular usecase is a table where rows correspond to data examples and columns correspond to dimensions or features. vecn = c("John Smith","Jane Doe") veca = c(42, 45) vecs = c(50000, 55000) R = data.frame(name = vecn, age = veca, salary = vecs) R ## name age salary ## 1 John Smith 42 50000 ## 2 Jane Doe 45 55000 names(R) = c("NAME", "AGE", "SALARY") # modify column names R ## NAME AGE SALARY ## 1 John Smith 42 50000 ## 2 Jane Doe 45 55000
  16. 16. Datasets Example: Iris dataset (in datasets package) names(iris) # lists the dimension (column) names head(iris, 4) # show first four rows iris[1,] # first row iris$Sepal.Length[1:10] # sepal length of first ten samples # allow replacing iris£Sepal.Length with shorter Sepal.Length attach(iris, warn.conflicts = FALSE) mean(Sepal.Length) # average of Sepal.Length across all rows colMeans(iris[,1:4]) # means of all four numeric columns subset(iris, Sepal.Length < 5 & Species != "setosa") # count number of rows corresponding to setosa species dim(subset(iris, Species == "setosa"))[1] summary(iris)
  17. 17. If-Else a = 10; b = 5; c = 1 if (a < b) { d = 1 } else if (a == b) { d = 2 } else { d = 3 } print(d) ## [1] 3 AND: &&, OR: ||, equality: ==, inequality: !=
  18. 18. Loops For, repeat, and while loops: sm=0 # repeat for 100 iteration, with num taking values 1:100 for (num in seq(1, 100, by = 1)) { sm = sm + num } repeat { sm = sm - num num = num - 1 if (sm == 0) break # if sm == 0 then stop the loop } a = 1; b = 10 while (b>a) { sm = sm + 1 a = a + 1 b = b - 1 }
  19. 19. Functions By default, arguments flow into the parameters according to their order at the call site. Providing parameter names allow out of order binding. foo(10, 20, 30) # parameter bindings by order foo(y = 20, x = 10, z = 30) # out of order parameter bindings foo(z = 30) # missing parameters assigned default values
  20. 20. # myPower(.,.) raises the first argument to the power of the # second. The first argument is named bas and has default value # The second parameter is named pow and has default value 2. myPower = function(bas = 10, pow = 2) { res = bas^pow # raise base to a power return(res) } myPower(2, 3) # 2 is bound to bas and 3 to pow (in-order) # same binding as above (out-of-order parameter names) myPower(pow = 3, bas = 2) myPower(bas = 3) # default value of pow is used
  21. 21. Vectorized Code Vectorized code runs much faster than loops due to R interpreter overhead. a = 1:10000000; res = 0 system.time(for (e in a) res = res + e^2) ## user system elapsed ## 3.742 0.029 3.800 system.time(sum(a^2)) ## user system elapsed ## 0.180 0.032 0.250
  22. 22. External/Native API Often, 10% percent of the code is responsible for 90% of computing time. Implementing bottlenecks in C/C++ allows staying mostly within the R environment. dyn.load("fooC2.so") # load compiled C code A = seq(0, 1, length = 10) B = seq(0, 1, length = 10) .Call("fooC2", A, B) Newer packages: Rcpp, RcppArmadillo, RcppEigen
  23. 23. ## [1] 13.34 17.48 21.21 24.71 28.03 31.24 34.34 37.37 40.33 43 ## [1] 13.34 17.48 21.21 24.71 28.03 31.24 34.34 37.37 40.33 43 0.0 0.5 1.0 1.5 0 250 500 750 1000 array size computationtime(sec) language C R
  24. 24. Graphing Data with R Guy Lebanon September 22, 2015
  25. 25. Goals I Learn how to use base graphics I Learn how to use base ggplot2 I Understand basic graph types and when to use them Module will be separated to 4 parts: (a) base graphics, (b) ggplot2, (c) datasets, (d) basic graph types and case studies.
  26. 26. Base Graphics Base graphics syntax: plot function followed by helper functions for annotating the graph. plot(x = dataframe$col_1, y = dataframe$col_2) title(main = "figure title") # add title Examples of low-level functions in the graphics package are: I title adds or modifies labels of title and axes, I grid adds a grid to the current figure, I legend displays a legend connecting symbols, colors, and line-types to descriptive strings, and I lines adds a line plot to an existing graph.
  27. 27. GGPLOT2 Philosophy: (a) Grammar of graphics, (b) logical separation of graphics and data, (c) concise and maintainable code. Option 1: Use the qplot function. Pass dataframe column names, dataframe name, geometry, and graphing options. qplot(x = x1, y = x2, data = DF, main = "figure title", geom = "point") Remember to install and load package using install.packages('ggplot2') library(ggplot2)
  28. 28. Option 2: Use the ggplot function. Pass dataframe, column names through aes function. Compose function output with additional layers using + operator. ggplot(dataframe, aes(x = x, y = y)) + geom_line() + geom_point() Function (and addition operator) returns an object that can be printed (using the print function) or saved for later.
  29. 29. Datasets We will use the three datasets below. I faithful: eruption time and waiting time to next eruption (both in minutes) of the Old Faithful geyser in Yellowstone National Park, Wyoming, USA. I mtcars: model name, weight, horsepower, fuel e ciency, and transmission type of cars from 1974 Motor Trend magazine. I mpg: fuel economy and other car attributes from http://fueleconomy.gov (similar to mtcars but larger and newer).
  30. 30. names(faithful) ## [1] "eruptions" "waiting" names(mtcars) ## [1] "mpg" "cyl" "disp" "hp" "drat" "wt" "qsec" "vs" ## [11] "carb" names(mpg) ## [1] "manufacturer" "model" "displ" "year" ## [5] "cyl" "trans" "drv" "cty" ## [9] "hwy" "fl" "class"
  31. 31. Strip Plot Strip plots graph one-dimensional numeric data as points in a two-dimensional space, with one coordinate corresponding to the index of the data point, and the other coordinate corresponding to its value. plot(faithful$eruptions, xlab = "sample number", ylab = "eruption times (min)", main = "Old Faithful Eruption Times")
  32. 32. ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0 50 100 150 200 250 1.52.53.54.5 Old Faithful Eruption Times sample number eruptiontimes(min)
  33. 33. I We conclude from the figure above that Old Faithful has two typical eruption times — a long eruption time around 4.5 minutes, and a short eruption time around 1.5 minutes. I It also appears that the order in which the dataframe rows are stored is not related to the eruption variable.
  34. 34. Histograms Histograms graph one-dimensional numeric data by dividing the range into bins and counting number of occurrences in each bin. It is critical to set the bin width value correctly. qplot(x = waiting, data = faithful, binwidth = 3, main = "Waiting time to next eruption (min)") ggplot(faithful ,aes(x = waiting)) + geom_histogram(binwidth = 1)
  35. 35. 0 10 20 30 40 40 60 80 100 waiting count Waiting time to next eruption (min)
  36. 36. There are clearly two typical eruption times – one around 2 minutes and one around 4.5 minutes. y values can be replaced with probability/frequency using the following syntax. ggplot(faithful, aes(x = waiting, y = ..density..)) + geom_histogram(binwidth = 4) Selecting the best bandwidth to use when graphing a specific dataset is di cult and usually requires some trial and error.
  37. 37. 0 10 20 30 40 40 60 80 100 waiting count Waiting time to next eruption (min)
  38. 38. 0 5 10 15 40 60 80 waiting count Waiting time to next eruption (min)
  39. 39. 0 20 40 60 80 50 75 100 waiting count Waiting time to next eruption (min)
  40. 40. Line Plot Line plot: a graph displaying a relation between x and y as a line in a Cartesian coordinate system. The relation may correspond to an abstract mathematical function or to a relation between two samples (for example, dataframe columns) x = seq(-2, 2, length.out = 30) y = x^2 qplot(x, y, geom = "line") # line plot qplot(x, y, geom = c("point", "line")) # line and point plot dataframe = data.frame(x = x, y = y) ggplot(dataframe, aes(x = x, y = y)) + geom_line() + geom_point() # same as above but with ggplot
  41. 41. S = sort.int(mpg$cty, index.return = T) # x: city mpg # ix: indices of sorted values of city mpg plot(S$x, # plot sorted city mpg values with a line plot type = "l", lty = 2, xlab = "sample number (sorted by city mpg)", ylab = "mpg") lines(mpg$hwy[S$ix] ,lty = 1) # add dashed line of hwy mpg legend("topleft", c("highway mpg", "city mpg"), lty = c(1, 2))
  42. 42. 0 50 100 150 200 101520253035 mpg highway mpg city mpg
  43. 43. Smoothed Histograms Denoting n values by x(1), . . . , x(n), the smoothed histogram is the following function fh : R ! R+ fh(x) = 1 n nX i=1 Kh(x x(i) ) where the kernel function Kh : R ! R typically achieves its maximum at 0, and decreases as |x x(i)| increases. We also assume that the kernel function integrates to one R Kh(x) dx = 1 and satisfies the relation Kh(r) = h 1 K1(r/h). We refer to K1 as the base form of the kernel and denote it as K.
  44. 44. Four popular kernel choices are the tricube, triangular, uniform, and Gaussian kernels, defined as Kh(r) = h 1K(r/h) where the K(·) functions are respectively K(r) = (1 |r|3 )3 · 1{|r|<1} (Tricube) K(r) = (1 |r|) · 1{|r|<1} (Triangular) K(r) = 2 1 · 1{|r|<1} (Uniform) K(r) = exp( x2 /2)/ p 2⇡ (Gaussian). As h increases the kernel functions Kh become wider.
  45. 45. h=1 h=2 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 gaussiantriangulartricubeuniform −2 0 2 −2 0 2 x K_h(x)
  46. 46. −2 0 2 4 6 8 0.000.100.200.30 x f_h(x) Smoothed histogram (h=1/6)
  47. 47. −2 0 2 4 6 8 0.000.050.100.150.200.25 x f_h(x) Smoothed histogram (h=1/3)
  48. 48. −2 0 2 4 6 8 0.050.100.15 x f_h(x) Smoothed histogram (h=1)
  49. 49. In ggplot2: ggplot(faithful, aes(x = waiting, y = ..density..)) + geom_histogram(alpha = 0.3) + geom_density(size = 1.5, color = "red")
  50. 50. 0.00 0.02 0.04 40 60 80 100 waiting density
  51. 51. Scatter Plot A scatter plot graphs the relationships between two numeric variables. It graphs each pair of variables as a point in a two dimensional space whose coordinates are the corresponding x, y values. plot(faithful$waiting, faithful$eruptions, pch = 17, col = 2, cex = 1.2, xlab = "waiting times (min)", ylab = "eruption time (min)")
  52. 52. 50 60 70 80 90 1.52.53.54.5 waiting times (min) eruptiontime(min)
  53. 53. I We conclude from the two clusters in the scatter plot above that there are two distinct cases: short eruptions and long eruptions. I Furthermore, the waiting times for short eruptions are typically short, while the waiting times for the long eruptions are typically long. I This is consistent with our intuition: it takes longer to build the pressure for a long eruption than it does for a short eruption.
  54. 54. The relationship between two numeric variables and a categorical variable can be graphed using a scatter plot where the categorical variable controls the size, color, or shape of the markers. plot(mtcars$hp, mtcars$mpg, pch = mtcars$am, xlab = "horsepower", cex = 1.2, ylab = "miles per gallon", main = "mpg vs. hp by transmission") legend("topright", c("automatic", "manual"), pch = c(0, 1))
  55. 55. ●● ● ● ● ● ● ● ● ● ● ● ● 50 100 150 200 250 300 1015202530 mpg vs. hp by transmission horsepower milespergallon ● automatic manual
  56. 56. We draw several conclusions from this graph. I There is an inverse relationship between horsepower and mpg. I For a given horsepower amount, manual transmission cars are generally more fuel e cient. I Cars with the highest horsepower tend to be manual (the two highest horsepower cars in the dataset are Maserati Bora and Ford Pantera, both sports cars with manual transmissions).
  57. 57. Changing marker size in a scatter plot qplot(x = wt, y = mpg, data = mtcars, size = cyl, main = "MPG vs. weight (x1000 lbs) by cylinder")
  58. 58. ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● 10 15 20 25 30 35 2 3 4 5 wt mpg cyl ● ● ● ● ● 4 5 6 7 8 MPG vs. weight (x1000 lbs) by cylinder
  59. 59. I When data is noisy, it is useful to add a smoothed line curve to visualize median trends I One technique to address this issue is to add a smoothed line curve yS , which is a weighted average of the original data (y(i), x(i)) i = 1, . . . , n: yS (x) = nX i=1 Kh(x x(i)) Pn i=1 Kh(x x(i)) y(i) . where the Kh functions above are the kernel functions described earlier I yS (x) is an average the y(i) values, weighted in a way that emphasizes y(i) values whose corresponding x(i) values are close to x. I The denominator in the definition of yS ensures that the weights defining the weighted average sum to 1.
  60. 60. qplot(disp, mpg, data = mtcars, main = "MPG vs Eng. Displacement") + stat_smooth(method = "loess", degree = 0, span = 0.2, se = TRUE) span parameter influences the value of h in the slide before and can make the line more or less smooth. Optional argument se adds standard errors as shaded region.
  61. 61. ●● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● 10 15 20 25 30 35 100 200 300 400 disp mpg MPG vs Eng. Displacement
  62. 62. Facets I Facets are a way to display multiple graphs next to each other in the same scale with shared axes. I This is an e↵ective way to visualize data that has higher dimensionality than 2 (mixed numeric-categorical). I The argument facets in qplot or ggplot takes a formula a ⇠ b where a, b specify the variables according to which the rows and columns are organized. qplot(x = wt, y = mpg, facets = .~amf, data = mtcars, main = "MPG vs. weight by transmission")
  63. 63. automatic manual ● ●● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 10 15 20 25 30 35 2 3 4 5 2 3 4 5 wt mpg MPG vs. weight by transmission
  64. 64. ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 10 15 20 25 30 35 10 15 20 25 30 35 flatV−shape 2 3 4 5 wt mpg MPG vs. weight by engine
  65. 65. automatic manual ● ● ●● ● ●● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 10 15 20 25 30 35 10 15 20 25 30 35 flatV−shape 2 3 4 5 2 3 4 5 wt mpg MPG vs. weight by transmission and engine
  66. 66. I Manual transmission cars tend to have lower weights and be more fuel e cient I Cars with V-shape engines tend to weigh less and be more fuel e cient I Manual transmission and V-engine cars tend to be lighter and more fuel e cient. Automatic transmission and non V-engine are heavier and less fuel e cient. “All pairs” plot: DF = mpg[, c("cty", "hwy", "displ")] library(GGally) ggpairs(DF)
  67. 67. ctyhwydispl cty hwy displ 10 15 20 25 30 35 Corr: 0.956 Corr: −0.799 20 30 40 ● ● ●● ●●●●● ●● ●●●●●● ● ● ● ● ●● ● ● ●●● ● ●● ● ● ● ● ● ● ●● ●● ●● ● ●● ●● ●●●● ●● ● ● ● ●● ● ●● ● ●● ● ●●● ● ●●● ● ●●●● ● ● ●● ●●●●●● ● ● ●●● ● ●●●● ● ●●● ● ● ● ●● ● ●● ●● ●● ● ● ●●● ●●● ● ●● ● ● ●● ●● ●● ● ●● ●● ●● ● ● ● ●● ●●●●● ●● ● ● ●●●● ●●● ● ●● ● ●●●●● ● ● ● ●●● ● ● ● ● ● ●● ●● ●● ● ●● ●●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ●●●● ●● ● ● ● ● ●●●●●● ●●● Corr: −0.766 2 3 4 5 6 7 10 15 20 25 30 35 ● ●●● ●● ● ●● ●● ●● ●● ● ● ● ●● ● ● ● ●● ●● ● ●● ● ● ● ● ● ●● ● ● ●●●●● ●●●● ●●●● ●●● ●● ● ●● ● ● ●● ●● ●●●● ●● ●● ● ●● ●●●● ● ● ●● ●●● ●● ●●●● ●●●● ● ●●●●●●●●● ●● ●●●● ● ●●●● ●●● ● ● ● ●● ● ● ● ●●●● ●●● ●● ● ● ●● ●● ●● ●● ●●● ● ● ● ●●● ● ●●●●●● ●● ●●●●●●●● ●● ● ● ●●●● ●● ● ●●●● ●● ● ●●●●● ● ● ●●● ●● ●● ●●●● ● ●●● ●● ●● ●● ●●●● ●● ●●●● ●● ● 20 30 40 ●●●● ●● ● ●●●● ●● ●● ● ● ● ●● ● ● ● ●● ●● ● ●● ● ● ●● ● ●● ● ● ●●●●● ●●●● ●●●● ●●● ●● ● ●● ● ● ●● ●● ●●●● ●● ●● ● ●● ●●●● ● ● ●● ●●● ●● ●●●● ●●●● ● ●●●●●●●●● ●●●●●● ● ●●●● ●●● ● ● ● ●● ● ● ● ●●●● ●●● ●● ● ● ●● ●● ●● ●● ●●● ● ● ● ●●● ● ●●●●●● ●● ●●●●●●●● ●● ● ● ●● ●● ●● ● ●●●● ●● ● ●●●●● ● ● ●●● ●● ●● ●●●● ● ●●●●● ●● ●● ●●●● ●● ●●●● ●● ● 2 3 4 5 6 7
  68. 68. Contour Plots Contour plots graph relationship between three numeric variables: z as a fuction of x, y. Steps: (a) create a grid for x values, (b) create a grid for y values, (c) create an expanded x ⇥ y grid, (d) compute values of z on the expanded grid, (e) graph data. x_grid = seq(-1, 1, length.out = 100) y_grid = x_grid R = expand.grid(x_grid, y_grid) names(R) = c('x', 'y') R$z = R$x^2 + R$y^2 ggplot(R, aes(x = x,y = y, z = z)) + stat_contour()
  69. 69. −1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0 x y
  70. 70. Quantiles and Box-Plots Box plots are an alternative to histograms that are usually more “lossy” but emphasize quantiles and outliers in a way that a histogram cannot. I The r-percentile of a numeric dataset is the point at which approximately r percent of the data lie underneath, and approximately 100 r percent lie above. I Another name for the r percentile is the 0.r quantile. I The median or 50-percentile is the point at which half of the data lies underneath and half above. I The 25-percentile and 75 percentile are the values below which 25% and 75% of the data lie. These points are also called the first and third quartiles (the second quartile is the median). I The interval between the first and third quartiles is called the inter-quartile range (IQR) (region covering the central 50% of data).
  71. 71. The box plot is composed of; I box denoting the IQR, I an inner line bisecting the box denoting the median, I whiskers extending to the most extreme point no further than 1.5 times IQR length away from the edges of the box, I points outside the box and whiskers marked as outliers. ggplot(mpg, aes("",hwy)) + geom_boxplot() + coord_flip() + scale_x_discrete("")
  72. 72. ●●● 20 30 40 hwy
  73. 73. ggplot(mpg, aes(reorder(class, -hwy, median), hwy)) + geom_boxplot() + coord_flip() + scale_x_discrete("class")
  74. 74. ● ●● ● ●● ● ●● ●● ●●●● ●●● ● compact midsize subcompact 2seater minivan suv pickup 20 30 40 hwy class
  75. 75. I The graph suggests the following fuel e ciency order among vehicle classes: pickups, SUV, minivans, 2-seaters, sub-compacts, midsizes, and compacts. I The compact and midsize categories have almost identical box and whiskers but the compact category has a few high outliers. I The spread of subcompact cars is substantially higher than the spread in all other categories. I We also note that SUVs and two-seaters have almost disjoint values (the box and whisker ranges are completely disjoint) leading to the observation that almost all 2-seater cars in the survey have a higher highway mpg than SUVs.
  76. 76. QQ-Plots I Quantile-quantile plots are useful for comparing two datasets, one of which may be sampled from a certain distribution. ggplot(R, aes(sample = samples)) + stat_qq(distribution = qt, dparams = pm) I They are essentially scatter plots of the quantiles of one dataset vs. the quantiles of another dataset. I The shape of the scatter plot implies the following conclusions (the proofs are straightforward applications of probability theory).
  77. 77. I A straight line with slope 1 that passes through the origin implies that the two datasets have identical quantiles, and therefore that they are sampled from the same distribution. I A straight line with slope 1 that does not pass through the origin implies that the two datasets have distributions of similar shape and spread, but that one is shifted with respect to the other. I A straight line with slope di↵erent from 1 that does not pass through the origin implies that the two datasets have distributions possessing similar shapes but that one is translated and scaled with respect to the other. I A non-linear S shape implies that the dataset corresponding to the x-axis is sampled from a distribution with heavier tails than the other dataset. I A non-linear reflected S shape implies that the dataset whose quantiles correspond to the y-axis is drawn from a distribution having heavier tails than the other dataset.
  78. 78. D = data.frame(samples = c(rnorm(200, 1, 1), rnorm(200, 0, 1), rnorm(200, 0, 2))) D$parameter[1:200] = 'N(1,1)'; D$parameter[201:400] = 'N(0,1)'; D$parameter[401:600] = 'N(0,2)'; qplot(samples, facets = parameter~., geom = 'histogram', data = D)
  79. 79. 0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50 N(0,1)N(0,2)N(1,1) −4 0 4 8 samples count
  80. 80. D = data.frame(samples = c(rnorm(200, 1, 1), rnorm(200, 0, 1), rnorm(200, 0, 2))); D$parameter[1:200] = 'N(1,1)'; D$parameter[201:400] = 'N(0,1)'; D$parameter[401:600] = 'N(0,2)'; ggplot(D, aes(sample = samples)) + stat_qq() + facet_grid(.~parameter)
  81. 81. N(0,1) N(0,2) N(1,1) ● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ● ● ● ● ● ●●●●●● ●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●● ●●●●● ● ● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ● −5.0 −2.5 0.0 2.5 5.0 −3 −2 −1 0 1 2 3−3 −2 −1 0 1 2 3−3 −2 −1 0 1 2 3 theoretical sample
  82. 82. x_grid = seq(-6, 6, length.out = 200) R = data.frame(density = dnorm(x_grid, 0, 1)) R$tdensity = dt(x_grid, 1.5) R$x = x_grid ggplot(R, aes(x = x, y = density)) + geom_area(fill = I('grey')) + geom_line(aes(x = x, y = tdensity)) + labs(title = "N(0,1) (shaded) and t-distribution (1.5 dof)")
  83. 83. 0.0 0.1 0.2 0.3 0.4 −6 −3 0 3 6 x density N(0,1) (shaded) and t−distribution (1.5 dof)
  84. 84. x_grid = seq(-6, 6, length.out = 200) R = data.frame(density = dnorm(x_grid, 0, 1)) R$samples = rnorm(200, 0, 1) pm = list(df = 1.5) ggplot(R, aes(sample = samples)) + stat_qq(distribution = qt, dparams = pm)
  85. 85. ● ● ● ● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ● ●●● ● ● ● ● −2 0 2 −30 −20 −10 0 10 20 30 theoretical sample
  86. 86. Preprocessing Data Guy Lebanon September 30, 2015
  87. 87. Goals I Learn how to handle missing data I Learn how to handle outliers I Learn when and how to transform data I Learn standard data manipulations techniques Module will be separated to 4 parts based on the four goals above.
  88. 88. Missing Data Data may be missing for a variety of reasons. I corrupted during its transfer or storage I some instances in the data collection process were skipped due to di culty or price associated with obtaining the data Di↵erent features in di↵erent samples may be missing: first sample (row) may have third feature (column) missing while the second sample may have the fifth feature missing.
  89. 89. Examples of Missing Data I Recommendation systems recommend to users items from a catalog based on historical user rating. Often, there are a lot of items in the catalog and each user typically indicates their star ratings for only a small subset of them. I In longitudinal studies some of the subjects may not be able to attend each of the surveys throughout the study period. The study organizers may also have lost contact with some of the subjects, in which case all measurements beyond a certain time point are missing. I In sensor data, some of the measurements may be missing due to sensor failure, battery discharge, or electrical interference. I In user surveys, users may choose to not respond to some of the questions for privacy reasons.
  90. 90. Missing Completely at Random I If a variable (dataframe column) is as likely to be missing as all other variables, we say that it is MCAR. I For example, in the case of users rating movies using 1-5 stars, we consider ratings of specific movies as dataframe columns and ratings associated with specific users as dataframe rows. Since some movies are more popular than others, some columns are more likely to be missing than others, violateing the MCAR definition.
  91. 91. Missing at Random (MAR) I MAR occurs when the probability that a variable is missing depends only on the other information available in the dataset. I For example, in a survey recording gender, race, and income, gender and race are not very objectionable questions, so we assume for now that the survey respondents answer these questions fully. The income question is more sensitive and users may choose to not respond for privacy reasons. I The tendency to report income or to not report income typically varies from person to person. If it only depends on gender and race, then the data is MAR. I If the decision whether to report or not report income depends also on other variables that are not in the dataframe (such as age or profession), the data is not MAR.
  92. 92. Handling Missing Data Most methods are designed to work with fully observed data. Below are some general ways to convert missing data to non-missing data. I Remove all data instances (for example dataframe rows) containing missing values. I Replace all missing entries with a substitute value, for example the mean of the observed instances of the missing variable. I Estimate a probability model for the missing variable and replace the missing value with one or more samples from that probability model. In the case of MCAR, all three techniques above are reasonable in that they may not introduce systematic errors. In the more likely case of MAR or non-MAR data the methods above may introduce systematic bias into the data analysis process.
  93. 93. Missing Data and R I R represents missing data using the NA symbol. I The function is.na returns a data structure having TRUE values where the corresponding data is missing and FALSE otherwise. I complete.cases() returns a vector whose components are FALSE for all samples (dataframe rows) containing missing values and TRUE otherwise. I na.omit() returns a new dataframe omitting all samples (dataframe rows) containing missing values. I Some functions have an na.rm argument, which if set to TRUE changes the function behavior so that it proceeds to operate on the supplied data after removing all dataframe rows with missing values.
  94. 94. The code below analyzes the dataframe movies in the ggplot2 package, which contains 24 attributes (genre, year, budget, user ratings, etc.) for 58788 movies obtained from the website http://www.imdb.com with some missing values. mean(movies$length) # average length ## [1] 82.34 mean(movies$budget) # average budget ## [1] NA # average budget (removing missing values) mean(movies$budget, na.rm = TRUE) ## [1] 13412513 mean(is.na(movies$budget)) # frequency of non-missing budget ## [1] 0.9113
  95. 95. moviesNoNA = na.omit(movies) qplot(rating, budget, data = moviesNoNA, size = I(1.2)) + stat_smooth(color = "red", size = I(2), se = F) ●● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●●●● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ●●●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●●● ●●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ●● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ●● ●●●● ● ● ● ●● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ●● ● ● ●● ● ● ●●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ●● ● ●● ●● ●●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ●● ●●● ● ● ●● ● ●● ●● ●● ● ●● ● ● ●● ● ●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ●●● ●● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ●● ●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●●● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ●● ●● ●● ● ● ●● ●● ●● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ●● ● ● ● ● ● ● ● ● ●● ●●● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●● ●● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ●●● ● ● ● ●● ●● ● ● ● ● ● ●●●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ●●●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ●●● ●●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ●● ●●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●●● ● ● ●●● ● ● ● ● ● ●● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ●● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ●● ● ●● ● ●● ●● ● ● ● ● ● ●● ● ● ●●●● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ●● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ●● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ● ●● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ●● ● ● ● ●●● ● ●● ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ●● ●● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●●● ● ●● ● ● ●● ● ● ● ●● ● ●● ●●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ●● ●● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ●●● ● ● ●●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ●● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ●●● ● ●● ● ● ● ● ● ● ● ● ●● ●● ● ●● ● ●● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●●●● ● ●● ●● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●●●●● ● ● ● ● ● ●● ●● ●●● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●●● ● ● ● ● ● ●● ●● ● ● ● ●●● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ●● ● ● ● ● ● ●● 0.0e+00 5.0e+07 1.0e+08 1.5e+08 2.0e+08 2.5 5.0 7.5 10.0 rating budget
  96. 96. moviesNoNA = na.omit(movies) qplot(rating, votes, data = moviesNoNA, size = I(1.2)) ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●●● ● ● ●● ● ● ● ● ● ● ● ●●●● ● ● ●● ●● ●● ● ● ● ●● ● ● ● ●● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ●● ●● ● ● ● ● ● ●● ●● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●●● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ●● ● ● ●● ●● ● ●● ●● ●● ● ● ● ●● ●●●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●● ● ● ● ●●●● ● ● ●● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ● ●●●● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●●● ●● ● ● ●● ● ● ●● ●● ●● ●● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●●● ●● ● ●● ●● ● ●● ● ● ● ●● ●● ● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●●● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ●●● ●● ● ● ●● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●● ●● ●● ●● ● ● ● ●●●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ●● ●●● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●●● ●●●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ●● ●●●● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ●● ●● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●●● ●● ● ●● ● ● ● ● ●● ● ●●● ● ● ● ● ● ●●●● ● ● ●● ● ●● ● ● ●● ●●●● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ●● ●● ●● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ●●● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●●●●●● ● ● ● ●● ●● ● ● ● ● ● ● ●●● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ●● ● ●●● ● ●● ● ● ● ●● ●● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ●● ● ● ●● ●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ●●● ● ●● ● ● ● ● ● ●● ●● ●●●● ● ● ● ● ●● ● ● ● ● ●● ●● ●● ● ●● ● ●●● ●●● ● ●● ●● ● ●● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ●● ●● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ●●● ● ● ● ● ● ●● ● ● ●● ● ●●● ● ●● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ● ● ● ●● ●●● ● ●●● ●●● ● ● ● ● ● ●●● ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ●●●● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ●● ●●● ●● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●●● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ●●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ●●● ●●● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ●● ● ● ●●● ● ●● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ●● ● ● ● ● ●● ●● ●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ●●● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ●● ● ● ●● ●● ●● ● ● ● ● ● ● ●●● ●● ● ● ● ●●● ●● ● ● ● ● ● ● ● ●● ●● ● ●● ●● ● ● ●● ● ●● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ●● ● ● ● ●●●● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ●● ● ● ● ● ●●● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●●●● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ●● ● ● ● ● ● ●● ● ● ●●● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ●● ●●● ● ● ● ● ● ● ● ● ● ●●●● ● ● ●●● ● ● ● ● ● ●● ●● ●●● ●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ●● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ●● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ●● ●●●●● ● ● ● ● ● ● ● ●●●● ● ● ● ● ●●●● ●● ●● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ●● ● ● ● ● ●● ●● ● ● ● ● ●● ●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ●●● ●● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ●●●●● ●● ● ● ● ●●●● ● ● ● ● ●● ●●● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●● ● ●● ● ● ●● ● ●●● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ●● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ●● ●●●● ● ● ● ● ●● ●●● ● ● ● ●● ● ● ●● ● ● ●●● ● ●● ● ●●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ●●● ●●● ● ●● ● ● ● ●● ● ● ●● ●● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ●●●● ●● ● ● ● ●● ● ●● ● ● ● ● ●● ● ● ●● ●● ● ● ●●●● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ●●● ● ● ● ●●● ● ● ●●● ● ● ● ● ● ●● ●●● ●● ●● ●● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ●● ● ●● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●●● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ●●●● ● ● ● ● ●● ●● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ●● ●● ● ●● ● ● ●● ● ●● ● ● ● ●● ●●● ●● ●● ● ● ●● ●● ●● ● ● ●●● ● ● ● ● ●● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ●● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ●●● ●●● ●●● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ●● ●● ● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ●●● ● ● ● ● ● ●● ●●● ●● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ●● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ●● ● ● ●●●● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ●● ●● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ●● ●●● ● ●● ●● ●● ● ●● ● ● ● ●●● ● ●● ● ● ●● ●●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ●● ● ● ● ● ●●● ●●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ●● ● ●● ● ●●● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ●●● ● ● ●● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ●● ● ●● ●● ● ● ●● ●●● ● ● ● ●● ●● ●●●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ●● ●●● ● ● ● ● ●● ●● ● ● ● ●● ● ● ●● ●● ● ●● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ●● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●●●● ● ● ●● ● ● ● ●●● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ●● ●● ●● ● ● ● ● ● ● ●●● ● ●● ●● ●● ●● ● ● ● ●● ● ●● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ●● ● ● ● ●●● ● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ●● ● ● ● ● ●● ●● ●● ● ● ● ●● ● ● ●●●●● ●● ●●● ● ●● ●●● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ●● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ●● ● ● ●● ●● ●● ● ● ● ● ● ●● ●●● ● ● ● ● ● ●●● ●●● ●● ● ●●● ●● ● ● ● ● ● ● ● ●●● ● ● ●● ● ●● ●●● ● ● ● ● ●● ● ● ● ●●●● ● ● ● ● ●● ●● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ●● ●● ● ● ● ●● ● ●● ●●●● ● ● ● ●●●● ● ● ●● ●● ●●● ●● ● ● ●● ●● ● ● ● ● ● ●● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ●●●● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●●●● ● ● ●● ●● ● ● ●●●● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●●● ● ●● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ●● ●●●● ● ● ●●●● ● ●● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ●● ● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ●● ● ●● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ●● ●● ●● ●● ● ● ● ● ●● ●● ● ● ● ● ● ● ●●● ● ● ●● ● ● ● ●● ●●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ● ● ● ●● ●●● ● ● ●●●● ● ●● ● ● ● ●● ● ●● ●● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ●●●● ●● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ●● ● ● ●● ● ● ● ●●● ● ● ● ●● ● ●● ● ●● ● ● ● ● ●● ● ●● ●● ●● ● ●● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ●● ●●●● ●● ● ● ●●● ● ● ● ● ● ● ● ● ●● ●●● ● ●● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ●● ●● ●● ● ● ● ●● ● ●● ●● ● ●● ● ● ● ● ● ●●● ● ●●●●● ● ● ● ● ●●●● ● ● ●● ● ● ● ●● ●● ●● ● ● ●●● ● ● ● ● ● ● ● ●● ●● ●●● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ●●●●● ● ●● ●● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ●●● ●● ● ● ●●● ●●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ●● ● ●●●●●● ● ●● ● ● ● ●● ●●● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ●● ● ●●● ● ● ●● ●●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●●● ● ● ●● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ●●● ●● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ●●● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ●●● ● ● ● ● ● ● ● 0 50000 100000 150000 2.5 5.0 7.5 10.0 rating votes
  97. 97. I Number of votes (which can be used as a surrogate for popularity) tend to increase as the average rating increase. I Spread in the number of votes increases with the average rating. I Movies featuring the highest average ratings have a very small number of votes. Note that users tend to see movies that they think they will like, and thus the observed ratings tend to be higher than ratings gathered after showing users random movies.
  98. 98. Outliers Two types of outliers I Corrupted values, for example, human errors during a manual process of entering measurements in a spreadsheet. I Substantially unlikely values given our modeling assumptions, for example Black Monday stock crash on October 19, 1987, when the Dow Jones Industrial Average lost 22% in one day. In both cases, data analysis based on outliers may result in drastically wrong conclusions.
  99. 99. library(Ecdat) data(SP500, package = 'Ecdat') qplot(r500, main = "Histogram of log(P(t)/P(t-1)) for SP500 (1981-91)", xlab = "log returns", data = SP500) 0 300 600 900 −0.2 −0.1 0.0 0.1 log returns count Histogram of log(P(t)/P(t−1)) for SP500 (1981−91)
  100. 100. qplot(seq(along = r500), r500, data = SP500, geom = "line", xlab = "trading days since January 1981", ylab = "log returns", main = "log(P(t)/P(t-1)) for SP500 (1981-91)") −0.2 −0.1 0.0 0.1 0 1000 2000 logreturns log(P(t)/P(t−1)) for SP500 (1981−91)
  101. 101. Robustness Robustness describes a lack of sensitivity of data analysis procedures to outliers. I The mean of n numbers is a non-robust procedure while the median is a robust procedure. I Assuming a symmetric distribution of samples around 0, we expect the mean to be zero, or at least close to it. But, the presence of a single outlier (very positive value or very negative value) may substantially a↵ect the mean calculation and drive it far away from zero, even for large n. I In contrast the median will not change its value.
  102. 102. Dealing with Outliers Truncating. Remove all values deemed as outliers. Winsorization. Replace outliers with the most extreme of the remaining values. Robustness. Analyze the data using a robust procedure.
  103. 103. Removing Outliers To remove outliers we need to first detect them. I Values below the ↵ percentile or above the 100 ↵ percentile for some small ↵ > 0. I Values more than c standard deviations away from the mean. I Chicken-and-egg problem since standard deviation and mean calculations above will be corrupted by outliers. One solution is computing the mean and standard deviation after removing the most extreme values (see next slide). Alternatively percentile (that are more robust) can be used.
  104. 104. originalData = rnorm(20) originalData[1] = 1000 sortedData = sort(originalData) originalData = originalData[3:18] lowerLimit = mean(sortedData) - 5 * sd(sortedData) upperLimit = mean(sortedData) + 5 * sd(sortedData) noOutlierInd = (lowerLimit < originalData) & (originalData < upperLimit) dataWithoutOutliers = originalData[noOutlierInd]
  105. 105. library(robustHD) originalData = c(1000, rnorm(10)) print(originalData[1:5]) ## [1] 1000.0000 -0.6265 0.1836 -0.8356 1.5953 print(winsorize(originalData[1:5])) ## [1] 3.2060 -0.6265 0.1836 -0.8356 1.5953
  106. 106. Data Transformations: Skewness and Power Transformation I In many cases, data is drawn from a highly-skewed distribution that is not well described by one of the common statistical distributions. I A simple transformation may map the data to a form that is well described by common distributions, such as the Gaussian or Gamma distributions I A suitable model can then be fitted to the transformed data (if necessary, predictions can be made on the original scale by inverting the transformation). Power Transformation Family: replace non-negative data x by f (x) = 8 >< >: (x 1)/ > 0 log x = 0 (x 1)/ < 0 x > 0, 2 R.
  107. 107. I Intuitively, the power transform maps x to x , up to multiplication by a constant and addition of a constant. I This mapping is convex for > 1 and concave for < 1. I A choice of < 1 removes right-skewness (data has a heavy tail to the right) with smaller values of resulting in a more aggressive removal of skewness. Similarly, a choice of > 1 removes left-skewness. I Subtracting 1 and dividing by makes f (x) continuous in as well as in x. I One way to select is to try di↵erent values, graph the resulting histograms, and select one of them. There are also more sophisticated methods for selecting based on the maximum likelihood method.
  108. 108. print(diamonds[1:10,1:8]) ## carat cut color clarity depth table price x ## 1 0.23 Ideal E SI2 61.5 55 326 3.95 ## 2 0.21 Premium E SI1 59.8 61 326 3.89 ## 3 0.23 Good E VS1 56.9 65 327 4.05 ## 4 0.29 Premium I VS2 62.4 58 334 4.20 ## 5 0.31 Good J SI2 63.3 58 335 4.34 ## 6 0.24 Very Good J VVS2 62.8 57 336 3.94 ## 7 0.24 Very Good I VVS1 62.3 57 336 3.95 ## 8 0.26 Very Good H SI1 61.9 55 337 4.07 ## 9 0.22 Fair E VS2 65.1 61 337 3.87 ## 10 0.23 Very Good H VS1 59.4 61 338 4.00
  109. 109. diamondsSubset = diamonds[sample(dim(diamonds)[1], 1000),] qplot(price, data = diamondsSubset) 0 50 100 150 200 0 5000 10000 15000 20000 price count
  110. 110. qplot(log(price), size = I(1), data = diamondsSubset) 0 20 40 6 7 8 9 10 log(price) count
  111. 111. I Power transformations are useful also for examining the relationship between two or more data variables. I The following plot shows the relationship between diamond price and diamond carat. It is hard to draw much information from that plot beyond the fact that there is a non-linear increasing trend. I Transforming both variables using a logarithm shows a striking linear relationship on a log-log scale.
  112. 112. qplot(log(price), size = I(1), data = diamondsSubset) 0 20 40 6 7 8 9 10 log(price) count
  113. 113. qplot(carat, price, size = I(1), data = diamondsSubset) ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● 0 5000 10000 15000 1 2 3 4 carat price
  114. 114. qplot(carat, log(price), size = I(1), data = diamondsSubset) ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 6 7 8 9 10 1 2 3 4 carat log(price)
  115. 115. qplot(log(carat), price, size = I(1), data = diamondsSubset) ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● 0 5000 10000 15000 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 log(carat) price
  116. 116. qplot(log(carat), log(price), size = I(1), data = diamondsSubset) ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 6 7 8 9 10 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 log(carat) log(price)

×