1.
Computational Techniques for the Statistical
Analysis of Big Data in R
A Case Study of the rlme Package
Herb Susmann, Yusuf Bilgic
April 12, 2014

2.
Workflow
Identify
Rewrite
Benchmark
Test
Case Study: rlme
Identify
Wilcoxon Tau Estimator
Pairup
Covariance Estimator
Summary
Keeping Ahead

3.
Motivation
Case study: rlme package
Rank based regression and estimation of two- and three- level
nested effects models.
Goals: faster, less memory, more data
Before: 5,000 rows of data
After: 50,000 rows of data

4.
Section 1
Workflow

5.
Workflow
Identify
Rewrite
Benchmark
Test

6.
Identify
Know your big O!

7.
Identify
Know your big O! (O(n2) memory usage? probably not so
good for big data)

8.
Identify
Know your big O! (O(n2) memory usage? probably not so
good for big data)
Look for error messages

9.
Identify
Know your big O! (O(n2) memory usage? probably not so
good for big data)
Look for error messages
Profiling with RProf

10.
Rewrite
High level design
Algorithm design

11.
Rewrite
High level design
Algorithm design
Statistical techniques: bootstrapping

12.
Rewrite
Microbenchmarking
Know what R is good at

13.
Rewrite
Microbenchmarking
Know what R is good at
Avoid loops in favor of vectorization

14.
Rewrite
Microbenchmarking
Know what R is good at
Avoid loops in favor of vectorization
Preallocation

15.
Rewrite
Microbenchmarking
Know what R is good at
Avoid loops in favor of vectorization
Preallocation
Arguments are by value, not by reference

16.
Rewrite
Microbenchmarking
Know what R is good at
Avoid loops in favor of vectorization
Preallocation
Arguments are by value, not by reference
Embrace C++

17.
Rewrite
Microbenchmarking
Know what R is good at
Avoid loops in favor of vectorization
Preallocation
Arguments are by value, not by reference
Embrace C++
Be careful!

18.
Vectorizing
## Bad
vec = 1:100
for (i in 1:length(vec)) {
vec[i] = vec[i]^2
}
## Better
sapply(vec, function(x) x^2)
## Best
vec^2

19.
Preallocation
## Bad
vec = c()
for (i in 1:0) {
vec = c(vec, i)
}
## Better
vec = numeric(100)
for (i in 1:0) {
vec[i] = i
}

20.
Pass by value
square <- function(x) {
x <- x^2
return(x)
}
x <- 1:100
square(x)

21.
Benchmark
Write several versions of a slow function

22.
Benchmark
Write several versions of a slow function
Test them against each other

23.
Benchmark
Write several versions of a slow function
Test them against each other
Package: microbenchmark

24.
Test
Regressions

25.
Test
Regressions
Unit Testing

26.
Test
Regressions
Unit Testing
Package: testthat

27.
Test
Regressions
Unit Testing
Package: testthat

28.
Section 2
Case Study: rlme

29.
Identify
Over to R!
Rprof("profile")
fit.rlme = rlme(...)
Rprof(NULL)
summaryRprof("profile")

30.
Wilcoxon Tau Estimator
Rank based scale estimator of residuals
Uses pairup (so already O(n2))

31.
Wilcoxon Tau Estimator
Original:
dresd <- sort(abs(temp[, 1] - temp[, 2]))
dresd = dresd[(p + 1):choose(n, 2)]

32.
Wilcoxon Tau Estimator
Original:
dresd <- sort(abs(temp[, 1] - temp[, 2]))
dresd = dresd[(p + 1):choose(n, 2)]
What’s wrong?

33.
Wilcoxon Tau Estimator
Original:
dresd <- sort(abs(temp[, 1] - temp[, 2]))
dresd = dresd[(p + 1):choose(n, 2)]
What’s wrong? Bad algorithm (the sort is at least O(nlogn)),
variable gets copied multiple times

34.
Wilcoxon Tau Estimator
Original:
dresd <- sort(abs(temp[, 1] - temp[, 2]))
dresd = dresd[(p + 1):choose(n, 2)]
What’s wrong? Bad algorithm (the sort is at least O(nlogn)),
variable gets copied multiple times
Updated with C++
dresd = remove.k.smallest(dresd)

35.
Wilcoxon Tau Estimator
Test with 2,000 residuals: better!

36.
Wilcoxon Tau
But what about really huge inputs?

37.
Wilcoxon Tau
But what about really huge inputs?
Bootstrapping: when over 5,000 rows, repeat estimate on
1000 sampled points 100 times

38.
Wilcoxon Tau
But what about really huge inputs?
Bootstrapping: when over 5,000 rows, repeat estimate on
1000 sampled points 100 times
Not about speed, but about memory

39.
Pairup
Pairup function: generates every possible pair from input
vector
Some rank-based estimators require pairwise operations
O(n2) complexity

40.
Pairup
Original version: vectorized (14 LOC)

41.
Pairup
Original version: vectorized (14 LOC)
Loop version (12 LOC)

42.
Pairup
Original version: vectorized (14 LOC)
Loop version (12 LOC)
”Combn” version (core R function, 1 LOC)

43.
Pairup
Original version: vectorized (14 LOC)
Loop version (12 LOC)
”Combn” version (core R function, 1 LOC)
C++ version (12 LOC)

44.
Over to R!

45.
Covariance Estimator
n × n covariance matrix
change to preallocation

46.
Covariance Estimator

47.
Summary
Identify
Rewrite
Benchmark
Test

48.
Keeping Ahead
Parallelism

49.
Keeping Ahead
Parallelism
Cluster: RMpi, snow

50.
Keeping Ahead
Parallelism
Cluster: RMpi, snow
GPU: rpud

51.
Keeping Ahead
Parallelism
Cluster: RMpi, snow
GPU: rpud
Probably not Hadoop, maybe Apache Spark?

52.
Keeping Ahead
Parallelism
Cluster: RMpi, snow
GPU: rpud
Probably not Hadoop, maybe Apache Spark?
Julia Language

53.
Keeping Ahead
Parallelism
Cluster: RMpi, snow
GPU: rpud
Probably not Hadoop, maybe Apache Spark?
Julia Language
Hadley Wickham (plyr, ggplot, testthat, ...)

54.
Keeping Ahead
Parallelism
Cluster: RMpi, snow
GPU: rpud
Probably not Hadoop, maybe Apache Spark?
Julia Language
Hadley Wickham (plyr, ggplot, testthat, ...)
“Advanced R Programming”

55.
Keeping Ahead
Parallelism
Cluster: RMpi, snow
GPU: rpud
Probably not Hadoop, maybe Apache Spark?
Julia Language
Hadley Wickham (plyr, ggplot, testthat, ...)
“Advanced R Programming”

56.
Questions?