Your SlideShare is downloading. ×
  • Like
Graphical Inference
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Graphical Inference

  • 894 views
Published

 

Published in Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
894
On SlideShare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
10
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. A graphical grammar + graphical inference = a grammar of graphical inference? Hadley Wickham, Rice University
  • 2. 1. Motivation 2. Resampling methods (graphical inference) 3. Graphical display (grammar of graphics) 4. Future work
  • 3. 5 8 6 ● ● ● 4 ● ● ● ● ● ● ● ● ● ● ● ● ● ●● 2 ● species ● ● ● ● Concinna ● ● ● ● ● Heikert. ● 0 ● ● ● ● ● ● ● ● Heptapot. ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● −2 ● ● ● ● ● ● ● ● ● ● ● ● ● −4 −6 −6 −4 −2 0 2 4 6
  • 4. 1 2 3 4 8 6 4 ● ●● ● ● 2 ●● ●●● ● ●● ● ●● ● ● ● ● ●●● ●● ● ●● ● ● ● ● ● ● ● ● ●●●●● ● ● ● ●● ●●● ●● ● ● ● ● ●● ● ●●● ●● ● ● ● ● ● ● ●● ● ● ● ● ●●●● ●● ● ● ●● ● ●●● ● ● ● ● ●● ●●● ●●●●● ● ●●● ●● ●●● ● ●●●● ● ● ●●●● ● ● 0 ● ● ● ●● ● ●● ● ●● ●● ●●● ● ●● ● ●● ● ● ●● ●● ●● ●● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ●●● ● ●● ●● ●● ● ●● ●● ●● ● ●● ● ●● ●● ●● ●● ● ● ● ● ●● ●●● ● ● ● ● ●● ●● ● ● ●●●●● ● ●● ●●● ● ●● ● ● −2 ●● ● ● ● ● −4 −6 5 6 7 8 8 6 4 ●● ● ● ● species ●●● ●● ● ●● ● ● ● ● ● 2 ●● ● ● ● ●● ● ● ●● ● ● ● ● ●● ●● ● Concinna ● ● ● ● ● ● ●● ●● ● ● ● ●● ●● ● ● ● ● ●● ●●●● ● ● ● ● ●● ● ●● ● ●● ● ● ●● ● ●● 0 ●● ●● ● ● ● ● ●●●● ● ●●● ●●● ● ●●●●● ●●●● ● ● ●● ●● ● ●● ● ● ● ●● ● ● ●● ● ●● ● ●●●● ●● ● ●● ●● ●● ● ● ● ●● ● ● ●●● ●● ● ● ●●●●● ● ● Heikert. ● ● ● ● ●● ● ●● ● ● ●● ● ●● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ●●●● ● ●●●● ● ●●● ●● ●● ● ● ●●●●● ● ● ● ● ● ● ● ● ●●● ● ● ● ●● ● ● ●●● ● ●● ● −2 ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ●● ● Heptapot. −4 −6 9 10 11 12 8 6 4 ● ● 2 ●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ●●●●● ● ● ●●● ● ●● ● ● ●●● ● ● ● ● ●● ●●● ● ● ● ●● ●● ● ● ● ● ● ●●●● ●● ●●● ● ● ● ● ● ● ●●●● ● ●● ●● ● ●●●●● ●● ● ●● ●●●● ●●●● ●●●●● ● ● ●● ● ●● ● 0 ●●● ●● ● ● ● ●● ●● ● ● ●● ● ● ●●●●● ● ●●●●● ● ● ●● ● ● ● ●●●● ● ● ● ● ●● ●● ● ● ● ●●●● ●● ●● ● ●● ●● ●● ●● ● ● ● ●● ● ●● ● ●●● ●● ●● ● ● ● ● ●●●● ● ● ● ● ●● ● ● ● ●●●● ● ●●● ●● ● ●●●● ● ● ●● ●●● ● ●●● ●● ●●● ● ●● ● ● ●● −2 ● ● ● −4 −6 −6 −4 −2 0 2 4 6 −6 −4 −2 0 2 4 6 −6 −4 −2 0 2 4 6 −6 −4 −2 0 2 4 6
  • 5. 1 2 3 4 ● 4 ● ● ● ● ● ●● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● 2 ● ● ● ● ● ● ● ●● ● ● ● ● ●● ●● ● ● ●●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ● ●●● ● ● ● ●● ● ● ● ●● ● ●● ● ●● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ●● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● 0 ● ● ● ● ● ● ● ●●●● ●●●● ● ●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ●● ● ●● ●●●● ●● ●● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●●● ● −2 ● ●● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● −4 5 6 7 8 ● 4 ● ● ● ● ● ● ● ● ●● ● ● ● species ● ● ● ● ● ● ●● ● 2 ● ●● ● ●● ●● ●● ● ● ● ●● ● ● ●● ● ● ●● ● ● ● ● ●●● ●●● ● ●● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ●● ●● ● ● ● ● ●● ● ●● ● ● ●● ● ● ●● ● ● ● Concinna ● ●● ● ● ●● ● ● ● ● ●●● ●●● ● ● ● ● ●● ● ● ●● 0 ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ●● ● ●● ● ● ●● ● ● ●● ●●● ●● ● ● ●● ● ● ● ● ● ● ● Heikert. ●● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ●● ●● ● ● ● ● −2 ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● Heptapot. ● ● ● ● ● ● −4 9 10 11 12 4 ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● 2 ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ●●● ● ● ●● ● ● ● ●● ● ● ●● ● ●● ● ● ●● ●● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ●●● ● ● ● ● ● ●●● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● 0 ●● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ●● ●●●●●● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ●● ●● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●● −2 ● ● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● −4 −4 −2 0 2 4 −4 −2 0 2 4 −4 −2 0 2 4 −4 −2 0 2 4
  • 6. Problem Want to make it very easy to make these types of display - should be able to create as easy as doing as simple numerical test. With the right tools, it turns out to be really easy! Two basic problems: how to do resampling under the null, and how to define the graphic.
  • 7. Resampling Many resampling methods have particularly elegant expression in R code. For example, if null hypothesis is that of independence: df <- transform(df, resp = sample(resp)) However, better to generalise the pattern.
  • 8. Generalisation permute_var <- function(var) { function(df) { df[[var]] <- sample(df[[var]]) df } } f <- permute_var(quot;mpgquot;) f(mtcars) permute_var(quot;mpgquot;)(mtcars)
  • 9. Advantages Separate description from implementation. Call describes action. Can later replace implementation if better approach discovered. Unimportant details concealed.
  • 10. n resamples Each method gives us a single resample, but we need n. Trivial to repeat n times with rdply() from the plyr package. (rdply() is a generalisation of replicate() that returns a data frame with a column that labels each replicate.)
  • 11. Overall resamp <- function(true, method, n = 19, pos = sample(n + 1, 1)) { samples <- rdply(n, method(true)) if (missing(pos)) { message(quot;True data in position quot;, pos) } add_true(samples, true, pos) } add_true <- function(samples, true, n) { samples$.n <- with(samples, ifelse(.n >= n, .n + 1, .n)) true$.n <- n all <- rbind(samples, true) all[order(all$.n), ] } Will see application shortly
  • 12. Other nulls Need functions for other common null hypotheses. Experimental null_model(), which computes rotation residuals given a specified linear model. (Demo a little later)
  • 13. Display How is the plot of the simulated data with true data different from a single plot? Just need to repeat the same display n times and label appropriately. This is easy if we have a description of the plot, independent of the data.
  • 14. Grammar of Graphics This is one principle of the grammar of graphics: should describe the graphic we want, not how to create it. Implemented with the ggplot2 package in R. Easy to modify a graphic after it has been created. For graphical inference, we need to change the data and facetting.
  • 15. Example
  • 16. ● 35 ● 30 ● ● ● ● ● ● 25 ● ● ● ● ● ● ● ● ● factor(year) ●● ● cty ● ● ● ● ● ●● ● ● 1999 ● ●● ●●●● ●● ● ● ●● ● ● 20 ● ● ● ● ● ● 2008 ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ●● ● ● ● ● ●● ●●● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● 15 ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ●● ●● ● ●● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 10 ● ● ● ● 2 3 4 5 6 7 displ
  • 17. ● 10 ● ● ●● ●●● ● ● ● ● ● ● ● ● ● 8 ●● ●● ● ● ● 1/cty * 100 ● ● ●● ● ● ●● ● factor(year) ● 1999 ●● ● ●● ●● ● ● ● ● ● ● 2008 ● ●● ● ● ● ● ● ● ● 6 ●● ●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● 4 ● ● ● ● ● ● ● ● 2 3 4 5 6 7 displ
  • 18. 1 2 3 4 5 ● ● ● ● ● 10 ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● 6 ● ●●●● ●● ●●● ● ● ●● ●●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ● ●●● ● ●●● ● ● ●● ●● ● ●● ● ●● ●● ● ●●●● ●● ●●● ● ● ●●● ●●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ● ●●● ● ●●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ●● ●●●● ● ● ● ●● ●● ● ●● ● ●● ● ● ●● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●●● ● ● ● ●● ●● ● ● ●●● ● ● ●● ●●● ● ● ●● ●●● ● ● ●● 4 ●● ● ●● ● ●● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● 6 7 8 9 10 ● ● ● ● ● 10 ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●●● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● 6 ● ●●●● ●● ● ●●● ● ● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ●● ●●●● ● ●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ●●● ● ● ●●● ●● ● ● ●● ●● ● ●● ● ●● ●● ● ●●●● ●● ●●● ● ● ●●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ●●●● ● ● ● ●● ●● ● ●● ● ●● ●● ● ● ●● ● ● ● ● ●● ● ●● ● ● ●● ● ● ●● ●●● ●● ● ● ●● ● ● ● ● ●● ● ●● ● ● ●●● ● ● ● ●●● ● ● ● ●● ● ●● ●●● ● ● ● ●●● ● ● ● 4 1/cty * 100 ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● factor(year) 11 12 13 14 15 ● 1999 ● ● ● ● ● ● 2008 10 ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● 6 ● ●●●● ●● ● ●●● ● ●●● ● ● ●● ●● ● ●● ● ●●●● ●● ●●● ● ● ●● ●●● ● ● ●● ●● ●●●●● ● ● ●●●● ●● ●●● ● ● ●●● ● ● ●● ●● ● ●● ● ●●●● ●● ●●● ●●●● ● ● ● ●● ●● ●●●● ● ●●●● ●● ●●● ●●●● ● ●● ● ● ●● ●● ●●●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● 4 ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● 16 17 18 19 20 ● ● ● ● ● 10 ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● 6 ● ●●●● ●● ●● ●●●● ● ● ● ●● ●● ●●●●● ● ● ●●●● ●● ● ●●● ● ● ● ● ●● ●● ● ●● ● ●●●● ●● ●●● ● ● ●●● ● ● ●● ●● ●●●●● ● ●●●● ●● ● ●●● ● ●● ● ● ● ●● ●● ● ●● ● ●●●● ●● ●●● ● ● ●● ●●● ● ● ●● ●● ● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● 4 ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 displ
  • 19. 1 2 3 4 5 ● ● ● ● ● 10 ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● 6 ● ●●●● ●● ●●● ● ● ●● ●●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ● ●●● ● ●●● ● ● ●● ●● ● ●● ● ●● ●● ● ●●●● ●● ●●● ● ● ●●● ●●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ● ●●● ● ●●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ●● ●●●● ● ● ● ●● ●● ● ●● ● ●● ● ● ●● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●●● ● ● ● ●● ●● ● ● ●●● ● ● ●● ●●● ● ● ●● ●●● ● ● ●● 4 ●● ● ●● ● ●● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● 6 7 8 9 10 ● ● ● ● ● 10 ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●●● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● 6 ● ●●●● ●● ● ●●● ● ● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ●● ●●●● ● ●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ●●● ● ● ●●● ●● ● ● ●● ●● ● ●● ● ●● ●● ● ●●●● ●● ●●● ● ● ●●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ●●●● ● ● ● ●● ●● ● ●● ● ●● ●● ● ● ●● ● ● ● ● ●● ● ●● ● ● ●● ● ● ●● ●●● ●● ● ● ●● ● ● ● ● ●● ● ●● ● ● ●●● ● ● ● ●●● ● ● ● ●● ● ●● ●●● ● ● ● ●●● ● ● ● 4 1/cty * 100 ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● factor(year) 11 12 13 14 15 ● 1999 ● ● ● ● ● ● 2008 10 ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● 6 ● ●●●● ●● ● ●●● ● ●●● ● ● ●● ●● ● ●● ● ●●●● ●● ●●● ● ● ●● ●●● ● ● ●● ●● ●●●●● ● ● ●●●● ●● ●●● ● ● ●●● ● ● ●● ●● ● ●● ● ●●●● ●● ●●● ●●●● ● ● ● ●● ●● ●●●● ● ●●●● ●● ●●● ●●●● ● ●● ● ● ●● ●● ●●●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● 4 ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● 16 17 18 19 20 ● ● ● ● ● 10 ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● 6 ● ●●●● ●● ●● ●●●● ● ● ● ●● ●● ●●●●● ● ● ●●●● ●● ● ●●● ● ● ● ● ●● ●● ● ●● ● ●●●● ●● ●●● ● ● ●●● ● ● ●● ●● ●●●●● ● ●●●● ●● ● ●●● ● ●● ● ● ● ●● ●● ● ●● ● ●●●● ●● ●●● ● ● ●● ●●● ● ● ●● ●● ● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● 4 ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 displ
  • 20. qplot(displ, cty, data = mpg, colour = factor(year)) qplot(displ, 1 / cty * 100, data = mpg, colour = factor(year)) mpg_perm <- resamp(mpg,permute_var(quot;yearquot;)) last_plot() %+% mpg_perm + facet_wrap(~ .n)
  • 21. Is a linear model with displacement as single predictor adequate?
  • 22. 1 2 3 4 5 ● ● ● ●● ● ●● 10 ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ●● ● ●●● ●● ● ● ● ● ● ● ●●● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ● ●● ● ● ●●●●● ● ● ●●● ● ●● ●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ●●● ● ● ● ●● ● 8 ●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ●●● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ●●●●● ● ●●●● ●● ● ● ●● ● ● ●●● ● ● ●●● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ●●●●●●● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ●●●● ● ● ●●●● ●● ● ●●●● ●● ● ● ●● ●● ● ● ●● ●● ● ● ● ● ●● ●●●●● ● ●●● ● ● ●● ● ● ●● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ●●●● ● ● 6 ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ●● ● ●● ● ● ● ●●●●● ●● ● ●● ● ●● ● ● ●●●● ●● ● ● ● ●●● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ●●●●● ● ● ●● ● ● ● ●●●●●●● ●●●●●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ●● ● ● ● ●●●● ● ● ●● ●● ● ●●●● ● ● ● ● ● ●●● ● ● ●● ● ●●● ● ●●●● ●● ●●● ● ● ●●●● ●● ● ● ● ●● ● ●● ● ●●●● ● ● ●●●● ●● ● ● ● ● ●●●●●● ●● ● ● ●● ● ●●● ● ● ● ●● ●●●● ● ● ● ● ●● ●●●● ● ● ●●● ● ● ●● ●●●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ●●● ● ● ● ● ●●● ● ●● ●●● ● 4 ● ● ● ●● ●●● ●● ●● ● ● ●●● ● ● ● ● ● ● ●● ●●● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ●●●● ●● ● ● ●● ●● ● ● ●●● ● ● ●● 2 ● ● 6 7 8 9 10 ● ● ● ● ●● ●● ● ● ● ● 10 ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ●● ● ● ● ● ● ●● ● ●● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●●●● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ● ●●●●●● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ●● ● ●● ● ● ● ●● ● ●● ● ● ●●●●●● ● ● ●● ● ●● ● ●● ● ● ● ●●●●● ● ● ●●●● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ●●●● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ●● ● ● ●●● 6 ● ● ●●● ● ● ● ●●● ● ● ● ●● ● ●●●●● ● ● ● ●●●● ● ● ● ● ●●●● ● ● ● ● ●●● ● ● ●●●● ● ● ● ● ●●●● ● ● ●●●●● ● ● ●●● ● ● ●●● ● ●●●●● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ●●●● ●● ● ● ●● ● ● ● ● ●●●●●● ● ●●●● ●● ● ● ●● ● ● ● ●●●● ● ● ●● ● ● ●● ● ● ● ● ● ●●● ● ●●● ● ● ● ● ●●● ● ● ●●● ● ● ● ●●● ● ● ● ● ●●● ●● ●● ●●●● ●● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ●● ●●● ● ●●●●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ●●● ● ● ● ●●●●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● 4 ● ●● ● ● ●● ● ●● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ●● ● ●● ● ●● ● ● ●● ● ● ● ● ●●● ● ● ●● ● ● ● ● ● ● ● 2 factor(year) gp100m ● 11 12 13 14 15 ● 1999 ● ● ● ● ● ● ● ● ● ● 2008 10 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●●● ● ● ● ●●● ●● ● ●● ● ● ● ● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● 8 ● ● ● ● ● ● ●●●● ● ●●● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ●●●● ● ● ● ● ● ●●● ●● ● ● ●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●●● ● ● ● ● ●● ● ●●●● ●● ●●● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ●●● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●●●●●●● ●● ● ● ● ●● ● ●●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●●● ● ●● ● ●●●●●●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●●●● ● ●●●● ●● ● ● ● ● ●●●●● ● ● ● ●●● ●● ● ●● ●● ● ● ● ● ● ●●●● ● ● ● ● ● ● ●●●●● ● ● ● ●●● ● ● ●●● ● ● ●● ●●●● ● ●● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● 6 ● ● ●●●●● ● ●● ● ●● ● ● ●● ● ● ● ●● ● ● ●●●●● ● ● ●● ● ● ● ●● ● ● ● ●● ●●●●● ● ● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ● ●●●● ● ● ●●●● ●● ● ● ● ● ● ●● ●●●● ●● ● ● ●● ●● ● ● ● ●● ● ●●●● ●● ● ● ●●● ●● ● ●● ● ● ●●●●●● ● ● ● ● ● ● ●● ●●●● ● ● ● ● ●● ●●●● ● ● ● ● ● ● ● ● ● ● ● ●●●●●● ● ● ● ●● ● ●● ●●●● ● ● ● ● ●● ● ● ●● ● ●● ●●●● ●●●●● ● ● ● ●●●● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ●●● ●● ● ●● ● ● ●● ● ● ● ●● ●●● ● ● ●● ● ● ● ●●● 4 ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ● ● ●● ●● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ●● ● ● ●● ●● ● ● 2 ● ● 16 17 18 19 20 ● ● ● ● ● 10 ●● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●●● ● ● ● 8 ●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●●● ● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●●●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ●●●● ● ●● ● ●●● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ●●● ●● ● ● ●● ● ●● ● ● ● ●●● ● ● ● ● ●●●● ● ● ● ● ●●●● ●● ●●●● ●● ●●●● ● ● ● ●●●●●● ● ● ● ● ● ● ●● ●●● ● ●● ●● ● ● ● ●● ● ●●●● ● ● ● ●● ● ● ●● ● ●●●● ● ● ● ● ● ●●● ●●●● ● ● ● ●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ●●●● ● ● ● ● ● ● ●●●●●● ●● ● ● ● ●●●● ●● 6 ●●● ● ●● ●● ● ●● ● ● ● ●●●● ●● ● ● ● ●● ● ● ●● ● ●●●● ● ● ● ● ●●● ● ● ● ● ● ●● ● ●● ● ●●● ●● ● ● ● ● ● ● ●●●●●● ●● ● ●●● ● ● ● ●● ●●●● ●● ● ● ●●●● ● ● ● ● ● ●●●● ● ● ● ● ● ● ●● ● ● ● ● ● ●●●●● ● ● ●● ●● ● ●● ● ● ● ●●● ● ● ● ● ●● ● ● ● ●●● ● ● ●●● ● ● ●●●●●●● ● ● ● ●● ●●●● ● ● ● ●● ● ●●●● ● ●● ● ● ● ●●●● ● ● ●● ● ● ● ●●● ● ● ● ● ● ●● ● ● ●●● ●● ● ● ● ●● ●● ● ● ● ●● ● ●●● ● ●●●● ● ● ● ● ● 4 ● ●● ● ●● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ●● ●● ● ●●●● ● ● ● ●● ●● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● 2 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 displ
  • 23. 1 2 3 4 5 ● ● ● ●● ● ●● 10 ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ●● ● ●●● ●● ● ● ● ● ● ● ●●● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ● ●● ● ● ●●●●● ● ● ●●● ● ●● ●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ●●● ● ● ● ●● ● 8 ●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ●●● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ●●●●● ● ●●●● ●● ● ● ●● ● ● ●●● ● ● ●●● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ●●●●●●● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ●●●● ● ● ●●●● ●● ● ●●●● ●● ● ● ●● ●● ● ● ●● ●● ● ● ● ● ●● ●●●●● ● ●●● ● ● ●● ● ● ●● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ●●●● ● ● 6 ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ●● ● ●● ● ● ● ●●●●● ●● ● ●● ● ●● ● ● ●●●● ●● ● ● ● ●●● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ●●●●● ● ● ●● ● ● ● ●●●●●●● ●●●●●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ●● ● ● ● ●●●● ● ● ●● ●● ● ●●●● ● ● ● ● ● ●●● ● ● ●● ● ●●● ● ●●●● ●● ●●● ● ● ●●●● ●● ● ● ● ●● ● ●● ● ●●●● ● ● ●●●● ●● ● ● ● ● ●●●●●● ●● ● ● ●● ● ●●● ● ● ● ●● ●●●● ● ● ● ● ●● ●●●● ● ● ●●● ● ● ●● ●●●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ●●● ● ● ● ● ●●● ● ●● ●●● ● 4 ● ● ● ●● ●●● ●● ●● ● ● ●●● ● ● ● ● ● ● ●● ●●● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ●●●● ●● ● ● ●● ●● ● ● ●●● ● ● ●● 2 ● ● 6 7 8 9 10 ● ● ● ● ●● ●● ● ● ● ● 10 ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ●● ● ● ● ● ● ●● ● ●● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●●●● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ● ●●●●●● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ●● ● ●● ● ● ● ●● ● ●● ● ● ●●●●●● ● ● ●● ● ●● ● ●● ● ● ● ●●●●● ● ● ●●●● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ●●●● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ●● ● ● ●●● 6 ● ● ●●● ● ● ● ●●● ● ● ● ●● ● ●●●●● ● ● ● ●●●● ● ● ● ● ●●●● ● ● ● ● ●●● ● ● ●●●● ● ● ● ● ●●●● ● ● ●●●●● ● ● ●●● ● ● ●●● ● ●●●●● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ●●●● ●● ● ● ●● ● ● ● ● ●●●●●● ● ●●●● ●● ● ● ●● ● ● ● ●●●● ● ● ●● ● ● ●● ● ● ● ● ● ●●● ● ●●● ● ● ● ● ●●● ● ● ●●● ● ● ● ●●● ● ● ● ● ●●● ●● ●● ●●●● ●● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ●● ●●● ● ●●●●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ●●● ● ● ● ●●●●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● 4 ● ●● ● ● ●● ● ●● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ●● ● ●● ● ●● ● ● ●● ● ● ● ● ●●● ● ● ●● ● ● ● ● ● ● ● 2 factor(year) gp100m ● 11 12 13 14 15 ● 1999 ● ● ● ● ● ● ● ● ● ● 2008 10 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●●● ● ● ● ●●● ●● ● ●● ● ● ● ● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● 8 ● ● ● ● ● ● ●●●● ● ●●● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ●●●● ● ● ● ● ● ●●● ●● ● ● ●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●●● ● ● ● ● ●● ● ●●●● ●● ●●● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ●●● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●●●●●●● ●● ● ● ● ●● ● ●●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●●● ● ●● ● ●●●●●●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●●●● ● ●●●● ●● ● ● ● ● ●●●●● ● ● ● ●●● ●● ● ●● ●● ● ● ● ● ● ●●●● ● ● ● ● ● ● ●●●●● ● ● ● ●●● ● ● ●●● ● ● ●● ●●●● ● ●● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● 6 ● ● ●●●●● ● ●● ● ●● ● ● ●● ● ● ● ●● ● ● ●●●●● ● ● ●● ● ● ● ●● ● ● ● ●● ●●●●● ● ● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ● ●●●● ● ● ●●●● ●● ● ● ● ● ● ●● ●●●● ●● ● ● ●● ●● ● ● ● ●● ● ●●●● ●● ● ● ●●● ●● ● ●● ● ● ●●●●●● ● ● ● ● ● ● ●● ●●●● ● ● ● ● ●● ●●●● ● ● ● ● ● ● ● ● ● ● ● ●●●●●● ● ● ● ●● ● ●● ●●●● ● ● ● ● ●● ● ● ●● ● ●● ●●●● ●●●●● ● ● ● ●●●● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ●●● ●● ● ●● ● ● ●● ● ● ● ●● ●●● ● ● ●● ● ● ● ●●● 4 ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ● ● ●● ●● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ●● ● ● ●● ●● ● ● 2 ● ● 16 17 18 19 20 ● ● ● ● ● 10 ●● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●●● ● ● ● 8 ●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●●● ● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●●●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ●●●● ● ●● ● ●●● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ●●● ●● ● ● ●● ● ●● ● ● ● ●●● ● ● ● ● ●●●● ● ● ● ● ●●●● ●● ●●●● ●● ●●●● ● ● ● ●●●●●● ● ● ● ● ● ● ●● ●●● ● ●● ●● ● ● ● ●● ● ●●●● ● ● ● ●● ● ● ●● ● ●●●● ● ● ● ● ● ●●● ●●●● ● ● ● ●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ●●●● ● ● ● ● ● ● ●●●●●● ●● ● ● ● ●●●● ●● 6 ●●● ● ●● ●● ● ●● ● ● ● ●●●● ●● ● ● ● ●● ● ● ●● ● ●●●● ● ● ● ● ●●● ● ● ● ● ● ●● ● ●● ● ●●● ●● ● ● ● ● ● ● ●●●●●● ●● ● ●●● ● ● ● ●● ●●●● ●● ● ● ●●●● ● ● ● ● ● ●●●● ● ● ● ● ● ● ●● ● ● ● ● ● ●●●●● ● ● ●● ●● ● ●● ● ● ● ●●● ● ● ● ● ●● ● ● ● ●●● ● ● ●●● ● ● ●●●●●●● ● ● ● ●● ●●●● ● ● ● ●● ● ●●●● ● ●● ● ● ● ●●●● ● ● ●● ● ● ● ●●● ● ● ● ● ● ●● ● ● ●●● ●● ● ● ● ●● ●● ● ● ● ●● ● ●●● ● ●●●● ● ● ● ● ● 4 ● ●● ● ●● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ●● ●● ● ●●●● ● ● ● ●● ●● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● 2 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 displ
  • 24. mpg$gp100m <- 100 / mpg$cty qplot(displ,gp100m, data = mpg, colour = factor(year)) mpg_rotate <- resamp(mpg, null_model(gp100m ~ displ)) last_plot() %+% mpg_rotate + facet_wrap(~ .n)
  • 25. Maybe there are fewer bigger cars?
  • 26. 35 30 25 factor(year) count 20 1999 2008 15 10 5 2 3 4 5 6 7 displ
  • 27. 1 2 3 4 5 40 30 20 10 6 7 8 9 10 40 30 20 10 factor(year) count 11 12 13 14 15 1999 40 2008 30 20 10 16 17 18 19 20 40 30 20 10 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 displ
  • 28. qplot(displ, data = mpg, colour = factor(year) geom = quot;freqpolyquot;, binwidth = 1) last_plot() %+% mpg_perm + facet_wrap(~ .n)
  • 29. Future work Methods for more null hypotheses. Is it possible to guess plausible null hypotheses from the plot specification?