Your SlideShare is downloading. ×
0
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Graphical Inference
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Graphical Inference

907

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
907
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. A graphical grammar + graphical inference = a grammar of graphical inference? Hadley Wickham, Rice University
  • 2. 1. Motivation 2. Resampling methods (graphical inference) 3. Graphical display (grammar of graphics) 4. Future work
  • 3. 5 8 6 ● ● ● 4 ● ● ● ● ● ● ● ● ● ● ● ● ● ●● 2 ● species ● ● ● ● Concinna ● ● ● ● ● Heikert. ● 0 ● ● ● ● ● ● ● ● Heptapot. ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● −2 ● ● ● ● ● ● ● ● ● ● ● ● ● −4 −6 −6 −4 −2 0 2 4 6
  • 4. 1 2 3 4 8 6 4 ● ●● ● ● 2 ●● ●●● ● ●● ● ●● ● ● ● ● ●●● ●● ● ●● ● ● ● ● ● ● ● ● ●●●●● ● ● ● ●● ●●● ●● ● ● ● ● ●● ● ●●● ●● ● ● ● ● ● ● ●● ● ● ● ● ●●●● ●● ● ● ●● ● ●●● ● ● ● ● ●● ●●● ●●●●● ● ●●● ●● ●●● ● ●●●● ● ● ●●●● ● ● 0 ● ● ● ●● ● ●● ● ●● ●● ●●● ● ●● ● ●● ● ● ●● ●● ●● ●● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ●●● ● ●● ●● ●● ● ●● ●● ●● ● ●● ● ●● ●● ●● ●● ● ● ● ● ●● ●●● ● ● ● ● ●● ●● ● ● ●●●●● ● ●● ●●● ● ●● ● ● −2 ●● ● ● ● ● −4 −6 5 6 7 8 8 6 4 ●● ● ● ● species ●●● ●● ● ●● ● ● ● ● ● 2 ●● ● ● ● ●● ● ● ●● ● ● ● ● ●● ●● ● Concinna ● ● ● ● ● ● ●● ●● ● ● ● ●● ●● ● ● ● ● ●● ●●●● ● ● ● ● ●● ● ●● ● ●● ● ● ●● ● ●● 0 ●● ●● ● ● ● ● ●●●● ● ●●● ●●● ● ●●●●● ●●●● ● ● ●● ●● ● ●● ● ● ● ●● ● ● ●● ● ●● ● ●●●● ●● ● ●● ●● ●● ● ● ● ●● ● ● ●●● ●● ● ● ●●●●● ● ● Heikert. ● ● ● ● ●● ● ●● ● ● ●● ● ●● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ●●●● ● ●●●● ● ●●● ●● ●● ● ● ●●●●● ● ● ● ● ● ● ● ● ●●● ● ● ● ●● ● ● ●●● ● ●● ● −2 ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ●● ● Heptapot. −4 −6 9 10 11 12 8 6 4 ● ● 2 ●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ●●●●● ● ● ●●● ● ●● ● ● ●●● ● ● ● ● ●● ●●● ● ● ● ●● ●● ● ● ● ● ● ●●●● ●● ●●● ● ● ● ● ● ● ●●●● ● ●● ●● ● ●●●●● ●● ● ●● ●●●● ●●●● ●●●●● ● ● ●● ● ●● ● 0 ●●● ●● ● ● ● ●● ●● ● ● ●● ● ● ●●●●● ● ●●●●● ● ● ●● ● ● ● ●●●● ● ● ● ● ●● ●● ● ● ● ●●●● ●● ●● ● ●● ●● ●● ●● ● ● ● ●● ● ●● ● ●●● ●● ●● ● ● ● ● ●●●● ● ● ● ● ●● ● ● ● ●●●● ● ●●● ●● ● ●●●● ● ● ●● ●●● ● ●●● ●● ●●● ● ●● ● ● ●● −2 ● ● ● −4 −6 −6 −4 −2 0 2 4 6 −6 −4 −2 0 2 4 6 −6 −4 −2 0 2 4 6 −6 −4 −2 0 2 4 6
  • 5. 1 2 3 4 ● 4 ● ● ● ● ● ●● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● 2 ● ● ● ● ● ● ● ●● ● ● ● ● ●● ●● ● ● ●●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ● ●●● ● ● ● ●● ● ● ● ●● ● ●● ● ●● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ●● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● 0 ● ● ● ● ● ● ● ●●●● ●●●● ● ●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ●● ● ●● ●●●● ●● ●● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●●● ● −2 ● ●● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● −4 5 6 7 8 ● 4 ● ● ● ● ● ● ● ● ●● ● ● ● species ● ● ● ● ● ● ●● ● 2 ● ●● ● ●● ●● ●● ● ● ● ●● ● ● ●● ● ● ●● ● ● ● ● ●●● ●●● ● ●● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ●● ●● ● ● ● ● ●● ● ●● ● ● ●● ● ● ●● ● ● ● Concinna ● ●● ● ● ●● ● ● ● ● ●●● ●●● ● ● ● ● ●● ● ● ●● 0 ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ●● ● ●● ● ● ●● ● ● ●● ●●● ●● ● ● ●● ● ● ● ● ● ● ● Heikert. ●● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ●● ●● ● ● ● ● −2 ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● Heptapot. ● ● ● ● ● ● −4 9 10 11 12 4 ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● 2 ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ●●● ● ● ●● ● ● ● ●● ● ● ●● ● ●● ● ● ●● ●● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ●●● ● ● ● ● ● ●●● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● 0 ●● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ●● ●●●●●● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ●● ●● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●● −2 ● ● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● −4 −4 −2 0 2 4 −4 −2 0 2 4 −4 −2 0 2 4 −4 −2 0 2 4
  • 6. Problem Want to make it very easy to make these types of display - should be able to create as easy as doing as simple numerical test. With the right tools, it turns out to be really easy! Two basic problems: how to do resampling under the null, and how to define the graphic.
  • 7. Resampling Many resampling methods have particularly elegant expression in R code. For example, if null hypothesis is that of independence: df <- transform(df, resp = sample(resp)) However, better to generalise the pattern.
  • 8. Generalisation permute_var <- function(var) { function(df) { df[[var]] <- sample(df[[var]]) df } } f <- permute_var(quot;mpgquot;) f(mtcars) permute_var(quot;mpgquot;)(mtcars)
  • 9. Advantages Separate description from implementation. Call describes action. Can later replace implementation if better approach discovered. Unimportant details concealed.
  • 10. n resamples Each method gives us a single resample, but we need n. Trivial to repeat n times with rdply() from the plyr package. (rdply() is a generalisation of replicate() that returns a data frame with a column that labels each replicate.)
  • 11. Overall resamp <- function(true, method, n = 19, pos = sample(n + 1, 1)) { samples <- rdply(n, method(true)) if (missing(pos)) { message(quot;True data in position quot;, pos) } add_true(samples, true, pos) } add_true <- function(samples, true, n) { samples$.n <- with(samples, ifelse(.n >= n, .n + 1, .n)) true$.n <- n all <- rbind(samples, true) all[order(all$.n), ] } Will see application shortly
  • 12. Other nulls Need functions for other common null hypotheses. Experimental null_model(), which computes rotation residuals given a specified linear model. (Demo a little later)
  • 13. Display How is the plot of the simulated data with true data different from a single plot? Just need to repeat the same display n times and label appropriately. This is easy if we have a description of the plot, independent of the data.
  • 14. Grammar of Graphics This is one principle of the grammar of graphics: should describe the graphic we want, not how to create it. Implemented with the ggplot2 package in R. Easy to modify a graphic after it has been created. For graphical inference, we need to change the data and facetting.
  • 15. Example
  • 16. ● 35 ● 30 ● ● ● ● ● ● 25 ● ● ● ● ● ● ● ● ● factor(year) ●● ● cty ● ● ● ● ● ●● ● ● 1999 ● ●● ●●●● ●● ● ● ●● ● ● 20 ● ● ● ● ● ● 2008 ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ●● ● ● ● ● ●● ●●● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● 15 ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ●● ●● ● ●● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 10 ● ● ● ● 2 3 4 5 6 7 displ
  • 17. ● 10 ● ● ●● ●●● ● ● ● ● ● ● ● ● ● 8 ●● ●● ● ● ● 1/cty * 100 ● ● ●● ● ● ●● ● factor(year) ● 1999 ●● ● ●● ●● ● ● ● ● ● ● 2008 ● ●● ● ● ● ● ● ● ● 6 ●● ●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● 4 ● ● ● ● ● ● ● ● 2 3 4 5 6 7 displ
  • 18. 1 2 3 4 5 ● ● ● ● ● 10 ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● 6 ● ●●●● ●● ●●● ● ● ●● ●●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ● ●●● ● ●●● ● ● ●● ●● ● ●● ● ●● ●● ● ●●●● ●● ●●● ● ● ●●● ●●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ● ●●● ● ●●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ●● ●●●● ● ● ● ●● ●● ● ●● ● ●● ● ● ●● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●●● ● ● ● ●● ●● ● ● ●●● ● ● ●● ●●● ● ● ●● ●●● ● ● ●● 4 ●● ● ●● ● ●● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● 6 7 8 9 10 ● ● ● ● ● 10 ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●●● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● 6 ● ●●●● ●● ● ●●● ● ● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ●● ●●●● ● ●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ●●● ● ● ●●● ●● ● ● ●● ●● ● ●● ● ●● ●● ● ●●●● ●● ●●● ● ● ●●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ●●●● ● ● ● ●● ●● ● ●● ● ●● ●● ● ● ●● ● ● ● ● ●● ● ●● ● ● ●● ● ● ●● ●●● ●● ● ● ●● ● ● ● ● ●● ● ●● ● ● ●●● ● ● ● ●●● ● ● ● ●● ● ●● ●●● ● ● ● ●●● ● ● ● 4 1/cty * 100 ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● factor(year) 11 12 13 14 15 ● 1999 ● ● ● ● ● ● 2008 10 ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● 6 ● ●●●● ●● ● ●●● ● ●●● ● ● ●● ●● ● ●● ● ●●●● ●● ●●● ● ● ●● ●●● ● ● ●● ●● ●●●●● ● ● ●●●● ●● ●●● ● ● ●●● ● ● ●● ●● ● ●● ● ●●●● ●● ●●● ●●●● ● ● ● ●● ●● ●●●● ● ●●●● ●● ●●● ●●●● ● ●● ● ● ●● ●● ●●●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● 4 ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● 16 17 18 19 20 ● ● ● ● ● 10 ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● 6 ● ●●●● ●● ●● ●●●● ● ● ● ●● ●● ●●●●● ● ● ●●●● ●● ● ●●● ● ● ● ● ●● ●● ● ●● ● ●●●● ●● ●●● ● ● ●●● ● ● ●● ●● ●●●●● ● ●●●● ●● ● ●●● ● ●● ● ● ● ●● ●● ● ●● ● ●●●● ●● ●●● ● ● ●● ●●● ● ● ●● ●● ● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● 4 ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 displ
  • 19. 1 2 3 4 5 ● ● ● ● ● 10 ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● 6 ● ●●●● ●● ●●● ● ● ●● ●●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ● ●●● ● ●●● ● ● ●● ●● ● ●● ● ●● ●● ● ●●●● ●● ●●● ● ● ●●● ●●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ● ●●● ● ●●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ●● ●●●● ● ● ● ●● ●● ● ●● ● ●● ● ● ●● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ●●● ● ● ● ●● ●● ● ● ●●● ● ● ●● ●●● ● ● ●● ●●● ● ● ●● 4 ●● ● ●● ● ●● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● 6 7 8 9 10 ● ● ● ● ● 10 ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●●● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● 6 ● ●●●● ●● ● ●●● ● ● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ●● ●●●● ● ●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ●●● ● ● ●●● ●● ● ● ●● ●● ● ●● ● ●● ●● ● ●●●● ●● ●●● ● ● ●●● ● ● ●● ●● ● ●● ● ●● ● ●●●● ●● ●●●● ● ● ● ●● ●● ● ●● ● ●● ●● ● ● ●● ● ● ● ● ●● ● ●● ● ● ●● ● ● ●● ●●● ●● ● ● ●● ● ● ● ● ●● ● ●● ● ● ●●● ● ● ● ●●● ● ● ● ●● ● ●● ●●● ● ● ● ●●● ● ● ● 4 1/cty * 100 ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● factor(year) 11 12 13 14 15 ● 1999 ● ● ● ● ● ● 2008 10 ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● 6 ● ●●●● ●● ● ●●● ● ●●● ● ● ●● ●● ● ●● ● ●●●● ●● ●●● ● ● ●● ●●● ● ● ●● ●● ●●●●● ● ● ●●●● ●● ●●● ● ● ●●● ● ● ●● ●● ● ●● ● ●●●● ●● ●●● ●●●● ● ● ● ●● ●● ●●●● ● ●●●● ●● ●●● ●●●● ● ●● ● ● ●● ●● ●●●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● 4 ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● 16 17 18 19 20 ● ● ● ● ● 10 ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ● 6 ● ●●●● ●● ●● ●●●● ● ● ● ●● ●● ●●●●● ● ● ●●●● ●● ● ●●● ● ● ● ● ●● ●● ● ●● ● ●●●● ●● ●●● ● ● ●●● ● ● ●● ●● ●●●●● ● ●●●● ●● ● ●●● ● ●● ● ● ● ●● ●● ● ●● ● ●●●● ●● ●●● ● ● ●● ●●● ● ● ●● ●● ● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● ●●● ● ● ● 4 ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 displ
  • 20. qplot(displ, cty, data = mpg, colour = factor(year)) qplot(displ, 1 / cty * 100, data = mpg, colour = factor(year)) mpg_perm <- resamp(mpg,permute_var(quot;yearquot;)) last_plot() %+% mpg_perm + facet_wrap(~ .n)
  • 21. Is a linear model with displacement as single predictor adequate?
  • 22. 1 2 3 4 5 ● ● ● ●● ● ●● 10 ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ●● ● ●●● ●● ● ● ● ● ● ● ●●● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ● ●● ● ● ●●●●● ● ● ●●● ● ●● ●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ●●● ● ● ● ●● ● 8 ●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ●●● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ●●●●● ● ●●●● ●● ● ● ●● ● ● ●●● ● ● ●●● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ●●●●●●● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ●●●● ● ● ●●●● ●● ● ●●●● ●● ● ● ●● ●● ● ● ●● ●● ● ● ● ● ●● ●●●●● ● ●●● ● ● ●● ● ● ●● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ●●●● ● ● 6 ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ●● ● ●● ● ● ● ●●●●● ●● ● ●● ● ●● ● ● ●●●● ●● ● ● ● ●●● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ●●●●● ● ● ●● ● ● ● ●●●●●●● ●●●●●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ●● ● ● ● ●●●● ● ● ●● ●● ● ●●●● ● ● ● ● ● ●●● ● ● ●● ● ●●● ● ●●●● ●● ●●● ● ● ●●●● ●● ● ● ● ●● ● ●● ● ●●●● ● ● ●●●● ●● ● ● ● ● ●●●●●● ●● ● ● ●● ● ●●● ● ● ● ●● ●●●● ● ● ● ● ●● ●●●● ● ● ●●● ● ● ●● ●●●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ●●● ● ● ● ● ●●● ● ●● ●●● ● 4 ● ● ● ●● ●●● ●● ●● ● ● ●●● ● ● ● ● ● ● ●● ●●● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ●●●● ●● ● ● ●● ●● ● ● ●●● ● ● ●● 2 ● ● 6 7 8 9 10 ● ● ● ● ●● ●● ● ● ● ● 10 ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ●● ● ● ● ● ● ●● ● ●● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●●●● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ● ●●●●●● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ●● ● ●● ● ● ● ●● ● ●● ● ● ●●●●●● ● ● ●● ● ●● ● ●● ● ● ● ●●●●● ● ● ●●●● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ●●●● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ●● ● ● ●●● 6 ● ● ●●● ● ● ● ●●● ● ● ● ●● ● ●●●●● ● ● ● ●●●● ● ● ● ● ●●●● ● ● ● ● ●●● ● ● ●●●● ● ● ● ● ●●●● ● ● ●●●●● ● ● ●●● ● ● ●●● ● ●●●●● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ●●●● ●● ● ● ●● ● ● ● ● ●●●●●● ● ●●●● ●● ● ● ●● ● ● ● ●●●● ● ● ●● ● ● ●● ● ● ● ● ● ●●● ● ●●● ● ● ● ● ●●● ● ● ●●● ● ● ● ●●● ● ● ● ● ●●● ●● ●● ●●●● ●● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ●● ●●● ● ●●●●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ●●● ● ● ● ●●●●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● 4 ● ●● ● ● ●● ● ●● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ●● ● ●● ● ●● ● ● ●● ● ● ● ● ●●● ● ● ●● ● ● ● ● ● ● ● 2 factor(year) gp100m ● 11 12 13 14 15 ● 1999 ● ● ● ● ● ● ● ● ● ● 2008 10 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●●● ● ● ● ●●● ●● ● ●● ● ● ● ● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● 8 ● ● ● ● ● ● ●●●● ● ●●● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ●●●● ● ● ● ● ● ●●● ●● ● ● ●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●●● ● ● ● ● ●● ● ●●●● ●● ●●● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ●●● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●●●●●●● ●● ● ● ● ●● ● ●●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●●● ● ●● ● ●●●●●●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●●●● ● ●●●● ●● ● ● ● ● ●●●●● ● ● ● ●●● ●● ● ●● ●● ● ● ● ● ● ●●●● ● ● ● ● ● ● ●●●●● ● ● ● ●●● ● ● ●●● ● ● ●● ●●●● ● ●● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● 6 ● ● ●●●●● ● ●● ● ●● ● ● ●● ● ● ● ●● ● ● ●●●●● ● ● ●● ● ● ● ●● ● ● ● ●● ●●●●● ● ● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ● ●●●● ● ● ●●●● ●● ● ● ● ● ● ●● ●●●● ●● ● ● ●● ●● ● ● ● ●● ● ●●●● ●● ● ● ●●● ●● ● ●● ● ● ●●●●●● ● ● ● ● ● ● ●● ●●●● ● ● ● ● ●● ●●●● ● ● ● ● ● ● ● ● ● ● ● ●●●●●● ● ● ● ●● ● ●● ●●●● ● ● ● ● ●● ● ● ●● ● ●● ●●●● ●●●●● ● ● ● ●●●● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ●●● ●● ● ●● ● ● ●● ● ● ● ●● ●●● ● ● ●● ● ● ● ●●● 4 ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ● ● ●● ●● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ●● ● ● ●● ●● ● ● 2 ● ● 16 17 18 19 20 ● ● ● ● ● 10 ●● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●●● ● ● ● 8 ●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●●● ● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●●●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ●●●● ● ●● ● ●●● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ●●● ●● ● ● ●● ● ●● ● ● ● ●●● ● ● ● ● ●●●● ● ● ● ● ●●●● ●● ●●●● ●● ●●●● ● ● ● ●●●●●● ● ● ● ● ● ● ●● ●●● ● ●● ●● ● ● ● ●● ● ●●●● ● ● ● ●● ● ● ●● ● ●●●● ● ● ● ● ● ●●● ●●●● ● ● ● ●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ●●●● ● ● ● ● ● ● ●●●●●● ●● ● ● ● ●●●● ●● 6 ●●● ● ●● ●● ● ●● ● ● ● ●●●● ●● ● ● ● ●● ● ● ●● ● ●●●● ● ● ● ● ●●● ● ● ● ● ● ●● ● ●● ● ●●● ●● ● ● ● ● ● ● ●●●●●● ●● ● ●●● ● ● ● ●● ●●●● ●● ● ● ●●●● ● ● ● ● ● ●●●● ● ● ● ● ● ● ●● ● ● ● ● ● ●●●●● ● ● ●● ●● ● ●● ● ● ● ●●● ● ● ● ● ●● ● ● ● ●●● ● ● ●●● ● ● ●●●●●●● ● ● ● ●● ●●●● ● ● ● ●● ● ●●●● ● ●● ● ● ● ●●●● ● ● ●● ● ● ● ●●● ● ● ● ● ● ●● ● ● ●●● ●● ● ● ● ●● ●● ● ● ● ●● ● ●●● ● ●●●● ● ● ● ● ● 4 ● ●● ● ●● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ●● ●● ● ●●●● ● ● ● ●● ●● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● 2 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 displ
  • 23. 1 2 3 4 5 ● ● ● ●● ● ●● 10 ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ●● ● ●●● ●● ● ● ● ● ● ● ●●● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ● ●● ● ● ●●●●● ● ● ●●● ● ●● ●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ●●● ● ● ● ●● ● 8 ●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ●●● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ●●●●● ● ●●●● ●● ● ● ●● ● ● ●●● ● ● ●●● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ●●●●●●● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ●●●● ● ● ●●●● ●● ● ●●●● ●● ● ● ●● ●● ● ● ●● ●● ● ● ● ● ●● ●●●●● ● ●●● ● ● ●● ● ● ●● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ●●●● ● ● 6 ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ●● ● ●● ● ● ● ●●●●● ●● ● ●● ● ●● ● ● ●●●● ●● ● ● ● ●●● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ●●●●● ● ● ●● ● ● ● ●●●●●●● ●●●●●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ●● ● ● ● ●●●● ● ● ●● ●● ● ●●●● ● ● ● ● ● ●●● ● ● ●● ● ●●● ● ●●●● ●● ●●● ● ● ●●●● ●● ● ● ● ●● ● ●● ● ●●●● ● ● ●●●● ●● ● ● ● ● ●●●●●● ●● ● ● ●● ● ●●● ● ● ● ●● ●●●● ● ● ● ● ●● ●●●● ● ● ●●● ● ● ●● ●●●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ●●● ● ● ● ● ●●● ● ●● ●●● ● 4 ● ● ● ●● ●●● ●● ●● ● ● ●●● ● ● ● ● ● ● ●● ●●● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ●●●● ●● ● ● ●● ●● ● ● ●●● ● ● ●● 2 ● ● 6 7 8 9 10 ● ● ● ● ●● ●● ● ● ● ● 10 ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● 8 ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ●● ● ● ● ● ● ●● ● ●● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●●●● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ● ●●●●●● ●● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ●● ● ●● ● ● ● ●● ● ●● ● ● ●●●●●● ● ● ●● ● ●● ● ●● ● ● ● ●●●●● ● ● ●●●● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ●●●● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ●● ● ● ●●● 6 ● ● ●●● ● ● ● ●●● ● ● ● ●● ● ●●●●● ● ● ● ●●●● ● ● ● ● ●●●● ● ● ● ● ●●● ● ● ●●●● ● ● ● ● ●●●● ● ● ●●●●● ● ● ●●● ● ● ●●● ● ●●●●● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ●●●● ●● ● ● ●● ● ● ● ● ●●●●●● ● ●●●● ●● ● ● ●● ● ● ● ●●●● ● ● ●● ● ● ●● ● ● ● ● ● ●●● ● ●●● ● ● ● ● ●●● ● ● ●●● ● ● ● ●●● ● ● ● ● ●●● ●● ●● ●●●● ●● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ●● ●●● ● ●●●●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ●●● ● ● ● ●●●●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● 4 ● ●● ● ● ●● ● ●● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ●● ● ●● ● ●● ● ● ●● ● ● ● ● ●●● ● ● ●● ● ● ● ● ● ● ● 2 factor(year) gp100m ● 11 12 13 14 15 ● 1999 ● ● ● ● ● ● ● ● ● ● 2008 10 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●●● ● ● ● ●●● ●● ● ●● ● ● ● ● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● 8 ● ● ● ● ● ● ●●●● ● ●●● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ●●●● ● ● ● ● ● ●●● ●● ● ● ●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●●● ● ● ● ● ●● ● ●●●● ●● ●●● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ●●● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●●●●●●● ●● ● ● ● ●● ● ●●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●●● ● ●● ● ●●●●●●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●●●● ● ●●●● ●● ● ● ● ● ●●●●● ● ● ● ●●● ●● ● ●● ●● ● ● ● ● ● ●●●● ● ● ● ● ● ● ●●●●● ● ● ● ●●● ● ● ●●● ● ● ●● ●●●● ● ●● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● 6 ● ● ●●●●● ● ●● ● ●● ● ● ●● ● ● ● ●● ● ● ●●●●● ● ● ●● ● ● ● ●● ● ● ● ●● ●●●●● ● ● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ● ●●●● ● ● ●●●● ●● ● ● ● ● ● ●● ●●●● ●● ● ● ●● ●● ● ● ● ●● ● ●●●● ●● ● ● ●●● ●● ● ●● ● ● ●●●●●● ● ● ● ● ● ● ●● ●●●● ● ● ● ● ●● ●●●● ● ● ● ● ● ● ● ● ● ● ● ●●●●●● ● ● ● ●● ● ●● ●●●● ● ● ● ● ●● ● ● ●● ● ●● ●●●● ●●●●● ● ● ● ●●●● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ●●● ●● ● ●● ● ● ●● ● ● ● ●● ●●● ● ● ●● ● ● ● ●●● 4 ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ● ● ●● ●● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ●● ● ● ●● ●● ● ● 2 ● ● 16 17 18 19 20 ● ● ● ● ● 10 ●● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●●● ● ● ● 8 ●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●●● ● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●●●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ●●●● ● ●● ● ●●● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ●●● ●● ● ● ●● ● ●● ● ● ● ●●● ● ● ● ● ●●●● ● ● ● ● ●●●● ●● ●●●● ●● ●●●● ● ● ● ●●●●●● ● ● ● ● ● ● ●● ●●● ● ●● ●● ● ● ● ●● ● ●●●● ● ● ● ●● ● ● ●● ● ●●●● ● ● ● ● ● ●●● ●●●● ● ● ● ●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ●●●● ● ● ● ● ● ● ●●●●●● ●● ● ● ● ●●●● ●● 6 ●●● ● ●● ●● ● ●● ● ● ● ●●●● ●● ● ● ● ●● ● ● ●● ● ●●●● ● ● ● ● ●●● ● ● ● ● ● ●● ● ●● ● ●●● ●● ● ● ● ● ● ● ●●●●●● ●● ● ●●● ● ● ● ●● ●●●● ●● ● ● ●●●● ● ● ● ● ● ●●●● ● ● ● ● ● ● ●● ● ● ● ● ● ●●●●● ● ● ●● ●● ● ●● ● ● ● ●●● ● ● ● ● ●● ● ● ● ●●● ● ● ●●● ● ● ●●●●●●● ● ● ● ●● ●●●● ● ● ● ●● ● ●●●● ● ●● ● ● ● ●●●● ● ● ●● ● ● ● ●●● ● ● ● ● ● ●● ● ● ●●● ●● ● ● ● ●● ●● ● ● ● ●● ● ●●● ● ●●●● ● ● ● ● ● 4 ● ●● ● ●● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ●● ●● ● ●●●● ● ● ● ●● ●● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● 2 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 displ
  • 24. mpg$gp100m <- 100 / mpg$cty qplot(displ,gp100m, data = mpg, colour = factor(year)) mpg_rotate <- resamp(mpg, null_model(gp100m ~ displ)) last_plot() %+% mpg_rotate + facet_wrap(~ .n)
  • 25. Maybe there are fewer bigger cars?
  • 26. 35 30 25 factor(year) count 20 1999 2008 15 10 5 2 3 4 5 6 7 displ
  • 27. 1 2 3 4 5 40 30 20 10 6 7 8 9 10 40 30 20 10 factor(year) count 11 12 13 14 15 1999 40 2008 30 20 10 16 17 18 19 20 40 30 20 10 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 displ
  • 28. qplot(displ, data = mpg, colour = factor(year) geom = quot;freqpolyquot;, binwidth = 1) last_plot() %+% mpg_perm + facet_wrap(~ .n)
  • 29. Future work Methods for more null hypotheses. Is it possible to guess plausible null hypotheses from the plot specification?

×