Description of interactive freely accessible online quantitative toolkit for linguistic data analysis and visualization. Step-by-step guide how to use this online tool and interpret results.

1. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Optimizing Language Variation Analysis:
Language Variation Suite
Olga Scrivner, Manuel D´ıaz-Campos and Rafael Orozco
obscrivn@indiana.edu mdiazcam@indiana.edu rorozc1@lsu.edu
Indiana University and Louisiana State University
NWAV45, 2016
1 / 93

2. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Goal
Provide researchers with a variety of quantitative methods to
advance language variation studies.
PositionSentence
p < 0.001
1
ind, pre post
Heaviness
p = 0.003
2
≤ 1 > 1
Period
p < 0.001
3
≤ 1 > 1
Node 4 (n = 81)
VOOV
0
0.2
0.4
0.6
0.8
1
Node 5 (n = 119)
VOOV
0
0.2
0.4
0.6
0.8
1
Node 6 (n = 181)
VOOV
0
0.2
0.4
0.6
0.8
1
Period
p < 0.001
7
≤ 2 > 2
Node 8 (n = 221)
VOOV
0
0.2
0.4
0.6
0.8
1
Focus
p < 0.001
9
cf nf
Node 10 (n = 66)
VOOV
0
0.2
0.4
0.6
0.8
1
Main_Verb_Structure
p < 0.001
11
ACIOther, Restructuring
Node 12 (n = 43)
VOOV
0
0.2
0.4
0.6
0.8
1
Node 13 (n = 265)
VOOV
0
0.2
0.4
0.6
0.8
1
2 / 93

3. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Objectives
1 Introduce a novel sociolinguistic toolkit
2 Develop practical quantitative analytical skills
3 Understand and interpret advanced statistical models
3 / 93

4. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
What is LVS?
Language Variation Suite
It is a Shiny web application designed for data analysis in
sociolinguistic research.
It can be used for:
Processing spreadsheet data
Reporting in tables and graphs
Analyzing means, regression, conditional trees and much
more
4 / 93

5. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Background
LVS is built in R using Shiny package:
1 R - a free programming language for statistical computing
and graphics
2 Shiny App - a web application framework for R
Computational power of R + Web interactivity
5 / 93

6. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Background
http://littleactuary.github.io/blog/Web-application-framework-with-Shiny/
6 / 93

7. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Data Preparation
Important things to consider before data entry:
File format:
Comma separated value (CSV) facilitates faster processing
Excel format will slow processing
Column names should not contain spaces
Permitted: non-accented characters, numbers, underscore,
hyphen, and period
One column must contain your dependent variable
The rest of the columns contain independent variables
7 / 93

8. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Workspace
Browser
Chrome, Firefox, Safari - recommendable
Explorer may cause instability issues
Accessibility
PC, Mac, Linux
Data files can be uploaded from any location on your
computer
Smart Phone
Data files must be on a cloud platform connected to your
phone account (e.g. dropbox)
8 / 93

9. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Terminology Review
a. Categorical - non-numerical data with two values
yes - no; deletion - retention; perfective - imperfective
b. Continuous - numerical data
duration, age, chronological period
c. Multinomial - non-numerical data with three or more
values
deletion - aspiration - retention
d. Ordinal - scale: currently not supported
9 / 93

10. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Workshop Files
https://languagevariationsuite.wordpress.com/
1 categoricaldata.csv: categorical dependent - Labov New
York 1966 study
2 continuousdata.csv: continuous dependent - Intervocalic
/d/ in Caracas corpus (D´ıaz-Campos et al.)
3 LVS web site:
https://languagevariationsuite.shinyapps.io/Pages/
10 / 93

11. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Workshop Files
https://languagevariationsuite.wordpress.com/
1 categoricaldata.csv: categorical dependent - Labov New
York 1966 study
2 continuousdata.csv: continuous dependent - Intervocalic
/d/ in Caracas corpus (D´ıaz-Campos et al.)
3 LVS web site:
https://languagevariationsuite.shinyapps.io/Pages/
10 / 93

12. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Language Variation Suite - Structure
1 Demo
Brief introduction
2 Data
Upload file, data summary, adjust data, cross tabulation
3 Visual Analysis
Plotting - histograms, frequencies, cluster plots
4 RBRUL
New version by Daniel Johnson!
5 Inferential statistics
Modeling, regression, conditional trees, random forest,
model comparison
11 / 93

13. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Language Variation Suite - Structure
1 Demo
Brief introduction
2 Data
Upload file, data summary, adjust data, cross tabulation
3 Visual Analysis
Plotting - histograms, frequencies, cluster plots
4 RBRUL
New version by Daniel Johnson!
5 Inferential statistics
Modeling, regression, conditional trees, random forest,
model comparison
11 / 93

14. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Language Variation Suite - Data
1 Upload CSV file
2 Upload Excel file
12 / 93

15. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Excel Format
1 Slow processing
2 Requires the name of your excel sheet
13 / 93

16. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Save Excel as CSV Format
To optimize speed - Save as CSV prior upload
14 / 93

17. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Server
Since LVS is hosted on a server, Shiny idle time-out settings
may stop the application when it is left inactive (it will grey
out).
Solution: Click reload and re-upload your csv data
15 / 93

18. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Upload File
Upload categoricaldata.csv
16 / 93

19. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Table
Table displays our dataset and allows for sorting columns in
descending/ascending order.
17 / 93

20. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Summary
Summary provides a quantitative summary for each variable,
e.g. frequency count, mean, median.
18 / 93

21. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Data Structure
19 / 93

22. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Data Structure
1 Total number of observations
2 Number of variables
3 Variable types
Factor - categorical values
Num - numeric values (0.95, 1.05)
Int - integer values (1, 2, 3)20 / 93

23. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Questions?
21 / 93

24. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Language Variation Suite - Structure
1 Demo
Brief introduction
2 Data
Upload file, data summary, adjust data, cross tabulation
3 Visual Analysis
Plotting, cluster classification
4 RBRUL
New version by Daniel Johnson!
5 Inferential statistics
Modeling, regression, conditional trees, random forest,
model comparison
22 / 93

25. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Visual Analytics
Visual Analytics: “The science of analytical reasoning
facilitated by visual interactive interfaces”
(Thomas et al. 2005)
23 / 93

26. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
One Variable Plot
24 / 93

27. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
One Variable Plot
25 / 93

28. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Two Variables Plot
26 / 93

29. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Two Variables Plot
27 / 93

30. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Three Variables Plot
28 / 93

31. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Three Variables Plot
29 / 93

32. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Cluster Plot
Classification of data into sub-groups is based on
pairwise similarities
Groups are clustered in the form of a tree-like
dendrogram
Independent variable must have at least THREE values
(e.g. store - Saks, Kleins, Macy’s)
30 / 93

33. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Cluster Plot
31 / 93

34. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Cluster Plot
Saks (upper middle-class store), Macy’s (middle-class store), Kleins
(working-class)
32 / 93

35. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Inferential Statistics
33 / 93

36. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Language Variation Suite - Structure
1 Demo
Brief introduction
2 Data
Upload file, data summary, adjust data, cross tabulation
3 Visual Analysis
Plotting, cluster classification
4 RBRUL
New version by Daniel Johnson!
5 Inferential statistics
Modeling, regression, conditional trees, random forest,
model comparison
34 / 93

37. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Modeling
35 / 93

38. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Modeling
35 / 93
We are interested in RETENTION
= Application

39. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Regression Types
Model
a.) Fixed effects
b.) Mixed effects - individual speaker/token variation (within
group)
Type of Dependent Variable
a.) Binary/categorical (only two values)
b.) Continuous (numeric)
c.) Multinomial - categorical with more than two values
36 / 93

40. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Regression
37 / 93

41. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Model Output
38 / 93

42. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Interpretation
1 Estimate: reported in log-odds: negative or positive effect
closer to zero - lesser effect
2 P - significance (p < 0.05)
39 / 93

43. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Interpretation
Lexical item Fourth has a negative effect on retention and is
significant
Normal style has a slightly negative effect on retention but its
coefficient is not significant
Macy’s and Saks have a positive and significant effect on
retention. Saks (upper middle class store) is more significant
than Macy’s (middle class store)
http://www.free-online-calculator-use.com/scientific-notation-converter.html40 / 93

44. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Interpretation
Lexical item Fourth has a negative effect on retention and is
significant
Normal style has a slightly negative effect on retention but its
coefficient is not significant
Macy’s and Saks have a positive and significant effect on
retention. Saks (upper middle class store) is more significant
than Macy’s (middle class store)
http://www.free-online-calculator-use.com/scientific-notation-converter.html40 / 93
exponential notation:
1.48e-8
0.0000000148

45. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Questions?
41 / 93

46. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
New Tools of Linguistic Analysis (Baayen 2008,
Tagliamonte 2014, Gries 2015)
Conditional inference trees and Random Forests
“Proves to be more stable than stepwise variable selection
approaches available for logistic regression” (Strobl
2009:325)
Can handle skewed data that often violate the
assumptions of regression approaches
42 / 93

47. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Conditional Tree
Conditional tree: a simple non-parametric regression analysis,
commonly used in social and psychological studies
Linear regression: all information is combined linearly
Conditional tree regression: visual splitting to capture
interaction between variables
Recursive splitting (tree branches)
43 / 93

48. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Conditional Tree - Tagliamonte and Baayen 2012
1 The distribution of was/were is split into two groups by
individuals.
2 The variant were occurs significantly more frequently with the
first group.
44 / 93

49. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Conditional Tree - Tagliamonte and Baayen (2012)
1 Polarity is relevant to the second group of individuals.
2 The variant were occurs significantly more often with negative
polarity
45 / 93

50. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Conditional Tree - Tagliamonte and Baayen (2012)
1 Affirmative Polarity is conditioned by Age.
2 The variant was is produced significantly more often by
Individuals of 46 and younger.
46 / 93

51. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Conditional Tree
47 / 93

52. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Conditional Tree
1 Store is the most significant factor for R-use
Kleins (working class store) - more R-deletion
2 R-use in Macy’s and Saks is conditioned by lexical item:
Floor shows more R-retention than Fourth
3 Style is not significant
48 / 93

53. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Random Forest
1 Variable importance for predictors
2 Robust technique with small n large p data
3 All predictors considered jointly (allows for inclusion of
correlated factors)
49 / 93

54. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Random Forest
50 / 93

55. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Random Forest
1 Store is the most important predictor
2 Lexical Item is the second predictor
3 Style is irrelevant: close to zero and red dotted line (cut-off
value).
51 / 93

56. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Let’s Have a Short Break
52 / 93

57. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Preparing Data
1 Download continuousdata.csv
2 Upload this file on LVS
53 / 93

58. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Table
54 / 93

59. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Summary
55 / 93

60. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Summary
56 / 93

61. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Changing Class from Integer to Factor
57 / 93

62. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Change Class
58 / 93

63. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Adjusted Dataset
59 / 93

64. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Summary - New Dataset
60 / 93

65. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Continuous Variable - Histogram
61 / 93

66. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Density - Histogram
Density: a non-parametric model of the distribution of points based
on a smooth density estimate
http://scikit-learn.org/stable/modules/density.html
62 / 93

67. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Frequency Plot
63 / 93

68. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Frequency Plot - Word Cloud
64 / 93

69. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Frequency Plot
65 / 93

70. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Questions?
66 / 93

71. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Fixed and Mixed Effects Models
67 / 93
I’m ready for Mixed Models!

72. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Fixed and Mixed Models
Fixed Effects Model : All predictors are treated independently.
Underlying assumption - no group-internal
variation between speakers or tokens
Mixed Effects Model : Allows for evaluation of individual- and
group-level variation
68 / 93

73. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Fixed and Mixed Models: Errors
Fixed Regression Model - ignoring individual variations
(speakers or words) may lead to Type I Error:
“a chance effect is mistaken for a real difference
between the populations”
Mixed Regression Model - prone to Type II Error:
“if speaker variation is at a high level, we cannot
discern small population effects without a large
number of speakers” (Johnson 2009, 2015)
69 / 93

74. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Mixed Effect Regression
Mixed Model = fixed effects + random effects
Fixed-effects factor - “repeatable and a small number of levels”
Random-effects factor - “a non-repeatable random sample
from a larger population” (Wieling 2012)
70 / 93

75. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Mixed Effect Regression
Mixed Model = fixed effects + random effects
Fixed-effects factor - “repeatable and a small number of levels”
Random-effects factor - “a non-repeatable random sample
from a larger population” (Wieling 2012)
walk, sleep, study, finish, eat, etc
aspectual verb, stative verb
speaker1, speaker3, speaker3, etc
male, female
70 / 93

76. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Mixed Effect Regression
Mixed Model = fixed effects + random effects
Fixed-effects factor - “repeatable and a small number of levels”
Random-effects factor - “a non-repeatable random sample
from a larger population” (Wieling 2012)
walk, sleep, study, finish, eat, etc
aspectual verb, stative verb
speaker1, speaker3, speaker3, etc
male, female
70 / 93

77. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Mixed Effect Modeling
71 / 93
NULL when the dependent variable is continuous
Fixed Effects - independent variables

78. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Mixed Effect Modeling
72 / 93
Mixed Effects - group-internal variation

79. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Regression Results
73 / 93

80. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Interpretation - Random Effects
1 Standard Deviation: a measure of the variability for each
random effect (speakers and tokens)
2 Residual: random variation that is not due to speakers or
tokens (residual error)
74 / 93

81. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Interpretation - Fixed Effects
1 Estimate/coefficient: reported in log-odds (negative or
positive)
2 P-value: tells you if the level is significant
75 / 93

82. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Language Variation Suite - Structure
1 Demo
Brief introduction
2 Data
Upload file, data summary, adjust data, cross tabulation
3 Visual Analysis
Plotting, cluster classification
4 RBRUL
New version by Daniel Johnson!
5 Inferential statistics
Modeling, regression, varbrul analysis, conditional trees,
random forest
76 / 93

83. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
RBRUL 3.0 Beta
77 / 93

84. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Upload File
78 / 93

85. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Model Selection
79 / 93

86. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Model Selection
80 / 93

87. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Model Selection
81 / 93

88. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Output
82 / 93

89. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Application Values
83 / 93

90. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Questions?
84 / 93

91. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Appendix 1: Cross-Tabulation
Cross-tabulation examines the relationship between two
variables (their interaction).
85 / 93

92. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Cross-Tabulation: One Dependent and One
Independent Variables
86 / 93

93. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Cross-Tabulation Output
Raw frequency / Proportion by column / Proportion across row
87 / 93

94. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Appendix 2: Data Modification
88 / 93

95. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Adjust Data
Retain: Select data subset
Exclude: Exclude variables from a factor group
Recode: Combine and rename variables
Change class: Numeric → factor; factor → numeric
Transform: Apply log transformation to a specific column
ADJUSTED DATASET:
Run - to apply all above changes
Reset - to reset to the original dataset
89 / 93

96. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Exclude: Emphatic Style
90 / 93

97. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Adjusted Dataset
91 / 93

98. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
Adjusting Dataset
To revert to the original data, select RESET:
92 / 93

99. Introduction
Data
Preparation
Language
Variation
Suite
Working with
Data
Visual
Analytics
Inferential
Analysis
Data
Modification
Mixed Effects
RBRUL
Appendix
Cross Tabulation
Data
Modification
References
References I
[1] Baayen, Harald. 2008. Analyzing linguistic data: A practical introduction to statistics. Cambridge:
Cambridge University Press
[2] Bentivoglio, Paola and Mercedes Sedano. 1993. Investigaci´on socioling¨u´ıstica: sus m´etodos aplicados a
una experiencia venezolana. Bolet´ın de Ling¨u´ıstica 8. 3-35
[3] Gries, Stefan Th. 2015. Quantitative designs and statistical techniques. In Douglas Biber Randi
Reppen (eds.), The Cambridge Handbook of English Corpus Linguistics. Cambridge: Cambridge
University Press
[4] Labov, W. 1966. The Social Stratification of English in New York City. Washington: Center for Applied
Linguistics
[5] http://gifsanimados.espaciolatino.com/x bob esponja 8.gif
[6] https://daniellestolt.files.wordpress.com/2013/01/are-you-ready1.gif
[7] http://www.martijnwieling.nl/R/sheets.pdf
93 / 93