1.
#pLASMA @SMLABTO
1
Learning in the Wild: Coding Reddit for
Learning and Practice
Priya Kumar, Ryerson University
Anatoliy Gruzd, Ryerson University
Caroline Haythornthwaite, Syracuse University
Sarah Gilbert, University of British Columbia
Marc Esteve del Valle, University of Groningen
Drew Paulin, University of California Berkeley
Hawaii International Conference on System Sciences (HICSS-51)
January 3-6, 2018

2.
How do we make sense of the vast amount of data
generated by learners, especially in the “wild”?
‘Learning analytics is concerned with
collecting data from learners’ actions,
developing techniques to analyze
these data, and making the results
useful to practitioners’ (Long &
Siemens, 2011).
Learning analytics focuses on making
sense of “big data” collected from
learning management systems (LMS),
MOOCs and social media
#pLASMA @SMLABTO 2

3.
We wanted to know
1. How are social media sites used in informal learning?
2. How are people engaging in informal learning processes on Reddit?
3. What are the different types of exploratory dialogue, behaviours and talks
that occur with learning processes on Reddit?
4. Does informal learning differ across subreddit communities?
#pLASMA @SMLABTO 3

4.
Introducing Reddit
• “the front page of the Internet”
• Reddit ranks 17th in terms of global traffic
and 4th in the U.S
• Diversity of niche communities called
‘subreddits’ maintain distinct subcultures,
thematic interests and subject focus
• Subreddit communities are user-generated
and moderated by Reddit members
• Content curation (Upvoting/downvoting)
#pLASMA @SMLABTO 4
‘AskHistorians’ Subreddit (May 15, 2017)

5.
Sampled Subreddits
Subreddit Mandate Guidelines and Norms
Askscience • Forum for answering science questions
• Promoting scientific literacy
• Help people understand scientific process
• No medical advice, speculation, anecdotes
• No abusive, offensive comments
• Answers should be supported with reputable
sources
Ask_politics • Promote political knowledge
• Disseminate knowledge about law and policy
• Be civil and on topic
• Reputable and sourced
• Free of layman speculation
Askhistorians • To provide serious academic-level answers to
questions about history
• Be civil, polite and courteous (no racist/sexist
talk)
• No current events, only human past
• Sources are highly encouraged
Askacademia • Questions and discussions about
Academia/Academic life
• Directed toward people interested in academia
(undergrad, grad, postdocs, profs, laymen)
• No off-topic unproductive discussions
• No rudeness, ad hominin attacks
• Treat strangers as you would in person

6.
Data Collection
Subreddit
Community
Number of
Moderators
Number of
Subscribers
Number of
Posts from 2016
Coded Sample
(1%)
Askscience 433 14M 223K 2,235
Ask_politics 8 26K 46K 464
AskHistorians 40 600K 122K 1,227
AskAcademia 3 32K 26.9K 269
#pLASMA @SMLABTO 6

7.
Analytic Framework: Discourse Analysis
Exploratory Dialogue Schema from Ferguson et al. (2013)
#pLASMA @SMLABTO 7
Cue Phrase Linguistic Dialogue Example
Critiques ‘However’, ‘I’m not sure’, ‘maybe’
Discussion of Resource ‘Have you read, ‘more links’
Evaluations ‘Good Example’, ‘good point’
Explanations ‘Means that’, ‘our goals’
Explicit Reasoning ‘Next step’, ‘relates to’, ‘that’s why’
Justifications ‘I mean’, ‘we learned’, ‘we observed’
Others’ Perspectives ‘Agree’, ‘here is another’

8.
Our Proposed Coding Schema
Code Definition
1. Explanation with Disagreement Expresses a NEGATIVE take on the content of the previous posts by adding new ideas or facts to
discussion thread.
2. Explanation with Agreement Expresses a POSITIVE take on the content of the previous posts by adding new ideas or facts to
discussion thread.
3. Explanation with Neutral
Presentation
Expresses a NEUTRAL explanation/judgment/reasoning/etc. with neither negative nor positive
reference to the content of the previous posts, nor necessarily any reference to previous posts.
4. Socializing with Negative Intent Socializing that expresses negative affect through tone, words, insults, expletives intended as
abusive.
5. Socializing with Positive Intent Socializing that expresses positive affect tone, praise, humor, irony intended in a positive way.
6. Information Seeking Postings asking questions or soliciting opinions, resources, etc. (‘Does anyone know …?’ ‘How
does this work?’). This does not include questions answered rhetorically within the post, e.g., if
a question is asked and answered.
7. Providing Resources Postings that include direct reference to a URL, book, article, etc.; postings that call upon a
well-known theory or the name of a well-known figure.
8. Subreddit Rules and Norms Postings on topics such as what is the appropriate sub-reddit for a particular discussion, what
language is appropriate to use, how to back up claims by using resources, etc.

9.
Example: Code 1 – Explanation with Disagreement
From ‘askpolitics’:
#pLASMA @SMLABTO 9
Definition Linguistic Dialogue Example
Expresses a NEGATIVE take on the content of the
previous posts by adding new ideas or facts to
discussion thread.
‘But’, ‘I disagree’, ‘not sure’, ‘not exactly’ with
explanation/ judgment/ reasoning/ etc.

10.
Code 2 – Explanation with Agreement
From ‘askacademia’:
Yep. Not every company is the same. In fusion research the three private companies I think of
are all headed by nerds and geeks who want to bring commercial fusion to people not
necessarily to make themselves bajillionaires.
#pLASMA @SMLABTO 10
Definition Linguistic Dialogue Example
Expresses a POSITIVE take on the content of the
previous posts by adding new ideas or facts to
discussion thread.
‘Indeed’, ‘also’, ‘I agree’, with explanation/
judgment/ reasoning/ etc.

11.
Code 3 – Explanation w/ Neutral Presentation
From ‘askhistorians’:
The division into light and dark elves is from Snorri's Prose Edda. The Poetic Edda barely makes a
distinction between gods and elves, and often mentions them together.
#pLASMA @SMLABTO 11
Definition Linguistic Dialogue Example
Expresses a NEUTRAL
explanation/judgment/reasoning/etc. with neither
negative nor positive reference to the content of the
previous posts, nor necessarily any reference to previous
posts.
Posts with non-judgmental language. Advice,
brainstorming and first hand experiences are
framed neutrally. ‘I can understand’, ‘interesting’,
‘depends on…’ or statement responses.

12.
Code 4 – Socializing w/ Negative Intent
From ‘askpolitics’:
Your lack of understanding of communism is disgusting especially when you feel qualified enough
to talk about it.
#pLASMA @SMLABTO 12
Definition Linguistic Dialogue Example
Socializing that expresses negative affect through
tone, words, insults, expletives intended as
abusive.
‘no’, ‘you’re an idiot’, ‘this has been
explained multiple times’

13.
Code 5 – Socializing w/ Positive Intent
From ‘askacademia’:
Your response is EXACTLY how I feel thank you.
#pLASMA @SMLABTO 13
Definition Linguistic Dialogue Example
Socializing that expresses positive affect tone,
words, praise, humor, irony intended in a positive
way.
‘thanks’, ‘great feedback’, ‘you’re correct’

14.
Code 6 – Information Seeking
From ‘askscience’:
So then what happens when the aircraft’s velocity vector shifts closer to the nose-up attitude on
takeoff Doesn’t the aircraft then lose the angle-of-attack lift increase.
#pLASMA @SMLABTO 14
Definition Linguistic Dialogue Example
Postings asking questions or soliciting opinions, resources,
etc. (‘Does anyone know …?’ ‘How does this work?’). This
does not include questions answered rhetorically within
the post, e.g., if a question is asked and answered.
‘First you have to think what happens if …?’ and
then you can see what happens’, ‘does anyone
know’, ‘can anyone explain’

15.
Code 7 – Providing Resources
From ‘askhistorians’:
I don't know of any that engage in the debate, but the collection *The Medieval City Under Siege*
has a good essay by Bert S. Hall which covers the sanitation and supply problems a large army
would have faced. *Logistics of Warfare in the Age of the Crusades* covers similar ground.
#pLASMA @SMLABTO 15
Definition Linguistic Dialogue Example
Postings that include direct reference to a URL,
book, article, etc.; postings that call upon a well-
known theory or the name of a well-known figure.
Link to resource copied to post (book, URL,
article, audio/video file). Referencing
theory/theorists, scholar or public work
(Einstein, Newton, Freud).

16.
Code 8 – Subreddit Rules and Norms
From ‘askacademia’:
I know this is r AskAcademia and I should clarify this is for an industry position. I didn t know
where else on reddit I could ask but this seminar is fairly similar to academic ones I guess and I
figured this would be relevant for other folks doing the same . I think what is freaking me out
the most is I will giving a talk to folks who have pioneered some of the analyses I use in my
dissertation.
#pLASMA @SMLABTO 16
Definition Linguistic Dialogue Example
Postings on topics such as what is the appropriate
sub-reddit for a particular discussion, what
language is appropriate to use, how to back up
claims by using resources, etc.
‘See/don’t forget subreddit link’, ‘this post
doesn’t belong here’, upvote/downvote
mentions, acknowledging OP redditors, and
bots.

17.
Coding Results
48%
44%
43%
50%
17%
5%
8%
17%
4%
7%
7%
12%
22%
11%
18%
10%
6%
18%
9%
6%
21%
9%
12%
4%
5%
1
2
5%
1
1
ASKHISTORIANS
ASK_POLITICS
ASKSCIENCE
ASKACADEMIA
3.Explanation with Neutral Presentation 5.Socializing with Positive Intent 2.Explanation with Agreement
6.Information Seeking 1.Explanation with Disagreement 7.Providing Resources
4.Socializing with Negative Intent 8.Subreddit Rules and Norms

18.
Summary of Results
Different Types of Discourse around Learning in Reddit
Differences across Subreddits
• ‘Ask_politics’ shows a greater proportion of posts with negative intent
• ‘AskAcademia’ has the highest neutral explanation with positive socializing
• ‘AskHistorians’ and ‘askscience’ are Q&A learning spaces, with direct
references to external resources
#pLASMA @SMLABTO 18

19.
Future Work
• Apply machine learning techniques to automate the coding process
• Extend to other social media datasets
• Twitter, Facebook, YouTube
• LinkedIn, ResearchGate
#pLASMA @SMLABTO 19

20.
#pLASMA @SMLABTO
20
Learning in the Wild: Coding Reddit for
Learning and Practice
Priya Kumar, Ryerson University
Anatoliy Gruzd, Ryerson University
Caroline Haythornthwaite, Syracuse University
Sarah Gilbert, University of British Columbia
Marc Esteve del Valle, University of Groningen
Drew Paulin, University of California Berkeley
Hawaii International Conference on System Sciences (HICSS-51)
January 3-6, 2018