A keyword analysis between
lone posts and initiating
posts in MOOC discussions
Shi Min Chua @shiminchua
CMC and Social Media Corpora 2018, Antwerp
Why did
nobody reply
to my post?
The CMC setting - MOOCs
CoursesOpen OnlineMassive
Large number of learners
-registered learners: up to 40000
-learners who take part in the
discussion: up to 5000
-Free
-No pre-requisite
-No face-to-face
-Asynchronous
-various subjects
-evolving learning
design
Post
Reply
Types of Posts
Initiating Post
Lone Post
Reply
Number of Posts Number of Tokens
117863 6162230
32080 2401795
54172 2642195
Why?
● Usenet groups (Burke et al., 2007; Himelboim, Gleave, & Smith, 2009)
● Online news commenting spaces (Ziegele, Breiner, & Quiring, 2014)
● Distance learning online discussion (Dennen & Wieland, 2007)
Overabundance of Lone Posts in CMC
● Levelling ground, not constrained by face-to-face conversation (Cavangah, 2007)
● Content (Joyce, Anand, Kraut, 2014; Ziegele et al., 2014)
● Time of posting (Jeong & Frazier, 2008)
● Size & topic (Himelboim, 2008)
Stylistic and
linguistic
features
Motivation for examining
Initiating Posts vs. Lone Posts
● Probability of posting again increased (Joyce & Kraut, 2006)
● Join the discussion for interactive purposes rather than cognitive gains
(Springer, Engelmann, & Pfaffinger, 2015)
● Initiate direct and explicit interaction,
●which is essential for dialectical, critical discourse and argumentation (Rourke
& Kanuka, 2007)
●Meaning-making and co-construction (Stahl, 2003)
●MOOC learners may feel frustrated if their posts are seldom responded to
(Hew & Cheung, 2014)
Motivation for examining
Initiating Posts vs. Lone Posts
● Reflection: Conversation with oneself or learning materials (Laurillard, 2012).
Keyword Analysis: Cut-off Criteria
● Comparing initiating posts to lone posts
● the log-likelihood ratio test, p <
0.000000000001 (Flowerdew, 2015)
● Keyness indicator (effect size): Bayes
Factor > 10 (Wilson, 2013)
● Normalized frequency > 5 per 100,000
(McEnery, 2016)
● Dispersion Measure, Gries’ Deviation of
Proportion (Gries’ DP, Gries, 2008; Lijffijt &
Gries, 2012) < 0.30
Keyword Analysis Comparison:
Initiating posts v.s. Lone posts
Number of posts Number of token
Abbreviation
Initiating
posts
Lone
posts
Replies Initiating
posts
Lone
posts
Replies
Technology 1151 4731 1966 92680 262140 89620
History 4176 8652 10280 330828 567627 459203
Technology 1900 4822 3588 105238 169263 144498
Linguistics 5500 4449 5000 501186 297458 304662
Social
Sciences 4841 30792 5542 406832 1652576 265959
Finances 1845 2612 5576 188831 232226 399772
Management 1207 5905 1303 73449 310497 60348
Astronomy 4386 11724 7974 201854 420635 307103
Health 2661 23109 5153 197137 1024804 221502
Sciences 1650 4331 2427 93420 216450 109408
Healthcare 720 8525 1273 70113 572934 63330
Geography 2043 8211 4090 140227 435620 216790
Total 32080 117863 54172 2401795 6162230 2642195
Analysis of Keywords
Step 1
● Concordance lines of each keyword
● Salient meaning/function (McEnery, 2016)
● Biber et al (1998), Wmatrix (Rayson, 2017),
Step 2
● Functional grouping of keywords
● Interpretations based on groups of keywords
● Categorized based on function related to dialogic learning and MOOCs
Step 3
● Discourse/conversation analysis of selected keywords (O’Keeffe & Walsh, 2016)
Labelling
Grouping
Case
Studies
Learners Lone
Posts
Initiating
Posts
Keywords 77 70
Dialogic
Expansion
● Heteroglosia (Baktin, 1986)
● Alternative voices
● “Represents the proposition as but
one of a range of possible positions”
● Allows dialogic alternatives
● “In my view the banks have been
greedy.”
Dialogic
Contraction
● Monoglossic
● Categorical or bare assertion
● No reference of others’
viewpoint
● “The banks have been greedy.”
● Martin & White (2004)
v.s.
● Modals, Hedges, Boosters, Pronouns, Evaluatives (Fairclough, 2003)
● Stance and intersubjectivity (Du Bois, 2007)
Stance Expression
Initiating Posts Lone Posts
Modal expression might, would, could will, need, able
Hedging perhaps, seems, sort
Quantifier any all, lot, much, every
Epistemic expression wonder, wondering aware, understanding, learned
…this is perhaps because we tend to…
…This might mean actually walking…
…I wonder would the microbial diversity
also mirror…
We need to be more exact…
…I need to be ambidextrous…
...money taken in by a Company is not all
down to their own effort, it relies on…
Stance Expression
Initiating Posts Lone Posts
Booster surely, just, rather, else really, very, definitely, always
Mental verbs feel, feeling, think, agree, keen, hope, hoping,
looking, forward, enjoy, enjoyed, love
Evaluative wrong difficult, easy, excellent, better, interesting,
informative, great, important, good, new
Negation cannot, ca, n't
…Really looking forward to learn…
…I enjoyed this course and definitely
learned a lot in…
Excellent range of resources, thanks!
I agree with this definition regarding
health
…I really don’t see the point of…
Pronouns
Initiating Posts Lone Posts
Pronouns he I, my, our, their
Indefinite pronouns anybody, anyone everyone
Addressing general audience, instead of “you” which is used
in one-to-one setting (Tagg, 2012)
Arbitrary instead of all inclusive (Biber et al, 1998)
Keyword: anybody in initiating posts
Request for recommendation
“…So, does anybody have a good suggestion for a text book on Anaconda, Python and
Pandas?”
Offering recommendation
“…Suggest anybody else that may still be receiving 'file not found' for exercise 6 could try the
same thing…”
Looking for shared experience
“Is it just me or does anybody else have issues find <sic> the PwC paper?” Is this a frequent
phrase?
Greetings:
Among 4375 instances, 975 “Hi everyone”, 868 “Hello everyone”
Appreciation:
428 “Thank(s)… everyone”
Making stances
-104 “not everyone…”
-”…we need everyone to control our daily waste…”
Narrative
“…especially when considering the spread of disease and how closely everyone
lived in the cities…”
Keyword: everyone in lone posts
Questions and Request
● “Question: does anybody knows what kind …?”
Keywords in initiating posts:
anybody, anyone, question, please, ?,
wonder, wondering, why
Initiating Posts Lone Posts
Discourse Particles please, sorry thanks, thank
Punctuation ,…-();?"': !.
Meta-language question, article information, course, knowledge
Connectors
Initiating Posts Lone Posts
Connectors if, or, then, example, e.g. also, and
● qualify or elaborate on a proposition by specifying a condition
● ….bulldozing the homes and 're building luxury houses in its place. (A fine example can be
found in Barnett, the Sweets Way estate).
● …obesity can be past<sic> on as a learnt behaviour to children through parents. An example
of this could be that, a person with depression that comfort eats and does very little else thus
causing them to be obese…
● …I need both hands to propel my wheelchair, so (for example) on the Underground, I find it
difficult to use the automatic gates because they don't give me time to put my ticket away….
Conclusions
●Initiating posts:
Questions and hedging to show uncertainty
Addressing with indefinite pronouns, so anyone could respond, give agency to others
Negation to voice out alternatives
●Lone posts:
Self-mention
Appreciation, Emotion, Evaluation
Agreement
Others
Initiating Posts Lone Posts
Comparative
terms/relational
than, same more
Grammatical
the, that, there, here, does, did, was,
were, 's, on, by
am, 'm, have, for, about, with, to
Punctuation ,…-();?"': !.
Speech act
mean, explain, tell, says, say, told,
called
Verbs in past
tense/passive form
used, tried, came joined
Verbs in present
tense/infinite form
affects, helps, achieve, work , gain, meet,
improve
Uncategorized
1, one, two, numbers, missing,
following, why, whether
like, well, week, main, currently, working,
opportunity, education, environment, mind
Acknowledgement
● Dr Caroline Tagg, Prof Mike Sharples, Prof Bart Rienties
● Leverhulme Trust
● Course Providers
● shimin.chua@open.ac.uk

A keyword analysis between initiating posts and lone posts in MOOC discussion

  • 1.
    A keyword analysisbetween lone posts and initiating posts in MOOC discussions Shi Min Chua @shiminchua CMC and Social Media Corpora 2018, Antwerp Why did nobody reply to my post?
  • 2.
    The CMC setting- MOOCs CoursesOpen OnlineMassive Large number of learners -registered learners: up to 40000 -learners who take part in the discussion: up to 5000 -Free -No pre-requisite -No face-to-face -Asynchronous -various subjects -evolving learning design
  • 4.
  • 5.
    Types of Posts InitiatingPost Lone Post Reply Number of Posts Number of Tokens 117863 6162230 32080 2401795 54172 2642195
  • 6.
    Why? ● Usenet groups(Burke et al., 2007; Himelboim, Gleave, & Smith, 2009) ● Online news commenting spaces (Ziegele, Breiner, & Quiring, 2014) ● Distance learning online discussion (Dennen & Wieland, 2007) Overabundance of Lone Posts in CMC ● Levelling ground, not constrained by face-to-face conversation (Cavangah, 2007) ● Content (Joyce, Anand, Kraut, 2014; Ziegele et al., 2014) ● Time of posting (Jeong & Frazier, 2008) ● Size & topic (Himelboim, 2008) Stylistic and linguistic features
  • 7.
    Motivation for examining InitiatingPosts vs. Lone Posts ● Probability of posting again increased (Joyce & Kraut, 2006) ● Join the discussion for interactive purposes rather than cognitive gains (Springer, Engelmann, & Pfaffinger, 2015) ● Initiate direct and explicit interaction, ●which is essential for dialectical, critical discourse and argumentation (Rourke & Kanuka, 2007) ●Meaning-making and co-construction (Stahl, 2003) ●MOOC learners may feel frustrated if their posts are seldom responded to (Hew & Cheung, 2014)
  • 8.
    Motivation for examining InitiatingPosts vs. Lone Posts ● Reflection: Conversation with oneself or learning materials (Laurillard, 2012).
  • 9.
    Keyword Analysis: Cut-offCriteria ● Comparing initiating posts to lone posts ● the log-likelihood ratio test, p < 0.000000000001 (Flowerdew, 2015) ● Keyness indicator (effect size): Bayes Factor > 10 (Wilson, 2013) ● Normalized frequency > 5 per 100,000 (McEnery, 2016) ● Dispersion Measure, Gries’ Deviation of Proportion (Gries’ DP, Gries, 2008; Lijffijt & Gries, 2012) < 0.30
  • 10.
    Keyword Analysis Comparison: Initiatingposts v.s. Lone posts Number of posts Number of token Abbreviation Initiating posts Lone posts Replies Initiating posts Lone posts Replies Technology 1151 4731 1966 92680 262140 89620 History 4176 8652 10280 330828 567627 459203 Technology 1900 4822 3588 105238 169263 144498 Linguistics 5500 4449 5000 501186 297458 304662 Social Sciences 4841 30792 5542 406832 1652576 265959 Finances 1845 2612 5576 188831 232226 399772 Management 1207 5905 1303 73449 310497 60348 Astronomy 4386 11724 7974 201854 420635 307103 Health 2661 23109 5153 197137 1024804 221502 Sciences 1650 4331 2427 93420 216450 109408 Healthcare 720 8525 1273 70113 572934 63330 Geography 2043 8211 4090 140227 435620 216790 Total 32080 117863 54172 2401795 6162230 2642195
  • 11.
    Analysis of Keywords Step1 ● Concordance lines of each keyword ● Salient meaning/function (McEnery, 2016) ● Biber et al (1998), Wmatrix (Rayson, 2017), Step 2 ● Functional grouping of keywords ● Interpretations based on groups of keywords ● Categorized based on function related to dialogic learning and MOOCs Step 3 ● Discourse/conversation analysis of selected keywords (O’Keeffe & Walsh, 2016) Labelling Grouping Case Studies Learners Lone Posts Initiating Posts Keywords 77 70
  • 12.
    Dialogic Expansion ● Heteroglosia (Baktin,1986) ● Alternative voices ● “Represents the proposition as but one of a range of possible positions” ● Allows dialogic alternatives ● “In my view the banks have been greedy.” Dialogic Contraction ● Monoglossic ● Categorical or bare assertion ● No reference of others’ viewpoint ● “The banks have been greedy.” ● Martin & White (2004) v.s. ● Modals, Hedges, Boosters, Pronouns, Evaluatives (Fairclough, 2003) ● Stance and intersubjectivity (Du Bois, 2007)
  • 13.
    Stance Expression Initiating PostsLone Posts Modal expression might, would, could will, need, able Hedging perhaps, seems, sort Quantifier any all, lot, much, every Epistemic expression wonder, wondering aware, understanding, learned …this is perhaps because we tend to… …This might mean actually walking… …I wonder would the microbial diversity also mirror… We need to be more exact… …I need to be ambidextrous… ...money taken in by a Company is not all down to their own effort, it relies on…
  • 14.
    Stance Expression Initiating PostsLone Posts Booster surely, just, rather, else really, very, definitely, always Mental verbs feel, feeling, think, agree, keen, hope, hoping, looking, forward, enjoy, enjoyed, love Evaluative wrong difficult, easy, excellent, better, interesting, informative, great, important, good, new Negation cannot, ca, n't …Really looking forward to learn… …I enjoyed this course and definitely learned a lot in… Excellent range of resources, thanks! I agree with this definition regarding health …I really don’t see the point of…
  • 15.
    Pronouns Initiating Posts LonePosts Pronouns he I, my, our, their Indefinite pronouns anybody, anyone everyone Addressing general audience, instead of “you” which is used in one-to-one setting (Tagg, 2012) Arbitrary instead of all inclusive (Biber et al, 1998)
  • 16.
    Keyword: anybody ininitiating posts Request for recommendation “…So, does anybody have a good suggestion for a text book on Anaconda, Python and Pandas?” Offering recommendation “…Suggest anybody else that may still be receiving 'file not found' for exercise 6 could try the same thing…” Looking for shared experience “Is it just me or does anybody else have issues find <sic> the PwC paper?” Is this a frequent phrase?
  • 17.
    Greetings: Among 4375 instances,975 “Hi everyone”, 868 “Hello everyone” Appreciation: 428 “Thank(s)… everyone” Making stances -104 “not everyone…” -”…we need everyone to control our daily waste…” Narrative “…especially when considering the spread of disease and how closely everyone lived in the cities…” Keyword: everyone in lone posts
  • 18.
    Questions and Request ●“Question: does anybody knows what kind …?” Keywords in initiating posts: anybody, anyone, question, please, ?, wonder, wondering, why Initiating Posts Lone Posts Discourse Particles please, sorry thanks, thank Punctuation ,…-();?"': !. Meta-language question, article information, course, knowledge
  • 19.
    Connectors Initiating Posts LonePosts Connectors if, or, then, example, e.g. also, and ● qualify or elaborate on a proposition by specifying a condition ● ….bulldozing the homes and 're building luxury houses in its place. (A fine example can be found in Barnett, the Sweets Way estate). ● …obesity can be past<sic> on as a learnt behaviour to children through parents. An example of this could be that, a person with depression that comfort eats and does very little else thus causing them to be obese… ● …I need both hands to propel my wheelchair, so (for example) on the Underground, I find it difficult to use the automatic gates because they don't give me time to put my ticket away….
  • 20.
    Conclusions ●Initiating posts: Questions andhedging to show uncertainty Addressing with indefinite pronouns, so anyone could respond, give agency to others Negation to voice out alternatives ●Lone posts: Self-mention Appreciation, Emotion, Evaluation Agreement
  • 21.
    Others Initiating Posts LonePosts Comparative terms/relational than, same more Grammatical the, that, there, here, does, did, was, were, 's, on, by am, 'm, have, for, about, with, to Punctuation ,…-();?"': !. Speech act mean, explain, tell, says, say, told, called Verbs in past tense/passive form used, tried, came joined Verbs in present tense/infinite form affects, helps, achieve, work , gain, meet, improve Uncategorized 1, one, two, numbers, missing, following, why, whether like, well, week, main, currently, working, opportunity, education, environment, mind
  • 22.
    Acknowledgement ● Dr CarolineTagg, Prof Mike Sharples, Prof Bart Rienties ● Leverhulme Trust ● Course Providers ● shimin.chua@open.ac.uk

Editor's Notes

  • #2 Information packaging Feedback: what is the difference between initiating post and lone post which use the same keywords (explicitly ask for replies) Explicit vs. not intending to ask for reply: manual coding
  • #3 Set the stage for further analysis of the discussion discussion
  • #5 16712 replies by facilitators 996 in post by facilitators
  • #16 Addressing general audience with these unspecified pronouns suggested that learners were trying to engage with other learners in a dialogue in their initiating posts with a personal touch despite not knowing any name, and it could be weird to use “you”, which was not a keyword in initiating posts   Yes, and this stands in contrast to personal one-to-one text messaging, for example, in which the most frequent word is ‘you’ – Tagg (2012)! 508 instances of “anyone” (60%) and 45 instances of “anybody” (32%) were used in a sentence ended with a question mark  75 instances of “anybody” (53%) were modal, “be” verb or auxiliary verbs, suggesting question format. For example, “can anybody”, “does anybody”, “has anybody”, “did anybody”, “could anybody”, “is anybody” and “would anybody
  • #17 involvement, agency
  • #18 Every inclusive quantifier (Biber et al, 1998) Addressing general audience with these unspecified pronouns suggested that learners were trying to engage with other learners in a dialogue in their initiating posts with a personal touch despite not knowing any name, and it could be weird to use “you”, which was not a keyword in initiating posts   Yes, and this stands in contrast to personal one-to-one text messaging, for example, in which the most frequent word is ‘you’ – Tagg (2012)! 508 instances of “anyone” (60%) and 45 instances of “anybody” (32%) were used in a sentence ended with a question mark  75 instances of “anybody” (53%) were modal, “be” verb or auxiliary verbs, suggesting question format. For example, “can anybody”, “does anybody”, “has anybody”, “did anybody”, “could anybody”, “is anybody” and “would anybody
  • #20 Addressing general audience with these unspecified pronouns suggested that learners were trying to engage with other learners in a dialogue in their initiating posts with a personal touch despite not knowing any name, and it could be weird to use “you”, which was not a keyword in initiating posts   Yes, and this stands in contrast to personal one-to-one text messaging, for example, in which the most frequent word is ‘you’ – Tagg (2012)! 508 instances of “anyone” (60%) and 45 instances of “anybody” (32%) were used in a sentence ended with a question mark  75 instances of “anybody” (53%) were modal, “be” verb or auxiliary verbs, suggesting question format. For example, “can anybody”, “does anybody”, “has anybody”, “did anybody”, “could anybody”, “is anybody” and “would anybody