SlideShare a Scribd company logo
1 of 21
Prevalence of
Convenient
Sampling
An investigation into
sampling trends in
online learning
research
Patrick R. Lowenthal & Ross Perkins
Boise State University
Online Growth
+ million
post-secondary
students
Source: Class Differences: Online Education in the United States, 2010
Poor
Quality
Research
Studies ā€œsuffered from
weaknesses such as small
sample sizes; failure to
report retention rates for
students in the conditions
being contrasted; and, in
many cases, potential bias
stemming from the
authorsā€™ dual roles as
experimenters and
instructorsā€
(Means et al., 2009, p. xviii)
Online learning research is
ā€œseverely wanting in terms
of depth of reportingā€
Bias
Context matters: A
description and
typology of the
online learning
landscape.
www.patricklowenthal.com/publications/AE
CT2009TypologyOnlineLearning.pdf
Balance
What are the
sampling trends of
online learning
research?
What are the
sampling trends of
online learning
research focused on
social presence?
ā€œsocial presenceā€
&
ā€œonline learningā€
Ebsco--Academic Search Premier,
Applied Science & Tech full text
(H.W. Wilson), Communication &
Mass Media Complete, Education
Research Complete, ERIC,
PsycINFO
Key
Words
ā€œsocial presenceā€
&
ā€œonline learningā€
102
studies
Results [Type of Publication]
4%
23%
73%
Journal
Dissertation
Journal
Conference
Results [Popular Outlets]
Journal of Interactive Online Learning
ā€¢ 8 articles
Internet and Higher Education
ā€¢ 8 articles
International Review of Research in
Open & Distance Learning
ā€¢ 4 articles
Journal of Asynchronous Learning
ā€¢ 4 articles
Computers & Education
ā€¢ 4 articles
Results [Methodology]
16% Mixed
methods
Results [Subject]
79%
9%
6%
6%
Education
Buisness
Nursing
Psychology
Results [Format]
19%
15%
1%
65%
Blended
Both
F2F
Online
Results [Level]
High School 4
Undergraduate 25
Graduate 35
6%
39%
55%
High School
Undergraduate
Graduate
Results [Sample Size]
ā€¢ 13%: 10 or less students
ā€¢ 24 %: 20 or less students
ā€¢ 34%: 50 or more students
ā€¢ 25%: 100 or more students
ā€¢ Mean: 101
ā€¢ Mode: 6
ā€¢ Median: 43
ā€¢ Range: 2 to 709
More time
reporting on
sampling
rationale
Study under-
represented
groups &
subjects
Replicate
studies in
different
contexts
Share &
publish more
dissertation
research
More time
reporting on
sampling
rationale
Study under-
represented
groups &
subjects
Replicate
studies in
different
contexts
Share &
publish more
dissertation
research
More time
reporting on
sampling
rationale
Study under-
represented
groups &
subjects
Replicate
studies in
different
contexts
Share &
publish more
dissertation
research
More time
reporting on
sampling
rationale
Study under-
represented
groups &
subjects
Replicate
studies in
different
contexts
Share &
publish more
dissertation
research
patricklowenthal@boisestate.edu
slides @ patricklowenthal.com

More Related Content

What's hot

Group 7 thesis
Group 7 thesisGroup 7 thesis
Group 7 thesis
IV_St_Teresa
Ā 
Copyright And Plagiarism
Copyright And PlagiarismCopyright And Plagiarism
Copyright And Plagiarism
fridayinstitute
Ā 

What's hot (20)

Bowdoin and Bushallow, "Wrestling with Access and Authentication: George Maso...
Bowdoin and Bushallow, "Wrestling with Access and Authentication: George Maso...Bowdoin and Bushallow, "Wrestling with Access and Authentication: George Maso...
Bowdoin and Bushallow, "Wrestling with Access and Authentication: George Maso...
Ā 
Rethinking academic publishing through multimedia scholarship
Rethinking academic publishing through multimedia scholarshipRethinking academic publishing through multimedia scholarship
Rethinking academic publishing through multimedia scholarship
Ā 
Dissertation Presentation
Dissertation PresentationDissertation Presentation
Dissertation Presentation
Ā 
Research Design for the Study of Social Media Use by Dutch Development Organi...
Research Design for the Study of Social Media Use by Dutch Development Organi...Research Design for the Study of Social Media Use by Dutch Development Organi...
Research Design for the Study of Social Media Use by Dutch Development Organi...
Ā 
Group 7 thesis
Group 7 thesisGroup 7 thesis
Group 7 thesis
Ā 
ProQuest
ProQuestProQuest
ProQuest
Ā 
NORSLIS 2010 - PhD Research Social Networks in a Learning Experience
NORSLIS 2010 - PhD Research Social Networks in a Learning ExperienceNORSLIS 2010 - PhD Research Social Networks in a Learning Experience
NORSLIS 2010 - PhD Research Social Networks in a Learning Experience
Ā 
Conferences virtual referencesummit2011
Conferences virtual referencesummit2011Conferences virtual referencesummit2011
Conferences virtual referencesummit2011
Ā 
Copyright And Plagiarism
Copyright And PlagiarismCopyright And Plagiarism
Copyright And Plagiarism
Ā 
Research Paper On Correlation
Research Paper On CorrelationResearch Paper On Correlation
Research Paper On Correlation
Ā 
Social Networking Tools for Academic Libraries
Social Networking Tools for Academic LibrariesSocial Networking Tools for Academic Libraries
Social Networking Tools for Academic Libraries
Ā 
MSM-28-191
MSM-28-191MSM-28-191
MSM-28-191
Ā 
The Evolution of Faculty and Expertise in Higher Education
The Evolution of Faculty and Expertise in Higher EducationThe Evolution of Faculty and Expertise in Higher Education
The Evolution of Faculty and Expertise in Higher Education
Ā 
Identifying and facilitating a community of practice in IL - McCluskey Dean
Identifying and facilitating a community of practice in IL - McCluskey DeanIdentifying and facilitating a community of practice in IL - McCluskey Dean
Identifying and facilitating a community of practice in IL - McCluskey Dean
Ā 
Let me tell you about my grandpa: a content analysis of user annotations to o...
Let me tell you about my grandpa: a content analysis of user annotations to o...Let me tell you about my grandpa: a content analysis of user annotations to o...
Let me tell you about my grandpa: a content analysis of user annotations to o...
Ā 
How Children Search the Internet with Keyword Interfaces
How Children Search the Internet with Keyword InterfacesHow Children Search the Internet with Keyword Interfaces
How Children Search the Internet with Keyword Interfaces
Ā 
(January 2012) ALISE 2012
(January 2012) ALISE 2012(January 2012) ALISE 2012
(January 2012) ALISE 2012
Ā 
Facebook psychology
Facebook psychologyFacebook psychology
Facebook psychology
Ā 
Who Are They
Who Are TheyWho Are They
Who Are They
Ā 
Studying Cybercrime: Raising Awareness of Objectivity & Bias
Studying Cybercrime: Raising Awareness of Objectivity & BiasStudying Cybercrime: Raising Awareness of Objectivity & Bias
Studying Cybercrime: Raising Awareness of Objectivity & Bias
Ā 

Similar to AERA 2014 - Prevalence of Convenient Sampling: An investigation into sampling trends

Examining Internet Use Among Low-Income Students
Examining Internet Use Among Low-Income StudentsExamining Internet Use Among Low-Income Students
Examining Internet Use Among Low-Income Students
Jason Seliskar
Ā 
Reliability vs. Authority: Credibility Assessment of Highschool Students
Reliability vs. Authority: Credibility Assessment of Highschool StudentsReliability vs. Authority: Credibility Assessment of Highschool Students
Reliability vs. Authority: Credibility Assessment of Highschool Students
Axel Maireder
Ā 
Context matters: A description and typology of the online learning landscape
Context matters: A description and typology of the online learning landscapeContext matters: A description and typology of the online learning landscape
Context matters: A description and typology of the online learning landscape
Patrick Lowenthal
Ā 

Similar to AERA 2014 - Prevalence of Convenient Sampling: An investigation into sampling trends (20)

A study of sixth gradersā€™ critical evaluation of Internet sources
A study of sixth gradersā€™ critical evaluation of Internet sourcesA study of sixth gradersā€™ critical evaluation of Internet sources
A study of sixth gradersā€™ critical evaluation of Internet sources
Ā 
AECT 2012 -- Sampling Trends in Social Presence Research
AECT 2012 -- Sampling Trends in Social Presence ResearchAECT 2012 -- Sampling Trends in Social Presence Research
AECT 2012 -- Sampling Trends in Social Presence Research
Ā 
First monday
First mondayFirst monday
First monday
Ā 
The Power of Massive Informal Learning Environments
The Power of Massive Informal Learning EnvironmentsThe Power of Massive Informal Learning Environments
The Power of Massive Informal Learning Environments
Ā 
First monday
First mondayFirst monday
First monday
Ā 
Asmcue nuts and bolts
Asmcue   nuts and boltsAsmcue   nuts and bolts
Asmcue nuts and bolts
Ā 
Assigning Inquiry How Handouts For Research Assignments Guide Today S Colleg...
Assigning Inquiry  How Handouts For Research Assignments Guide Today S Colleg...Assigning Inquiry  How Handouts For Research Assignments Guide Today S Colleg...
Assigning Inquiry How Handouts For Research Assignments Guide Today S Colleg...
Ā 
Social research methods and open educational resources: a literature review (...
Social research methods and open educational resources: a literature review (...Social research methods and open educational resources: a literature review (...
Social research methods and open educational resources: a literature review (...
Ā 
The Role of Social Media in Today's College Student Experience
The Role of Social Media in Today's College Student ExperienceThe Role of Social Media in Today's College Student Experience
The Role of Social Media in Today's College Student Experience
Ā 
Social Presence, Identity and Online Learning
Social Presence, Identity and Online LearningSocial Presence, Identity and Online Learning
Social Presence, Identity and Online Learning
Ā 
The Net Generation: Myths, Realities and Implications for Higher Education
The Net Generation: Myths, Realities and Implications for Higher EducationThe Net Generation: Myths, Realities and Implications for Higher Education
The Net Generation: Myths, Realities and Implications for Higher Education
Ā 
Addressing Plagiarism In A Digital Age
Addressing Plagiarism In A Digital AgeAddressing Plagiarism In A Digital Age
Addressing Plagiarism In A Digital Age
Ā 
Learning Theories and Instructional Pathways for Adult Learners in the Online...
Learning Theories and Instructional Pathways for Adult Learners in the Online...Learning Theories and Instructional Pathways for Adult Learners in the Online...
Learning Theories and Instructional Pathways for Adult Learners in the Online...
Ā 
Tcetc2010
Tcetc2010Tcetc2010
Tcetc2010
Ā 
Tcetc2010
Tcetc2010Tcetc2010
Tcetc2010
Ā 
Access To Online Databases Predicate For Faculty Research Output
Access To Online Databases  Predicate For Faculty Research OutputAccess To Online Databases  Predicate For Faculty Research Output
Access To Online Databases Predicate For Faculty Research Output
Ā 
Course development by higher education partners of the OERu: Case studies us...
Course development by higher education partners of the OERu:  Case studies us...Course development by higher education partners of the OERu:  Case studies us...
Course development by higher education partners of the OERu: Case studies us...
Ā 
Examining Internet Use Among Low-Income Students
Examining Internet Use Among Low-Income StudentsExamining Internet Use Among Low-Income Students
Examining Internet Use Among Low-Income Students
Ā 
Reliability vs. Authority: Credibility Assessment of Highschool Students
Reliability vs. Authority: Credibility Assessment of Highschool StudentsReliability vs. Authority: Credibility Assessment of Highschool Students
Reliability vs. Authority: Credibility Assessment of Highschool Students
Ā 
Context matters: A description and typology of the online learning landscape
Context matters: A description and typology of the online learning landscapeContext matters: A description and typology of the online learning landscape
Context matters: A description and typology of the online learning landscape
Ā 

More from Patrick Lowenthal

AECT 2015 Creating an intentional web presence
AECT 2015 Creating an intentional web presenceAECT 2015 Creating an intentional web presence
AECT 2015 Creating an intentional web presence
Patrick Lowenthal
Ā 
Reaching out and Being There: What we know and don't know about social presence
Reaching out and Being There: What we know and don't know about social presenceReaching out and Being There: What we know and don't know about social presence
Reaching out and Being There: What we know and don't know about social presence
Patrick Lowenthal
Ā 

More from Patrick Lowenthal (20)

Developing construct maps for attentiveness
Developing construct maps for attentivenessDeveloping construct maps for attentiveness
Developing construct maps for attentiveness
Ā 
In search of a better understanding of social presence: An investigation into...
In search of a better understanding of social presence: An investigation into...In search of a better understanding of social presence: An investigation into...
In search of a better understanding of social presence: An investigation into...
Ā 
10 simple guidelines to improve your PowerPoint
10 simple guidelines to improve your PowerPoint10 simple guidelines to improve your PowerPoint
10 simple guidelines to improve your PowerPoint
Ā 
Northwest eLearn 2016 Keynote
Northwest eLearn 2016 KeynoteNorthwest eLearn 2016 Keynote
Northwest eLearn 2016 Keynote
Ā 
Open Access Journals in Educational Technology: Results of a Survey of Exper...
Open Access Journals in Educational Technology:  Results of a Survey of Exper...Open Access Journals in Educational Technology:  Results of a Survey of Exper...
Open Access Journals in Educational Technology: Results of a Survey of Exper...
Ā 
Social Presence in Online Courses -- Webinar
Social Presence in Online Courses -- WebinarSocial Presence in Online Courses -- Webinar
Social Presence in Online Courses -- Webinar
Ā 
AECT 2015 - Defining social presence
AECT 2015 - Defining social presenceAECT 2015 - Defining social presence
AECT 2015 - Defining social presence
Ā 
AECT 2015 Creating an intentional web presence
AECT 2015 Creating an intentional web presenceAECT 2015 Creating an intentional web presence
AECT 2015 Creating an intentional web presence
Ā 
In Search of Quality: Using Quality Matters to Analyze the Quality of Massive...
In Search of Quality: Using Quality Matters to Analyze the Quality of Massive...In Search of Quality: Using Quality Matters to Analyze the Quality of Massive...
In Search of Quality: Using Quality Matters to Analyze the Quality of Massive...
Ā 
Getting graphic About Infographics: Design Lessons Learned From Popular Infog...
Getting graphic About Infographics: Design Lessons Learned From Popular Infog...Getting graphic About Infographics: Design Lessons Learned From Popular Infog...
Getting graphic About Infographics: Design Lessons Learned From Popular Infog...
Ā 
Distance Teaching & Learning 2015 -- Social Presence and Online Video
Distance Teaching & Learning 2015 -- Social Presence and Online VideoDistance Teaching & Learning 2015 -- Social Presence and Online Video
Distance Teaching & Learning 2015 -- Social Presence and Online Video
Ā 
Intentional Web Presence for Research and Technology Professionals
Intentional Web Presence for Research and Technology ProfessionalsIntentional Web Presence for Research and Technology Professionals
Intentional Web Presence for Research and Technology Professionals
Ā 
Reaching out and Being There: What we know and don't know about social presence
Reaching out and Being There: What we know and don't know about social presenceReaching out and Being There: What we know and don't know about social presence
Reaching out and Being There: What we know and don't know about social presence
Ā 
Learn it2015 keynote infographics
Learn it2015   keynote infographicsLearn it2015   keynote infographics
Learn it2015 keynote infographics
Ā 
AERA 2015 Instructional Design Lessons Learned From Reviewing Popular Infogra...
AERA 2015 Instructional Design Lessons Learned From Reviewing Popular Infogra...AERA 2015 Instructional Design Lessons Learned From Reviewing Popular Infogra...
AERA 2015 Instructional Design Lessons Learned From Reviewing Popular Infogra...
Ā 
AERA 2015 social presence and video
AERA 2015 social presence and videoAERA 2015 social presence and video
AERA 2015 social presence and video
Ā 
Intentional Web Presence for Educational Technology Professionals
Intentional Web Presence for Educational Technology ProfessionalsIntentional Web Presence for Educational Technology Professionals
Intentional Web Presence for Educational Technology Professionals
Ā 
PASSHE 2015: If You Record It, Will They Watch It? And Will It Matter? Explor...
PASSHE 2015: If You Record It, Will They Watch It? And Will It Matter? Explor...PASSHE 2015: If You Record It, Will They Watch It? And Will It Matter? Explor...
PASSHE 2015: If You Record It, Will They Watch It? And Will It Matter? Explor...
Ā 
WCET 2014
WCET 2014WCET 2014
WCET 2014
Ā 
Boise State Blackboard Conference -- Building Learning Community
Boise State Blackboard Conference -- Building Learning CommunityBoise State Blackboard Conference -- Building Learning Community
Boise State Blackboard Conference -- Building Learning Community
Ā 

Recently uploaded

Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in DelhiRussian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
kauryashika82
Ā 
Gardella_PRCampaignConclusion Pitch Letter
Gardella_PRCampaignConclusion Pitch LetterGardella_PRCampaignConclusion Pitch Letter
Gardella_PRCampaignConclusion Pitch Letter
MateoGardella
Ā 
1029 - Danh muc Sach Giao Khoa 10 . pdf
1029 -  Danh muc Sach Giao Khoa 10 . pdf1029 -  Danh muc Sach Giao Khoa 10 . pdf
1029 - Danh muc Sach Giao Khoa 10 . pdf
QucHHunhnh
Ā 
Activity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfActivity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdf
ciinovamais
Ā 

Recently uploaded (20)

Ecological Succession. ( ECOSYSTEM, B. Pharmacy, 1st Year, Sem-II, Environmen...
Ecological Succession. ( ECOSYSTEM, B. Pharmacy, 1st Year, Sem-II, Environmen...Ecological Succession. ( ECOSYSTEM, B. Pharmacy, 1st Year, Sem-II, Environmen...
Ecological Succession. ( ECOSYSTEM, B. Pharmacy, 1st Year, Sem-II, Environmen...
Ā 
APM Welcome, APM North West Network Conference, Synergies Across Sectors
APM Welcome, APM North West Network Conference, Synergies Across SectorsAPM Welcome, APM North West Network Conference, Synergies Across Sectors
APM Welcome, APM North West Network Conference, Synergies Across Sectors
Ā 
SECOND SEMESTER TOPIC COVERAGE SY 2023-2024 Trends, Networks, and Critical Th...
SECOND SEMESTER TOPIC COVERAGE SY 2023-2024 Trends, Networks, and Critical Th...SECOND SEMESTER TOPIC COVERAGE SY 2023-2024 Trends, Networks, and Critical Th...
SECOND SEMESTER TOPIC COVERAGE SY 2023-2024 Trends, Networks, and Critical Th...
Ā 
How to Give a Domain for a Field in Odoo 17
How to Give a Domain for a Field in Odoo 17How to Give a Domain for a Field in Odoo 17
How to Give a Domain for a Field in Odoo 17
Ā 
Key note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdfKey note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdf
Ā 
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptx
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptxSOCIAL AND HISTORICAL CONTEXT - LFTVD.pptx
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptx
Ā 
Measures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SDMeasures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SD
Ā 
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in DelhiRussian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Ā 
Gardella_PRCampaignConclusion Pitch Letter
Gardella_PRCampaignConclusion Pitch LetterGardella_PRCampaignConclusion Pitch Letter
Gardella_PRCampaignConclusion Pitch Letter
Ā 
ICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptxICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptx
Ā 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The Basics
Ā 
Grant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy ConsultingGrant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy Consulting
Ā 
CĆ³digo Creativo y Arte de Software | Unidad 1
CĆ³digo Creativo y Arte de Software | Unidad 1CĆ³digo Creativo y Arte de Software | Unidad 1
CĆ³digo Creativo y Arte de Software | Unidad 1
Ā 
1029 - Danh muc Sach Giao Khoa 10 . pdf
1029 -  Danh muc Sach Giao Khoa 10 . pdf1029 -  Danh muc Sach Giao Khoa 10 . pdf
1029 - Danh muc Sach Giao Khoa 10 . pdf
Ā 
Unit-IV; Professional Sales Representative (PSR).pptx
Unit-IV; Professional Sales Representative (PSR).pptxUnit-IV; Professional Sales Representative (PSR).pptx
Unit-IV; Professional Sales Representative (PSR).pptx
Ā 
Activity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfActivity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdf
Ā 
Sports & Fitness Value Added Course FY..
Sports & Fitness Value Added Course FY..Sports & Fitness Value Added Course FY..
Sports & Fitness Value Added Course FY..
Ā 
Unit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptxUnit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptx
Ā 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introduction
Ā 
Mixin Classes in Odoo 17 How to Extend Models Using Mixin Classes
Mixin Classes in Odoo 17  How to Extend Models Using Mixin ClassesMixin Classes in Odoo 17  How to Extend Models Using Mixin Classes
Mixin Classes in Odoo 17 How to Extend Models Using Mixin Classes
Ā 

AERA 2014 - Prevalence of Convenient Sampling: An investigation into sampling trends

Editor's Notes

  1. In the fall of 2009, 5.6 million students took at least one online course according to Sloan-C
  2. Lowenthal, P. R., Wilson, B., & Parrish, P. (2009, October). Context matters: A description and typology of the online learning landscape. Paper presented at the 2009 AECT International Convention, Louisville, KY. http://www.patricklowenthal.com/publications/AECT2009TypologyOnlineLearning.pdf
  3. We posit that researchers need to do a better job of researching various contexts of online learning and investigating how the context shapes the experience. We are not arguing as some have that we simply need more experimental or multi-institutional studies with samples in the thousands but rather that we need to do a better job of investigating various and diverse samples of online learning (which then might lead to larger multi-institutional studies).
  4. Initially found 194 results (1994-2011) After deleting duplicates, 156 studiesCouldnā€™t locate 16 dissertations Couldnā€™t locate 11 journal articlesCouldnā€™t locate 4 journal articlesRemoved 10 articles that were reviews
  5. Journal of Interactive Online learning: 8Internet and Higher Ed: 8International Review of Research in Open and Distance Learning: 4Journal of Asynchronous Learning Networks: 4Computers & Education: 4
  6. 16 out of 102
  7. 12 werenā€™t listed
  8. 12 werenā€™t listed
  9. 12 werenā€™t listed
  10. 12 werenā€™t listed
  11. 12 werenā€™t listed
  12. 12 werenā€™t listed
  13. 12 werenā€™t listed