Challenges in Defining,
Designing, and
Measuring
“Digital Literacy”
Development in
Learners:
A Proposed Framework
AERA, PH...
“Digital Literacy” in Naturalistic vs. Prescriptive Contexts
Naturalistic Phenomena in General Population
• Digital Divide...
“Digital Literacy” in Naturalistic vs. Prescriptive Contexts
Prescriptive Instructional Contexts
• Strand 2 of the U.S. FC...
Definitions and Operationalizations Vary
• Example 1, scholarship: Web-use skill
(Hargittai,2005): “a user’s ability to lo...
Definitions and Operationalizations Vary
• Example 2, scholarship: Barron (2004) defines
technological fluency as learners...
Definitions and Operationalizations Vary
Example 3, scholarship: Eshet & Aviram
• Skeptical hypothesis 2 centers on techno...
Definitions and Operationalizations Vary
Example 1, policy: AASL
Standards: “The definition of
information literacy has be...
Definitions and Operationalizations Vary
Example 2, policy: NETS Standards
Definitions and Operationalizations Vary
Example 3, scholarship: Eshet & Aviram
• Conservative hypothesis in most studies ...
Critical Issues
• Reynolds (2008): Research and policy definitions hold significant
limitations for advancement of researc...
Critical Issues
• To take it 1 step further, a learning sciences approach to
designing digital literacy intervention requi...
This Study
• The definition and framework for digital literacy that follows
emerged out of an existing learning innovation...
1
2
3
4
Contemporary Learning Ability
(CLA)
Practices Representing Each CLA & How They Are Articulated/Integrated in Globa...
5
6 Figure 1. Globaloria learning objectives: Promote development of six contemporary learning abilities (CLAs)
Inquiry, i...
Constructionist, knowledge-building conditions supporting development
of 6-CLAs (Reynolds & Harel Caperton 2009; Reynolds ...
This Study
Building evidence base for the six dimensions of the CLAs
Research Questions and Hypotheses
RQ1: What is the em...
INTERVENTION: Guided discovery-based game design program
and curriculum offered by the WorldWide Workshop. MS, HS
teachers...
Learning Management System
as Information System
Learning Management System
as Information System
Methods
Non-experimental pre/post survey design, 2011/2012
school year of Globaloria
• Out of 1,356 middle and high school...
Methods
Operationalization of Digital Literacy:
• Self-reported frequency of engagement in twenty different
Globaloria act...
Results
• Factor analysis, reliability analysis indicate that the given survey
items for each factor hang together as 6 fa...
Regression Model Results
• We ran multiple regression models investigating how
change in program engagement in all 6 CLA d...
Regression Model Results
Regression Model Results
• Neither Gender nor proxy for SES (parent education)
predicted change in engagement @Home (very ...
Results
• Factors predicting change in Create @ Home:
• Change in Create @ School
• Change in Socialize @ School
• Change ...
Discussion and Implications
• Claim that the practices are contributing to each other in this
social cognitivist learning ...
Discussion and Implications
• This general learning objectives approach of digital literacy
conceptualization can support ...
Rebecca.reynolds@gmail.com
Rutgers University website
http://comminfo.rutgers.edu/directory/rbreynol/index.html
Thanks to ...
Upcoming SlideShare
Loading in...5
×

Challenges in Defining, Designing, and Measuring “Digital Literacy” Development in Learners: A Proposed Framework

175

Published on

This presentation discusses scholarly definitions for the research construct “digital literacy,” identifies limitations in conceptualizations to-date, fand presents a proposed framework of Six Contemporary Learning Abilities (or 6-CLAs: Create, Manage, Publish, Socialize, Research, Surf). This explicated framework offers a more structured definition based on student-centered social constructivist learning theory. The article then presents an empirical investigation of digital literacy development, drawing on the framework, and its proposed approach for operationalizing technology activities (whether as research constructs or instructional activities). The empirical analysis is situated in the context of an innovative educational program implementation of game design based learning for middle and high school students offered in a U.S. state, in the 2011/2012 school year. The study explores how student engagement in activities representing the 6-CLA dimensions factor, inter-correlate, change from pre- to post-program, and bring about student transfer of that engagement, from school to home environments. Findings reveal that the dimensions proposed hang together well, students change in their engagement as a result of the intervention across multiple dimensions in both school and home contexts, and at-school engagement in the dimensions contributes to at-home engagement in them (in various ways as reported). The study offers support for the proposed framework, provides some evidence of digital divide effects for the intervention, presents questions for further inquiry, and offers a conceptual and research design stake in the ground for other researchers interested in the digital literacy construct.

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
175
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Challenges in Defining, Designing, and Measuring “Digital Literacy” Development in Learners: A Proposed Framework

  1. 1. Challenges in Defining, Designing, and Measuring “Digital Literacy” Development in Learners: A Proposed Framework AERA, PHL, April 2014 Rebecca Reynolds, Assistant Professor School of Communication & Information Dept. of Library and Information Science Rutgers University John Wolf, Ph.D., Instructor, NJIT
  2. 2. “Digital Literacy” in Naturalistic vs. Prescriptive Contexts Naturalistic Phenomena in General Population • Digital Divide, Levels 1 and 2 (Hargittai & DiMaggio, 2002; Barzilai- Nahon, 2006) • Important cultural, social and political activities occur in online environments, and participation in such activities requires digital skills and knowledge (Mossberger, Tolbert, and McNeal, 2007; Jenkins, 2009, Hobbs, 2010; Horrigan, 2011) • More sophisticated forms of content creation, participatory engagement, digital knowledge and cultural benefits have been associated with privilege: higher socio-economic status and level of education (Pew, 2007; Hargittai and Walejko, 2008) • Connects digital literacy to capacity for social mobility (Hargittai, 2010)
  3. 3. “Digital Literacy” in Naturalistic vs. Prescriptive Contexts Prescriptive Instructional Contexts • Strand 2 of the U.S. FCC’s NBP of 2010 focuses on awareness raising and digital literacy efforts, esp. in schools • NETP of 2010: Optimizing learning and instructional design in US schools with technology-integrated project / problem- based learning interventions. • Prepare learners for 21st century work cultures • Role-taking, epistemic learning • Learning (engage & empower), Assessment (measure what matters), Teaching (prepare & connect), Infrastructure (access & enable), Productivity (redesign & transform) • AASL’s Standards for 21st Century Learner and ISTE National Education Technology Standards for students and teachers: Discuss student-level dispositions and latest versions bring in creativity and productivity
  4. 4. Definitions and Operationalizations Vary • Example 1, scholarship: Web-use skill (Hargittai,2005): “a user’s ability to locate content on the web effectively and efficiently” with no definition offered in her updated work in 2009; operationalizes digital literacy as a user’s familiarity with terminology reflecting certain technology concepts (JPEG, frames, preference settings, newsgroups, PDF, refresh/ reload, advanced search, weblog, bookmark, bookmarklet, spyware, bcc (on e-mail), blog, tagging, tabbed browsing, RSS, wiki, malware, social bookmarking, pod-casting, phishing, web feeds, firewall, cache, widget, favorites, torrent)
  5. 5. Definitions and Operationalizations Vary • Example 2, scholarship: Barron (2004) defines technological fluency as learners’ ability “to express themselves creatively, to reformulate knowledge, and to synthesize new information. [The concept] entails a process of lifelong learning in which individuals continually apply what they know to adapt to change and acquire more knowledge to be more effective at applying technology to their work and personal lives.” Operationalization: students’ self - reported frequency of participation in the following twelve constructive activities
  6. 6. Definitions and Operationalizations Vary Example 3, scholarship: Eshet & Aviram • Skeptical hypothesis 2 centers on technology situatedness of the definition one uses of digital literacy. • The definer must prioritize particular technology contexts of the moment, and any definition reflects the norms and values of the definer in assigning important to given technologies • “Clash of civilizations” ”Old, modern, rationalist, linear, conceptual, book-based culture of Western societies is CLASHING with new emerging post modern, branching, multi- media based reproduction oriented culture that has been developed in the last 20 years largely due to the spread of technology”
  7. 7. Definitions and Operationalizations Vary Example 1, policy: AASL Standards: “The definition of information literacy has become more complex as resources and technologies have changed. Information literacy has progressed from the simple definition of using reference resources to find information. Multiple literacies, including digital, visual, textual, and technological, have now joined information literacy as crucial skills for this century.”
  8. 8. Definitions and Operationalizations Vary Example 2, policy: NETS Standards
  9. 9. Definitions and Operationalizations Vary Example 3, scholarship: Eshet & Aviram • Conservative hypothesis in most studies rests on assumption that digital literacy is a set of distinct skills that we can measure in the population. • Theoretical questions (are skills distinct or are they really something else, like intelligence or motivation?) • Operation-oriented questions (how to measure) • Didactic questions (can they be taught?) • Development and design questions (how are they best taught?) • Skeptical hypothesis 1: No such thing as “digital literacy;” it is orthogonal with individual level capacities: • Motivation / Curiosity, Learning styles, Creativity, Intelligences, Capacities, Personality types • This hypothesis places the individual and his/her expertise at forefront
  10. 10. Critical Issues • Reynolds (2008): Research and policy definitions hold significant limitations for advancement of research because of the definitional variation, technology determinedness of operationalizations resulting in a need for constant update, and in policy, limited structural, material means for realizing ends specified (e.g., time, curricula, budgets). • Yet, given Digital Divide, and learning opportunities technology poses, the need for intervention clearly remains. • To design, implement and study processes and effects of interventions that inculcate “digital literacy” in learners requires clearly defining learning objectives and outcomes for the intervention, and developing the appropriate pedagogy. • Eshet & Aviram’s skeptical hypotheses advise (a) a focus on individual expertise, and (b) problematization of technology situatedness
  11. 11. Critical Issues • To take it 1 step further, a learning sciences approach to designing digital literacy intervention requires theoretical grounding of the instructional design principles
  12. 12. This Study • The definition and framework for digital literacy that follows emerged out of an existing learning innovation called Globaloria that applies social cognitive theory (e.g., Vygotsky) and Constructionism (e.g., Papert & Harel, 1991) in its design. • Reynolds & Harel Caperton 2009 outline the theoretical linkages between G’s instructional design principles and Constructionism • Reynolds & Hmelo-Silver (2013) identify conceptual linkages among Globaloria’s design, Constructionism and the knowledge building literature of Scardamalia & Bereiter. • The definition of “Contemporary Learning Abilities” we propose is student centered, and clearly distinguishes between the abilities and the technology contexts out of which they may emerge.
  13. 13. 1 2 3 4 Contemporary Learning Ability (CLA) Practices Representing Each CLA & How They Are Articulated/Integrated in Globaloria Invention, creation and completion of a digital project stemming from an original idea Brainstorming and developing game and simulation ideas and storylines using Web2.0 tools Writing an original game narrative and a proposal to explain it Generating creative ideas for designs to express the subject of the game and the user experience Planning/programming/completing a game demo that illustrates the original game design and functionality Developing knowledge of the game's domain or topic through game invention/creation/research Project planning, project management, teamwork (e.g., role- taking, task delegation), problem-solving Coordinating and managing the design/creation/programming of game elements Managing the project’s execution by creating/organizing a wiki and by sharing project assets and progress updates Managing team work by defining and assigning team roles/coordinating tasks/executing roles Project troubleshooting for self and others Gaining leadership experience through the project management of all game production elements Publishing and distribution of self-created digital media artifacts to an audience and/or community of peers Creating a wiki profile page and project pages Integrating and publishing text/video/photos/audio/programming code/animations/digital designs on wiki pages Posting completed assignments/game design iteration and assets/notes and reflections about projects to wiki Developing a blog Giving and getting feedback about project through social interaction, participation, exchange Collaborating by using Web2.0 tools such as posting to wikis/blogs/open source help forums/instant messaging Exchanging/sharing feedback and resources by posting information/links/source code questions/ answers Reading and commenting on others’ blogs and wiki pages Presenting final digital projects for others both virtually in game galleries and in person in live game demonstrations
  14. 14. 5 6 Figure 1. Globaloria learning objectives: Promote development of six contemporary learning abilities (CLAs) Inquiry, information-seeking, agentive use of resources (human and text/digital content), to support the artifact’s topic/message, and design/execution Searching the Web for answers and help on specific issues related to programming games Searching and finding resources on MyGLife.org network, website, and wiki Searching the Web for new Flash design, animation and programming resources Searching for information in support of the game’s educational subject and storyline Surfing, experimentation and play with existing Web applications and tools Surfing to MyGLife.org starter kit site and other game sites and playing games online Keeping track of and bookmarking surfing results that are relevant to projects Browsing Web2.0 content sites such as YouTube, Flickr, blogs, Google tools
  15. 15. Constructionist, knowledge-building conditions supporting development of 6-CLAs (Reynolds & Harel Caperton 2009; Reynolds & Hmelo-Silver, 2013)
  16. 16. This Study Building evidence base for the six dimensions of the CLAs Research Questions and Hypotheses RQ1: What is the empirical relationship among the six proposed dimensions of contemporary learning? RQ2: How does the introduction of Globaloria intervention in school affect frequency of engagement in CLA-based activities and practices in non-school settings? H1: The intervention will have a larger effect on frequency of engagement for the more constructionist dimensions of contemporary learning.
  17. 17. INTERVENTION: Guided discovery-based game design program and curriculum offered by the WorldWide Workshop. MS, HS teachers and students gain experience and expertise in a range of agentive digital practices.
  18. 18. Learning Management System as Information System
  19. 19. Learning Management System as Information System
  20. 20. Methods Non-experimental pre/post survey design, 2011/2012 school year of Globaloria • Out of 1,356 middle and high school student participants, a total of 1,063 completed the pre-survey (78.4%), and approximately 670 completed the post-survey (49.4%). • Sample includes pre/post only students: 282 middle school and 781 high school students participated. • The pre-intervention sample was 29.0% female and 71.0% male.
  21. 21. Methods Operationalization of Digital Literacy: • Self-reported frequency of engagement in twenty different Globaloria activities, mapped to the CLAs • Improvements: Definition and operationalization of digital literacy are theoretically grounded in the social cognitivist, Constructionist and knowledge-building inspired instructional design features • That is . . . The chosen technology activities included in our operationalization of digital literacy reflect the theory that inspired the instructional technology context’s design About this, we are explicit.
  22. 22. Results • Factor analysis, reliability analysis indicate that the given survey items for each factor hang together as 6 factors (note, all CLAs had 3 or more items, except the Publish factor (CLA 3) which had only 1 item). • Bivariate correlational analyses indicate that at school, pre- program results indicated the strongest apparent correlations among the more constructionist dimensions of contemporary learning (CLAs 1-3) • Repeated measures ANOVAs indicate that students increased in their frequency of engagement in all CLA dimensions @School from pre- to post-program except Surfing (CLA 6) • Repeated measures ANOVAs indicate that students increased in their frequency of engagement in CLA dimensions 1,2, 3 and 5 (Create, Manage, Publish, Research) @Home from pre- to post- program
  23. 23. Regression Model Results • We ran multiple regression models investigating how change in program engagement in all 6 CLA dimensions @School contributed to change in the 6 dimensions @Home. For instance: • Does change in Creation @School contribute to change in Creation @Home? • Does change in Research @School contribute to change in Creation @Home?
  24. 24. Regression Model Results
  25. 25. Regression Model Results • Neither Gender nor proxy for SES (parent education) predicted change in engagement @Home (very interesting in light of digital divide research; Globaloria factors may mitigate digital divide factors; Reynolds & Chiu, 2013 in fact bear this out) • Engagement @School led to an increase in engagement @Home, even though no homework.
  26. 26. Results • Factors predicting change in Create @ Home: • Change in Create @ School • Change in Socialize @ School • Change in Surf @ School • Factors predicting change in Manage @ Home: • Change in Manage @ School • Change in Socialize @ School • Change in Surf @ School • Factors predicting change in Publish @ Home: • Change in Socialize @ School • Change in Surf @ School • Factors predicting change in Socialize @ Home: • Change in Socialize @ School • Change in Surf @ School • Factors predicting change in Research @ Home: • Change in Research @ School • Change in Surf @ School • Factors predicting change in Surf @ Home: • Change in Research @ School • Change in Surf @ School
  27. 27. Discussion and Implications • Claim that the practices are contributing to each other in this social cognitivist learning experience are supported by the findings • Surfing, experimentation, play, along with social engagement in the context drive transfer of engagement to home (measure of motivation) • For the more effortful, creative work (Creation, Publish), experiencing those practices @School is influencing students to engage at home, too. • Contribution: • Study shows how when Digital Literacy expertise domains, technology contexts and instructional design features are more clearly articulated and identified, it creates opportunities to explicate factors of influence
  28. 28. Discussion and Implications • This general learning objectives approach of digital literacy conceptualization can support improvements in more pragmatic and executable educational policy recommendations, instructional design, measurement and assessment
  29. 29. Rebecca.reynolds@gmail.com Rutgers University website http://comminfo.rutgers.edu/directory/rbreynol/index.html Thanks to IMLS! Thanks to my partners! Globaloria.org Worldwideworkshop.org Thank you!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×