A general overview of my research agenda, as of Dec. 2013

1. Supporting guided
discovery based learning
with informational
learning management
systems: Challenges and
Opportunities
Nina Wacholder Class Guest Lecture
Rebecca Reynolds, Assistant Professor
School of Communication & Information, Dept. of Library and
Information Science, Rutgers University

2. Alignment: Learning System Design
Across Theoretical Perspectives
Discipline
Theoretical Main Principles of
Important Research Constructs
Perspective Curricular Design
Learning
ProblemStudent centered, active, Effectiveness of varying types of
Sciences
based learning self-directed learning that scaffolds for collaboration and
starts with a problem and problem resolution; cognitive load
involves peer collaboration
alongside human expert
and/or system scaffolding
Educational Project based Constructing projects in a Role-taking, communication,
Psychology; learning
situated epistemic (real life) collaboration, iteration, standards of
human
context leads to deep(er) performance, assessment
development
learning; driven by an
open-ended question or
challenge
Information Inquiry-based Driven by a question,
Question- vs. task-driven contexts;
Science
learning;
scenario, problem; learner naturalistic vs. imposed learning
Guided
pursues answers through contexts; prerequisite expertise;
Inquiry
information-seeking
emotion; cognitive load; info literacy

3. Constructionism:
My Own Path to These Ideas
• Idit Harel, MIT Media Lab, World Wide Workshop, Globaloria
•
In the Constructionist framework for educative action (Papert & Harel,
1991; diSessa & Cobb, 2004), learners:
•
•
•
a) create a computational artifact in a workshop-based learning
context, working in teams
b) engage with and learn a core content domain, AND,
c) use resources to support their learning
•
Creative production, design and programming are the central purposes
(tasks) that drive the inquiry and collaboration activity.
•
History of research charting meta-cognitive, affective, learning gains
(Papert & Harel, 1991; Harel, 1991; Kafai, 1995; Bruckman & Resnick, 1995; Kafai &
Resnick, 1996; Urrea, 2001, 2002; Cavallo, 2004; Kafai & Ching, 2004; Kafai, 2006;
Peppler, Kafai & Chiu, 2007; Klopfer, 2008; Reynolds, 2008)

4. Common Threads
• Social constructivist theoretical underpinnings (Dewey; Piaget /
•
•
•
•
•
Vygotsky)
Pedagogy is optimally developed to both leverage and cultivate
human agency
Situated, distributed cognition (class- and team-levels)
Involves inquiry, information seeking, and application of
information into some kind of product (problem answer,
project, creation/presentation)
Converging evidence base (E.g., Kuhlthau, Todd, Chu, HmeloSilver, Martin, Kapur & Kinzer, Blumenfeld, Eccles, and
Constructionist cites)
HOWEVER, STILL DEBATES . . . “discovery-based learning” &
cognitive load (Kirschner, Sweller, Clark, 2006)

5. INTERVENTION: Guided discovery-based game design program
and curriculum offered by the World Wide Workshop. MS, HS
teachers and students gain experience and expertise in a range of
agentive digital practices.

6. Globaloria is currently active
in 4 U.S. states: CA, TX, NY, WV, >2000 students

7. Learning Management System
as Information System

8. Learning Management System
as Information System

9. Learning Supports for Students and Educators:
Flash software, Wiki Environment, Curriculum,
Tutorials
“Hands On” Training Sessions (virtual, local)
• Globaloria Academy – In-person, intensive trainings (3)
• Online Mini Webinars - Web-based workshops (7)
Globaloria Mentors Program
Experienced educators take on a leadership role by
supporting other educators
“24/7” Virtual Support
• Expert Support via wikis, blogs, email, WebEx
• Educator Community Development – private
educators community wiki, peer-to-peer mentoring, weekly
educators newsletter, sharing teaching & learning reports
Rewards and Recognition
•Teachers: Stipends and Graduate credits are earned
•Students: Nationally-Recognized Game Design
Competitions

10. Domains of Learning and Expertise
•
Game Example
•
Constructionist digital literacy (skills needed in knowledge economy =>
6-CLAs)
•
Computational thinking through game design in Flash and programming in
Actionscript
•
Core curricular subject matter:
o When game subjects are linked to core curriculum and students deepen
knowledge about topic through online research and design
•
STEM career interests: Technology & Engineering; Computer Science
•
Motivation, Affect, Attitudes, Life Choices, New Possibilities and Horizons

11. Globaloria Game Design Program Learning Objectives:
Cultivate the Six Contemporary Learning Abilities (6CLAs)
(Reynolds & Harel, 2009)
Developing games in a social e-learning system cultivates participatory
practices that simulate productive engagement in today’s digital cultures
and knowledge-based economy

12. Research Agenda: Investigate Collaborative Information
Behavior in Globaloria
o Collaboration & information seeking are meta processes supporting the
primary game design task
o Teams of 2-4 collaborate to complete a game (& some indivs)
o Unstructured collaboration
o Little scaffolding for effective engagement (e.g., no inquiry circles
(Kuhlthau, Maniotes, Caspari, 2007)
o Role taking, though, is encouraged
o Learning management system:
o Information resources
o Supports social engagement and project management (e.g., the
team page)
o Simulates future professional work contexts studied in areas such as
CSCW, therefore, the CIS literature may be relevant to phenomena
occurring in this Constructionist, knowledge building project among youth

13. My Research: Issues of Structure and Agency
in Socio-Technical e-Learning Systems Research
• In Constructionist / discovery-based / inquiry-based learning:
Learners
• a) engage with and learn a core content domain, AND,
• b) engage with and find resources to support their
learning
• Simulates informal, naturalistic learning
• Gives students experience with autonomous, agentive
inquiry and creation
• In Constructionist learning, in particular, creative production,
design and programming are the central purposes that drive
the inquiry.
• GIVEN DEBATES about effectiveness of guided discoverybased learning interventions. . . .

14. My Research: Issues of Structure and Agency
in Socio-Technical e-Learning Systems Research
• I am investigating student
engagement in these
general inter-related
spheres of activity
• My research considers
the design of the learning
management system as a
support (or hindrance) for
activity in these spheres,
in this game design
implementation context.

15. Select results so far in the Globaloria context:
Reynolds (2012), AERA:
• Especially for practices representing the more
Constructionist engagement categories, survey analysis (ttest) indicates increases in student motivation towards, and
self reported understanding of these practices as a result of
participation.
• For three other less-Constructionist but active technologyuse categories (e.g., information seeking), the results were
more varied with regard to statistically significant increases.
Ceiling effects may have played a role in this.
• Indicates a measure of success in the program at meeting
the stated learning objectives.

16. Results so far in the Globaloria context:
o Reynolds & Harel Caperton (ETR&D, 2011): Qualitative,
student self-reports of experience => Student affect and
attitudes vary towards the autonomy-supportive, semi- and
ill-structured activities in Globaloria
• Some +, some • Why the variation?

17. Intrinsic Motivation in Globaloria Students
o Reynolds (ICA, 2011): intrinsic motivation positively
correlated with game quality (measured thru content
analysis); extrinsic motivation negatively correlated
o Reynolds & Chiu (ICLS, 2012): Multi-level analysis model at
team level of analysis supports 2011 findings for intrinsic
motivation positive contribution to team outcome scores
o Results qualify Kirschner et al‟s critique:
o Intrinsic motivation contributes positively to outcomes
in guided discovery-based learning
o Such programs may support those with this orientation

18. Digital Divide
o Reynolds & Chiu, 2013 (submitted): Student participation
attenuates digital inequality effects (factors that predict DL in
prior research also influence students‟ self-reported
technology engagement prior to Globaloria participation, but
no longer contribute after the fact).

19. What are the mechanisms?
Self determination theory: Self-determined, fulfilling
intrinsically motivated engagement coincides with
perceived competence, autonomy, social relatedness
(in individual and as qualities supported by
environment) (Deci & Ryan, U-Rochester)
Latest Model findings:
Student resource uses including their use of online tutorials
(text-based), use of online tutorials (video-based), use of
self-sought online resources on the wider internet
contributed positively to team outcome scores, AND, to
CHANGES IN INTRINSIC MOTIVATION
• Inquiry plays a role. . .
• But we need to investigate *when* inquiry may also
detract.
• Multi-level analyses continue

20. Collaborative Information Behavior in Globaloria
(Reynolds, Hmelo-Silver, Sorenson, & Van Ness, 2013; Reynolds, Baik, Li, 2013).
o Collaboration & information seeking are meta processes
supporting the primary game design task
o Teams of 2-4 collaborate to complete a game (& some indivs)
o Unstructured collaboration
o Little scaffolding for effective engagement (e.g., no
inquiry circles (Kuhlthau, Maniotes, Caspari, 2007)
o Role taking, though, is encouraged
o Learning management system:
o Information resources
o Supports social engagement and project management
(e.g., the team page)
o Simulates future professional work contexts studied in areas
such as CSCW, therefore, the CIS literature may be relevant
to phenomena occurring in this Constructionist, knowledge
building project among youth

21. Research Questions, Methods
(Reynolds, Hmelo-Silver, Sorenson, & Van Ness, 2013; Reynolds, Baik, Li, 2013).
o “What collaborative information behaviors do middle school
students evidence when given the chance to self-organize
their game design task-driven teamwork?”
o Step 1: Categorical analysis
o Data sources:
o video recorded face-to-face and virtual interviews with
student teams, transcripts, coded in 2 waves in
Dedoose
o Step 2: Case study analysis of 4 distinctly dissimilar
teams (Firestone, 1993)
o Data sources:
o wiki log files, wiki history, teacher quarterly progress
reports, game design evaluation results, and individual
student blogs

22. Peer Help
o Many peer help examples; this was very common
o Assessment and perceptions of cheating: “…cause
usually other teachers…when you ask a student, they think
you are cheating „cause students do the same thing. But in
this class you can‟t really cheat, because we all work on
different things... the other two partners work on astronomy
and you work on math ….so how are you gonna cheat?”
o Challenges norms around individualized universal
education where all students learn the same material
o Other evidence existed, though, around issues of
intellectual property – students holding on to ideas and not
posting them online; conflicts between teams about idea
stealing, etc.
o Not about assessment  innovation and authorship

23. Role-Taking and Division of Labor
o Some teams adopted the strategy of individuals taking on
multiple roles; others remained in single roles throughout
o Distribution of work based on perceived expertise, with
some students gaining recognition at class level:
o "I‟m a little bit better drawer than he is and a better searcher and
he‟s better with the code and getting things to work than I am. So I
usually do the drawing and save to link, and he usually does the
coding and makes things to work.”
o “We are honest with each other, so say somebody performs
something better than somebody, we will tell them, say, William did
buttons better than Justin. We got to be honest or we‟re not going to
get a full game. I‟m good at flash and he‟s good at buttons…at first
we were kind of raw, but we all worked together.”
o Assessment implications of specialization

24. Role-Taking and Division of Labor
o Distributing the work to meet deadlines:
“We were trying to work with each other at first, and do all
the levels on our own but I didn‟t get what my level was
supposed to be. So what I did was put it on my flash drive
and on my wiki, and I started working on my own, so it
was split in two levels. So she had that one and I have
mine. Yeah, so we can do whatever we want on our own
level. We just split apart like that, split the work into
halves.”

25. Version Control
o Developing (misguided) project management
workaround:
o “Well, what we usually do is use that one computer just for
flash and the other one for the wiki and to code video things
and we usually switch computers for flash. We usually just
follow the team page and when we start to make ourselves
confused, we do that.”
o “Well what we do is usually…she logs in as me cause I keep
all my stuff there. I save it first and then she saves it then we
just kinda like…”

26. Videos
o Example 1
o 2 ninth graders (male, female)
o Individual level work to create mobile games (pilot)
o Male student‟s project was a track and field racing
game including nutritional information for athletes as
quiz content (personal interest of the student)
o Off-task collaborative information seeking behavior is
demonstrated
o Related, but peripheral to main game topic, which
o Male student was searching for information on a
sports injury
o Female student seated adjacently takes over the
search

27. Opportunity:
Collaborative Information-Seeking is Engaging
Vid omitted

28. Videos
o Example 1 Observations:
o Dialogue, conversation, possibly flirtation
o Search is used as a constructive tool for social
discourse across genders, race, ethnicity in a school
where racial tensions exist
o Off-task engagement might distract….
o Might also support social engagement, attitudes
towards school / project, sense of belonging
o Empathy
o Information credibility?
o Health Q&A
o Opportunities, and Challenges
o Information literacy instruction . . .

29. Videos
o Example 2
o Informal discussion w educators on system limitations
for information search

30. Challenge:
Information Use in Complex PBL/Constructionist
Interventions is Under-Structured
Vid omitted

31. How does Globaloria structure and scaffold students?
(Reynolds, Hmelo-Silver, Sorenson, & Van Ness, 2013; Reynolds, Baik, Li, 2013)
Qualitative research findings:
• Primary task (game design) is supported; secondary tasks
(collab; inquiry) under-supported
• Collaboration & Inquiry meta-processes are
under-structured
• Discovery process not scaffolded. . .
• Guided Inquiry model of Kuhlthau may inform
ongoing modifications for information literacy;
collaboration (e.g., reciprocal teaching / inquiry
circles)

32. Knowledge Building as Conditions to
Cultivate in Instructional Design
Reynolds & Hmelo-Silver. (2013). Areas of Convergence in Constructionism,
Knowledge Building and Guided Discovery Based Learning in the Globaloria
Game Design Initiative. Presented at AERA 2013 in San Francisco, CA.
Scardamalia & Bereiter (2006) state these in brief as:

Knowledge advancement as a community rather than individual
achievement
 Knowledge advancement as idea improvement rather than as progress
toward true or warranted belief
 Knowledge of in contrast to knowledge about
 Discourse as collaborative problem solving rather than as argumentation
 Constructive use of authoritative information
 Understanding as emergent

33. Information Sciences Contribution
 Inter-disciplinary cross-
walks are important
 IS perspectives such as
CIS/CIB can help strengthen
understanding of
collaborative and
information meta-processes
in IBL/PBL contexts
 Contribute to the design of
curricular supports

34. Scaffolding Information Literacy and Collaboration
• Leverage agency inherent to inquiry; cultivate agency in
students
• But don‟t lose the learning and knowledge production needed
during the inquiry process
• Creating information literacy modifications in Globaloria
• Help students track sources, find and use information more
effectively
• Delicious; NoodleTools; others?
• Ross Todd: “Transportation vs. Transformation”
• Internet reading comprehension literature; Donald Leu
• School librarians can help; professional development of SLs
as information literacy experts and curriculum developers =>
Common Core emphasis on non-fiction information texts;
digital environments

35. References
•
•
•
•
•
•
•
•
Reynolds, R., & Chiu, M. (2013). Context matters: The effect of formal and informal context differences
upon pre- to post-program changes in student engagement in a program of game design
learning. Journal of Learning, Media & Technology.
Reynolds, R.; Baik, EB & Li, X. (2013). Collaborative information seeking in the wild: Middle-schoolers’
self-initiated teamwork strategies to support game design. Paper presented at the annual convention of
the American Society for Information Science and Technology (ASIST), 2013.
Reynolds, R., & Chiu, M. (2013). How sustained engagement in game design and social media use
among diverse students can mitigate effects of the digital divide. Paper presented at the annual
convention of the American Education Research Association (AERA), San Francisco, CA, April, 2013.
Reynolds, R., Hmelo-Silver, C., Sorenson, L., & Van Ness, C. (2013). Interview findings on middle
schoolers’ collaboration in self-organizing game design teams. Poster presented at the International
Conference of Computer-Supported Collaborative Learning, July 2013, Madison, WI.
Reynolds, R., and I. Harel Caperton. 2011. Contrasts in student engagement, meaning-making, dislikes,
and challenges in a discovery-based program of game design learning. Educational Technology
Research and Development 59 (2): 267–289.
Reynolds, R. (2011). Children's game design learning in discovery-based contexts: Contribution of
intrinsic and extrinsic motivational orientations to student outcomes. Paper presented at the annual
International Communication Association (ICA) conference, May 2011, Boston, MA.
Reynolds, R. (2012). Changes in student attitudes towards 6 dimensions of digital engagement in a
program of game design learning. Paper presented at the annual conference of the American Education
Research Association (AERA), April 2012, Vancouver, Canada.
Reynolds, R., & Chiu, M. (2012). Contribution of motivational orientations to student outcomes in a
discovery-based program of game design learning. Paper presented at the annual conference of the
International Conference of the Learning Sciences (ICLS), July 2012, Sydney, Australia.

36. Thank you!
Rebecca.reynolds@gmail.com
Rutgers University website
http://comminfo.rutgers.edu/directory/rbreynol/index.html
Thanks to IMLS!
Thanks to my partners!
Globaloria.org
Worldwideworkshop.org