SlideShare a Scribd company logo
1 of 64
Cindy E. Hmelo-Silver
Rutgers University
Complex Learning
Providing Scaffolds and Creating Contexts
Problem-based Learning: STELLAR
Creating contexts with the VMC
Complex Systems: Systems and Cycles
We live in a complex and dynamic world
Need to go beyond learning isolated knowledge
and facts
   Useable knowledge
   Soft skills (Derry & Fischer, 2007)
Preparation for lifelong learning, reasoning, and
problem solving (Fischer & Sugimoto, 2006)
   Useable knowledge
Transfer, from a range of perspectives
Such learning:
   Often situated in problem-based and inquiry
   learning environments (Hmelo-Silver, Duncan, &
   Chinn, 2007)
   Potential for excessive cognitive load (van
   Merriënboer, Kirschner, & Kester, 2003)
Appropriate scaffolding and contexts to deal
with cognitive and social challenges
Provide support to allow learners to
   Competently do task
   Learn from task
Builds on notion of ZPD (Vygotsky, 1978)
Scaffolding complex tasks through
   Structuring
   Problematizing
Three primary kinds of scaffolding
   Communicating process
   Coaching
   Eliciting articulation
Hard and soft scaffolds (Saye & Brush, 2002)
Contextual support
    Video, Hypermedia in STELLAR (Teacher Education)
    VideoMosaic Collaborative Repository (VMC)
    Systems and Cycles simulations
Collaboration spaces
    STELLAR whiteboards, threaded discussion
Access to structured information
   STELLAR Knowledge Web
   VMC metadata
   Systems and Cycles hypermedia
Scaffolding through Interface and activity structures
   pbl online in STELLAR
   VMC analytic
   Systems and cycles curriculum materials
Processes
  Collaborative knowledge building
  Engagement
  Learning trajectories
Outcomes
  Application
  Transfer
    Expert perspective (e.g., Barnett & Ceci, 2002)
    Preparation for future learning (e.g., Schwartz &
    Bransford, 1998; Schwartz & Martin, 2004)
    Actor-oriented transfer (Lobato, 2006)
With Sharon Derry, Anandi
Nagarajan, Ellina
Chernobilsky
Initial implementation (Hmelo-Silver, 2000)
  Paper cases
  One wandering facilitator for 6-7 groups
Limitations
  Cases were oversimplification
  One wandering facilitator for 6-7 groups
  Difficulty identifying fruitful learning issues because
  of limited and variable prior knowledge
Creating Context:
   Provide rich video cases of practice
   Concepts in context
Scaffolds
   PBL online activity structure: extend skilled
   facilitation resources
   Knowledge Web: CFT Hypermedia support for
   generating learning issues
Pre-post across institutions using video analysis
task
   Significant pre to post gains for both sites, with
   different implementations
Quasi-experimental design over 3 semesters
   Participants:
     101 STELLAR PBL students in 18 groups
     126 comparison students from Educational Psych
     subject pool Tracer concepts “Understanding”
     on video analysis
   Moderate to large effects over three years
   Between group variability striking
Group 1
                                                                 Facilitator
                                                                 CHS
                                                                 Ann
 50                                                              Fran
                                                                 Cathy
                                                                 Fauna
                                                                 Luke
 45                                                              Other monitoring
                                                                 SDL
                                                                 Group monitoring
                                                                 Individual monitoring
 40                                                              Grounded beliefs
                                                                 Personal beliefs
                                                                 Elaborations
 35                                                              Explanations
                                                                 Transforming
                                                                 Elaborated telling
                                                                 Telling
 30                                                              Acknowledgement
                                                                 Summary
Codes




                                                                 Disagreements
 25                                                              Agreements
                                                                 Modifications
                                                                 New Ideas
                                                                 Metacognitive questions
 20                                                              Explanation questions
                                                                 Information questions
                                                                 Personal talk
                                                                 Concept talk
 15
                                                                 Tools as help
                                                                 Tools as a problem
                                                                 Task talk
 10                                                              View Other Proposals
                                                                 Research Library
                                                                 White Board
                                                                 Discussion Board
   5                                                             Notebook
                                                                 KW
                                                                 Video
   0
    300   400   500   600   700        800   900   1000   1100
Group 2                                      Facilitator
                                                                                     CHS
                                                                                     Matt
                                                                                     Bob
        50                                                                           Carla
                                                                                     Caitlin
                                                                                     Liz
                                                                                     Helen
        45                                                                           Other monitoring
                                                                                     SDL
                                                                                     Group monitoring

        40                                                                           Individual monitoring
                                                                                     Grounded beliefs
                                                                                     Personal beliefs
                                                                                     Elaborations
        35                                                                           Explanations
                                                                                     Transforming
                                                                                     Elaborated telling
                                                                                     Telling
        30                                                                           Acknowledgement
                                                                                     Summary
Codes




                                                                                     Disagreement
        25                                                                           Agreement
                                                                                     Modifications
                                                                                     New Ideas
                                                                                     Metacognitive questions
        20                                                                           Explanation questions
                                                                                     Information questions
                                                                                     Personal talk
        15                                                                           Concept talk
                                                                                     Tools as help
                                                                                     Tools as a problem
                                                                                     Task Talk
        10                                                                           View Other Proposals
                                                                                     Research Library
                                                                                     White Board

         5                                                                           Discussion Board
                                                                                     Notebook
                                                                                     KW
                                                                                     Video
         0
          450   550   650   750   850   950       1050   1150   1250   1350   1450
                                        Lines
Similar type of rubric developed to measure
“transfer” (Hmelo-Silver et al, 2009) on 0-3
scale
Components of transfer rubric:
1. requires understanding,
2. involves activating appropriate prior knowledge
   and applying something learned in a new situation,
3. involves abstraction and cognitive flexibility,
4. can be near or far transfer, and
5. can be preparation for future learning.
Pretest and Posttest Scores by Class Type

       Class        N        Pretest            Posttest

    STELLAR         33     0.71 (0.31)         2.02 (0.69)

    Traditional     37     0.61 (0.36)         0.68 (0.34)


        F (1,67) = 114.323, p < .001, d=2.55
Large variability in groups
Examined STELLAR whiteboards for contrasting
cases analyses
  Engagement with “Transfer”
Group A
  6 female students who had some difficulty
  Mean gain= 1.40, SD=0.89
Group B
  6 female students, rarely needed any assistance
  Mean gain= 1.33, SD= 0.61
Discussed transfer in 3 of 4 problems
   In Problem 1, Jenny proposed explanation for
   enduring understanding that child in video
   developed:
“In the case of Brandon, he needed to have an
understanding of how and why he was able to solve
the block problem in order to transfer his ideas onto
the pizza problem. "The first factor that influences
successful transfer is degree of mastery of the original
subject" (How People Learn, 53). Brandon was able to
continue to solve such a problem because of his
complete understanding of how he was able to arrive
at the solution for the block problem.”
In problem 2, Rina used the concept of transfer in
   thinking about assessment as she offered this proposal:
…The portfolio should have a final summary of the students'
work and questions regarding the students' learning, so that
the students can explain and evaluate their own thinking.
(knowlege web [sic]) The students should be able to transfer
their prior knowledge of concepts such as force and motion
in order to create their vehicle, while also allowing the
activity to expand on that knowledge. …another important
facet of understanding is application (sic). Ms. Baker will
know whether the students acquired enduring
understanding by how much they can apply this knowledge
to real world problems. One way of doing is to have Ms.
Baker create another problem that will use the same
concepts in a real world setting, and evaluating whether the
students were able to apply the concepts they had learned.
All students mentioned transfer in P1; 2-4 students in
   subsequent problems
   Kathy wrote:
…Second, Brandon was able to recognize a connection
between the pizza problem and the tower problem that he
did weeks earlier. Moreover, he made this connection
relatively quickly and without much effort. He was able to
show us, by using his chart and the manipulatives (blocks),
exactly how the pizza problem mapped into the tower
problem. His understanding of the pizza problem therefore
facilitated a new, and deeper, understanding of the block
problem; this process is called transfer. Brandon’s seemingly
effortless use of transfer provides evidence that he
understood the problem, because “transfer and wide
application of learning are most likely to occur when
learners achieve an organized and coherent understanding
of the material.” (How People Learn, p. 238-239)…
On later problems, fluid application of concept part of shared
    understanding
Micki writes:
…another way to look for enduring understanding would be the students'
transfer and application of principles of force and motion especially to
real world situations. This would show the student's understandings of
information previously and transfer it to the problem at hand, which is a
real world problem that allows students to work with hands-on material.

    Mimi followed this up by incorporating Micki’s comment and a
    previous proposal from another group member:
To put these two ideas together, [t]he teacher could bring together
individual explanation and transfer as evidence of enduring
understanding. An activity could be created at the end of each project
that would ask the individual members of the group to use the principles
gained to explain a real world scenario. Likewise, an activity could be
designed to facilitate transfer of the instructional objectives. For instance,
one of the objectives was learning the scientific inquiry process. The
teacher could present a real world problem that would require the
students to use the same scientific process to solve. (This would also
facilitate transfer)
Technology:
  Scaffolds
  Context
Complex measures:
  Knowledge in use
  More and less
  productive
  collaborations
with Carolyn Maher, Marjory Palius,
Grace Agnew, Robert Sigley, Chad
Mills
www.videomosaic.org
Preserves a major video data collection on student
reasoning
   From diverse schools settings
   To be available as open source
   From 40 doctoral dissertations
Makes available new tools for
   Teachers
   Educators
   Researchers
From longitudinal/cross sectional studies spanning
25 years
Videos following the same student cohort from
elementary school through high school and beyond
Over 4500 hours of video
Video Mosaic Collaborative
(VMC)
Importance of making sense of students’
     conversations and how tools mediate learning
     (Hmelo-Silver, 2003)
     Being aware of the contextual resources
     (media, other Ss, prior experience) that Ss use
     influence collaborative knowledge construction
     (Arvaja et al., 2006)
     Attending to social interactions in collaborative
     knowledge construction (Palincsar, 1998)



30
Graduate                Online Design
mathematics ed             eCollege CMS
hybrid course           Streaming video /
Four online groups      linked papers
2+ week unit            Minimal online
  In class problem      instructor intervention
  solving               Data
  Individual study of      Postings from online
  videos and related       threaded discussions
  readings                 Pre and post tests
  Group discussion         (math, Ss reasoning)
  questions
To what extent do videos and readings
     promote online discussion within and across
     groups?
     How do learners relate practice to online
     discussion?
     To what extent do Ss relate videos to readings
     in their online discussions?



32
All posts coded for comments related to
          Video (V)
          Readings (R)
     Additional sub categories of comments relating
       videos/readings to:
          Own problem solving (PV/PR)
          Others’ problem solving (OV/OR)
          Earlier interventions (EV/ER)
          Affect (AV/AR)
          Practice (TV/TR)

33
Shows two groups of 10th graders (2 in one &
         3 in other) working on the problem:
         How many different block towers can be built,
         four tall, selecting from three colors of blocks
         such that the towers have at least one block of
         each color?
         Approximately 8 minutes
     http://hdl.rutgers.edu/1782.1/rucore00000001201.Video.000062055




34
(1) Describe Romina’s strategy for solving the Ankur’s
    Challenge problem.
(2) In your opinion, is this solution a convincing one?
    Why or why not?
(3) According to the Yackel & Hanna chapter, both von
    Glaserfeld and Thompson equate reasoning with
    learning (p. 227). From this perspective, in what ways
    do explaining and justifying contribute to learning
    mathematics?
Percent of Posted Discussions
                                    Related to Video Activity       Overall

SS Relate Videos       N
                           a

and Readings to:
                               A        B         C        D
Own problem            27      100.0%   100.0%    100.0%   83.3%    95.7%
solving (PV/PR)
Others’ problem        13      100.0%   100.0%    100.0%    85.7%   92.3%
solving (OV/OR)
Earlier                10      66.6%    100.0%    100.0%    33.3%   70.0%
interventions in the
class (EV/ER)
Enjoyment              20      100.0%    83.3%    100.0%    80.0%   90.0%
(AV/AR)
Practice (TV/TR)       24      25.0%     71.4%    75.0%     12.5%   40.7%

a
    N= Number of posts
Percent of Posted Discussions
                         Relating Videos to Assigned
      Topic                       Readings

                              Discussion Group
                       A         B        C         D

SS relate videos to   16.7%    16.7%   45.5%*     7.1%
assigned readings
(RV)

Number of posts        18       18        11       14
Across all groups:
   Studying videos generated reflections about own and
   classmates’ problem solving
   Studying videos of students’ reasoning was enjoyable

But still left us with question of ways in which
which Ss related resources to their practice
Create multimedia artifacts using
                            the VMC repository
                            Narrative with video for purpose
                            Variety of uses by instructors,
                            researchers
                            Goal to classify and identify what
                            differentiates high quality and low
                            quality “analytics”
                            Data sources: 27 VMCanalytics from
                            several different classes and
                            researchers
                            Work very much in progress




Agnew et al, 2010; Hmelo-
Silver et al., in press
VMC Analytic by Hmelo-Silver (2011)




VMC Analytic by Horwitz (2011)
Math    LS
    Class   Depth   Depth   Clarity   Coherence
Design-
based
Research    0.78    1.00    1.22         1.33
Intro to
Math Ed     1.96    1.80    2.36         2.20
Critical
Thinking    2.00    2.40    2.30         2.00
Practicum   3.00    2.25    2.63         2.63
No Class    1.17    1.67    3.00         3.00
Looking deeper with contrasting case analysis
Example 1: Analytic illustrating how students can move
from particular to general
   Concepts from both learning sciences and mathematics
   clearly articulated
   Indicative of designer's understanding of students’ learning
   trajectory
Example 2: Analytic illustrates teacher questioning during
early algebra exploration
   Students claims not supported by video segments selected
   Textual descriptions of events were vague
   Video not well chosen for intended purpose of relationship
   of teacher questioning and student engagement
VMCAnalytics coded for
emergent themes
Explored use of word
clouds as a learning
analytic
  Mathematics Education     Word Cloud of coded Mathematics themes

  ideas
  Learning Sciences ideas

Allow us to see dominant
themes within the two       Word Cloud of coded Learning Sciences themes


areas of interest
with Rebecca Jordan, Catherine Eberbach,
Suparna Sinha, Lei Liu, Steven Gray, Wes
Brooks, Yawen Yu
Goals
  Learning about ecosystems
  Reasoning about evidence
  Modeling
6 week curriculum
Creating Contexts
Scaffolding complex
learning
Understanding how and
why along with what
Aquarium Design

Eutrophication in
local pond

Marines problems
caused by ocean
acidification
Provide context for:     Help focus on function
   Discussions           and behavior
   Science practices     Make invisible visible
   Engage with complex   and open for inspection
   systems phenomena
Use the following arrows to help you decide how the evidence relates to the different
explanations:

                                      Solid Arrow
                                                               Evidence supports the
                                                                      model
                                                                                           Conceptual
                                     Wavy Arrow
                                                             Evidence strongly supports
                                                                     the model

                                                                Evidence contradicts
                                                                                           representation
            X                       Crossed Arrow


                                     Dashed Arrow
                                                              (disagrees with) the model
                                                             Evidence neither supports
                                                             nor contradicts the model
                                                                                             SBF CMP
Link each evidence box to each of the model boxes, using the arrows. On the next
page, write your reasons for the three most interesting or important arrows.
                                                                                           Worksheets
                                                                                           EMT
                                                                                           Teacher
                                                                           Model A



              Evidence 1




                                                                           Model B




              Evidence 2

                                                                           Model C
Reliable pre to post test gains on systems idea
   Connections across system levels
     SBF
     Macro-micro
Individual and group variability
Learning trajectories (Eberbach, Hmelo-Silver et
al., 2012)
Engagement with content (Sinha et al., 2012)
Participation in practices (Eberbach & Hmelo-
Silver, 2010; in prep)
Transfer
   Multiple perspectives (Yu, Hmelo-Silver et al., 2013;
   Sinha, in progress)
Developing ecosystems understanding is
multidimensional
   AbioticBiotic
   MacroMicro
   StructureFunction/Behavior
Dimensions may develop differently (Wilson,
2009)
How are they influenced by particular aspects of
instruction?
Classroom microgenetic analysis (Chinn, 2006)
3




                             B:A
                             M:M
2
                             Extraneous
                             SBF
                             Coherence



1
    Pre   AA1   AA2   Post
Often a goal of schooling– but hard to find in
the lab
Do students transfer ecosystems concepts from
one context to another?
   Aquatic Rainforest
Focus on tracer concepts that were targets of
instruction
   Photosynthesis
   Cellular Respiration
   Decomposition
Example of presence
              of “plants use
              energy” and
Example of    “organic compounds
presence of   are produced”
“gas is
exchanged”
AOT perspective focuses on how students
perceive similarities (Lobato, 2006; Sinha et
al., 2010)

Pre and post interviews of 38 students from 2
schools
Task
   Label groups EMT model in terms of CMP
   Label EMT model of
Frequencies of students’ generalization of CMP
                          Levels of CMP transfer
Total n   No       C            C&P        C&M        CMP
          Transfer transfer     transfer   transfer   transfer
38        3 (8%)     15 (39%) 7 (18%)      1 (3%)     12 (31%)
Work in progress (Sinha et al, in prep)
Goal: Relating engagement to transfer
   Social coordination
   Behavioral
   Task
   ConceptualConsequential
Coding of video as students work on modeling
and simulations
   5 min intervals
Coding Conceptual-Consequential (CC)
Engagement
           High (3)                   Medium (2)                    Low (1)



   Connects to other           Focused on content           Focus on low-level
   sources of knowledge        connections and              declarative knowledge;
   and experiences             conceptual                   facts
                               understanding, but do
   Reflects on larger          not necessarily reflect or
   question or problem (e.g.   go back the central
   why do fish die).           question or relate to the
                               real world.



 ‘0 ‘ Code was assigned when teacher was addressing the entire class.
•   Creating need to know
                 Context
 Scaffolds                 •  Technology allows engagement
             Learner
                              with complex phenomena
             activity
                             • Video
                             • Simulations
                           • Distributed scaffolding help
Complex understanding         manage complexity
                           • BUT need to examine both
                              participation and outcomes
National Science Foundation
Institute for Education Sciences


Questions?
   cindy.hmelo-silver@gse.rutgers.edu

More Related Content

Featured

2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by HubspotMarius Sescu
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTExpeed Software
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsPixeldarts
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthThinkNow
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfmarketingartwork
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024Neil Kimberley
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)contently
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024Albert Qian
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsKurio // The Social Media Age(ncy)
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Search Engine Journal
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summarySpeakerHub
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next Tessa Mero
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentLily Ray
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best PracticesVit Horky
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project managementMindGenius
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...RachelPearson36
 

Featured (20)

2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best Practices
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
 

technology to support complex learning

  • 2. Complex Learning Providing Scaffolds and Creating Contexts Problem-based Learning: STELLAR Creating contexts with the VMC Complex Systems: Systems and Cycles
  • 3. We live in a complex and dynamic world Need to go beyond learning isolated knowledge and facts Useable knowledge Soft skills (Derry & Fischer, 2007) Preparation for lifelong learning, reasoning, and problem solving (Fischer & Sugimoto, 2006) Useable knowledge Transfer, from a range of perspectives
  • 4. Such learning: Often situated in problem-based and inquiry learning environments (Hmelo-Silver, Duncan, & Chinn, 2007) Potential for excessive cognitive load (van Merriënboer, Kirschner, & Kester, 2003) Appropriate scaffolding and contexts to deal with cognitive and social challenges
  • 5. Provide support to allow learners to Competently do task Learn from task Builds on notion of ZPD (Vygotsky, 1978) Scaffolding complex tasks through Structuring Problematizing Three primary kinds of scaffolding Communicating process Coaching Eliciting articulation Hard and soft scaffolds (Saye & Brush, 2002)
  • 6. Contextual support Video, Hypermedia in STELLAR (Teacher Education) VideoMosaic Collaborative Repository (VMC) Systems and Cycles simulations Collaboration spaces STELLAR whiteboards, threaded discussion Access to structured information STELLAR Knowledge Web VMC metadata Systems and Cycles hypermedia Scaffolding through Interface and activity structures pbl online in STELLAR VMC analytic Systems and cycles curriculum materials
  • 7. Processes Collaborative knowledge building Engagement Learning trajectories Outcomes Application Transfer Expert perspective (e.g., Barnett & Ceci, 2002) Preparation for future learning (e.g., Schwartz & Bransford, 1998; Schwartz & Martin, 2004) Actor-oriented transfer (Lobato, 2006)
  • 8. With Sharon Derry, Anandi Nagarajan, Ellina Chernobilsky
  • 9. Initial implementation (Hmelo-Silver, 2000) Paper cases One wandering facilitator for 6-7 groups Limitations Cases were oversimplification One wandering facilitator for 6-7 groups Difficulty identifying fruitful learning issues because of limited and variable prior knowledge
  • 10. Creating Context: Provide rich video cases of practice Concepts in context Scaffolds PBL online activity structure: extend skilled facilitation resources Knowledge Web: CFT Hypermedia support for generating learning issues
  • 11.
  • 12.
  • 13.
  • 14.
  • 15. Pre-post across institutions using video analysis task Significant pre to post gains for both sites, with different implementations Quasi-experimental design over 3 semesters Participants: 101 STELLAR PBL students in 18 groups 126 comparison students from Educational Psych subject pool Tracer concepts “Understanding” on video analysis Moderate to large effects over three years Between group variability striking
  • 16. Group 1 Facilitator CHS Ann 50 Fran Cathy Fauna Luke 45 Other monitoring SDL Group monitoring Individual monitoring 40 Grounded beliefs Personal beliefs Elaborations 35 Explanations Transforming Elaborated telling Telling 30 Acknowledgement Summary Codes Disagreements 25 Agreements Modifications New Ideas Metacognitive questions 20 Explanation questions Information questions Personal talk Concept talk 15 Tools as help Tools as a problem Task talk 10 View Other Proposals Research Library White Board Discussion Board 5 Notebook KW Video 0 300 400 500 600 700 800 900 1000 1100
  • 17. Group 2 Facilitator CHS Matt Bob 50 Carla Caitlin Liz Helen 45 Other monitoring SDL Group monitoring 40 Individual monitoring Grounded beliefs Personal beliefs Elaborations 35 Explanations Transforming Elaborated telling Telling 30 Acknowledgement Summary Codes Disagreement 25 Agreement Modifications New Ideas Metacognitive questions 20 Explanation questions Information questions Personal talk 15 Concept talk Tools as help Tools as a problem Task Talk 10 View Other Proposals Research Library White Board 5 Discussion Board Notebook KW Video 0 450 550 650 750 850 950 1050 1150 1250 1350 1450 Lines
  • 18. Similar type of rubric developed to measure “transfer” (Hmelo-Silver et al, 2009) on 0-3 scale Components of transfer rubric: 1. requires understanding, 2. involves activating appropriate prior knowledge and applying something learned in a new situation, 3. involves abstraction and cognitive flexibility, 4. can be near or far transfer, and 5. can be preparation for future learning.
  • 19. Pretest and Posttest Scores by Class Type Class N Pretest Posttest STELLAR 33 0.71 (0.31) 2.02 (0.69) Traditional 37 0.61 (0.36) 0.68 (0.34) F (1,67) = 114.323, p < .001, d=2.55
  • 20. Large variability in groups Examined STELLAR whiteboards for contrasting cases analyses Engagement with “Transfer” Group A 6 female students who had some difficulty Mean gain= 1.40, SD=0.89 Group B 6 female students, rarely needed any assistance Mean gain= 1.33, SD= 0.61
  • 21. Discussed transfer in 3 of 4 problems In Problem 1, Jenny proposed explanation for enduring understanding that child in video developed: “In the case of Brandon, he needed to have an understanding of how and why he was able to solve the block problem in order to transfer his ideas onto the pizza problem. "The first factor that influences successful transfer is degree of mastery of the original subject" (How People Learn, 53). Brandon was able to continue to solve such a problem because of his complete understanding of how he was able to arrive at the solution for the block problem.”
  • 22. In problem 2, Rina used the concept of transfer in thinking about assessment as she offered this proposal: …The portfolio should have a final summary of the students' work and questions regarding the students' learning, so that the students can explain and evaluate their own thinking. (knowlege web [sic]) The students should be able to transfer their prior knowledge of concepts such as force and motion in order to create their vehicle, while also allowing the activity to expand on that knowledge. …another important facet of understanding is application (sic). Ms. Baker will know whether the students acquired enduring understanding by how much they can apply this knowledge to real world problems. One way of doing is to have Ms. Baker create another problem that will use the same concepts in a real world setting, and evaluating whether the students were able to apply the concepts they had learned.
  • 23. All students mentioned transfer in P1; 2-4 students in subsequent problems Kathy wrote: …Second, Brandon was able to recognize a connection between the pizza problem and the tower problem that he did weeks earlier. Moreover, he made this connection relatively quickly and without much effort. He was able to show us, by using his chart and the manipulatives (blocks), exactly how the pizza problem mapped into the tower problem. His understanding of the pizza problem therefore facilitated a new, and deeper, understanding of the block problem; this process is called transfer. Brandon’s seemingly effortless use of transfer provides evidence that he understood the problem, because “transfer and wide application of learning are most likely to occur when learners achieve an organized and coherent understanding of the material.” (How People Learn, p. 238-239)…
  • 24. On later problems, fluid application of concept part of shared understanding Micki writes: …another way to look for enduring understanding would be the students' transfer and application of principles of force and motion especially to real world situations. This would show the student's understandings of information previously and transfer it to the problem at hand, which is a real world problem that allows students to work with hands-on material. Mimi followed this up by incorporating Micki’s comment and a previous proposal from another group member: To put these two ideas together, [t]he teacher could bring together individual explanation and transfer as evidence of enduring understanding. An activity could be created at the end of each project that would ask the individual members of the group to use the principles gained to explain a real world scenario. Likewise, an activity could be designed to facilitate transfer of the instructional objectives. For instance, one of the objectives was learning the scientific inquiry process. The teacher could present a real world problem that would require the students to use the same scientific process to solve. (This would also facilitate transfer)
  • 25. Technology: Scaffolds Context Complex measures: Knowledge in use More and less productive collaborations
  • 26. with Carolyn Maher, Marjory Palius, Grace Agnew, Robert Sigley, Chad Mills www.videomosaic.org
  • 27. Preserves a major video data collection on student reasoning From diverse schools settings To be available as open source From 40 doctoral dissertations Makes available new tools for Teachers Educators Researchers From longitudinal/cross sectional studies spanning 25 years Videos following the same student cohort from elementary school through high school and beyond Over 4500 hours of video
  • 29. Importance of making sense of students’ conversations and how tools mediate learning (Hmelo-Silver, 2003) Being aware of the contextual resources (media, other Ss, prior experience) that Ss use influence collaborative knowledge construction (Arvaja et al., 2006) Attending to social interactions in collaborative knowledge construction (Palincsar, 1998) 30
  • 30. Graduate Online Design mathematics ed eCollege CMS hybrid course Streaming video / Four online groups linked papers 2+ week unit Minimal online In class problem instructor intervention solving Data Individual study of Postings from online videos and related threaded discussions readings Pre and post tests Group discussion (math, Ss reasoning) questions
  • 31. To what extent do videos and readings promote online discussion within and across groups? How do learners relate practice to online discussion? To what extent do Ss relate videos to readings in their online discussions? 32
  • 32. All posts coded for comments related to Video (V) Readings (R) Additional sub categories of comments relating videos/readings to: Own problem solving (PV/PR) Others’ problem solving (OV/OR) Earlier interventions (EV/ER) Affect (AV/AR) Practice (TV/TR) 33
  • 33. Shows two groups of 10th graders (2 in one & 3 in other) working on the problem: How many different block towers can be built, four tall, selecting from three colors of blocks such that the towers have at least one block of each color? Approximately 8 minutes http://hdl.rutgers.edu/1782.1/rucore00000001201.Video.000062055 34
  • 34.
  • 35. (1) Describe Romina’s strategy for solving the Ankur’s Challenge problem. (2) In your opinion, is this solution a convincing one? Why or why not? (3) According to the Yackel & Hanna chapter, both von Glaserfeld and Thompson equate reasoning with learning (p. 227). From this perspective, in what ways do explaining and justifying contribute to learning mathematics?
  • 36. Percent of Posted Discussions Related to Video Activity Overall SS Relate Videos N a and Readings to: A B C D Own problem 27 100.0% 100.0% 100.0% 83.3% 95.7% solving (PV/PR) Others’ problem 13 100.0% 100.0% 100.0% 85.7% 92.3% solving (OV/OR) Earlier 10 66.6% 100.0% 100.0% 33.3% 70.0% interventions in the class (EV/ER) Enjoyment 20 100.0% 83.3% 100.0% 80.0% 90.0% (AV/AR) Practice (TV/TR) 24 25.0% 71.4% 75.0% 12.5% 40.7% a N= Number of posts
  • 37. Percent of Posted Discussions Relating Videos to Assigned Topic Readings Discussion Group A B C D SS relate videos to 16.7% 16.7% 45.5%* 7.1% assigned readings (RV) Number of posts 18 18 11 14
  • 38. Across all groups: Studying videos generated reflections about own and classmates’ problem solving Studying videos of students’ reasoning was enjoyable But still left us with question of ways in which which Ss related resources to their practice
  • 39.
  • 40. Create multimedia artifacts using the VMC repository Narrative with video for purpose Variety of uses by instructors, researchers Goal to classify and identify what differentiates high quality and low quality “analytics” Data sources: 27 VMCanalytics from several different classes and researchers Work very much in progress Agnew et al, 2010; Hmelo- Silver et al., in press
  • 41. VMC Analytic by Hmelo-Silver (2011) VMC Analytic by Horwitz (2011)
  • 42. Math LS Class Depth Depth Clarity Coherence Design- based Research 0.78 1.00 1.22 1.33 Intro to Math Ed 1.96 1.80 2.36 2.20 Critical Thinking 2.00 2.40 2.30 2.00 Practicum 3.00 2.25 2.63 2.63 No Class 1.17 1.67 3.00 3.00
  • 43. Looking deeper with contrasting case analysis Example 1: Analytic illustrating how students can move from particular to general Concepts from both learning sciences and mathematics clearly articulated Indicative of designer's understanding of students’ learning trajectory Example 2: Analytic illustrates teacher questioning during early algebra exploration Students claims not supported by video segments selected Textual descriptions of events were vague Video not well chosen for intended purpose of relationship of teacher questioning and student engagement
  • 44. VMCAnalytics coded for emergent themes Explored use of word clouds as a learning analytic Mathematics Education Word Cloud of coded Mathematics themes ideas Learning Sciences ideas Allow us to see dominant themes within the two Word Cloud of coded Learning Sciences themes areas of interest
  • 45. with Rebecca Jordan, Catherine Eberbach, Suparna Sinha, Lei Liu, Steven Gray, Wes Brooks, Yawen Yu
  • 46. Goals Learning about ecosystems Reasoning about evidence Modeling 6 week curriculum Creating Contexts Scaffolding complex learning Understanding how and why along with what
  • 47. Aquarium Design Eutrophication in local pond Marines problems caused by ocean acidification
  • 48. Provide context for: Help focus on function Discussions and behavior Science practices Make invisible visible Engage with complex and open for inspection systems phenomena
  • 49. Use the following arrows to help you decide how the evidence relates to the different explanations: Solid Arrow Evidence supports the model Conceptual Wavy Arrow Evidence strongly supports the model Evidence contradicts representation X Crossed Arrow Dashed Arrow (disagrees with) the model Evidence neither supports nor contradicts the model SBF CMP Link each evidence box to each of the model boxes, using the arrows. On the next page, write your reasons for the three most interesting or important arrows. Worksheets EMT Teacher Model A Evidence 1 Model B Evidence 2 Model C
  • 50. Reliable pre to post test gains on systems idea Connections across system levels SBF Macro-micro Individual and group variability
  • 51. Learning trajectories (Eberbach, Hmelo-Silver et al., 2012) Engagement with content (Sinha et al., 2012) Participation in practices (Eberbach & Hmelo- Silver, 2010; in prep) Transfer Multiple perspectives (Yu, Hmelo-Silver et al., 2013; Sinha, in progress)
  • 52. Developing ecosystems understanding is multidimensional AbioticBiotic MacroMicro StructureFunction/Behavior Dimensions may develop differently (Wilson, 2009) How are they influenced by particular aspects of instruction? Classroom microgenetic analysis (Chinn, 2006)
  • 53.
  • 54. 3 B:A M:M 2 Extraneous SBF Coherence 1 Pre AA1 AA2 Post
  • 55. Often a goal of schooling– but hard to find in the lab Do students transfer ecosystems concepts from one context to another? Aquatic Rainforest Focus on tracer concepts that were targets of instruction Photosynthesis Cellular Respiration Decomposition
  • 56.
  • 57. Example of presence of “plants use energy” and Example of “organic compounds presence of are produced” “gas is exchanged”
  • 58. AOT perspective focuses on how students perceive similarities (Lobato, 2006; Sinha et al., 2010) Pre and post interviews of 38 students from 2 schools Task Label groups EMT model in terms of CMP Label EMT model of
  • 59.
  • 60. Frequencies of students’ generalization of CMP Levels of CMP transfer Total n No C C&P C&M CMP Transfer transfer transfer transfer transfer 38 3 (8%) 15 (39%) 7 (18%) 1 (3%) 12 (31%)
  • 61. Work in progress (Sinha et al, in prep) Goal: Relating engagement to transfer Social coordination Behavioral Task ConceptualConsequential Coding of video as students work on modeling and simulations 5 min intervals
  • 62. Coding Conceptual-Consequential (CC) Engagement High (3) Medium (2) Low (1) Connects to other Focused on content Focus on low-level sources of knowledge connections and declarative knowledge; and experiences conceptual facts understanding, but do Reflects on larger not necessarily reflect or question or problem (e.g. go back the central why do fish die). question or relate to the real world. ‘0 ‘ Code was assigned when teacher was addressing the entire class.
  • 63. Creating need to know Context Scaffolds • Technology allows engagement Learner with complex phenomena activity • Video • Simulations • Distributed scaffolding help Complex understanding manage complexity • BUT need to examine both participation and outcomes
  • 64. National Science Foundation Institute for Education Sciences Questions? cindy.hmelo-silver@gse.rutgers.edu

Editor's Notes

  1. Will now present some examples in the context of my work– will talk about technology
  2. ----- Meeting Notes (9/5/12 17:17) -----STELLAR was an attempt to support problem-based learning in teacher education through the uses of technology.
  3. - analyze only steps 3 to 6 of the activity for each group, the line counts begin at different numbers for each group because Group 2 engaged more with the STELLAR tools in the earlier phases of the activity than did Group 1.The vertical axis shows the categories of tool hits, discourse codes, and speakers. The horizontal axis shows the number of tool-related events, either a log entry or a discourse turn. The bottom seven categories represent tool hits by any member of the group. The top six or seven categories represent the speakers.Group 1- weaker group, in Group 1, the facilitators were involved early and fairly frequently, and asked most of the explanatory and metacognitive questions. Ann and Fauna seemed to dominate the discourse though other students contributed. Task talk continued throughout duration of activity as they struggled to figure out what they were doingAnother aspect of how the CORDTRA diagrams help us distinguish the collaborative activity between two groups is by showing the overall relation between the discourse and the tool use. In Group 1, the students initially viewed the video and the Knowledge Web but after about line 650, none of the group members used these two resources until the very end of the collaborative phase at line 1000. The content of their online postings were intermixed conceptual, social, task and tool-related talk throughout their work on the problem. There was some discussion of tools as a problem midway through the discussion. ----- Meeting Notes (9/5/12 18:03) -----To better understand the variability, we created visual representation in which we examined the relation between discourse features and log data from using the various tools.CORDTRA-- builds on work of Rose Luckin-- Chronologically ordered representation of discourse and tool related activity all on a single timeline.
  4. Group 2In Group 2, the facilitators joined in much later and made infrequent contributions in the form of questions. This is because in this group the students themselves asked a number of questions throughout the duration of the problem. The group participated fairly evenly except for Matt who chimed in late in the group’s work. His contribution built on one of the other student’s ideas and was grounded in personal experience. His later contributions offered an important new idea for an activity that was grounded in psychological theory. What is interesting in Group 2 is that they went back to the video and the Knowledge Web at intervals throughout the discussion (e.g., around lines 850-925, 1050-1100, 1275). It appears that group members did this following several explanation questions as Figure 7 demonstrates (for example at about line 1050). This suggests that Group 2 was using the resources of the video and the Knowledge Web in a purposeful manner. That is, they seemed to be using the STELLAR resources to answer the questions at hand. They were bringing together the problems of practice and conceptual ideas repeatedly. We hypothesize that it is the repeated meshing of conceptual ideas from the Knowledge Web with the perceptual ideas from the video, which is a precondition for transfer (see Derry, 2006). Our current analysis, however, did not test to see whether transfer in fact occurs.The CORDTRA demonstrates several other distinctions between the two groups with regards to timing. Group 2 engaged in group monitoring throughout the activity whereas Group 1 did not engage in this kind of monitoring until relatively late in the activity. This was generally a request for feedback from the rest of the group. Inspection of the data shows that it was the facilitator doing most of the group monitoring in Group 1 whereas in Group 2, it was the students who were the ones monitoring themselves. The facilitators adapted their support to the needs of the group and thus differentially engaged with each group as needed. ----- Meeting Notes (9/5/12 18:03) -----There are other distinctions I could mention, but from a methodological perspective, this allowed us to see how the different groups engaged in their problem solving task.
  5. Participants viewed a brief video in which high school students learned about electricity, electrical circuits, and how a light bulb works. Before viewing the video, they received a brief written explanation describing how the video clip illustrated a problem the teacher had: that even top students were maintaining their pre-course misconceptions about electricity after instruction. The video explained how the teacher had spent a month covering advanced topics in electricity and provided hands-on experience designed to reinforce those concepts and illustrate how electricity enabled a light bulb to work. Video also showed an interview with a good student before and after instruction and demonstrated that she maintained the same misconceptions following instruction.They had thirty minutes to answer the following four questions: (1) How do you know that the student failed to learn? (2) Why did the student fail to learn? (3) What recommendations would you make to the teacher to help him improve his teaching? and (4) What else do you need to know to better understand the teaching-learning situation? What additional questions would you ask?
  6. Group A- difficultydemonstrated by the need for the teaching assistant to frequently intervene and facilitate this group’s workGroup B- Both groups engaged with the concept of transfer and they showed this in how they used the technology
  7. All students discussed in P1, number decreased in subsequent problems. All the students in this group used transfer in their independent proposals but did not use it as an idea in commenting on other proposals, suggesting that the members of this group used the concept in parallel without real knowledge building Jenny quote 1: This excerpt shows that the student brought in ideas about transfer through direct quotes that define the concept. This is much like knowledge telling (Bereiter &amp; Scardamalia, 1987), as Jenny shared relatively unprocessed information with her group. Although this group continued to use the concept in independent proposals, it later became encapsulated as part of the group’s language and no longer required detailed explanation.
  8. illustrates a more fluid use of the concept of transfer without needing to provide all the definitions, suggesting that the group members had achieved a shared understanding. This use of conceptual ideas was seen to an even greater degree in Group B.
  9. , similar to Group A, the group members began by incorporating direct quotes as they shared information, as this excerpt from Kathy shows. But unlike the other group– she more carefully mapped the definition to the evidence in the video.
  10. Preserves video and metadata by converting to digital formatOffers a research and teaching portalProvides analytic tools, customized ontologies, and personal spaceSupports individual work and group collaboration
  11. Different methods help deal with the complexity of theoretical perspectives
  12. Same pre and post tests as in other enactments– but here our research questions focused on how people used the VMC resources
  13. Posted Assignment This week&apos;s assignment for online work involves a video and two readings, with threaded discussion, that follows class work on problem solving for the Ankur&apos;s Challenge task.  The following questions are intended to guide discussion in your small groups (and will also be posted in the introduction to group discussion threads).
  14. ACROSS GROUPS, THE PERCENT OF POSTED DISCUSSIONS RELATING VIDEOS TO OWN AND OTHERS’ PROBLEM SOLVING WAS STATISTICALLY SIGNIFICANT.THE PERCENT OF POSTED DISCUSSIONS ABOUT ENJOYMENT ACROSS GROUPS WAS STATISTICALLY SIGNIFICANT.GROUPS A &amp; D MADE LESS REFERENCE TO EARLY INTERVENTIONS AND TO PRACTICE .
  15. TABLE 2 SHOWS THAT FOR GROUP C, THE PERCENTAGE OF SS WHO RELATED VIDEOS TO ASSIGNED READINGS WAS STATISTICALLY SIGNIFICANT,
  16. Answered this with qualitative analysis
  17. Purposes include:purpose (e.g., research, professional development)We also wanted, for classes, to better understand design features
  18. Part of goal was to get sense of terrain.
  19. Used this to look at high and lesser quality analytics to help future instructors better design tasks and evaluate analytics. Help us in discerning additional criteria that distinguished high and low quality
  20. This was derived from researcher-supplied themes– will be interesting to apply to the raw data.
  21. Talk a little bit about move
  22. Embedded in problem- design aquarium ,video of eutrophication of pond,
  23. Describe the screen – variables, controls, graph, outputs, fish… controls: sunlight, nutrient runoffDescribe the screen and the purpose of this simulation. At this point they have already identified inverse relationships between O2 and CO2. That when the O2 is down, more fish die. The teacher has just reminded the whole class that the goal of the activity is to model how the fish died in the pond—to reproduce the conditions in the pond that resulted in the fish dying. They have been modeling a system that keeps the fish alive or to die of old age. Why is the O2 decreasing?----- [NEXT SLIDE]
  24. Move from SBF CMP– need to explain this.
  25. Note that a great deal of this work is in progress.
  26. Typically, we look at an aggregate pre to post test gain, but in complex domains, this is not sufficient- need to know how students learn– where they start, which aspects of systems are harder than others.Question 1 we are beginning to gain traction on– analysis is ongoing– question 2 is on the burner/Microgenetic analysis
  27. Coded from Drawings taken over 4 time points in a year--
  28. To support our analysis, we constructed a graph) that depicts the group means for each systems dimension in Classroom 1. Students appear to have steep growth towards higher levels of Biotic/Abiotic and macro/micro dimensions but relatively more steady growth towards higher levels of SBF and Coherence dimensions that converge by posttest. From the start, students represented structures in terms of their behaviors or functions and made connections between phenomena (Coherence). However, these structures were primarily macroscopic and biotic. Students made more complex connections between Biotic/Abiotic structures (AA1) and between Macro/Micro structures (AA2) before reaching higher levels of SBF or Coherence. Finally, the inclusion of extraneous structures peaked at AA1 and then declined until no drawings included extraneous structures by posttest.Of course, means mask some of the variability but we are seeing some consistency in B:A preceding M;M in general and that making both of these are important for a coherent understanding– represented by both the coherence measure and sBF score.  
  29. We designed the interview to assess students’ generalization of mechanistic reasoning and to make sense of new problems related to aquatic ecosystems. In the first problem, students were shown a paper copy of their group-generated EMT model depicting factors that may have led to fish dying suddenly in a local pond. The students were then asked to label the model in terms of CMP and explain their reasoning. In the model (see Figure 1), the entire problem reflected the phenomena, the rectangular boxes represented components that were linked together by explanations of their mechanistic behavior. In the second problem, students were told that there has been a sudden increase in geese population around a lake that has resulted in changes to the aquatic ecosystem. They were shown three versions of EMT models (the first consisting only of components, the second had only mechanisms but no components, and the third consisted of numerous components, mechanisms connecting them and phenomena). Students were first asked to rank each model on a scale of one to three, with three being the most complete explanation about what happened to the lake ecosystem as a result of the overpopulation of geese. Next, students labeled the model they had ranked the highest and were asked to explain the criteria for making their selection.
  30. To trace generalization of mechanistic reasoning from an AOT lens, we compared each student’s labeled model in task 1 with that in task 2. We identified items they labeled as identical in terms of CMP in both tasks and also kept track of areas where they exhibited differences (i.e., identified phenomena in one task as the entire model and in the other had it labeled as a mechanism).
  31. As we compared students’ labeling of models, it was evident that a majority of students generalized the concept of components and phenomena. We observed that students would either circle the entire model or focus on the primary problem under investigation when asked to label phenomena. Additionally, almost all the students were quick to tag components as factors that may have led to the problem
  32. Social coordinationBehavioral engagement refers to the degree of the group’s on-task behavior.Task engagement refers to the focus of engagement on efficient planning and what steps to take next to accomplish the task. Can also refer to a focus on manipulating the technology tool (but the rating does not reflect whether concepts or content represented).
  33. Conceptual engagement is considered a continuum that ranges from content connections that are focused on the key question or task problem or relating to the real world/experiences to simple knowledge telling.  Note: The conceptual and consequential connections do not need to be accurate.To warrant a high rating, students need to go beyond referencing the larger question of why do fish die, for example, to making a connection with evidence or content.  Also, a connection to a simulation or piece of evidence is still a medium rating, unless connected to the larger question.Medium: Group discusses and identifies content relationships between visible and invisible components.Low: simply reading the hypermedia; only interpreting simulation data without content relation (nutrients go up).We are seeing trends that the simulations foster high conceptual consquential engagement, task engagement, and behavioral engagement– more variability with EMT modeling tool.
  34. Technology can play important role, but is synergistic with creating engaging contexts that create need for knowledge, bringing hard and soft scaffolds together– thinking about how to distribute scaffolding– also need to examine learner activity