Poster presented at the May 2015 Conference for Advancing Evidence-Based Teaching, Center for Advancing Learning and Teaching Through Research, Northeastern University, Boston, MA

1. Course and Assignment
Course: How People Learn is the gateway course for the eLearning and
Instructional Design M.Ed. and an elective for two other concentrations. It
introduces students to learning sciences research, including the cognitive
processes of learning and high impact practices that improve learning.
Assignment: In the first week, students create a visualization of how they
conceptualize “learning,” using the free online tool bubbl.us, and share it with
the class. In Week 10 they create a second map that depicts their revised
and refined understanding of “learning.” The images below illustrate one
student’s pre/post map set. This activity is both a learning strategy and an
opportunity for course assessment. This poster presents insights we gained
when we analyzed and compared pre/post maps.
Methodology
Sample:
We analyzed pairs of maps from 37 students in 2 different classes that occurred
during consecutive semesters. We omitted all maps created by students who
completed one, but not two maps.
Data Dimensions:
To create a data collection instrument, we selected half a dozen maps that we felt
demonstrated a range of sophistication and tried to identify salient characteristics.
Prominent features included content types, content relationships, shape, and design.
Norming:
We selected 12 maps that we both analyzed to compare our results, and used
discrepancies in our ratings to clarify our data form and our understandings of the
categories. When we were confident that our interpretations were reliably similar, we
divided the final maps and each analyzed a group.
Data Collection and Analysis:
Using a Google form for data collection enabled us to download results into an Excel
spreadsheet and sort to achieve multiple views. The images below illustrate our
analysis categories, data collection form, and graphs of corresponding results.
Making Student Learning Visible:
Using Concept Map Analysis as an Assessment Tool
Gail Matthews-DeNatale, Ph.D.
Assistant Teaching Professor, Graduate School of Education
College of Professional Studies
Laurie Poklop, Ed.D.
Associate Director
Center for Advancing Teaching and Learning Through Research
Kaleena Seeley, B.A.
Research and Program Manager
Center for Advancing Teaching and Learning Through Research
Research Questions
• Do the pre/post maps exhibit progression from novice to expert
knowledge? Assuming there is evidence of progression from novice to
expert, what are the markers and patterns of development?
• Are some areas of development more prominent than others? Are there
gaps or areas in which they continue, as a group, to show little progress?
FINDINGS
Surprises and Reflections
Prior Knowledge: We assumed the baseline maps would feature affective
dimensions (e.g., learning as “fun”) and post maps would emphasize cognitive
processes (e.g., memory, transfer). Few baseline maps referenced affective
terms, and most pre/post maps included process-oriented terms.
Indicators of Development: If most maps focus on process,
what are the markers of novice, intermediate, and advanced
maps? We need a more sophisticated framework for analysis.
Redesign Decisions
Provide Context: We will add a required reading that orients students to concept
maps as an instructional strategy, providing a rationale for the assignment. The
revised assignment directions will also prompt students to include connecting terms,
as depicted in the reading’s illustrations.
Guide the Revision: If every term is afforded equal weight, then nothing is
important. Students need strategies for approaching the tasks of assigning value and
articulating relationships. These are important steps in conceptualizing and
visualizing a model. The revised Week 10 assignment directions will prompt students
to sort, categorize, look for relationships, and consider process flow between
conceptual domains.
Streamline and Focus: Students will submit the revised map with a written narrative
that explicates the revisions and provides a rationale for their conceptual model. This
will be in lieu of a synthesis paper that was not explicitly connected to the mapping
exercise.
Align Tools with Task: We will revise the bubbl.us tutorial to provide step-by-step
directions for adding terms to the connecting lines.
TOTAL 1 2 3 4 5
PRE 8 10 17 2 0
POST 0 1 9 22 5
0
5
10
15
20
25
Content Areas Referenced
Student 30:
Post-course map
OURSTUDYIMPLICATIONS
Free
Association
Tiered/Hierar
chical
Conceptual
PRE 24 11 2
POST 7 19 11
0
5
10
15
20
25
30
Relationship Between Terms
Constellation Branching Synaptic
PRE 22 14 1
POST 5 22 10
0
5
10
15
20
25
Map Shape
Fallacy of Complexity as Indicator: Post maps show an increase in all
content areas, terms, and the percentage categorized as branching or
synaptic. But more is not necessarily better. Authors who distilled their
ideas into a conceptual model created simplified maps. Perhaps complex
maps indicate intermediate as opposed to advanced development?
Conceptual Relationships: Some authors apparently
placed terms at random. Other maps indicate intentional
arrangement. Several students put
connecting words between terms. This made it
easier to follow the author’s thinking process.
Student 30:
Pre-course map