Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Making Student Learning Visible: Using Concept Map Analysis as an Assessment Tool


Published on

Poster presented at the May 2015 Conference for Advancing Evidence-Based Teaching, Center for Advancing Learning and Teaching Through Research, Northeastern University, Boston, MA

Published in: Education
  • Login to see the comments

Making Student Learning Visible: Using Concept Map Analysis as an Assessment Tool

  1. 1. Course and Assignment Course: How People Learn is the gateway course for the eLearning and Instructional Design M.Ed. and an elective for two other concentrations. It introduces students to learning sciences research, including the cognitive processes of learning and high impact practices that improve learning. Assignment: In the first week, students create a visualization of how they conceptualize “learning,” using the free online tool, and share it with the class. In Week 10 they create a second map that depicts their revised and refined understanding of “learning.” The images below illustrate one student’s pre/post map set. This activity is both a learning strategy and an opportunity for course assessment. This poster presents insights we gained when we analyzed and compared pre/post maps. Methodology Sample: We analyzed pairs of maps from 37 students in 2 different classes that occurred during consecutive semesters. We omitted all maps created by students who completed one, but not two maps. Data Dimensions: To create a data collection instrument, we selected half a dozen maps that we felt demonstrated a range of sophistication and tried to identify salient characteristics. Prominent features included content types, content relationships, shape, and design. Norming: We selected 12 maps that we both analyzed to compare our results, and used discrepancies in our ratings to clarify our data form and our understandings of the categories. When we were confident that our interpretations were reliably similar, we divided the final maps and each analyzed a group. Data Collection and Analysis: Using a Google form for data collection enabled us to download results into an Excel spreadsheet and sort to achieve multiple views. The images below illustrate our analysis categories, data collection form, and graphs of corresponding results. Making Student Learning Visible: Using Concept Map Analysis as an Assessment Tool Gail Matthews-DeNatale, Ph.D. Assistant Teaching Professor, Graduate School of Education College of Professional Studies Laurie Poklop, Ed.D. Associate Director Center for Advancing Teaching and Learning Through Research Kaleena Seeley, B.A. Research and Program Manager Center for Advancing Teaching and Learning Through Research Research Questions • Do the pre/post maps exhibit progression from novice to expert knowledge? Assuming there is evidence of progression from novice to expert, what are the markers and patterns of development? • Are some areas of development more prominent than others? Are there gaps or areas in which they continue, as a group, to show little progress? FINDINGS Surprises and Reflections Prior Knowledge: We assumed the baseline maps would feature affective dimensions (e.g., learning as “fun”) and post maps would emphasize cognitive processes (e.g., memory, transfer). Few baseline maps referenced affective terms, and most pre/post maps included process-oriented terms. Indicators of Development: If most maps focus on process, what are the markers of novice, intermediate, and advanced maps? We need a more sophisticated framework for analysis. Redesign Decisions Provide Context: We will add a required reading that orients students to concept maps as an instructional strategy, providing a rationale for the assignment. The revised assignment directions will also prompt students to include connecting terms, as depicted in the reading’s illustrations. Guide the Revision: If every term is afforded equal weight, then nothing is important. Students need strategies for approaching the tasks of assigning value and articulating relationships. These are important steps in conceptualizing and visualizing a model. The revised Week 10 assignment directions will prompt students to sort, categorize, look for relationships, and consider process flow between conceptual domains. Streamline and Focus: Students will submit the revised map with a written narrative that explicates the revisions and provides a rationale for their conceptual model. This will be in lieu of a synthesis paper that was not explicitly connected to the mapping exercise. Align Tools with Task: We will revise the tutorial to provide step-by-step directions for adding terms to the connecting lines. TOTAL 1 2 3 4 5 PRE 8 10 17 2 0 POST 0 1 9 22 5 0 5 10 15 20 25 Content Areas Referenced Student 30: Post-course map OURSTUDYIMPLICATIONS Free Association Tiered/Hierar chical Conceptual PRE 24 11 2 POST 7 19 11 0 5 10 15 20 25 30 Relationship Between Terms Constellation Branching Synaptic PRE 22 14 1 POST 5 22 10 0 5 10 15 20 25 Map Shape Fallacy of Complexity as Indicator: Post maps show an increase in all content areas, terms, and the percentage categorized as branching or synaptic. But more is not necessarily better. Authors who distilled their ideas into a conceptual model created simplified maps. Perhaps complex maps indicate intermediate as opposed to advanced development? Conceptual Relationships: Some authors apparently placed terms at random. Other maps indicate intentional arrangement. Several students put connecting words between terms. This made it easier to follow the author’s thinking process. Student 30: Pre-course map