Virtual worlds and immersive environments have become an accepted method for teaching and learning (USDE, 2010).
The development and demonstration of higher-order cognitive skills involved in science inquiry are difficult to measure with open-response and multiple-choice tests (NRC, 2006; Quellmalz & Haertel, 2004; Resnick & Resnick, 1992). The focus of this research is not specifically about what inquiry is but how inquiry can be measured through VPAs.
authentic assessment exercises require students to apply scientific knowledge and reasoning to situations similar to those they will encounter in the world outside the classroom, as well as to situations that approximate how scientists do their work (NSES)
Virtual Performance Assessments (VPAs, IES Grant #R305A080141) use an immersive environment to situate students in a virtual setting that simulates a scientific challenge. These immersive environments enable authentic scientific inquiry where students are assessed based on performances captured as in-world interactions. Rich event-logs of student actions form the basis of observations on which conclusions are drawn about what inquiry skills each student knows. Rather than an ongoing narrative where there is a right answer, our assessments allow students to construct a narrative through a series of choices where each decision they make affects their in-world experience. Rich in-world observations enable us to make nuanced assessments of students’ misconceptions of science inquiry.
It is May, and students in Ms. Jones’ 8th grade science class have to take their achievement tests. She logs into the VPA teacher’s portal and creates accounts for her students, selecting the initial assessment she wants them to take. When class starts, the students sit at individual computers and login to begin their simulated experience.
The ability for students to choose their own avatar is a design decision that we hope will provide students with a sense of autonomy for the experience. Arielle sits at her computer and logs into the student portal. She opens the assessment and is immediately allowed to choose what her avatar looks like. She selects and avatar and enters the world.
The camera slowly provides an aerial view of the world to orient Arielle to the problem space. Arielle sees that there is a village and what appears to be farms with ponds. The camera then focuses in on a multi-colored frog with 6 legs. Arielle wonders, “What could be causing this frog to have 6 legs?” The assessment begins. A scientist and farmers who have just discovered this mutated frog greet Arielle. The farmers all offer competing hypotheses for why the frog is mutated.
The scientist turns to Arielle’s avatar and tells her that she must conduct and investigation and come up with her own theory and back it up with evidence. He asks her if she thinks any of the hypotheses are plausible.
The goal of VPA is for students to make choices based on sound science inquiry skills that advance the theory that they are attempting to build. In VPAs, a student’s measure of science inquiry performance is based on their in-world actions. Their actions and choices are given a range of scores and weightings that contribute to an ongoing student model of science inquiry. They are temporally evaluated based on past, present, and future actions. In other words, a choice is evaluated in terms of the previous actions, their actual choice within the context of the available choices, and the outcome of their choice that sets the stage for the next set of actions. For example, if a character asks a student what they think the problem is and the student responds that they think the mutant frog is a result of pollution, the character will ask the student to provide evidence for their claim. The evidence that a student gives will be weighted and evaluated based on their prior actions (data that they have previously collected) and by what they choose to present as evidence.
At this point in the assessment, Arielle has a choice. She could have gone to the lab and accessed information there such as research articles. However, she chose to go explore. This choice is recorded on the back-end. We are recording students’ choices and that are then be compiled into patterns. These patterns are then built and compared to profiles of students’ inquiry knowledge established during our cognitive task analyses. Because of this, the assessment has a built-in framework that enables us to examine students’ intent and interpret their actions.At the first farm, Arielle says she plans to collect a water sample. She enters the farm and collects a sample of the water. She also picks up a frog and a tadpole to bring back to the lab and run some tests.
She finds a research article and starts reading it. It contains information on tadpoles and viruses so she puts it in her backpack and decides to visit another farm. At this point, Arielle has collected 5 pieces of data. Her backpack will only allow her to hold 8 pieces of data at a time.
Arielle will be forced to make a choice about what data she thinks is the most important or that she wants to investigate first. If students were allowed to pick up every piece of data in the world then it would be difficult to make inferences about their knowledge of what data is important evidence in the investigation. If students were asked to evaluate a piece of data every time they collected it then the task would become boring. Thus, the design is requiring students to make a choice through actions. She can go to the lab at any times to run tests on the data (e.g. water tests, blood test, genetic test). Any piece of discarded data from the backpack will go back into the world and can be picked back up at any time (given there is space in the backpack).Arielle has collected 8 pieces of data from two farms. She does not want to discard any data and decides to go to the lab to run some tests. She arrives at the lab and examines the water samples. Her tests show that the lab water and water from one of the farms contains pesticides. However, one of the farms has clean water. She runs genetic tests on the 2 frogs she collects and sees that they are the same. She notes that both of the frogs have high counts of white blood cells. She decides that she needs more evidence and goes to collect water samples from the other two ponds. At this point, Arielle has spent her time collecting data and running tests.
21st Century Curriculum and Assessment
TIE*21 21st Century Curriculum and Assessment Dr. Valerie Irvine Dr. Jillianne Code @_valeriei or firstname.lastname@example.org @jilliannecode or email@example.com Assistant Professors, Educational Technology University of Victoria http://tie.uvic.ca
TIE*21 Why? • 21st Century Teaching and Learning See Christy Clark’s Throne Speech (pdf) » Pg 10 & 17-18 See Report on 21st Century Teaching and Learning by BC Premier’s Technology Council • Opportunity to improve learning methods
TIE*21 Why? • Assessment and evaluation opportunities – Using networking and multimedia tools • Increased engagement • Support for Special Needs • Just-in-time resources (often free or cheap)
TIE*21 21st Century CURRICULUM KEY POINTS • Incorporation of technology as a way of doing things…
TIE*21 21st Century CURRICULUM KEY POINTS Understanding how networks work is one of the • Digital Literacy most important literacies of • Understandingthe 21st century. Howard Rheingold (2010)
TIE*21 21st Century CURRICULUM KEY POINTS • Increased focus on skills as opposed to content
TIE*21 21st Century CURRICULUM KEY POINTS • Self-Regulation of Learning • see Allyson Hadwin at Uvic or recording at http://tie.uvic.ca
TIE*21 21st Century CURRICULUM KEY POINTS • Needs to allow for creativity as per PTC report
TIE*21 21st Century CURRICULUM KEY POINTS • Pull-out IRP document of technology- infused outcomes similar to grade-only pull-out IRPS • Supports district or school technology coordinators • Supports pre-service teacher educators
TIE*21 iPads as an example • Demonstration • Trend of iPads in Schools • iPads for Kindergarters • View Youtube
TIE*21 21st Century ASSESSMENT • Move from Assessment OF Learning to Assessment FOR Learning • Incorporate innovative ways of assessing learning
TIE*21 Our challenge • Simply using technology to deliver digitized versions of item-based paper-and-pencil tests does not realize the full power of technology in assessment • We need to create new types of measurement experiences that are purpose driven, reliable, and engaging • These will provide richer and more authentic observations of student learning27
TIE*21 Assessing Science Inquiry Learning The case of a 21st century assessment28
TIE*21 the problem Science inquiry is difficult to measure with open-response and multiple- choice tests29
TIE*21 authentic assessment of inquiry require students to apply scientific knowledge and reasoning to situations similar to those they will encounter in the world outside the classroom, as well as to situations that approximate how scientists do their work (NSES)30
TIE*21 what is a immersive virtual assessment (IVA) • an immersive environment used to situate students in a virtual setting that simulates a scientific challenge • allow students to construct a narrative through a series of choices • rich in-world observations enable us to make nuanced assessments of students’ misconceptions of science inquiry31
TIE*21 there’s a new frog in town assessment 232
TIE*21 ok, so how is science inquiry measured? • a student’s measure of science inquiry performance is based on their in-world actions • actions and choices are given a range of scores and weightings that contribute to an ongoing student model of science inquiry. • They are temporally evaluated based on past, present, and future actions.37
TIE*21 Resources • iPads/iPods in the Classroom Diigo http://bit.ly/mYwDct Apps in Education & Blog Posts on Do’s and Don’ts http://bit.ly/iD2woL Keith Rispin’s “The other side” http://t.co/eWUyY2i TechieAng Expanding Teaching/Exploring Technology http://bit.ly/sfejX1