Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Assessment:
Designs and Approaches
Gail Matthews-DeNatale
Northeastern University
Dave Consiglio
Bryn Mawr College
The Cycle
Identify
Scope
Interpret
Data
Share &
Act Upon
Findings
Form
Research
Question
Design &
Implement
Research
You A...
On a Dime
http://www.flickr.com/photos/jdhancock/3544855402
Gold Standard
http://www.flickr.com/photos/bullionvault/3591745555
Three Scenarios
Activity
Online Enrollment
Your Dean wants to raise the cap
on enrollment in online courses.
The cap limit will be revisited on an
...
Clickers
Your institution has been using
classroom response systems
(a.k.a. “clickers”) for several years.
Usage is robust...
Blended Learning
A pilot initiative is underway at
your school to integrate blended
courses across the curriculum.
What wi...
North Star Questions
What do you want to know/learn?
What’s the purpose?
What’s the population?
Who will receive the resul...
Scenario Planning
20
Minutes
What do you want to know?
What’s the population?
What’s the purpose?
Who will be involved?
Wh...
Gold Standard
http://www.flickr.com/photos/bullionvault/3591745555
Gold Standard
Reliability
and Validity
Gold Standard
Controls
Gold Standard
Example: Try Science
Qualitative Methods (Mostly)
Background information
(years teaching, education, prior OL)
Activity and work products
(disc...
Interviews
I thought I knew everything
about density. When I did my
physics unit in class, I thought I
was all set. As lon...
Turning Qual into Quant
96% Program Retention Rate
Cost: $600,000
Report: 121 pages
Some Numbers
Quantitative Example: CATS
Goal: Study effect of interactive instruction on student learning
Tools: Standardized concept i...
Your Task Over Lunch
What would these “gold standard”
methodologies would look like if
they were done “on a dime”?
How mig...
Upcoming SlideShare
Loading in …5
×

0

Share

Download to read offline

Assessment: Designs and Approaches

Download to read offline

Presentation given at the November 2011 NERCOMP Workshop on Seeking Evidence of Impact

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

Assessment: Designs and Approaches

  1. 1. Assessment: Designs and Approaches Gail Matthews-DeNatale Northeastern University Dave Consiglio Bryn Mawr College
  2. 2. The Cycle Identify Scope Interpret Data Share & Act Upon Findings Form Research Question Design & Implement Research You Are Here
  3. 3. On a Dime http://www.flickr.com/photos/jdhancock/3544855402
  4. 4. Gold Standard http://www.flickr.com/photos/bullionvault/3591745555
  5. 5. Three Scenarios Activity
  6. 6. Online Enrollment Your Dean wants to raise the cap on enrollment in online courses. The cap limit will be revisited on an annual basis. What research can you perform with your students and your faculty at your institution to inform these decisions?
  7. 7. Clickers Your institution has been using classroom response systems (a.k.a. “clickers”) for several years. Usage is robust. You’re leading a task force to assess the impact of clickers on learning.
  8. 8. Blended Learning A pilot initiative is underway at your school to integrate blended courses across the curriculum. What will be the research component for the pilot, how will the research be performed, and how will it be used?
  9. 9. North Star Questions What do you want to know/learn? What’s the purpose? What’s the population? Who will receive the results? Who could/should be involved?
  10. 10. Scenario Planning 20 Minutes What do you want to know? What’s the population? What’s the purpose? Who will be involved? Who will receive the results?
  11. 11. Gold Standard http://www.flickr.com/photos/bullionvault/3591745555
  12. 12. Gold Standard
  13. 13. Reliability and Validity Gold Standard
  14. 14. Controls Gold Standard
  15. 15. Example: Try Science
  16. 16. Qualitative Methods (Mostly) Background information (years teaching, education, prior OL) Activity and work products (discussion texts, projects, reports) Formative feedback responses Teacher follow-up interviews Program retention rates Pre-post thought experiments Self-reported increase in confidence Change in classroom practice (pre- post -observation/video) Tool NVIVO
  17. 17. Interviews I thought I knew everything about density. When I did my physics unit in class, I thought I was all set. As long as you know the formula, you know density -- that's the way I taught it before. Try Science blew my mind. At first I was a little bit angry because I found out that for the three years I taught density, in some cases, I know truly believe that I was teaching it incorrectly. The assumptions that I made from reading it instead of doing it were wrong.
  18. 18. Turning Qual into Quant 96% Program Retention Rate
  19. 19. Cost: $600,000 Report: 121 pages Some Numbers
  20. 20. Quantitative Example: CATS Goal: Study effect of interactive instruction on student learning Tools: Standardized concept inventory, demographic and attitudinal survey of students, teaching practice survey of instructors Population: Undergraduate students taking Astronomy 101 Study group: Thousands of students of dozens of higher education institutions from across the country Process: Pre and post tests using concept inventory, student survey demographic and attitudinal survey, measure classroom and institutional variables Results: Statistical analysis of individual and classroom influences on learning gains
  21. 21. Your Task Over Lunch What would these “gold standard” methodologies would look like if they were done “on a dime”? How might you adapt them for use in situations that are under- resourced and pressed for time?

Presentation given at the November 2011 NERCOMP Workshop on Seeking Evidence of Impact

Views

Total views

664

On Slideshare

0

From embeds

0

Number of embeds

2

Actions

Downloads

11

Shares

0

Comments

0

Likes

0

×