OPEN-BOOK ONLINE MCQS TO IMPROVE LEARNING OUTCOMES ON A LARGE DATA ANALYSIS MODULE DELIVERED TO BUSINESS
 

OPEN-BOOK ONLINE MCQS TO IMPROVE LEARNING OUTCOMES ON A LARGE DATA ANALYSIS MODULE DELIVERED TO BUSINESS

on

  • 206 views

 

Statistics

Views

Total Views
206
Views on SlideShare
206
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

OPEN-BOOK ONLINE MCQS TO IMPROVE LEARNING OUTCOMES ON A LARGE DATA ANALYSIS MODULE DELIVERED TO BUSINESS OPEN-BOOK ONLINE MCQS TO IMPROVE LEARNING OUTCOMES ON A LARGE DATA ANALYSIS MODULE DELIVERED TO BUSINESS Presentation Transcript

  • Open-book online MCQs Paula Carroll and Niall McDermott Open-book online MCQs to improve learning outcomes on a large Data Analysis module delivered to business students School of Business, UCD, Dublin
  • ABOUT ME Paula Carroll Dr School of Business, UCD Paula.carroll@ucd.ie I have used Blackboard for 11 years. My backgound is in engineering, I’ve been lecturing in the School of Business for over 11 years. I’m the Module Co-ordinator for a Data Analysis module delivered to over 500 undergrad business students.
  • ABOUT MY INSTITUTION University College Dublin • 21,000 undergraduates, • 9,000 post-graduates, • 7 Colleges spanning academic disciplines, • School of Business in College of Business and Law • Horizon Programme: Core modules plus electives • Semesterised
  • ABOUT MY INSTITUTION • Management Information Systems Subject Area • Business eLearning Team • School of Business main programmes: • Bachelor of Commerce • Bachelor of Commerce International • Bachelor of Business and Law • Data Analysis for Decision Makers module • Core for all stage 1 business students
  • WHAT WE ARE GOING TO LEARN TODAY Use of online Open book MCQs. 1. Pedagogic justification; 2. Student responses; 3. Technological and administrative issues.
  • OUR CHALLENGE Data Analysis Module learning outcomes: 1. Prepare spread-sheet models to store, manipulate and analyse quantitative data using common probability distributions and statistical functions, 2. Calculate, analyse and present useful statistical measurements from large-scale data sets, 3. Create and interpret inferential statistical statements about population parameters, 4. Interpret the results of data analyses with a view to informing decision making. Evidence-based decision making
  • OUR CHALLENGE Meaningful assessment for large numbers? Business students perceive maths, statistics and data analysis as boring and irrelevant!
  • OUR SOLUTION American Statistical Association Guidelines for Assessment and Instruction in Statistics Education recommendations: 1. Use technology for developing conceptual understanding and analysing data; 2. Use assessments to improve and evaluate student learning. Decision: Use open-book online MCQs as assessment component. Counts for 15% of module grade, held in weeks 7 and 9.
  • OUR SOLUTION Replicate real-world conditions – authentic formative assessment: “I don't need to know everything, I just need to know where to find it, when I need it” - Albert Einstein Move away from simply remembering to creating something new (Bloom’s Taxonomy). Discourage: Encourage: • Rote learning Discovery Learning • Memorisation Active engagement
  • HERE’S HOW WE USED BLACKBOARD • Decide MCQ strategy: • Random 20 questions, 40 minutes, feedback • Develop pools of MCQ questions: • Module Tools, Tests, Surveys and Pools • Create Question, Multiple Choice Question –Question text, –4 answers, –Feedback.
  • HERE’S HOW WE USED BLACKBOARD • Provide tutor and student guide.
  • OUR RESULTS Research Question: Are open-book online MCQs a reliable and effective assessment method? Methodology • Ethical approval for research project, • Assessment and profile data gathered (2012), • Analysis and interpretation.
  • OUR RESULTS Students come in to the module with a variety of quantitative skills
  • OUR RESULTS Open book MCQ mark is correlated with Closed Book Exam mark: Pearson Correlation Coefficient = 0.54, statistically significant
  • OUR RESULTS Average Open book MCQ marks are different than Closed book Exam: MCQ distinguishes between student quantitative abilities. Mean Range Std Dev Exam 49.13 87.00 15.42 MCQ 59.64 73.18 13.18
  • OUR RESULTS Closed book Exam pattern:
  • OUR RESULTS Survey of student attitudes: • 24% response rate • “Open-book MCQs helped me reach the learning outcomes” on a 5-point Likert scale Likert mean Likert Stdev 3.19 1.14
  • OUR RESULTS Survey of student attitudes: Which activity best aided your learning? “MCQs as answers were given straight away and so mistakes could be learned from while fresh in memory.”, “MCQ. it forces me to prepare in order to get higher score.”, “MCQ Encouraged revision and study.”
  • OUR RESULTS Implementation issues: • Device • Bring Your Own Device – Tablets • Non-Certified Browsers • Operating Systems • Infrastructure & Blackboard • Wireless versus LAN • Capacity issues
  • OUR RESULTS Implementation issues: • Administrative • Student technological savviness • Preventing and detecting cheating • Creating banks of good MCQ questions
  • DO THIS NEXT If you haven’t used open-book online MCQs: • Select MCQ design best practice guidelines; • Decide MCQ policy; • Pilot on small group! What we’re doing next: MCQ evaluation, ideas from measurement theory.
  • THANK YOU! Paula Carroll Dr School of Business, lecturer UCD Paula.carroll@ucd.ie Niall McDermott Mr School of Business, eLearning UCD Niall.mcdermott@ucd.ie