Use of online quizzes to support inquiry-based learning in chemical engineering

Use of online quizzes to support inquiry-based learning in chemical engineering Dr Diane Rossiter, CPE Dr Catherine Biggs, CPE Bob Petrulis, CiLASS

Contents

Why? - Background to implementation

How? - Creating online quizzes

Was it worth it?

A student’s perspective

Our perspective

Lessons learnt and future developments

Why? Background to implementation

Course aims – CPE1002

Inquiry based learning format

Need for online quizzes – CiLASS project

Course Aims

CPE1002 aims to provide an introduction to the principles of chemical engineering through discussion of the chemical industry and the development and application of material balances over a range of equipment and processes.

CPE1002 is a core subject in Year 1.

Inquiry based learning

First implemented in 2005/2006

Students not seeing the connection between core technical skills and chemical engineering practice

Opportunity for change – Associate Prof Paul Lant (PAL) (University of Queensland)

Increasing students numbers by x 2.5

(28 in 2003/04 to 70 in 2007/08)

What we tried to teach them

Core Technical skills

Material balances, units, system boundaries etc

Dealing with uncertainty

Personal Skills

Working and communicating in a group

Independent and self directed learning

Transferable Skills

Technical reporting

Presentations

Communication

Problem-based Tutorials (2 hours per week)

Group Assignments with real data, authentic,

not always one neat solution

Lectures/Keynotes (1 hour per week)

Overview and introduction

Directed learning

Homework Sheets and Paper-based Quiz

Independent study and re-enforcement

How we tried to teach them major shift of focus

Typical format of a tutorial

Feedback on homework – small class, 4 staff using flipcharts located in different corners

IBL Activity – working in groups, typically prepare OHP with their proposed solution

Feedback on IBL Activity – whole class

Homework/Assignment – study time, queries

Why introduce online quizzes?

Student Perspective

Independent self assessment of core skills – identify weaknesses and strengths

Increase student’s confidence to actively participate in group work, multi-cultural

Instant feedback and directed study

Why introduce online quizzes?

Our Perspective

Reduce staff marking load

Re-focus the scaffold of teaching

“ sage on the stage to guide on the side”

Opportunity for implementation – DR experience in online courses

CiLASS Project CiLASS Centre of Excellence in Inquiry based learning in Arts and Social Sciences

Contents

How? – Creating online quizzes

Was it worth it?

Our perspective

Development of Online Quizzes

CAA not new (see HEA resources (Seale, 2002 – LTSN Generic Centre) for review)

Variety of tools available from custom made (e.g. E3AN project Question Buddy, ASTutE)

Generic quiz tools embedded within VLEs

Our choice – WebCT Vista Quiz Tool, supported at institutional level with training.

9 types of question styles – calculated, MCQ etc

Creating Quiz Content

Mechanism with 6 step process using Respondus 3.5 as development tool

Reasons for choosing Respondus 3.5

Ease of use of interface, support available

Able to print databank of questions

Use of WORD templates for importing questions

Publish to various VLEs including WebCT Vista

Respondus 3.5, http://www.respondus.com/products/index.shtml

Import from MS Word

Can import multiple choice, true-false, paragraph, short answer, matching, and multiple response questions from a file.

The questions must be organized in a format that is acceptable to Respondus and the file must be stored in one of the following formats: plain text (.txt), rich-text (.rtf), MS Word (.doc) or tab/comma delimited (.csv) format.

Example – Mixer Flowsheet MCQ with figure

Figure 1 WORD file format for multiple choice style question Title: Mixer Flowsheet 2) Using the flowsheet for the mixer provided and assuming steady state conditions with no reaction occurring, determine the overall material balance and select the correct answer from the following options: @ For steady state conditions and no reaction, the overall material balance becomes: total mass flowrate IN = total mass flowrate OUT *a. A + B = P @ Correct. b. A - B = P @ Incorrect, B is an input stream. c. A + P = B @ Incorrect, P is an output stream. d. P + B = A @ Incorrect, B is an input stream. Not possible for calculated style questions! Mixer A (kg/h) B (kg/h) P (kg/h)

For Calculated Questions

Need to enter details directly into Respondus databank.

Example from excess air calculation relating to a chemical reaction taking place

Screen shot 1 – what students sees

Screen shot 2 – what designer sees

Generates up to 100 alternatives of same calculation!

Useful features of Calculated Questions

Generates up to 100 alternatives of same calculation! So facilitates repeated use.

Answer required to certain decimal places or significant figures – key skill for engineers!

Can request they include units with answer – another key skill!

Other question style - matching

Our Implementation

A databank of 78 online quiz questions relating to five core chemical principles topics was developed

unit conversions using the unity brackets approach (26);

mass to mole conversions (8);

calculations and definitions relating to material balances (13);

material balance calculations without reactions (20); and

material balances with reactions (11).

Questions types included calculated, multiple choice, short answer, matching pairs, fill-in-the-blanks, true-false.

Key steps Step 1. Brainstorming core concepts and relevant Q’s Step 4. Type question directly into Respondus v3.5 e.g calculated Step 2. Type question into MS Word e.g. MCQ, T/F Step 3. & 5 Combine in Respondus & preview Step 6. Publish in WebCT & do settings Based on what students found hard in the past

Implementation into CPE1002

Available to students in self-test mode within WebCT Vista (2008) Published to coincide with relevant key note lectures.

Feedback from individual questions directed to lecture notes and textbook

Same databank of questions was also used for WebCT Vista Quiz (10%)

In Week 9, this online test was made available with a time limit of 1 hour

The online test was released for 7 days and at the same time the self-test quizzes were made unavailable.

Out of 70 students, 63 did the online test.

The average score for the online test was 8.3% (out of 10%). Low marks for some due to not accessed self-test quizzes earlier and caught about by exactness of CAA responses required.

Contents

How? - Creating online quizzes

Was it worth it?

Our perspective

Was it worth it? Student focus group Student questionnaires Project leaders’ interview

Student questionnaire results 1

To what extent has your use of online self-test quizzes helped you …

1. End of semester 1 Feedback by Year 1 students, 15 th Feb 2008, 54 responses out of 69. 1.8% didn’t know, 3.7% didn’t use and 5.6% gave no response Total of 81.5% quizzes had helped to some extent in developing core skills 7.4% 31.5% 50% 9. to develop core skills required for IBL actvities… Very little Some Very much/ quite a bit

Student focus group 2 2. Eight students – Interviewer R. Petrulis (CiLASS) 3 rd March 2008 “ The homeworks made us work hard, but the quizzes really helped us learn how to do the homeworks.” Did everyone else do the quizzes and the homeworks as well? [Yes.] Did you find that one was more useful than the other? “ I tried the quizzes a couple times, and they weren’t very hard . The homeworks were more challenging, and you could talk to the teachers about them. That felt so much more helpful.” A couple of you said you didn’t use the quizzes at all. Could you say why? “ They were beneficial because of the immediate feedback . With the homework, it took a week or so. The online quizzes also referred you to the book and page for more information. If you just get a grade, that’s meaningless.” “ Because you could access the quiz more than once , that made it really helpful for revision.” How about the online quizzes? What worked?

Project leaders’ Interview 3 3. Interviewer – R. Petrulis (CiLASS), Interviewees – D. Rossiter and C. Biggs, 20 March 2008 DR: It reinforces that you could describe something in three different ways and it would have meaning to different people. It’s not that the quizzes, assignments and homework cover different things—they don’t. They make it accessible to different types of learners . I think that was highlighted by what you got in the focus group. CB: The point of the quizzes was to help those who needed the basics; not to challenge those who needed to be challenged, because the homework assignments were there to do that. I think this shows that we were right to set this up in the first place. BP: Those students said they preferred to do the homework. CB: That was quite interesting. BP: There were two responses (at focus group) that I got when I asked about the online quizzes. One was that the students found it helpful for learning the material and for revision. Another, smaller group, said they found them to be too basic.

Contents

How? - Showcase of online quizzes

Was it worth it?

Our perspective

Lessons learnt, conclusions and future developments

Lessons Learnt

Work as part of a team for creating quizzes it is much better for beta testing, being creative in question development and support.

3 levels of Bloom’s taxonomy of learning can be achieved with careful choice of question styles (see Cullen and Fielding, LTU at MMU “MCQ’s – testing higher order skills”, 2007). Quiz worth (10%)

Level 1 Knowledge reproduction – True/false, matching pairs , fill-in-blanks

Level 2 Comprehension – MCQ

Level 3 Application – calculated (invaluable for engineering)

Automated marking in WebCT Vista doesn’t allow for “marks for method” so need other assessment strategies for testing higher order skills in engineering.

Lessons Learnt (2)

Bloom’s Levels 4 & 5 Analysis and Evaluation

Homeworks (formative), IBL assignments (40%) and final exam questions (50%) test the students’ higher order levels of learning

Project time offset by saving time on Homework and paper-based test marking. Benefits likely to be more obvious in future years.

Handing control for their learning to students is effective for them and us!

Conclusions

Majority of students helped in developing their core skills by using self-test quizzes.

Staff also benefitted from developing own CAA skills and change of role!

Some referred in reflective statements for 2 nd IBL assignment that they accessed the quizzes repeatedly to help them solve the authentic IBL problems!!