Good afternoon everyone. Thank you very much for coming to my session. I hope you’ll find it interesting and informative. I’ve got a lot to cover in 20 minutes so I’ll get started right away! The module in question is a first year key skills module for Politics, History and International Relations students. It includes information-seeking and the application of critical thinking, plagiarism and referencing, traditional study skills – essay writing, etc.
Most important word on the slide is ‘together’. Together = collaboration! This is not something I could have successfully undertaken on my own.
The result? If nothing else, it was most definitely a celebration of teaching collaboration!
Module feedback had declined over the previous two years. Noticeably, there has been a drop in the average response to the question relating to how the module had helped them to develop their understanding of the subject. I was looking to be involved in a suitable research activity and here was my opportunity. I’d worked with Matt on this module for the past 3 years, we’d worked well together and both of us were really keen to turn things around. We were well aware that it was going to be a challenge as skills modules are rarely crowd-pleasers! We wanted to find ways to better engage the students and assess how successful we’d been (or not!) in doing so. So, I’ll explain briefly what we did in terms of overhauling the teaching component and then move on to how we tried to evaluate the impact.
Face-to-face discussion over the preceding summer to shake it up. Radical overhaul of our teaching methods. Let’s begin with the lecture component and look briefly at each of these changes in turn.
On the left-hand side of the screen we have the pre-shake up slides and on the right, some of the revised slides for the first introductory lecture. Quite a difference between the two. In fact, it was while delivering ‘this’ the previous year that I realised just how routine, stale and deadly dull it had become. So, out with the text and bullet-points and in with the graphics and one-liners.
In the same lecture I incorporated real-time polling using the freely-available Mentimeter. Here’s the first question I asked them and the 168 responses. Nicely illustrating what we as librarians know already!
We decided to be brave in the presentation skills lecture! Albeit a little tentatively, we incorporated deliberate mistakes in both our delivery and on the slides.
So what? We felt there was a need to put the content into context and make it relevant to both their studies and longer-term? So, we made a point of emphasizing all the things we perhaps sometimes assume the students will acquire almost by osmosis. Basically, we spelt it out at every opportunity. We wanted the module to feel to the students like there was progression and correlation between the various skills, so, for example, we drew their attention to the link between note-taking and effective academic reading. We were aiming to deliver the module as a ‘whole package’ rather than a disparate range of independent skills. We started off by explaining that they’re working at a much higher level now so they can’t rely on their existing academic skills but will need to enhance them to perform well.
We tried mixing it up a bit to make topics more relevant and more engaging. For example: Plagiarism: evolved into a two-handed question and answer session Note-taking: Matt’s mini-lecture on politics followed by questions Essay writing: political speech on YouTube followed by questions based on the arguments put forward to encourage them to think about argument construction.
That’s the lectures. There’s more….
Three more important component to the revised module:
1). SKILOD (Skills Intended Learning Outcomes Diary): to encourage reflection throughout the course of the module 2). Teaching linked to a well-regarded skills handbook (Stella Cottrell’s Study Skills Handbook) that we’d referred to in a number of sessions previously. It became a key component of the module with students being asked to read the relevant sections prior to lectures and sometimes afterwards. By doing so, the thinking was that they’d be able to read about the basics or essentials of these key academic skills, freeing us up to try different approaches to the norm in the lectures. 3). Mechanistic workshop on finding information and another on the application of critical thinking recreated as worksheets to be completed, submitted online, and formatively assessed by Matt and myself. I also created model answers for them to refer to on the VLE.
So, what was the impact of these changes to the module?
What did we do? Large-scale survey, pre- and post- module, the first one distributed immediately prior to the Library induction session and the follow-up survey at the start of Semester Two, at the start of two lectures in other modules (with the kind permission of another sympathetic lecturer), 197 pre-module surveys and 141 post-module surveys returned. Not all completed every section, especially in the follow-up survey. These are the main areas we evaluated (but not exclusively), the first one being more important to me than to Matt! Did the teaching impact on their understanding and awareness of my role? Did they feel their skill levels had improved in light of the teaching? What was their perception and awareness of plagiarism before and after the module and did the teaching adequately prepare them for the test they’re all required to pass?
Pre-module, 18% had heard of the term academic librarian. By the end of the module, 64% were able to fully name me. Others got my name confused with one of departmental lecturers, Susan Reid, others just put Sharon. Post-module, when asked about topics the academic librarian can help with, the only area where less than 90% of those who responded failed to identify it as one of our skills was on being able to conduct a literature review which scored 64%. How to avoid engaging in plagiarism scored 100% (very pleasing given that only 80% attended my plagiarism lecture!
We asked the students to assess their level of skill in all of these areas both before and after the module, on a scale of 1-5, 1 being poor and 5 excellent. Along the bottom axis we have the average score for each skill. There are lots of things we could draw from these outcomes but some of the most pertinent are: Their perceived level of skill had increased most significantly for referencing followed by presentations. A gratifying result for presentations given our new approach and pleasing for referencing given the amount of time we devote to it. In fact, it would have been good to have seen a higher score than 3.7. Their perceived level of skill had either increased or stayed the same for all categories apart from revising and exam-taking in which it had fallen. Given that their attendance at this lecture (the last on the module) was 23% that’s probably not altogether surprising! Interestingly, they felt fairly confident in their web evaluative skills on arrival despite our appraisal of these skills differing to theirs! Encouragingly, this too saw an increase after the teaching.
What can we conclude from this? Maybe their pre-module evaluations were unrealistically high. Perhaps our teaching made them more conscious of their actual ability level in areas such as essay writing and note-taking. Maybe they didn’t learn anything from these lectures….etc. Overall, however, a positive outcome borne out by the following chart….
Nearly three quarters of those who responded to this question felt that the teaching had improved their skill levels, 13% did not and the remaining 15% of those who responded to this question selected neither option (it was a ‘select ALL that apply’ type of question).
An interesting (frustrating?) outcome re. the usefulness of online worksheets and whether students actually prefer a hands-on approach. With a mean score of 2.9 and 41% scoring this one a ‘bang in the middle’ 3, it’s difficult to draw a conclusion one way or the other. This inconclusive result was furthered by the next two charts….
….the first of which indicates that 56% would have preferred a workshop to the online approach (but was this because there was so much work involved and a formative assessment?)….and an inconclusive 50:50 felt they learnt a lot from the online exercises (but is their perception correct? Maybe they learnt more than they realised or just don’t appreciate the importance yet of what they have learnt). In fact, the latter may be the case because this graph shows that a rather large 75% of those who responded felt their ability to evaluate web sources had increased as a direct result of this particular exercise.
What it doesn’t tell us if whether they would have learnt as much from the workshops. We simply don’t know at this stage.
The reflective diary which was Matt’s baby had disappointing results overall, although it’s important to stress that there were some positive comments about it’s usefulness (“Very forced me to reflect on the class, essentially a mini revision session and then evaluate my skills”, “Very, it helped with exam revision”, Only 24% of those who responded claimed to have completed it on a regular basis, citing one of the reasons on the right for their failure to do so.
From the initial survey, we know that 86% already knew about it (many having a good knowledge of what it means with others referring merely to the idea of copying rather than crediting). However this didn’t stop nearly half of them from cutting-and-pasting without acknowledgement despite half of these knowing it was wrong to do so.
51% of those who did cut-and-paste without acknowledgement.
Their feelings about the act of committing plagiarism hadn’t changed a great deal by the end of the module and you can see that the majority are not impressed by those that do so. Unfortunately, a stubborn 6% still insist that “It really doesn’t bother me”.
Mean score pre-module = 2.4 Mean score post-module = 2.5
In terms of the plagiarism test that all of the students have to pass, a pleasing 81% felt that their ability to do so was positively influenced by the teaching.
There were very definitely things that worked well and others that very definitely didn’t. The surveys, particularly the pre-module one with the captive audience were a great source of information. We’d definitely do it that way again rather than online surveys which have a comparatively poor take-up. Some of the lectures were pleasing, others need to be reworked or refined – the essay writing one didn’t quite hit-the-mark and the note-taking one still needs some tweaking. We need to be clearer about what they’re required to do themselves (i.e. the text-book reading prior / after the lecture to cover the basics).
Matt – “This year’s version has been the best so far, but still has some way to go. I do believe the changes we made especially changing the lecture style and type, and the introduction of the formative assessment in weeks 2 and 3 are points to develop. The Skilod was a good idea, but needs to be imbedded better by ALL staff. Part of this may be addressed with a specific introductory lecture in week 1 (which we did not have this week).”
My thoughts are similar to Matt’s. It would be great to build on this next year, implement the necessary changes and repeat the post-module survey to assess the effectiveness of the changes. So, going forward….
Matt re. the module feedback – “The positive thing about the negative comments is that there is nothing new – that is we have heard it all before. It is all positive in that the changes we have introduced have not created unforeseen problems. However, it is negative that the changes we have introduced have not impacted on the weaker sides of the module in these student’s eyes.”
New introductory lecture to help embed the SKILOD and clarify what’s expected of them in terms of reading about the basic skills in their own time. More of the same to draw comparisons between the two years but also implement changes where they’re clearly needed. Survey next year’s students, at least post-module to draw comparisons Develop the formatively assessed online materials? Or return to a hands-on workshop. Matt is in favour of the online approach for at least another year. Retain and refine the lecture style changes: Matt and I have reflected on this and decided exactly what we need to do to improve upon these. Compare qualitative module feedback with this year’s.
Shaken and stirred! The librarian, the academic and the case of the refreshed skills module - Sharon Reid
Shaken and stirred!
The librarian, the academic and
the case of the refreshed skills
…we worked together
to overhaul the teaching
the disengaged and
evaluate the impact of
role of the
Familiarity with the term academic
Able to name the academic
Role of the Academic Librarian
Post-module, student understanding of the role of the academic
librarian was high. 100% of those who responded identified
plagiarism avoidance as an area of expertise. The lowest score (64%)
was for offering advice on conducting a literature review.
2.5 3 3.5 4
Finding sources of information
Evaluating Web resources
Revising and exam-taking
Student self-assessment of skill level
Scale: 1 to 5
1 = poor
5 = excellent
“I do / do not feel the teaching I received has
improved my skill levels”
I do not
1 2 3 4 5
Rating from 1 (not useful at all) to 5 (extremely useful)
Student assessment of the usefulness of online
worksheets as a method of learning and
Mean score = 2.9
"I learnt a lot from the
"I would prefer to learn about these
topics in a hands-on workshop"
"I now feel better equipped to critically evaluate web sources
for reputability, reliability and quality"
Reflective diary (SKILOD)
"I completed the SKILOD on a
regular basis" Not helpful / didn’t
Issues with content or
Coursework pressures /
Plagiarism 86% learnt about it at school
49% cut-and-pasted from the web
51% of those who did, knew it was