TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
Making the connections: assessment, training, progression and performance. Harrison & Newton
1. University Library
Making the connections: assessment,
training, progression and performance
Dr A. Harrison: Institute of Psychological Sciences
A. Newton: Leeds University Library
2. Why this project?
BSc Psychology (Hons)
A progressive and integrated degree programme that
encourages students to become more autonomous thinkers
and practitioners of Psychology
Culminating in an independent, empirical project
reporting on a substantial piece of research
3. Do we need Information Literate
students?
Quality Assurance Agency
Generic Skills
Retrieve and organise information effectively:- books,
journals, computer and internet sources
Handle primary source material critically
Become more independent and pragmatic as learners
4. The story so far
Lilac 2005
Second year students are more information literate
than first year students
First year students have a strong preference for using
the internet to source information
Second year students use the Library Catalogue as
their primary literature searching tool
Both groups use the Library Catalogue ineffectively
5. Current study
Aims:
• Assess the reliability of the data produced by the information literacy
questionnaire
• Provide detailed information about the progression of information
literacy skills acquisition by students
• Assess the relevance of information literacy skills to academic
performance
6. A reliable tool?
Comparing results
No significant differences between the performance of the two
first year groups (2005-06/2006-07): t = 0.909, df = 382 p=0.364
No significant differences between the performance of the two
second year groups (2005-06/2006-07): t =1.58, df = 383, p>0.115
Year of entry 2005 2006
First year Students
Mean % correct +/- (SED)
48.94 (9.89) 47.96 (11.67)
Second Year Students
Mean % correct +/- (SED)
61.69 (12.80) 59.63 (13.22)
7. Tracking progression
Second year students do progress in Information
Literacy
0
10
20
30
40
50
60
70
80
90
100
Within Group Comparison (N=112) Betw een Group Comparison
(N=805)
Mean%Correct
First Year
Second Year
8. Progression
Areas of significant progression
Search strategies
Locating and accessing information
Organising and applying information
Areas of less progression
Recognising an information need
Identifying ways to fill an information need
Comparing and evaluating information
9. IL progression
Second year students are:
Better at keyword searches
Better users of Boolean logic
Search Strategy
0
10
20
30
40
50
60
70
80
90
100
First Year Second Year
Students
Mean%ofStudentsAnswering
Correctly
10. IL progression
Second year students are far more competent at referencing
Neither group have sufficient understanding of copyright
Organise and Apply
0
10
20
30
40
50
60
70
80
90
100
First Year Second Year
Students
Mean%ofStudentsAnswering
Correctly
11. IL progression
Both groups fail to use search facilities effectively
More second year students understand the usefulness of a
bibliography in a printed book
Locate And Access
0
10
20
30
40
50
60
70
80
90
100
First Year Second Year
Students
Mean%ofStudentsAnswering
Correctly
12. Less progression
Good performance by both groups:
Very good at acknowledging gaps in their information
Appropriate strategies for dealing with information overload
Room for improvement:
Lack skepticism
First year students use of online encyclopedias
Recognise Need
0
10
20
30
40
50
60
70
80
90
100
First Year Second Year
Students
Mean%ofStudentsAnswering
Correctly
13. Less progression
Good performance in both groups:
Understand purpose of journal article abstracts
Over ½ correctly identified differences between websites and peer reviewed
journal articles
Understand peer review process
Compare and Evaluate
0
10
20
30
40
50
60
70
80
90
100
First Year Second Year
Students
Mean%ofStudentsAnswering
Correctly
14. Less progression
Poor performance by both groups:
First year students source most information from internet search engines
Second year students are more discerning
Second year students look to the internet for recent academic information
Review articles: both groups unaware of appropriate use
Identify Ways to Fill Need
0
10
20
30
40
50
60
70
80
90
100
First Year Second Year
Students
15. Academic performance
First year Students
In general, Information Literacy ability of students entering Higher
Education is strongly related to their academic performance in year one
semester one at University
Mean Academic Performance:
(r =0.349, n =130, p<0.001) HIGHLY SIGNIFICANT
Multiple choice examination performance:
(r = 0.326, n=130, P<0.001) HIGHLY SIGNIFICANT
Practical report writing performance:
(r = 0.208, n = 130, p<0.05) SIGNIFICANT
Essay writing performance:
(r = 0.152, n = 130, P=0.084) NOT SIGNIFICANT
16. Academic performance
Second year students
After some initial Information Literacy training in Higher Education the
strong relationship between IL ability and academic performance is
enhanced
Mean Academic Performance:
(r =0.355, n =118, p<0.001) HIGHLY SIGNIFICANT
Multiple choice examination performance:
(r = 0.290, n=118, P=0.001) HIGHLY SIGNIFICANT
Practical report writing performance:
(r = 0.269, n = 118, p=0.03) SIGNIFICANT
Essay writing performance:
(r = 0.181, n = 118, P<0.05) SIGNIFICANT
17. Summary
Reliability
Data produced by the questionnaire is reliable across different
cohorts of students and the LILAC 2005 data suggesting a
progression of IL ability from year 1 to year 2 has been
replicated
Progression
The Questionnaire now provides detailed information about
student abilities including areas of substantial or no
progression
Academic performance
Information Literacy is strongly related to academic
performance
18. Newton-Harrison 7 pillars of
evidence based IL training
Pre-training
Recognisein
ofyourstude
Identifyways
needsofyou
Constructat
programme
Deliveryoftra
Compare&ev
studentabilit
progression
Analysestud
performance
Synthesised
evidence-bas
Post-training Pre-training
19. Future plans
3rd
year student data to be obtained
Re-examine IL in the Psychology curriculum
Further investigation into academic performance:
A-level subjects
Dissertation results (3rd
year students)
Final degree results
Editor's Notes
A new Information Literacy training programme for psychology undergraduate students at the University of Leeds was launched in 2003. In order to ensure that the training programme addressed the needs of the students we created a tool based on the 7 pillars model of IL and the Quebec study ( IL Qnn ) that would allow us to assess their IL skills ability, the progression of their skills acquisition and therefore the success of our training programme.
Although the necessity of IL skills is implicit in almost every degree programme the QAA has to some degree made these explicit in their benchmark statements
Substantial revision of the Qnn (summer 2005) to assess a range of abilities related to each of the six key skill areas and to ground it specifically in the discipline of psychology, thus encouraging student engagement in the study.
Now used the revised Qnn for two consecutive years and today will present the data from these two years highlighting the usefulness of regularly assessing the students skills as a means of assessing the effectiveness of your training and the progression of student skill acquisition
a) ensuring that given a similar population of respondents (for example two groups of first year students) similar data is obtained at a different time (t-tests L105 vs L106 & L2o5 vs L2o6)
b) Compare the performance of first and second year students using much larger group sizes thus making the statistics more powerful. By this we mean that we now have a large sample of first and second year respondents so we can have some confidence that our data accurately represents these populations. This analysis was also run to check that the data presented in LILAC 2005 was reliable and accurate
2. having run the study with the new Qnn for two consecutive years we can follow the students who entered in 2005 through the first year of acquisition of these skills, and thus compare the performance of the same students at the start of their first and second years of study.
3. And finally we have analysed whether the IL ability of students (as measured by our Qnn) is related to their academic performance as measured by a variety of assessment methods.
We have now run the Qnn for two consecutive years and so we have data from two first and second year cohorts of students. To assess the reliability of the Qnn we compared the results from these two groups and found that:
we can now follow a single group of students (N=112) from entry into our degree programme through to their second year of study. In this way we can now report data relating to the progression of skill acquisition of a single cohort of students (those that entered our programme in September 2005).
A similar pattern to the between group comparison (N=805) previously mentioned is seen with second year students performing better on the Qnn
we can now use the detailed data generated by the Qnn regarding student performance in the different skills areas to give us a clear picture of what skills they are learning and those that they are not or put in a different way what are we good at teaching and what are we not good at teaching OR how is our training programming failing to meet our students need
This analysis was conducted on the whole cohort of respondents from first year in 2005 and second year in 2006
Doesn’t mean to say that less progression means they were useless to begin with – might mean there is little room for improvement in the first place.
(however their understanding of the use of truncation is atrocious (26.8% got it correct) despite having been trained in this and used it as part of an online tutorial for PSYCINFO
GRAPH
first yearsecond year
Mean44.8661.52
Copyright, think they can print or p/copy as much as they want to (– indicates a more general attitude towards information i.e: they expect it to be free?)
GRAPH
first yearsecond Year
Mean48.8066.70
(both fail- they expect the library catalogue to behave like google- that is that all search boxes are holistic in that they will access all information , rather than being specific to particular datasets, for example they are particularly bad at knowing what part of a reference you use to find a journal article- L2= 17.5% correct)
(however, there is still room for improvement in this group)
Confusion of index with bibliography
Not all progression is necessarily IMPRESSIVE
GRAPH
first yearsecond Year
Mean24.3641.73
Good at taking action over inclusive search results – deal with information overload
Seem to trust information without giving sufficient consideration to issues of bias and fail to seek out independent corroborative evidence (50%) – question relating to government information – Ofcom study on teenagers and media literacy confirms this.
Second year students more likely to seek help from textbooks when they don’t understand concepts.
GRAPH
first yearsecond Year
Mean68.7072.20
- Both groups know the purpose of journal article abstracts
- More than half of the students in both groups understand the difference in quality/ validity of information contained in websites in comparison with peer reviewed J articles
- A similar proportion of the students have an understanding of the peer review process
BUT a third of second year students don’t know the quality differences between these sources of information or the peer review process- so why read journals???
GRAPH
first yearsecond Year
Mean71.8675.43
Consistent with our LILAC 2005 presentation
- First year students heavily dependent upon and show a strong preference towards inappropriately sourcing all information from the internet search engines
- second years students are becoming more discerning about the sources of their information but many are still look to the internet for the most recent academic information
- Most students in both groups are unaware of the appropriate use of review articles
GRAPH
first yearsecond Year
Mean38.2442.68
In Leeds, at the end of each 11 week semester students are examined
IL 1 measure weeek1 semester 1 Yr 1 was compared to semester 1 yr1 academic perf
IL 2 measured week 1 semester 1 yr 2 was compared to academic perf at end of semester 2 yr1 (closest assessment period and before they received any further training
To assess whether each students Il ability was related in any way to their academic performance corelative analysis was conducted between these two performance indicators.
IL 1 measure weeek1 semester 1 Yr 1 was compared to semester 1 yr1 academic perf
IL 2 measured week 1 semester 1 yr 2 was compared to academic perf at end of semester 2 yr1 (closest assessment period and before they received any further training
To assess whether each students Il ability was related in any way to their academic performance correlative analysis was conducted between these two performance indicators.
Pre-training
Recognise need: There is no point in creating a training programme without knowing what the needs of your students are- they may surprise you good OR bad. Remember not only do you need to assess the abilities of new incoming students, but also more advanced students e.g. those that you may assume have certain abilities. (our data clearly demonstrates that even after specific explicit training our second year students have not acquired certain skills- much to our dismay!!)
Identify: There are now many different teaching methods used to address the training issues associated with IL training (online tutorials, library treasure hunts, classroom activities), but also remember that people are resources, we feel that the success of this work is largely attributable to the consistent and direct interaction between library staff and academic staff. This interaction is critical if you want to follow best practice guidelines and integrate your skills training into the academic curriculum
Construct: Only when you know the needs of the students can you provide them with a, timely and progressive programme of training
Post-training:
After delivery there must be some assessment of the abilities of the students and analysis of this data should be used to inform future training thus producing evidence-based training
There is much still to explore, and we’re not done yet!