Lee, Y. Y., Lowe, M. S., Maxson, B. K. & Stone, S. M. (2017, May). Pedagogical v. pathfinder:
reimagining course and research guides for student success. Presented at the LOEX (formerly Library Orientation Exchange) annual conference, Lexington, KY.
Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
Pedagogical v. pathfinder: reimagining course and research guides for student success
1. Pedagogical v. Pathfinder:
R e i m a g i n i n g C o u r s e & R e s e a r c h G u i d e s f o r S t u d e n t
S u c c e s s
Y o o Y o u n g L e e
Digital User Experience Librarian
@ IUPUI University Library
Educational Development Librarian
@ IUPUI University Library
Humanities Librarian
@ IUPUI University Library
Dentistry Librarian
@ IU School of Dentistry
M . S a r a L o w e B r o n w e n K . M a x s o n S e a n M . S t o n e
2. POLL
h t t p s : / / r e s p o n d . c c
S e s s i o n K e y : 3 7 7 9 7 1
I n o n e w o r d o r
s h o r t p h r a s e ,
w h a t i s t h e
p u r p o s e o f
r e s e a r c h / c o u r s e
g u i d e s ?
h t t p s : / / m q l i c k e r . c o m / r / i 8 m 1R e s p o n s e s :
3. Does the design and
layout of course &
research guides
impact student
learning?
Q
4. h t t p : / / w w w. d i g i t a l p e d a g o g y l a b . c o m / h y b r i d p e d / l i b g u i d e s - p e d a g o g y - t o - o p p r e s s /
Why?
6. Learning Theory C o g n i t i v e
L o a d
h t t p s : / / t e a c h i n g h o w 2 s . c o m / b l o g / c o g n i t i v e - l o a d
7. Guide Design
P e d a g o g i c a l P a t h f i n d e r
F o r a n e x a m p l e o f a c u r r e n t p e d a g o g i c a l
g u i d e :
h t t p : / / i u p u i . c a m p u s g u i d e s . c o m / b r i d g e
10. Outside of the classroom
a k a s e l f - d i r e c t e d l e a r n i n g
11. Methodology
n = 2 2
0
1
2
3
4
5
6
7
F i r s t - Y e a rS o p h o m o r eJ u n i o r S e n i o r G r a d u a t e
W o r k f l o w
P a t h f i n d e r o r
P e d a g o g i c a l
G u i d e
P r e - S u r v e y
S i m u l a t e d
R e s e a r c h
A s s i g n m e n t
3 s c h o l a r l y
s o u r c e s
P o s t - S u r v e y
( U s a b i l i t y )
12. Methodology
L e a r n i n g O u t c o m e L e v e l o f A c h i e v e m e n t
Developed
3
E m e r g i n g
2
I n i t i a l
1
K e y w o r d s & S y n o n y m s Sufficient keywords and synonyms
● Multiple keywords identified
● Multiple, appropriate synonyms
identified for each keyword
Insufficient keywords or synonyms
● Limited identification of keywords
● Few or improper synonyms
Minimal keywords and no synonyms
● No keywords beyond those used in
the original question/topic
● No viable synonyms found for the
keywords
R e s e a r c h Q u e s t i o n Question is relevant to the topic Question is kind-of relevant to the topic –
containing elements of the topic but not
everything (e.g., Islam in America)
Question is not relevant to the topic, very
vague, OR only repeats the topic in
question form
A r t i c l e ( R e l e v a n c e ) Article content seems appropriate for the
proposed question/topic
Article is only tangentially related to the
proposed question/topic
Article is not related to the proposed
question/topic
A r t i c l e ( S c h o l a r l y ) Article is from an appropriate scholarly
journal
Article is from an authoritative source (e.g.,
NY Times) but not a scholarly journal
Article is not from an authoritative source
nor a scholarly journal
E v a l u a t i o n Well thought out explanation of how the
article was relevant to the topic or research
question
Explanation is not completely thought out
or clearly copied from article abstract
No explanation or no effort to explain (e.g.,
it’s scholarly, I found it in a database, etc.)
R u b r i c
13. Results
P e d a g o g i c a l P a t h f i n d e r
T 1 : H o m e
T 2 : 1 . Y o u r Q u e s t i o n
T 3 : 2 . B a c k g r o u n d
I n f o r m a t i o n & B o o k s
T 4 : 3 . F i n d M a t e r i a l s
T 4 : A r t i c l e s
T 3 : R e f e r e n c e M a t e r i a l s
T 4 : A r t i c l e s
G u i d e I n t e r a c t i o n
14. Results
P e d a g o g i c a l P a t h f i n d e r
G u i d e - D a t a b a s e I n t e r a c t i o n
16. Results
0 1 2 3 4 5
The resources listed on the guide were too many for me.
I was able to use the research guide without any additional instructions.
The research guide helped me improve my research skills.
I would recommend the research guide to a friend.
Overall, I am more confident about research.*
Overall, I am satisfied with the ease of using the research guide to find a scholarly source.
I quickly became skillful with it.‡
I learned to use the research guide quickly.*
The research guide helped me find scholarly sources.
P e d a g o g i c a l P a t h f i n d e r
P o s t - S u r v e y
17. Results P o s t - S u r v e y
0 1 2 3 4 5
The research guide worked the way I want it to work.
Overall, I am more confident about research.
I learned to use the research guide quickly.
I quickly became skillful with it.
I easily remembered how to use the research guide.
I would like to be able to refer to any subject research…
Pedagogical Pathfinder
p-value<0.05p-valuecloseto0.05
18. Student Perceptions
P e d a g o g i c a l
• I t i s h e l p f u l t o d e f i n e
w h a t a s c h o l a r l y a r t i c l e
i s a s I d i d n ’ t k n o w w h a t
i t r e a l l y m e a n t .
• T h i s i s s o c o o l !
[ I n f o g r a p h i c o f t h e
r e s e a r c h p r o c e s s ]
• I l i k e t h e w a y t h e
r e s o u r c e s a r e o r g a n i z e d .
P a t h f i n d e r
• I d o n ’ t k n o w w h e r e t o
s t a r t .
• I d o n ’ t k n o w h o w t o u s e
d a t a b a s e s a n d t h e
r e s e a r c h g u i d e d o e s n ’ t
h e l p m e f i g u r e o u t .
• S o c o n f u s i n g .
20. Methodology
I n s t r u c t i o n
• T w o s e c t i o n s o f f i r s t - y e a r s e m i n a r
• n = 4 0
• S a m e p r o f e s s o r
• S a m e l i b r a r i a n
• S a m e c o u r s e c o n t e n t
A s s e s s m e n t
• P r e - T e s t
• P o s t - T e s t & U s a b i l i t y Q u e s t i o n s
• 2 w e e k s a f t e r l i b r a r y s e s s i o n
• E n d - o f - S e m e s t e r T e s t
• A n n o t a t e d B i b l i o g r a p h i e s
• R u b r i c
21. Results
-10%
-5%
0%
5%
10%
15%
p r e - t e s t t o p o s t - t e s t p o s t - t e s t t o e n d - o f - s e m e s t e r
P e d a g o g i c a l P a t h f i n d e r
I m p r o v e m e n t B e t w e e n
A s s e s s m e n t s
22. Results
0
1
2
3
S o u r c e A n n o t a t i o n E v a l u a t i o n C i t a t i o n ‡ M e c h a n i c s
P e d a g o g i c a l P a t h f i n d e r
A n n o t a t e d B i b l i o g r a p h y R u b r i c
23. Results
0 1 2 3 4 5
know where to look to find differences and changes in a topic or discipline.
write using proper attributions and citations.
organize and begin to synthesize my resources for use in my research project.‡
effectively search for resources (choose keywords, use Booleans, limiters, etc.).‡
understand the difference in types of resources like scholarly and popular.
evaluate resources and choose the ones with the highest quality.‡
understand the research process and academic scholarship in general.‡
choose appropriate databases and places in which to look for resources.‡
choose and/or refine a research topic or research question.‡
P e d a g o g i c a l P a t h f i n d e r
“ T h e r e s e a r c h g u i d e h e l p e d m e … ”
24. Results
0 1 2 3 4 5
know where to look to find differences and changes in a topic or discipline.
write using proper attributions and citations.
organize and begin to synthesize my resources for use in my research project.‡
effectively search for resources (choose keywords, use Booleans, limiters, etc.).‡
understand the difference in types of resources like scholarly and popular.
evaluate resources and choose the ones with the highest quality.‡
understand the research process and academic scholarship in general.‡
choose appropriate databases and places in which to look for resources.‡
choose and/or refine a research topic or research question.‡
P e d a g o g i c a l P a t h f i n d e r
“ T h e r e s e a r c h g u i d e h e l p e d m e … ”
26. Conclusions
O u t s i d e t h e c l a s s r o o m s t u d y
Although no statistically significant
performance difference between the two,
students using the pedagogical guide reported
a more positive experience and greater self-
efficacy not seen in students using the
pathfinder guide.
I n - c l a s s r o o m s t u d y
Students using the pedagogical course guide
demonstrated better learning and retention
of concepts than students using the pathfinder
guide.
30. Student Perceptions
Discuss how you think your research process might differ from someone
who does not have access to the resources you have because you are a
student at IUPUI.
• I t h in k it wou ld be fairly diff i cu lt becau se t h ey wou ld h ave t o go
search for research at a library or somewh ere e ls e. T h ey wou ldn 't
h ave t h e on lin e resou rces w e u sed .
• I t h in k anyon e o ut side o f col leg e h as th e ex act same resou rces as
anyon e in colleg e . S ome databases ju st requ ire paymen t an d
u su ally [t h e] sch ool pays for it .
• T h ey may h ave t o fin d a differen t way t o access t h e in format ion
wh ich may be chall en gin g if y o u do not live n e ar a library o r t h e
people an d resou rces t o h elp y o u .
C r i t i c a l
P e d a g o g y
31. T h a n k Y o u !
Questions?
Y o o Y o u n g L e e
y o o y l e e @ i u p u i . e d u
M . S a r a L o w e
m l o w e @ i u p u i . e d u
B r o n w e n K . M a x s o n
m a x s o n b @ i u p u i . e d u
S e a n M . S t o n e
s m s t o n e @ i u . e d u
32. POLL
h t t p s : / / r e s p o n d . c c
S e s s i o n K e y : 2 9 9 9 7 6
W i l l y o u m a k e
c h a n g e s t o y o u r
g u i d e s b a s e d o n
t h i s
p r e s e n t a t i o n ?
I f s o , h o w ?
h t t p s : / / m q l i c k e r . c o m / r / s 8 m 3R e s p o n s e s :
Editor's Notes
BKM
Each section ~10 minutes ; ~10 min for discussion of crit lib @ end ; ~10 min for general questions
Session title (limit 20 words): Pedagogical v. Pathfinder: Reimagining Course and Research Guides for Student Success
Proposal Track: Innovations and Failures: Roots of Our Success
List of 1-3 learning outcomes ;
Participants will be able to RECOGNIZE how the design and organization of guides affects student learning IN ORDER TO design their own guides.
Participants will be able to INTERPRET student information-seeking behavior through observed and recorded data IN ORDER TO develop guides that aid student success.
Participants will be able to EXAMINE the critical pedagogical implications of student learning within the confines of guides IN ORDER TO build their own capacity in this area.
Type of audience(s) for which the session is intended: People with some experience in the topic, but looking to grow
Long description of the session (limit 500 words);
Course and research guides are widely used to extend instruction beyond the class session. Does the design or layout lead to transformative learning experiences for students? Traditionally designed to help students find library resources, guides (at many libraries, including ours) were organized in a pathfinder-style format, containing a static list of resources, such as find books, find articles. This type of guide might lead students to believe that research is easy, the answer is just one click away. But if we think of guides as an extension of the class session, serving as an aid for students to conduct research and develop critical thinking and information literacy skills, does a pedagogical design, where the guide walks a student through the complexities and nuances of the research process, aid in that dynamic process better than a traditional pathfinder guide? This hypothesis would require a radical re-design of our existing guides, with no evidence that the pedagogical style is more effective.
This presentation discusses two separate studies designed to measure if guides can be designed to improve learning both in and outside the classroom. Through separate methodologies, both seek to determine which is better - a pathfinder or pedagogical-style guide? The first is a usability study observing how students interact with a guide through a simulated research paper which sought to determine if there is a “better” design to encourage student learning outside the classroom. Students from freshman to graduate students (n=20) across disciplines participated in the study. Randomly assigned to either guide type, students were asked to complete a pre-survey, usability testing with a mock research paper assignment, and a post-survey.
The second study is an in-classroom study and involves a pre-test to determine a baseline, instruction using the two styles of course guides, and post-tests to measure retention. Using classes with multiple sections but the same instructor and librarian (n=40), in one section librarians present a pedagogical-style guide with written support and instruction to help students discover the process of research in its complexity. In another section, instruction occurs using a traditional pathfinder-style guide. Both sections are tested for retention with post-tests deployed throughout the semester (2 weeks post instruction and end-of-semester) as well as authentic, rubric-based assessment of final student work (research paper, annotated bibliography). In addition, to determine if critical pedagogy can live within a guide, the post-tests check not only for basic skills and processes but ask students to reflect upon their learning through the lens of information privilege.
Presenters will engage the audience through real-world examples as well as qualitative and quantitative study data which will encourage them to think more deeply about the impact of guide design on student learning both in and outside of the classroom.
Short description of the session (limit 100 words);
What is the best way to design course and research guides? Can the design of a guide impact student learning and transform student information-seeking processes? This presentation will detail results of two studies (one in-classroom, the other usability) which use qualitative and quantitative assessments to determine if guide design aids in student learning. Draw on our successes and failures to invigorate your course and research guides for transformative learning.
BKM
Short description of the session (limit 100 words);
What is the best way to design course and research guides? Can the design of a guide impact student learning and transform student information-seeking processes? This presentation will detail results of two studies (one in-classroom, the other usability) which use qualitative and quantitative assessments to determine if guide design aids in student learning. Draw on our successes and failures to invigorate your course and research guides for transformative learning.
BKM
This all started with a blog posting by Alison Hicks – Libguides:Pedagogy to Oppress? - http://www.digitalpedagogylab.com/hybridped/libguides-pedagogy-to-oppress/
Although some of our guides leaned pedagogical, there had not been a systematic effort to make all guides pedagogical, nor did we have evidence that pedagogical guides were any better than pathfinder-style guides. [And I'll define what we're calling pedagogical and pathfinder guides in a moment]
Yoo Young Lee was also interested in course and research guide design from a usability and consistency standpoint – in order to create best practices and make recommendations to librarians.
BKM
The pathfinder guide embodies what Paulo Freire called "the banking model of education". As illustrated by this cartoon, students can unfortunately get the impression that learning is about filling their heads with facts that they need to recall in order to get a good grade. Or clicking on the right link to get to the right database to download the articles that reinforce their ideas. Cherry picking sources in their writing and instructors nit-picking over citations can also be reflections of this impression of education.
Joan Wink (2005) Critical pedagogy: Notes from the real world: "Critical pedagogy is a way of thinking about, negotiating, and transforming the relationship among classroom teaching, the production of knowledge, the institutional structures of the school, and the social and material relationships of the wider community.”(Boston: Pearson/Allyn & Bacon.)
-Looping back to the Hicks blog post - One of our questions was – is this something you can “teach” or develop in students from a research guide? We want students to movie beyond the banking model of education. We also want to cultivate critical thinking in our students – the Framework certainly asks more of IL than box-checking.
Jacobs' Pedagogies of Possibility Within the Disciplines: Critical Information Literacy and Literatures in English: "By extension, our students also can see that their own work within libraries or in literary historical research is not value neutral but, rather, situated in a complex matrix of social, political, and cultural forces with which they may interact in numerous ways. In order to 'realize the full potential' of a broadly defined information literacy in literatures in English, we need to pose problems about libraries to our students (and to our selves) that interrogate all of the choices, values, actions, and inactions that shape our libraries and inform what we, as individuals and as a society, know and how we know it.”
Problem-posing: how do people access information without library access? What kind of information do you need to answer your research question?
It's also about us as librarians -Thinking reflectively about our practice/teaching – why are we teaching what we’re teaching?
BKM
Cognitive Load – This can be overwhelming for students
(Intrinsic Load > Extraneous Load > Germane Load = Total Cognitive Load)
(Intrinsic – inherent demand of the learning task ; Extraneous – activities unrelated to learning ; Germane – activities that promote learning)
examples – e.g., Guides in smaller subjects – breaking large topic down into smaller chunks – decreases intrinsic cognitive load; Guides for individual courses; steering clear of formal language and jargon
Little, Jennifer J. "Cognitive Load Theory and Library Research Guides.“ Internet Reference Services Quarterly 15, no. 1 (January 2010): 53-63.
Thinking of teaching to the guide – as an extension of the classroom – what are the learning outcomes?
BKM
The research process graphic was something Sara Lowe introduced to IUPUI and the pedagogical design was sort of a happy accident. You can see on the left, the pages or tabs in the guide are numbered. I had an instructor call me up and say, "I saw you moved the database list off the front page. I was worried students wouldn't know where to go, but then I looked at your new guide and I can see they will know exactly what to do."
On the right of course, is the traditional pathfinder guide with just a list of resources, often organized by format. Most LibGuides still exist in this form.
Let's take a closer look.
BKM
Explain what we mean when we say pedagogical and pathfinder – what does that look like
BKM
Explain what we mean when we say pedagogical and pathfinder – what does that look like
SL
SL
Usability study
SL
SL
On average, pedagogical clicked 4 pages ; pathfinder 3.
SL
On average, pedagogical used 6 resources ; pathfinder 3.
SL
No statistical significance in performance.
SL
1 = strongly disagree
5 = strongly agree
In every case (except the last which is negative – so higher is worse) pedagogical outperformed pathfinder.
SL
Yellow arrows = p-value less than 0.05
White arrows = p-value close to or slightly bigger than 0.05, although we can’t reject the null hypothesis
1 = strongly disagree
5 = strongly agree
SL
Participants’ verbal expressions were analyzed in order to see how they were feeling while doing the simulated research assignment. Both groups indicated that they were not sure in general about how to conduct research or how to use databases like “I don’t know how to find articles in general (Participant ID 1)” or “I’ve never heard of any databases except Wikipedia (Participant ID 13).”
However, the pedagogical group expressed more their positive feeling on research guide than the pathfinder group (Fig 4). The pedagogical group said like “It is helpful to define what a scholarly article is as I didn’t know what it really meant (Participant ID 1),” “This [the graphic] is so cool (Participant ID 5)” or “I like the way the resources are organized (Participant ID 19)” while the pathfinder group said like “I don’t know where to start (Participant ID 4),” “I don’t know how to use databases and the research guide doesn’t help me figure out (Participant ID 3)” or “So confusing (Participant ID 16).”
SS
SS
Same worksheet in each class
Each section used their respective guide to finish worksheet in class
LibGuide hits - pathfinder section=317 hits between August and December 2016; the pedagogical guide=253
SS
Guide integrated into course management software (Canvas)
Students encouraged to use the course research guide for post-test and end-of-semester test
Ped was more consistent at end of semester
NOTE: pedagogical students started lower than pathfinder students in pre-test -
Average % correct:
Pre-test – ped 40% ; path 48%
Post-test – ped 54% ; path 61%
End-of-semester – ped 53% ; path 55%
Gains were similar in pre- to post-test but pedagogical lost less learning (e.g., retained more) from post- to end-of-semester
SS
Discuss rubric categories – based on earlier rubrics
Graded by librarian/faculty pair after norming
Difference in scores in citation - p-value less than 0.05 (evaluation is close to 0.05 but we can’t reject the null hypothesis)
SS
Universal approval of pedagogical guide
1 = strongly disagree
5 = strongly agree
SS
Yellow arrows = ‡ p-value less than 0.05
SS
SS
SL
Presentations to librarians at IUPUI letting people know about the findings
LibGuides Task Force has made recommendations
Issues:
Every liaison controls their own guides, all we can do is lead them to the water. We can’t force them to drink.
It takes a lot of work and time to retool a guide - and some liaisons are unable or unwilling to put in that effort.
We are not at 100% compliance.
SL
Beginning to look at heat mapping (with product called Crazy Egg)
Multiple avenues to get at UX and student learning
SL
Because the outside of classroom study showed students only click about 4 tabs – we’ve modified the research process to help it align more with their preferences – but also trying not to lose the nuance or complexity of the process.
Encouraging creation of more course (rather than subject) guides.
BKM
Elmborg's Critical Information Literacy: Implications for Instructional Practice "Its most influential theorists, including Paolo Freire, Peter McLaren, and Henry Giroux, argue that schools enact the dominant ideology of their societies— either consciously or unconsciously. Viewed this way, education is a profoundly political activity. Educators must either accept the dominant ideology of their society or intentionally resist it and posit alternative models. Neutrality is not an option.”
Looping back around to the in-classroom study – this was a failure
End of semester quiz included a question to get at critical pedagogy “Think back on the research process you used to find sources for your paper this semester. In a brief paragraph, please discuss how you think your process might differ from someone who does not have access to the resources you have because you are a student at IUPUI.”
All answers are from pedagogical students.
No discernable difference between pedagogical and pathfinder students – although slightly more pedagogical students seemed to get the idea of information privilege.
More work is needed in this area. Pedagogical style guides might increase critical pedagogy – but our measurement didn’t prove this one way or the other.
