To Evaluation and Beyond: The Evolving Role of the Embedded Librarian

This presentation was prepared for the LibrariesThriving webinar series sponsored byCredo Reference, which took place on Jan. 14, 2013.In the presentation, we explore• the nature of evaluation in higher education, and how current trends are affecting the delivery of information literacy instruction.• the proposition that embedded librarianship offers an effective framework for delivering information literacy instruction, and also lends itself – more than any other option – to meaningful evaluation.The presentation also reviews the definition of embedded librarianship really is, anddiscusses some key factors that contribute to applying it successfully. Finally, othervalued roles beyond instruction are discussed. 1

Let’s focus on the connections between three elements: higher education,information literacy instruction, and evaluation. 2

Let’s start with the intersection of evaluation and higher education.Over the past few years, academic institutions have come under increasing scrutiny.They are being called on to demonstrate their effectiveness, justify their value, andadapt to radically new circumstances – and they haven’t always been up to the task.In the abstract of her address to the Library Assessment Conference last Fall, Dr.Judith Eaton, President of the Council for Higher Education Accreditation, had this tosay, “Higher education’s work in assessment … is not sufficiently responsive to today’senvironment in which greater transparency, reliable evidence of student achievementand clear and readily accessible information about our performance have takencenter stage.” 3

And really we have seen only the beginning. Predictions abound that new computingand communications technologies, along with new operational models, are poised tocause a fundamental disruption in higher education. Mark Andreessen, creator of thefirst graphical web browser and now a successful Silicon Valley venture capitalist, wasquoted in Wired magazine last May as saying that education was high on the list ofindustries to be disrupted. 4

No doubt you’re familiar with examples of the disruption taking place – initiatives likethe Khan Academy, the EdX joint venture of Harvard, M.I.T., and now others, theStanford University experiments that led to Coursera and Udacity – et cetera, etcetera. This list is going to continue to grow. 5

So it’s clear that university administrators have to sharpen the focus on questionslike: are we doing a good job? Are we worth what people are paying us? How do weknow, and how can we prove it? What changes will enable us to do even better?They’re going to ask us and every other unit on campus too. They recognize thatexpectations are rising continually, and the status quo is not an option. 6

Now let’s look at evaluation and information literacy instruction. 7

Academic libraries are no strangers to disruption. We have been living with disruptionof our traditional operational model for just exactly 20 years now. It was in 1993 thatAndreessen’s Mozilla graphical web browser, along with the opening of the WorldWide Web to all users, began to disrupt society’s information infrastructure in themost profound way since Gutenberg. The resulting disruption is evident in this graphof reference transactions reported by members of the Association of Researchlibraries in the first decade of the 21st century. The numbers are down 45% in 10years. 8

As Michael Stephens put it in Library Journal, People do not think of the library firstwhen they need information.(“Stuck in the Past.” LJ Apr 15, 2011, p. 54) 9

We’ve responded to the disruption in a variety of ways: creating and integratingdigital collections; repurposing our buildings; and taking on an increased instructionalrole are some of the major changes we have introduced. Information literacyinstruction has been an effective response by academic librarians to the disruptionwe have been facing. 10

But we haven’t done so well when it comes to evaluation. The authors of ARL Spec Kit318 stated in 2010 that they had found “shockingly little work that focuses oninvestigating whether use of library resources and services correlate with measures ofsuccess for library users.” Yet that’s exactly what our institutions are demanding of us,and will continue to demand, as society and new competitors demand it of them. 11

So, still we are left with the questions – the same ones our parent institutions arefacing: and not very good ways to answer them. 12

Note that what we need here is not just more evaluation, but different evaluation. Tohelp us analyze the options for evaluating our work, I’d like to cite Philip Kotler, ofNorthwestern University, Kellogg School of Management. Kotler is widely regarded asthe dean of academic thinkers in the field of Marketing. He identifies three levels ofevaluation, which he terms Output, Outcome, and Impact.Adapting these three levels to today’s discussion, I’d like to offer these definitions. 13

First, Output. We might expand this to include Inputs and Activities as well. When wemeasure our budgets, or the number of items in our collections, or the number ofitems we buy, or catalog, we are measuring inputs, activities, and outputs. Thesemeasures can tell us something about our value, indirectly, but their relationship tovalue is tenuous. They don’t tell us if those items we acquire, catalog, and hold areused; or if the use benefits anyone. 14

Then there are Outcomes. Outcomes tell us how people responded to our actions. Interms of traditional library services, measures like the number of circulations, ordatabase accesses, or reference questions, or interlibrary loan transactions are a formof Outcome measurement. They tell us that the students and faculty we have tried toreach used our services to a greater or lesser degree. Surveys are generally a form ofOutcome measurement as well. In surveys, we often ask whether our audiences arefamiliar with our services, whether they have used them, and what they think of thequality and effectiveness of our work. These are Outcomes. 15

Finally there are the Impacts. Impacts tell the story of how our work has changed theworld. Sadly, in traditional library work we rarely are able to gauge the Impacts of ourwork. We don’t know if a book that was checked out got read. We don’t know ifreading that book helped the student get an A on the paper, or an A for the course.We don’t know if a great article retrieved from one of our databases – or even thehelp of a reference librarian – changed the student’s major, or career choice.Edwin Perry was an academic reference librarian for 43 years. As he wrote in hisarticle in the December 2011 issue of Searcher magazine, as a reference librarian*quote+ “you rarely learn the end result of your work. Did your help really help toproduce the desired results? If you are lucky, you will experience a few situationswhere you get the answer to that question.” *end quote+ … But a few situations aren’tenough for the kind of evaluation demanded of us today.But a few situations aren’t enough for the kind of evaluation demanded of us today.What we need to do is to align our goals with the instructional goals of the coursesand programs of our institutions, and ultimately with their overarching goals forstudent learning and achievement. That way, our work will be evaluated as part ofthe overall institutional evaluation strategy. So, how do we do that? 16

To develop an answer, let’s start by analyzing information literacy instruction. Thenwe’ll connect up IL instruction with evaluation.Clearly, information literacy instruction is meeting a need. It’s needed because insome ways that disruption of our society’s information infrastructure that wementioned earlier has made it harder, not easier, to get the right information at theright time. So, learning about instructional design, developing instructional programs,and delivering them have become hot skills for academic librarians.There are several different ways that we can analyze the options for informationliteracy instruction. I’m going to label three important dimensions as the Technologydimension, the Mode dimension, and the Structural dimension. 17

Technology is probably the most obvious. We have an ever-proliferating array oftechnologies to choose from when we are designing IL instruction. Here’s a selectionof the ever-multiplying array of tools at our disposal. 18

Compared to the growing array of technologies at our disposal, what I’m calling theMode dimension is pretty simple. We can either deliver instruction when everyone istogether at one time – synchronously – or we can design instructional programs thatallow students to work through the instruction whenever they want to –asynchronously. Similarly, we have options for bringing everyone together in oneplace, or conducting our instruction across a distance, with remote students: co-located, or dispersed. 19

Finally there is what I’m calling the Structural dimension. What I have in mind here isthe administrative structure under which we deliver our IL instruction. There arethree main options. The first is ad hoc delivery. The most traditional form of ad hocinstruction takes place when we turn a reference interview into a teachable moment,and use the student-initiated interaction to conduct a little training exercise inresearch methods. Beyond that, we can post tutorials on the web, and promotethem, and hope students use them. We can offer workshops and seminars. All ofthese, because they are more or less informal and voluntary, I group together underthe label “Ad Hoc.” Next is the standalone, formal course. This can be either non-credit or for-credit; mandatory or voluntary. But it’s a formal learning program thatstudents register for, that requires them to go through a set of experiences, andincludes some sort of formal evaluation. It’s “standalone” because it’s unilaterallyadministered and led by the library organization. Then, finally, there’s the form ofinstruction that’s commonly referred to as Embedded. With the Embedded structure,IL instruction is incorporated into an academic program. Typically, this is at the levelof a specific subject course – so it’s sometimes called course-embedded. However, itdoesn’t have to be at the course level. Thus, there’s some overlap between thestandalone and embedded models. That is, you might have a separate informationliteracy course, but one that is developed to meet the needs of a specific academicprogram and is tailored to the needs of that program. 20

So, these are the three dimensions we can use to analyze approaches to ILinstruction: Technology, Mode of instruction, and administrative Structure. Thechoices we make on each dimension can affect the success of our work. But the pointI want to emphasize is this: They’re substantially independent of each other.In particular, it’s critically important to recognize that embedded librarianship isn’t atechnology, or a mode, and it isn’t dependent on any technology or mode of delivery.You can be an embedded librarian for a distance course and do all your work in acourse management system, or you can stand up in a classroom and lead a classsession. Embedded librarianship is something entirely independent of technology ormode. 21

Obviously, technology, mode of instruction, and structure are all going to affect howyou evaluate your instruction. But let’s focus on just the structural dimension, so wecan examine how the evaluation differs among the options of Ad Hoc, Standalone,and Embedded. I’ll use a series of tables to illustrate the issues we face. Rememberfrom our earlier discussion that it’s impacts we are truly after. Outputs and outcomeshave their place, and I’m not dismissing them, but ultimately what we also need toshow is that our work has an impact on student success, in terms of course goals andinstitutional goals for student achievement. 22

We’ve already discussed the point that traditional ad hoc instruction in the referenceinterview doesn’t give us much insight into outcomes and impacts. The same is truefor ad hoc tours, orientations, and guides. Web-based tutorials do offer at least thepossibility of assessing student learning, through the use of self-administered quizzes.However, almost all the ones I’ve seen are pretty simplistic. Usually you can just guessthe answer to a multiple-choice question – and if you guess wrong, you get to guessagain, so you “pass” the test. Plus, students taking ad hoc tutorials are oftenanonymous, so there would be no systematic way to relate their use of the tutorialsto academic results. That’s the reason why I’ve grayed out the Impacts of web-basedtutorials in the table. It’s pretty hard to tell if we’ve “changed the world” – that is,effected a lasting change in the students’ capabilities – from ad hoc instruction. 23

When it comes to standalone instruction, the situation is better. A formal course,whether non-credit or for-credit, enables the librarian to set up some form ofevaluation of student learning. A for-credit, graded course might be the strongestcandidate for demonstrating an impact, because it affords the librarian theopportunity to make a formal, recorded evaluation of student achievement. If the ILcourse is truly standalone – not connected to the needs of an academic program, orto overall institutional goals and mission – then we can correlate IL courseperformance with overall academic performance. 24

With course-embedded instruction, we get essentially the same measures as withStandalone Instruction. But we also get a few more. Since the IL instruction takesplace as part of the course, course assignments can be structured for students toapply, and be evaluated on, Information Literacy skills. The immediate application ofthe skills in context, and evaluation, provides a powerful learning and evaluationopportunity that’s missing from the Standalone model.Also, subject faculty gain direct, immediate insight and appreciation for the impact ofIL instruction. When it works well, they see the immediate improvement in studentperformance. This generates a “virtuous cycle” in which they then become advocatesfor embedded IL instruction, and reinforce it. There are examples of this in theliterature and I’ve encountered them in my own research. 25

And here’s just one vivid example – an observation by a Biology professor Iinterviewed in the course of my research. She had worked with an embeddedlibrarian in the first-year course and then had the following perception when she sawsome of the same students again as sophomores.“The information literacy skills developed in the Great Problems course *a first-yearcourse with an embedded librarian] stay with the students and enable them toperform better *as sophomores+.”This instructor has perceived a direct connection between a librarian’s effective ILinstruction and ongoing student academic performance. There are other exampleslike this in my own research and other literature as well. It seems to me that that isthe kind of change we are trying to create in the world as a result of our work – andthe kind of impact that we need to show to our academic administrators andinstitutional leaders.I do want to emphasize that general correlations have their place. But, as I’ll observein another context in a few minutes, direct perception and support from subjectfaculty are extremely powerful. You want them on your side! 26

Now let’s explore in more depth the nature of embedded librarianship, and how itaffords us opportunities to strengthen both our work and our ability to evaluate ourimpacts. 27

So here’s my definition of embedded librarianship:An embedded librarian is one who develops strong working relationships withmembers of a team or community; develops shared objectives with the communityand responsibility for achieving them; and makes customized, highly-valuedcontributions to the team, which play an important role in the success of the teamand the institution. Applied to IL instruction, this means that the embedded librarianbuilds strong working relationships with instructors and students. The relationshipslead to shared understanding – the librarian’s understanding of the course andacademic requirements; the instructor’s understanding of the role of informationliteracy in the course. From the basis of this understanding, the two can collaborateeffectively to provide a powerful learning opportunity for the students. For thelibrarian’s part, this means customized, relevant professional work. It’s theseelements that lead to achieving the learning objectives of the course and theoutcomes that the institution wants to produce for the students. 28

To develop this further, here are five key factors that differentiate embeddedlibrarianship from traditional librarianship. 29

As I talk to academic librarians and read accounts of embedded librarianship, I see aconsiderable number of so-called “embedded IL instruction” programs that don’t fullyachieve the promise of embedded librarianship. There’s a sense that something ismissing. So, now I’d like to address what works and what doesn’t. I’ll focus on two keyfactors.The first is the relationship between the librarian and the subject instructor, and thesecond is the key role of leadership on the part of senior library adminstrators. 30

Sometimes just because there’s a link to a librarian, library resources, or librarytutorials, in the course management system, or the librarian is able to participate inCMS-based activities, the instruction is considered “embedded”. Sometimes a one-shot lecture is moved from the library to the classroom, and it’s considered“embedded”. Sometimes librarians are designated (by the library administrator) asembedded, but instructors fail to engage. I recently read an account of a librarianwho quote embedded unquote a video introduction into some courses in a coursemanagement system. Only she didn’t tell the instructors ahead of time that she wasdoing this. Guess what? Some of them weren’t too happy about it. That was notembedded librarianship!So what was lacking in that example, is lacking in a substantial number of ILinstructional initiatives that aren’t really embedded: the fundamental collaborativerelationship that is the prerequisite for embedded librarianship. Or, the relationshipmay not have matured into a true collaboration.Here’s a set of questions that can help you evaluate the strength of collaborationbetween a librarian and a subject instructor. 31

If all the above sounds like a lot of work, it is. So, how are we going to get all of thatdone? That’s where the second factor comes in. It is effective strategic leadership.This probably involves the senior library administrator, though it may involve othersas well. In my research, I’ve seen counterproductive leadership behavior ranging frompassivity to outright undermining of embedded librarianship. On the other hand, I’veseen examples of active and effective strategic leadership. Here are four key elementsof this effective leadership. 32

I promised to close with some observations about other roles for librarians in highereducation. That might seem a bit disconnected from the emphasis on evaluation ofinformation literacy instruction in higher education, but it’s not. The connection isthat final point about evaluating our work by how we contribute to our parentuniversities.There’s a general formulation of the mission of institutions of higher learning, and itincorporates three things: teaching, research, and service.So far, all we’ve talked about in this presentation is the teaching mission. It’simportant to be sure, but what about research, and service? Increasingly, academiclibrarians are finding ways to embed themselves in these activities. 33

Many academic libraries are getting involved in curating research data. Embeddedlibrarians are in a position to contribute a great deal to these initiatives. Because oftheir relationships with faculty and researchers, they understand the data and cancontribute their understanding to its management.Another contribution that spans teaching, research, and service is knowledgemanagement. A higher education trend we haven’t talked about today is the trendtoward interdisciplinarity. In all aspects of academic life, the integration of disciplinesis increasingly important, and librarians are well positioned to contribute.Academic librarians are also able to leverage their embedded relationships to take aleadership role in service. 34

These are just a couple of examples of the trend. Barbara Dewey, then of theUniversity of Tennessee, gave an enduring, comprehensive vision of the potential inher 2004 article in “Resource Sharing & Information Networks.” She wrote:“The embedded librarian, who is truly integrated into the academic, administrative,athletic, cultural, research, teaching, and learning arenas of the university, providesquality and depth to the total campus experience.”That’s what we’re after. So, when you’ve established your embedded informationliteracy instruction program, and you’ve demonstrated its success in terms of impactson the university mission – don’t think you’re done. As an academic librarian, youhave more opportunities awaiting you! 35

And now I’d like to close by thanking you for your attention. I’d also like to thankLaura and the Credo Reference organization for sponsoring today’s session. If you’dlike to know more, here are three ways to follow up:1. Feel free to email me. I love to hear from those of you who are doing this work, orthinking about it. I’d love to hear from you.2. Check out my blog at www.embeddedlibrarian.com . I only post when I have timeand something to say, so if you subscribe to it you won’t be inundated.3. And finally, you might be interested in my book, published last summer byInformation Today. The second half of it is an in-depth guide to developing andsustaining embedded librarianship, based on my research and experience in the field.I hope it may be helpful to you. 36

Credits 37

To Evaluation and Beyond: The Evolving Role of the Embedded Librarian

by davidshumaker on Jan 18, 2013

Accessibility

Upload Details

Usage Rights

2 Embeds 2,362

Statistics

To Evaluation and Beyond: The Evolving Role of the Embedded Librarian Document Transcript

http://www.scoop.it	2357
http://erroladamsjdmls.tumblr.com	5