Designing training for circulation student workers
DESIGNING TRAININGFOR THE CUNNINGHAMMEMORIAL LIBRARY’SCIRCULATIONDEPARTMENTPresented to the College of EducationDepartment of Curriculum,Instruction, and Media TechnologyIndiana State UniversityTerre Haute, Indianain Partial Fulfillment of the CourseRequirements for CIMT 620 Andrea Boehme Indiana State University 4/30/2012
Designing Instruction for the Cunningham Memorial Library iContentsContents _____________________________________________________________________ iList of Figures ________________________________________________________________ ivList of Tables ________________________________________________________________ vChapter 1 INTRODUCTION ____________________________________________________ 1 Purpose ___________________________________________________________________ 1Chapter 2 LEARNING CONTEXT _______________________________________________ 3 Instructional Environment ____________________________________________________ 3 Characteristics of the Organization ____________________________________________ 12 Characteristics of the Trainers ________________________________________________ 12 Curriculum Materials _______________________________________________________ 14 Comparison of Options _____________________________________________________ 14 Summation _______________________________________________________________ 15Chapter 3 LEARNER ANALYSIS _______________________________________________ 16 Target Audience ___________________________________________________________ 16 Discussion _______________________________________________________________ 17 Implications for Design _____________________________________________________ 19Chapter 4 FOCUS GROUP ____________________________________________________ 20 Introduction ______________________________________________________________ 20 Agenda for Transportation ___________________________________________________ 20 Questions Asked During the Interview _________________________________________ 20 Clothing for Interview ______________________________________________________ 21 Interviewees ______________________________________________________________ 21Chapter 5 NEED ANALYSIS __________________________________________________ 22 Process __________________________________________________________________ 22 Phase I: Planning __________________________________________________________ 22 Phase II: Collecting Data ____________________________________________________ 24 Phase III: Analyzing the Data ________________________________________________ 24 Phase IV: Final Report ______________________________________________________ 28 Summation _______________________________________________________________ 29Chapter 6 TASK ANALYSIS ___________________________________________________ 30 Introduction ______________________________________________________________ 30 Method __________________________________________________________________ 30
Designing Instruction for the Cunningham Memorial Library iii Assessment Tool for Applicants_______________________________________________ 54Appendix 2 Learner Survey ____________________________________________________ 58Appendix 3 Pretest Rubric _____________________________________________________ 59Appendix 4 Surveys __________________________________________________________ 59 Subject Matter Expert _______________________________________________________ 59 Staff ____________________________________________________________________ 59Appendix 5 Task Analysis Flow Chart ____________________________________________ 61Appendix 6 Declarative Game __________________________________________________ 63 Shelve that material! For the timekeeper _______________________________________ 63 Creating the “shelves” ______________________________________________________ 63 Creating the shelving items __________________________________________________ 64 Creating the play area _______________________________________________________ 64 Shelve that Material! For the players ___________________________________________ 66 Score Card _______________________________________________________________ 68Appendix 7 Creating a New Resident Card ________________________________________ 69Appendix 8 Evaluation Pretest __________________________________________________ 71 Overview ________________________________________________________________ 71 Objectives ________________________________________________________________ 71 Instructions _______________________________________________________________ 71Appendix 9 Evaluation PowerPoint ______________________________________________ 78Appendix 10 Survey Backend with Results ________________________________________ 79References __________________________________________________________________ 80
Designing Instruction for the Cunningham Memorial Library ivList of FiguresFigure 2:1 Front of Room 028 _______________________________________________________________ 5Figure 2:2 Back of Room 028 ________________________________________________________________ 5Figure 2:3 Side View 028 ___________________________________________________________________ 5Figure 2:4 Computer Setup 028 ______________________________________________________________ 5Figure 2:5 Projector 028 ____________________________________________________________________ 6Figure 2:6 028 Layout ______________________________________________________________________ 7Figure 2:7 Instruction Lab __________________________________________________________________ 9Figure 2:8 Instruction Lab Layout ___________________________________________________________ 9Figure 2:9 Student Cube Layout ____________________________________________________________ 12Figure 5:1 Phases of Analysis ______________________________________________________________ 22Figure 5:2 Departmental Structure _________________________________________________________ 23Figure 5:3 Task Performance Graph ________________________________________________________ 25Figure 5:4 Perceived Ability Graph__________________________________________________________ 26Figure 5:5 Desired Training Graph _________________________________________________________ 26Figure 5:6 Method Graph __________________________________________________________________ 28Figure 5:7 Training Needed Graph _________________________________________________________ 28Figure 11:1 Motivation ____________________________________________________________________ 50Figure 11:2 Appropriateness _______________________________________________________________ 50Figure 11:3 Remember _____________________________________________________________________ 50
Designing Instruction for the Cunningham Memorial Library vList of TablesTable 2:1 Room 028 Features ________________________________________________________________ 3Table 2:2 Instructional Lab Features _________________________________________________________ 8Table 2:3 Individual Training Features _____________________________________________________ 10Table 3:1 Learner Analysis _________________________________________________________________ 17Table 9:1 Procedural Instruction ___________________________________________________________ 40Table 10:1 Cognitive Instruction ____________________________________________________________ 44
Designing Instruction for the Cunningham Memorial Library 1Chapter 1 INTRODUCTION Interacting with almost all departments and patrons on a regular basis, the circulationdepartment at Cunningham Memorial Library (CML) is one of the most visible on campus.From the time library opens until it closes, circulation is active in the library. It is vital,therefore, that staff is well trained. CML’s circulation department employs twenty-nine persons.Nineteen of these are students. Students are consistently hired and trained to make up for the older students whograduate, or leave the department. This year six new students were hired who all needed to learntheir job quickly. An orientation/training meeting informs students of their duties and outlinewhat is expected of them. This needs analysis is an evaluation specifically of the orientation, andgenerally the whole training program in the circulation department of CML.Purpose The training was created to meet a felt need (Morrison, Ross, & Kemp 2001, p 29) of thecirculation staff (in this case and all following “staff” refers to those employees who work fulltime in the circulation department. “Students” refer to those employees who are students ofIndiana State University and work part time in the department). The staff was unhappy with theperformance of the older students and knew the new students would need a firm foundation tooffset the inconsistent training they were currently receiving. Concern with student performance is not new to CML. In 2003, the head of circulationAlberta Comer (now the dean of the library) published a study on improving student jobperformance. The study noted that a lack of student motivation and poor communication was thecause of low performance. The study recommended the creation of a handbook, a recognition
Designing Instruction for the Cunningham Memorial Library 2system for outstanding students, formalizing peer training, utilizing email for communication,and expanding training methods as methods to combat the problems. The majority of these havebeen implemented to varying degrees of successfulness.
Designing Instruction for the Cunningham Memorial Library 3Chapter 2 LEARNING CONTEXTInstructional Environment Paula, the student supervisor, and the instructional designer (ID) gave the instruction thesecond week of the semester. The students were least busy academically at this time, and thenew students were hired. The instruction took place in a room on the Lower Level of the library,room 028 (see figures 2:1-2:6). This room is commonly used for staff meetings, andpresentations. The room is large with seating for around 100 people. Chairs were lined upacross the width of the room, in five rows of fifteen chairs, with two tables, seating seven people,behind that. The majority of students choose to sit in the rows of chairs towards the back of theroom. Projection of the presentation was done from a ceiling mounted projector in the middle ofthe room onto a standard screen. A computer terminal, located to the right of the screen, servedas the center of control and also as a podium. A laptop was used to run the presentation as theavailable computer did not have access to Millennium (the integrated library system which isused to check out books) which was needed for instruction. The following chart describes the environmental factors in the instructional context, andis taken from Morrison, Ross and Kemp (2011).Table 2:1 Room 028 Features Room 028 - original instruction environment. Lighting Lighting controls are accessed by two separate switches. One in the back controls the back lights. Another in the front controls the front. The lights in the room were turned off, to better view the presentation.
Designing Instruction for the Cunningham Memorial Library 4 However light filtered in from the outside through opaque glass and from the presentation.Noise The Lower Level is a “loud” floor, meaning patrons are allowed to speak normally. The doors of the meeting room were closed to muffle outside sound. However, there were no major distractions from noise during the presentation.Temperature Temperature is controlled offsite and there are two settings: on and off. That day was warm outside and it was hot in the room. The students did not complain, but it was uncomfortable.Seating There were enough seats that students could choose a vantage point that worked well for them. However, as there were only two tables most students would have had to take notes in their lap. We could have had tables brought in but decided to forgo them as we didn’t foresee much writing going on.Accommodations The student supervisor, Paula, brought food and drink. The supervisor was late, so students had to wait until the end of the presentation to eat.Equipment Projector and screen, laptop, visual aids (resident card, ILL reserve place card, ect.)Transportation Students regularly provide transportation to and from work, so it was not an issue. Unfortunately, some students had class. They were given a copy of the presentation, handouts, and instructor notes with encouragement to ask questions. Paula, however, was late as there was thick fog and she had to pick up the refreshments.
Designing Instruction for the Cunningham Memorial Library 5Figure 2:1 Front of Room 028Figure 2:2 Back of Room 028Figure 2:3 Side View 028Figure 2:4 Computer Setup 028
Designing Instruction for the Cunningham Memorial Library 6Figure 2:5 Projector 028
Designing Instruction for the Cunningham Memorial Library 7Figure 2:6 028 Layout Alternative option one Another option for instruction is the Instruction Lab (see Figures 2:7 and 2:8). Locatedon the second floor, this room is used by the Reference department to do their instruction. Thisroom has more technological options. It has the standard computer cart, to display instructionalmaterial on two screens and a SMARTboard, from three ceiling mounted projectors. Fiftycomputers are available for use and take up the majority of table space. This is a wide room withseating arranged across the length of the room. In the event that we added more technical aspects to the training we may need to use thisroom. It does have the Millennium software on the computers already which is nice. We did notchoose this room because we were worried that students would get distracted by the computers.
Designing Instruction for the Cunningham Memorial Library 8It felt a waste to have the instruction in a room with computers when we weren’t going to usethem. Food and drink are not allowed in this room, so it was less than ideal for instruction.Table 2:2 Instructional Lab Features Instructional Lab Lighting Lighting is controllable using a bank of light switches. As the room is sectioned off from the stacks by glass partitions, light comes in through the glass. This room is still light when the lights are turned off for this reason. If the computers and projectors are turned on that also adds to the brightness. Noise The second floor is also a “loud” floor that receives the same amount of activity as the Lower Level. The doors can be closed and the students have their backs to the glass aiding in concentration. Temperature The Instruction Lab is subject to the same heating and cooling as the rest of the library, and is not adjustable. The addition of computers would increase the temperature in the room. However, there are fans available to keep the discomfort minimal. Seating Each computer terminal is a seating area for students which makes for a roomier seating arrangement, but does not leave much space for traditional note taking. To aid in viewing of material, it can be projected onto three different screens so one does not have to crane their neck across the room. Accommodations There is no eating or drinking in the lab. Equipment instructor computer, individual students computers, SMARTboard, 3 projectors, 2 projection screens, fans, Millennium, polling clickers
Designing Instruction for the Cunningham Memorial Library 9 Transportation This site has the same transportation concerns as the original room on the Lower Level.Figure 2:7 Instruction LabFigure 2:8 Instruction Lab Layout
Designing Instruction for the Cunningham Memorial Library 10 Alternative option two Another option is giving instruction on a singular basis in the environment where workerswill do the job. This is the current method used in the library through peer training. Anexperienced student is paired with a new one for approximately a week (this varies on thestudent’s abilities and how urgently a replacement is needed). The more experienced student isgiven their regular schedule, and the new one follows behind learning as they go. Along withthis the new student could use part of their time to review instructional materials alone in thestudent cubicle located in circulation (see figure 2:9).Table 2:3 Individual Training Features Individual Training Lighting Lighting throughout the library and in the student cubicle is controlled by switches in the circulation area. The lights are always on when the library is open. Noise The basement and 3rd floor of the library are “quiet” floors. Patrons and staff are required to speak at a whisper, or locate to another floor. Peer training in these areas will have to be at a whisper. The student cubicle is located in the middle of the circulation area. During the day, there is quite a bit of noise from staff, machinery, and patrons. At night it is quieter as there are less staff and machines creating noise. Temperature Temperature concerns are the same as listed in above sections. Seating During peer training students sit two places: at the circulation
Designing Instruction for the Cunningham Memorial Library 11 desk, or in the student cube during tasks. At the circulation desk there are two chairs one for staff and another for students. If the staff decides to stay at the desk one of the students will have to stand for 1-2 hours. The student cube is cramped with room for 3-4 people maximum and two work stations. For independent training the student would have to share the student cube with other students on the job. This area is cramped and during the day they may have to find a corner of the library so work can get done. At night they should be able to have the cube to themselves.Accommodations In either situation, students are allowed to eat and drink but breaks are to be taken away from the circulation department.Equipment In both situations, students can be exposed to all the equipment they would use on the job. Computer, laptop, scanners, sensitizers, carts, disk cleaners, receipt printers, VCR, Millennium, walkie talkies,Transportation The same transportation issues are present as in the other areas.
Designing Instruction for the Cunningham Memorial Library 12Figure 2:9 Student Cube LayoutCharacteristics of the Organization As this curriculum was created using the suggestions of staff, the characteristics of theorganization are important. The general consensus was that students needed to cut down onerrors and learn to work independently so that staff could focus on doing their job. Also the circulation department is a laidback, friendly group. Students feel free toexpress their opinion on most subjects, and have no trouble joking around with staff. Seriousdiscussion is not hard to have, as long as you give time for jocularity.Characteristics of the Trainers Instruction was given by the ID and by Paula; the head of circulation, Susan was there aswell.
Designing Instruction for the Cunningham Memorial Library 13 Content experience Susan - has worked in circulation for almost two years, before that she was in theReference department. She doesn’t know every minutia of what the students do but she cananswer questions about overall policy, and was able to clarify why we are/are not allowed to dothings certain ways. As a fellow Department of Education student, Susan has a positive view oninstruction, and values training. Focusing on running the administration side of the department,Susan oversees the big picture. Paula – has been working in circulation as the student supervisor for three years.Students report directly to her in all of their activities. She knows everything that there is toknow about how students are supposed to do their job, what they know and don’t know, and whoto go to if she doesn’t have the answer. Paula has been responsible for the training of students.Due to a busy schedule, the ID took over for this session. Her focus is on procedure. Are thebooks shelved right, are students here when they are supposed to be, and so on. She worksmostly on the behind-the-scenes aspects of circulation Andrea (ID)– has been working in circulation for nine months. As the night supervisor,she can tell them how to do their tasks and can hold them responsible for not following policy(privacy, professionalism). Andrea defers to Paula in all other situations. Technology experience The ID, is knowledgeable about all technology used in the creation of the instructionalmaterials. Paula is not very technically oriented. She is unfamiliar with PowerPoint and knowsthe basics of how to use the computer. This lack of experience limits the technology availablefor use in creating and administering instruction.
Designing Instruction for the Cunningham Memorial Library 14 Instructor training If Paula is to take over training completely, she will have to know how to use PowerPointand how to operate the computer set up so she can display the presentation. Any newtechnologies that are added will have to be learned by all instructors.Curriculum Materials There was no official curricula to fit the newly designed into. The instructional materialsaddressed concerns felt in the circulation department. The previous approach, peer instruction,was not altered. That is still the main way students learn the majority of tasks. Additionalinstructions should be able to function within that system to define standards and to provide away for students to seek out answers themselves after training is done.Comparison of Options The room that is the most obvious for training is the Instruction Lab. It is the most hightech, and is set up specifically to give instruction. It has the Millennium program for use indemonstration, and has the best set up to view information as a group. They also havetechnology that can be used to enhance learning. Polling clickers and the SMARTboard could beused to advance the training and to make for a more dynamic session. The concern is thecomputers. They take up a lot of desk space, and may lead to distraction if the students havethem on. Also as there is no food allowed, refreshments would have to be served in anotherlocation in the library. The second most obvious choice is what we actually used. It is less formal and can holdthe most people. It has good technology options that were suitable for the instructional needs atthe time. Extra technology had to be brought in to show how to use Millennium, but it was
Designing Instruction for the Cunningham Memorial Library 15easily obtainable. There was no room for writing but students didn’t seem to mind. Overall, thisroom meets the needs of instruction. This also fits the informal culture of circulation. The more unconventional option that would be used in a peer training situation is trickybut offers the best access to the tools used on the job. Instruction would have to take place on anindividual basis which would form bonds between workers. However, seating would be aproblem especially during the day.Summation The environment used for training was perfect for our needs during the presentation: foodcould be served; everyone could sit comfortably (in regards to space, not to temperature) andparticipate. If changes were made in the curriculum the space needs may need to bereconsidered. Changes in technology may also be a problem down the line as well as the studentsupervisor is not familiar with most technology.
Designing Instruction for the Cunningham Memorial Library 16Chapter 3 LEARNER ANALYSISTarget Audience The primary audience for instruction is the nineteen student workers in the circulationdepartment. Instruction was created to further their understanding of what is expected of themwhen they work in circulation, as well as a clarification of tasks. The secondary audience isfuture circulation students and part-time staff who might undergo the training. In order to identify the learning characteristics of the primary audience, students weregiven a pretest (Appendix 1) and a survey (Appendix 2). These methods were originallyintended to evaluate the student curriculum; however they have proven useful in identifying priorlearning. All other information was obtained through personal knowledge of the studentsobtained by interacting with them on a daily basis. Using Smith and Ragan’s (2005) four categories of learner characteristics students can bedescribed as follows (see Table 3.1). The focus in on the primary audience; traits of thesecondary group are mentioned only if they differ from the primary.
Designing Instruction for the Cunningham Memorial Library 17Table 3:1 Learner Analysis Similarities Differences Primary Primary Sensory Capacities: Students Aptitudes: Unknown possess no hindrances to sight or Cognitive Styles: Unknown sound comprehension Psychosocial Traits: Information Processing: At least two Anxiety – no outliers students have ADHD related Locus of control – external and problems and as such have trouble internal are balanced in the groupStable concentrating for extended periods academic self conduct - generally of time positive Types and Conditions of learning: Demographics: Primarily Caucasian, peer training, independent learning 50/50 split on gender, 18-24 years Secondary old, generally middle class Information Processing: no outliers Secondary Demographics: (part time staff) Caucasian, Female, 26-50 years old concentrated to over 35. Primary Primary Developmental process: no outliers Developmental State: Students are a Intellectual development: no outliers mix of Erikson’s (1968) Identity vs. Language Development: no outliers Identity Confusion, and Intimacy vs. Psychosocial/Personality: students Isolation are highly social Prior Learning: Generally Secondary (knowledge of the world) studentsChanging Psychosocial: staff are from are at the same level of learning. different departments and have Specifically (knowledge of the job) various preferences to social only 5 of the 20 students are interaction “experts” at the job. Secondary Developmental State: Staff are in Erikson’s Generatively vs. Stagnation Prior: Staff are considered intermediate at job skillsDiscussion The pretest was given to eleven of the nineteen students. Originally, it was supposed tobe taken by all of the students, but due to a communications error it was given late. The resultswere separated by experience level. Six participants had over two years of experience, two had ayear or less experience, and three had only a few days of experience. The tests were graded and
Designing Instruction for the Cunningham Memorial Library 18each given a rating of expert, intermediate, or novice level. Expert level students made a fewerrors generally related to not examining the questions carefully enough; they were explicit intheir explanations and descriptions. Intermediate students made a few errors related toignorance, they gave short answers that did not fully detail their process. Novice students madeconsiderable errors due to ignorance, and did not give enough detail in their explanations. Pleasesee the rubric (Appendix 3) for more detail. The students with over two years of experiencegenerally preformed at an expert level, with only one falling to intermediate. The second grouppreformed at an intermediate level, with one bordering on novice. The third group, expectedly,preformed at a novice level. Originally, the majority of data on the students was to come from a focus group. Aftertwo separate calls for participants, the suggested group size of five people could not be achieved.At first, it was suspected this was because of scheduling problems. So a request was made forwilling participants to send times they were available and then a common opening would be usedto schedule the group. This method was also unsuccessful. Currently, data is being gatheredthrough an alternative method, a survey. The survey is still in the collection process, but isproving interesting. It was distributed through SurveyMonkey a week ahead of time. It consistsof eight questions pertaining to training and takes about two minutes to complete. Six responseshave been collected so far, however, one was answered all “no reply” and as such has beenremoved from the data. These responses are all from students with over 2 years of experience. This pattern of non-responsiveness by the students is troubling. Only a select group ofstudents are willing to participate in evaluation activities. This group is mostly female, and fromthe expert group of students. Data, obviously, will be skewed towards this group. It is unclearwhy students do not want to participate. They weren’t getting paid for the focus group, but pizza
Designing Instruction for the Cunningham Memorial Library 19and drinks were to be provided. Possibly the timing was not convenient, as students were askedto supply a time between 8am and 4pm. However, the response from a two minute online surveyonly accounts for a fourth of the students, and they were from that same two years or more ofexperience group. Informal information of importance about the students was gathered through conversationbetween Andrea, the night supervisor, and the students, staff, and student supervisor. Studentsare a highly social. The majority of new hires are recommended by students already employedby circulation and are often close friends or relatives. Students and staff also get along well.The more experienced students have babysat staff children and are friends with staff onFacebook. Newer students are introduced to staff through the students they know. There are afew staff members who students gravitate towards, but all students have at least a professionalcamaraderie with staff.Implications for Design A pattern is evident in the students: they are highly social, students with the mostexperience are more willing to participate in evaluation than those who have been working hereless, and students are performing at a level appropriate for their experience. In order to improve training an attitude learning approach (Smith and Ragan 2005, p 260-268) could be useful. The focus on changing an attitude could rectify the student’s lethargy inregards to willingness to participate in evaluation activities. If we can show them that their inputis valuable this could help. Also as a majority of tasks in circulation involve customer service,attitude learning will be helpful in creating good customer service skills. The instructionalconditions for attitude objectives put a focus on role-playing. This works well with the student’ssocial tendencies as it requires students to express themselves.
Designing Instruction for the Cunningham Memorial Library 20Chapter 4 FOCUS GROUPIntroduction In order to evaluate the circulation student training curriculum, the students, staff, andsubject matter expert (SME) of circulation were questioned. Originally, a focus group withstudents was the main method of data collection. Due to low response, the focus group had to becancelled. Instead an online survey was distributed through email. This survey was based offthe format from the staff survey. The SME was given a list of interview questions throughemail. This was arranged to fit both the interviewer and interviewee’s opposite schedules.Agenda for Transportation As the survey and interview were conducted over the Internet transportation was not afactor. The agenda for the interview was informal as well. The email, sent on February 6th,asked her to respond by Februrary10th.Questions Asked During the Interview Both types of questions, the survey and the interview, were structured. As neithermedium excels in allowing the participant to fully express their views, a structured format wasnecessary. The interview was intended to be a dialog between the SME and the researcher;however after the initial results no further questions were needed. These methods were used to discover the felt needs of the circulation department. Feltneeds are “a desire or want that an individual has to improve either his or her performance or thatof the target audience” (Morrison, Ross, & Kemp 2001, p 29). Another goal was to quantify the
Designing Instruction for the Cunningham Memorial Library 21subjects felt needs, that before had only been expressed in complaints at staff meetings.Surveying the student’s also served to get the other side of the story.Clothing for Interview In an experiment conducted in 1976 with eighty undergraduate students, it was shownthat dress had minimal effect on “perceived expertness” (Kerr & Dell). It was the conduct of theinterviewers that seemed to relay expertise. Hubble and Gelso (1978) also found that clients hadthe best reaction to counselors with a dress style that was a bit more formal than what the clientusually wears. So a person who usually dresses casually (casual shirt and jeans) responds best toa counselor who is business casual (nice slacks and a button down). These experiments show thatclothing does affect how interviewees perceive the interviewer. An important note is thatinterviewer behavior, and the preferences of the interviewee have the most impact on perception. More current discourse on attire is focused on a job interview. These articles recommend“professional” attire. However, in these situations the person is being questioned, not doing thequestioning. They are the one being asked to prove their worth, whereas in an interview, aimedat getting information, one wants to make the subject(s) feel at ease. The two types of intervieware not the same and should not be treated as such. Obviously, wearing pajamas to a focus groupis not appropriate, but the deciding factor on what to wear should come down to the audience.Since the interviews were given without face-to-face interaction dress was not an issue.However, any further interviews may be different. The majority of staff, students, and the SMEdress casually, therefore a business casual dress would be appropriate.Interviewees The only interview given was with the SME, Paula Huey.
Designing Instruction for the Cunningham Memorial Library 22Chapter 5 NEED ANALYSISProcess To conduct the needs analysis the following method from Morrison, Ross, and Kemp(2001) was utilized.Figure 5:1 Phases of Analysis Phase Phase II: Phase III: Phase IV: I:Planning Collecting data Data Analysis Final Report Target Sample Size Analysis Purpose audience Strategy Scheduling Prioritization Process Analysis Results Particpants ActionPhase I: Planning The target audience for the assessment is the students and the SME in the department.They will deal directly with the outcome of the assessment. The SME will have to implementand enforce any changes, and the students will have to adapt and perform under any newmethods. The secondary audience is the staff at large. Susan, the head of the circulationdepartment, will have final say on what will and will not happen, and can direct the departmentas a whole to take action in regards to training students. Other circulation staff will also have to
Designing Instruction for the Cunningham Memorial Library 23enforce changes and can report back the effectiveness. The following diagram shows thestructure of the department. One thing to note is Andrea has been separated and deemphasizedto show that, while a member of the regular staff, she conducted the study and was not a part ofthe data. Paula has been emphasized to show her importance in the outcome.Figure 5:2 Departmental Structure Susan Staff Paula Andrea Students In order to assess the training program at CML, the strategy was to conduct anassessment using the discrepancy model (Smith & Ragan 2005). In this model there is acurriculum in use and the model is used to find and address gaps in it. Gaps were found using apre/posttest over the meeting, and discussions with the staff and students. A survey of staff,interview with the SME, and focus group of students was the original plan. Due to a lack ofparticipants the focus group was changed to a survey. Analysis of the results happened as follows. The pre and posttest were compared usingthe rubric in Appendix 3. The tests were divided by the experience level of the students to showif any improvement occurred and at what levels. The surveys were compiled to comparestudents and staff, and then that data was compared to the interview with the SME.
Designing Instruction for the Cunningham Memorial Library 24Phase II: Collecting Data The evaluation of the orientation/meeting was a pre/posttest (see Appendix 1) . This testwas given to eleven students before the meeting and fifteen after the test. The test consisted ofthree short answer questions aimed at assessing customer service skills, fifty questions,consisting of multiple choice, matching, and a map labeling exercise, to assess their knowledgeof the department, and lastly forty questions on Library of Congress classification that is alsoused as an assessment during the hiring process. The other data collection method consisted of an interview with the subject-matter expert(SME) student supervisor, and a survey of staff and students. The interview with the SME wasconducted through email to accommodate both the interviewer’s, and interviewee’s oppositeschedules. The two surveys were also collected electronically using SurveyMonkey an onlinesurvey creation and distribution website. The staff was sent an email inviting them to take thesurvey; of the eight staff members, six responded. The students were sent a link to the website ina mass email. Of the twenty potential respondents, six responded. One of these surveys wasanswered all “No Response” and was excluded from the data, making five total. The five studentresponses were all from those who had worked in the department for two or more years. Thisskewed the results to the most experienced students in the department.Phase III: Analyzing the Data Pre/Post Examination of the pre/post data showed the students with over two years of experiencegenerally preformed at an expert level, with only one falling to intermediate. The second groupperformed at an intermediate level, with one bordering on novice. The third group, expectedly,performed at a novice level. After the orientation the score generally improved. The
Designing Instruction for the Cunningham Memorial Library 25experienced students stayed the same, those students working between a year and half and fivemonths improved with two people advancing to expert. The novice students improved greatly aswell. They had all improved to intermediate, with one achieving expert status. The rise in scores was a resulted from improved performance on the map labeling andmultiple choice questions. Students performed consistently on the short answer customer servicequestions before and after the meeting. Surveys A comparison of staff and student surveys shows that there is little agreement on what thestudents can do well and how to handle further training.Figure 5:3 Task Performance Graph 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 0 10-20 30-40 50 60-70 80-90 100 Staff Students
Designing Instruction for the Cunningham Memorial Library 26Figure 5:4 Perceived Ability Graph 6 5 4 3 2 Staff Students 1 0Figure 5:5 Desired Training Graph 7 6 5 4 3 Staff 2 Students 1 0 The most telling of the data is the Correct/Confident task performance chart and the dataon customer service. When asked to rate the students on “what percentage of tasks do studentperform correctly” staff say task correctness tops out at 90% and goes down to as little as 50%.
Designing Instruction for the Cunningham Memorial Library 27Whereas students have a more positive view of themselves: performing between 80 and 100%confidence. Granted this data is from students who have worked in the library the longest anddoes not reflect all student performances. Customer service is also a point of contention. Thestudents feel they have excellent customer service skills, and the SME agrees with them. Whenasked what students do well, customer service was the first thing the SME wrote (P. Huey,personal communication, February 10, 2012). Current training method When examining the effectiveness of current methods and what could be added staff wassplit between the same amount and more training was needed, however, students were generallyfine with the current offerings. This was also reflected in the available training methods withstudents tending towards what was currently in use (email, peer training, orientations). Studentsand staff did see the value in diversifying the training curriculum by adding a handbook andmanual. Staff wanted to also add periodic testing, but the students showed no interest.
Designing Instruction for the Cunningham Memorial Library 28Figure 5:6 Method Graph 7 6 5 4 3 2 1 0 handbook manual orientation peer emails testing lessons other Staff StudentFigure 5:7 Training Needed Graph Staff Students 4.5 4 3.5 Respondents 3 2.5 2 1.5 1 0.5 0 More Same LessPhase IV: Final Report The gaps in training can be seen in customer service. The survey shows that students andstaff disagree on what is proper customer service. A lack of change in the short answer on thepre/posttest also speaks to the problem. Future training will need to focus on that aspect. An
Designing Instruction for the Cunningham Memorial Library 29examination of comments on both the student and staff survey also show that peer training isinconsistent. Students are trained my multiple peers, and often do not get to experience tasksthat form the basis of tasks in the department. Another gap that was not actively studied butbecame a factor was motivation. As seen in the survey, students are fine with their currentperformance. There is also a lack of motivation in improving the department. Only a smallportion of students are willing to take time outside of work to discuss the problems. The need tocancel the focus group and the small number of respondents to the survey shows this. In order to fill these gaps the training process needs to be formalized in a series of lowtech solutions that happen as a part of the job. It was shown during an examination of thelearning environment that the SME, who will be ultimately responsible for implementing andadministering training, is not technologically adept and would need training on new methods.Also the on-the-job aspect of training is critical. Students show more willingness to contributewhen they are in the environment. Giving them materials (such as a handbook, or video) theycan utilize when they are dealing with a problem may be more effective than a reminder outsideof the work environment. This problem of motivation is an important, but was not the focus of this study. A deeperlook at this problem and possible solutions is needed to fully understand how it could beimproved. Another possible topic would be student/staff relations as a way to encouragecommunication.Summation Comer’s study of the students in 2003 still rings true today. The same problems arepresent. Hopefully, revisiting and revamping the solutions the study recommends will improvestudent performance.
Designing Instruction for the Cunningham Memorial Library 30Chapter 6 TASK ANALYSISIntroduction After conducting a Needs Assessment plan for the Cunningham Memorial Library’s(CML) circulation department, it was found that students needed training methods that they wereable to during their work hours. In order to facilitate those needs an analysis of tasks is needed.By examining what is involved an appropriate media, or method can be chosen. Students have amyriad of tasks that need to be done and examining their job as a whole is impractical. To thatend, this task analysis will be on a job that forms the backbone of student tasks: sorting andshelving materials.Method As Smith and Ragan (2005) report, the first part of analyzing a task is to create ordetermine a learning goal. This is a “statement of purpose... [of] what learners should be able todo at the conclusion of instruction” (Smith & Ragan 2005, p 77). It was discovered in the needsassessment that shelving was an area in which students can improve, creating the goal: Learnersneed to know the correct method for shelving the materials in the library. The second step is to determine the type of learning that task entails. For sorting/shelvingmaterial a declarative (Smith & Ragan 2005 pp 79-80) learning outcome is most appropriate.The student needs to memorize the indicators of location that are available and where to findthat information on the item. Some application of knowledge they receive is present. Forexample, a book that has “Ref” marked out, or incorrectly formatted call numbers can causeconfusion. However, there is nothing truly “intellectual” about recognizing that something iswrong and asking for clarification, which is the course of action taught by the student supervisor.
Designing Instruction for the Cunningham Memorial Library 31 The last part of creating a task analysis is the analysis itself. Sorting/Shelving is a taskthat begs for a procedural analysis. As explained by Morrison, Ross, and Kemp (2001, p 70) aprocedural analysis “is used to analyze tasks by identifying the steps required to complete [theprocess]. The [analysis] breaks tasks into the size of steps needed for learning.” To do this thetask of sorting and shelving will be described by a Subject Matter Expert, in this case thedesigner.Task Analysis Please see Appendix 5 for the flow chart for the task.Discussion In creating this chart a number of issues presented themselves. The first, that additionalsubject material needs to be incorporated into the design. An understanding of Dewey andLibrary of Congress classification models is required to shelve. Education on the systems neednot be very deep; however, a few key points need to be emphasized. For example, in bothsystems the dot in a number is not a period, but a decimal point. This means that the number .423comes before .43. The second issue is that this process does not happen as one continuous task. Sortinghappens at the circulation desk and again in preshelving, while shelving is yet another task.Workers have to know the whole process to understand why some parts of their job arenecessary. When they work at the desk, students are asked to put away materials in the sameorder they are on the shelf to speed up work flow further down the line. Hopefully, seeing thefull process can show them why this task is required of them. Also because this does not happenall at the same time it may make creating a lesson difficult. To that end, educators could sharethis model with the students when addressing work flow or use it to give the student an overview
Designing Instruction for the Cunningham Memorial Library 32of the task; then the students should participate in the sorting and the shelving task for the mainmode of education.
Designing Instruction for the Cunningham Memorial Library 33Chapter 7 LEARNING OBJECTIVESLearning Objectives Using Library of Congress classification, students will shelve stacks books correctly 90% of the time with no errors from misreading the call number before the Cutter. Using Dewey classification, students will shelve stacks books correctly 90% of the time with no errors in identifying subgroups (easy reader, biography, ect) Using alphabetical classification, students will make zero errors when shelving material.Analysis of Learning Objectives As stated in Morrison, Ross, and Kemp (2001), learning objectives fulfill three roles inthe learning process. They: offer a means for instructors to focus their instruction provide a method of evaluating student performance guide the learner so they know what to expect from instructionHaving well stated learning objectives is essential in shaping both the student and teacher’sexpectations of their level of performance and the goal of the coursework. The objectives listed above fulfill all of these roles. The objectives each have a specificfocus: teaching/learning about a specific classification system. Evaluation will occur in thestacks with specific criteria to assess how the learner performed. They also guide the learner bytelling them which classification systems they will be learning about.
Designing Instruction for the Cunningham Memorial Library 34Objective Types Objectives are created with one of a few domains in mind. Each of these domains helpsthe designer create objectives by identifying what type of learning will occur. Morrison, Ross,and Kemp (2001) list three domains: cognitive, psychomotor, and affective. The cognitivedomain as defined by Morrison, Ross, and Kemp (2001) deals with “information or knowledge,naming, solving, predicting and other intellectual aspects of learning” (p 86). An objective inthe cognitive domain addresses learning that challenges the learner to remember, comprehend,apply, analyze, synthesize, or evaluate material. These tasks were created by Bloom in 1956 inhis taxonomy of cognitive objectives (Morrison, Ross, & Kemp 2001). Cognitive domain is thefocus of the objectives above. Students are asked to comprehend, apply, and analyze, each of theshelving methods. For Smith and Ragan, the objectives are procedural ones. Psychomotor is the next domain. This domain is focused on physically completing atask. Tasks relating to using proper technique or manipulation of the body fall under this domain.While no objectives were created for this domain for this exercise, an example for this coursecould be how to properly lift heavy objects to reduce the risk of injury. The last domain is the affective domain. Tasks with the goal of changing or creatingattitudes are included in this domain (Morrison, Ross, and Kemp 2001, p 89). An example ofthis domain for a library student worker curriculum would be coursework on creating goodcustomer service skills. Instilling a willingness to be helpful and polite would require curriculumthat fosters that willingness.Learning Outcomes Smith and Ragan (2005) break down these domains into eight different types ofoutcomes: declarative, concepts, principles, procedure, domain-specific problem solving,
Designing Instruction for the Cunningham Memorial Library 35cognitive strategies, attitudes, and psychomotor skills. Attitudes and psychomotor skills aresimilar to their objective counterpoints. Declarative knowledge challenges the student to “knowthat” something (Smith & Ragan 2005, p 79). In this lesson students have to recall where a bookgoes in the library. Concept learning has students assess if an object is part of a particularconcept. In the library example student would be handed a material type and would have toplace it into one of the groups. Then in procedures the student would have to recall a procedureand then go through the steps. This would be the act of shelving in the library. Principles“describe the relationship between two or more concepts” (Smith & Ragan 2005, p 205). Thereare few principles in the student workers jobs. An example outside the library would bediminishing returns in economics. Domain-specific problem solving is exactly how it sounds.Learners have to solve a problem using the tools of that domain. For instance, where do I place areceiver dish on this tower to get the best signal? Lastly is cognitive strategies which teachlearners how to learn. A good example of this would be in teaching students to createmnemonics to remember a series of steps or suggestions. Knowledge of these domains and objectives types is important to creating objectives thathelp both learner and educator proceed with instruction. Knowing the domain helps designercreate material that enhances learning. For instance, objectives focused on the psychomotordomain of customer service will teach how to present one’s self. Does the staff member sit upstraight and make eye contact would be a way to evaluate. However, an affective objectivewould evaluate how often the student displays a positive attitude. Objectives should focus bothparties on what to expect from instruction and how they will be evaluated on what is learned.
Designing Instruction for the Cunningham Memorial Library 36Chapter 8 INSTRUCTIONAL STRATEGY: DECLARATIVE “A well-designed instructional strategy prompts or motivates the learner to actively make [connections] between what the learners already knows and the new information.” Morrison, Ross, & Kemp 2001, p 124 The Cunningham Memorial Library at Indiana State University employs twenty studentsin its circulation department. One of the keystone tasks is shelving. Students must be aware ofwhere books are in the library and how they are shelved for eight of eleven common tasks theyperform. The library has seven material types, two of which have multiple subtypes, and usesthree systems for shelving items. With such diversity in material and shelving systems, shelvingcan get complicated. To aid in creating connections, a game was created. This game is based ondeclarative learning strategies.What is Declarative Knowledge? Smith and Ragan (2005) define declarative knowledge as “[involving] ‘knowing that’something is the case. In is often what we mean when we say we want the learners to‘understand’ a content” (p 152). Within this knowledge type are three distinct subsections:labels and names, facts and lists, and organized discourse. In the subtype labels and namesstudents create connections between information. Facts and lists have learners memorize data aslong sets of related information, or as knowledge sets. Lastly, students are challenged tocomprehend large sections of text in organized discourse. The main goal of this game is to learn facts and lists. In this case the facts and lists are:what kind of materials does the library have, where are those materials shelved, and how do weshelve them.
Designing Instruction for the Cunningham Memorial Library 37Cognitive Process Declarative knowledge is learned through the cognitive process of “propositionalnetworks” (Smith & Ragan 2005, p 153). Prepositions are ideas and networks are the linkedideas. Therefore, declarative knowledge is created by imparting ideas on students so that theymay link those ideas to create a greater understanding of them.Linking with Existing Knowledge The game utilizes a few methods to link old and new knowledge. The first of which ismetaphoric devices (Smith and Ragan 2005, p 159). The game has students run books to theirappropriate shelves and floor. The floors are represented by book carts placed in areas markedout by floor (see game rules for a fuller description and map). The books the students areshelving are dummies that use the same labeling that the library currently employs to furtherconnect the dummy to the real item. This method creates an associational technique where in agreen label means this area of browsing.Organization/Elaboration To help the students organize the information in their minds, they will be given a pretestthat asks them as groups to name all the different types of materials the library has. This will becontinued at the end of the day when students will organize the material types into floors as aclass. These tasks of organization and elaboration are important to linking information instudent’s mindsConditions Supporting Learning The organization of instruction follows the suggestions of Smith and Ragan (2005). Atthe beginning of instruction session in order to gain attention and focus the students they will be
Designing Instruction for the Cunningham Memorial Library 38broken down into groups and asked to create a list of all the materials we have in the library.This is also a way to preview instruction and communicate instructional purpose. By showingstudent’s the many varieties of materials it will instill in them the complexity of their job. Thenthe main instruction will take place.Practice The last part of the puzzle is practicing the information to evaluate the student’s learningstudents. This will be done through participation in a game at the end of the session. Thisgame’s focuses on using declarative techniques and instructional organization to help circulationstudents learn how to shelve properly. Metaphor, association, and practice are all employed tohelp give the students mental tie-ins to the lesson.Evaluation Smith and Ragan (2005) recommend an evaluation based on recall to assess thestudent’s learning. The game’s scorecard will provide the medium to test the recall byasking students to put the book in the correct area. Points are subtracted when the playerplaces a book on the wrong “shelf”.
Designing Instruction for the Cunningham Memorial Library 39Chapter 9 INSTRUCTIONAL STRATEGY: PROCEDURALIntroduction Another area where students need assistance is in the creation of resident cards forpatrons. Smith and Ragan (2005) would classify this process as a simple procedure. The task isa step-by-step process where there is little ambiguity on how to perform each step. Additionally,this is a simple procedure, as there are no decision points where the procedure would branch intoseparate trees. Each time you complete this task it will happen the same way. In order to teach students how to create a resident card a tutorial will be created. Studentworkers will have access to this material during their training, but it will also be available forthem at the desk. The tutorial will be created in Microsoft Word with screencaps and detailedinstructions following the suggestions of Smith and Ragan described in the following sections(See Appendix 7 for tutorial)Simplified Procedure 1. Patron needs a card 2. Prepare materials a. Turn on laminator b. Get card c. Get Barcode d. Get new patron form 3. Have patron fill out all parts of form a. Confirm information against his/her driver’s license 4. Click “new patron” button a. Fill out all applicable fields b. Double check for correctness 5. Create card a. Patron signs card b. Date card with today’s date c. Attach barcode d. Laminate
Designing Instruction for the Cunningham Memorial Library 40 6. Complete a. Check out materials as normalInstructional Strategy/Procedural Considerations In their chapter on procedural strategies Smith and Ragan (2005) have many suggestionsfor how to create this type of lesson. They start by describing how to write the procedure.(Smith and Ragan describe strategies for both complex and simple procedures; this paper willonly focus on simple). use clear sentences each operation should represent and single...action operation steps should be stated as imperative sentences (beginning with a verb) With that written the next step is to present the material. For the sake of efficiency, Smithand Ragan (2005) recommend that the procedure should be told to the students rather thanrelying on a discovery method. Therefore, the instructor should first explain a step then letstudents practice it before proceeding to the next. With these recommendations in mind, thefollowing lesson was created.Table 9:1 Procedural InstructionIntroductionDeploy attentionEstablish Today you will learn how to create a resident card. Doing this correctlyInstructional purpose will save the patron, the billing clerk, and you the headache of having to ask again for information.Arouse interest and Most likely you will have to complete this task many times while youMotivation work here.Preview Lesson There are three major steps in this procedure: entering in the patron’s information, creating the card, and putting the completed form away.BodyRecall Relevant Students will have to access Millennium, and get the appropriate formPrior Knowledge from the cart.Process Information learning to determine if the procedure is required: Resident cards can beand Examples created for any resident of the state of Indiana and the Illinois Counties that surround Terre Haute. They must have a valid
Designing Instruction for the Cunningham Memorial Library 41 Indiana/Illinois driver’s license and be willing to give us their Social Security number. Members of Rose Hulman and Saint Mary of the Woods can use their cards at ISU and should not be issued a card. To avoid duplicate accounts ALWAYS perform a patron search before issuing a card. learning to complete the steps in the procedure: With computer screen projected onto viewing screen, go through handout (appendix 7) showing each step. For creating the card show students all the materials they need to properly laminate a card learning to check the appropriateness of a completed procedure: show students the confirmation screen and have them compare the form and the confirmation learning to list the steps in the procedure: As a group, start a second dummy account and have them walk instructor through steps.Focus Attention During the lesson students will be focused with use of the mouse on the screen to highlight areas that they need to focus on.Employ Learning Students will be told where the handout can be found if they forget a stepStrategiesPractice Each student will be asked to create a resident account according to a completed sample form. They must turn in the form to the instructor when they are complete.Evaluate Feedback Depending on the size of the class, and time left in the session, 2-5 accounts will be picked at random and the class will review other submissions for completeness.ConclusionSummarize and Based on feedback go over the problem areas and ask if there are anyReview additional questions.Remotivate and Thank you all for participating. I know that we will all save a lot of timeClose now that you all understand so wellAssessment of Procedural LearningAssess Learning To assess the students long term, the billing clerk will keep track of information that is commonly entered incorrectly when connecting the patron accounts to billing accounts. The clerk will relay this information to the instructor so that specific areas can be remediated.Formative Analysis Morrison, Ross, and Kemp (2001) suggest performing analysis of materials early in thecreation process to avoid costly redesign later. To test the material, the procedure was given tonew members of circulation. They were asked to read the procedure (see Appendix 7) and create
Designing Instruction for the Cunningham Memorial Library 42a record from start to finish. The students were able to follow along with the sheet very well.However, it was discovered that the student version of Millennium was not asking the students tofill out certain required information. While not an issue for training, the problem needed to beaddressed in Millennium.
Designing Instruction for the Cunningham Memorial Library 43Chapter 10 INSTRUCTIONAL STRATEGY: COGNITIVEIntroduction An important part of sorting books is being able to identify the markers of a particularmaterial type. A common problem is students giving patrons things they have no use for. Theywill give patrons dummy cases, place markers, and informational markers. Students are literallytrying to check out pieces of wood. In order to correct this, students need to learn how toexamine the material for shelving clues. To do this, cognitive strategies need to be employed. Smith and Ragan (2005 p 244)define cognitive strategies as “techniques that learners use to control and monitor their owncognitive processes”. They allow students to “organize, elaborate, manipulate, and retrieveknowledge” or “discover, invent, or create” (Smith & Ragan 2005 p 244). In this task we focuson retrieving knowledge.Strategy Selection The strategy used to teach this task is the cognitive domain strategy of organizing.Students will be asked to graphically organize material. This strategy was selected because itallows students to collate identifiers.Application Through guided instruction, students will be asked to create word webs to describematerial. They will be given an example material type, such as a DVD case, and will be asked todescribe what they know about it, and how they found out that information. The webs willprovide enough spaces for each identifier so that each space must be filled correctly to earn fullcredit.
Designing Instruction for the Cunningham Memorial Library 44Formative Evaluation Formative evaluation begins during the construction of a instructional unit (Morrison,Ross, & Kemp 2001). During the formative evaluation, Gooler suggests asking eight questionsto help instructional designers (ID) assess instruction (Morrison, Ross, & Kemp 2001 p 267).Using a test subject who knew nothing about sorting, evaluation of the webs was done. During the lesson the subject commented that the webs were repetitive as many materialtypes used similar identifiers, with only a few materials having any noticeable differences.Resulting webs ended up clustered on one section with little “web-y-ness” in that there were onlyone or two spokes. The subject suggested switching the focus from the material type to theidentifiers.To accommodate the suggestions the ID created a list of common elements. The elements werearranged in order from most obvious to least obvious. Physical Item Call number Color StickersThese elements were used to form a mnemonic: Pretty Cool Circulation Students. This will helpstudents could use to remember the identifiers.ApplicationTable 10:1 Cognitive InstructionIntroduction Deploy Attention To begin, students will be asked to discuss how they know where to shelve material. Arouse Interest and Tell learners how miss-shelving effects work flow motivation and cause additional frustrations. Establish instructional In order to correct this we will be looking at how to purpose look for clues on the material itself. Preview lesson Explain the PCCS mnemonic, and have students recite it.Body
Designing Instruction for the Cunningham Memorial Library 45 Recall prior knowledge Classification systems Process Information Using PowerPoint and examples have students describe how to Focus Attention Each of the examples will only have the element being discussed. Students won’t be distracted by other elements. Employ learning Before switching slides I would ask the students for strategies the next part of the mnemonic and what element it relates to. Practice After going through the PowerPoint, students will break into small groups and receive example and asked to describe what they know about the material using the PCCS model Evaluate Feedback Have each group describe the material and how they found that out in front of the large group.Conclusion Summarize and review Have students recite the mnemonic. Transfer knowledge Remotivate and close Thanks so much for your hard work. This will help you all identify materials with greater accuracy.Assessment Assess performance Observation of the group’s examples for nonuse of the mnemonic or incorrect application of an identifier Feedback and Correct lesson for common errors found in the remediation student’s explanationsSummative Evaluation To evaluate this strategy further, student worker errors will be analyzed. Each daystudents record any errors they find while shelf reading. This information will be analyzed forthe average number of errors found in each section. Data from the semester before, and thesemester after the instruction will be compared. The average will be used to account for patronsputting material back in the incorrect place. Another method will be of patron/staff reports ofstudents giving out incorrect materials. Lastly, during instruction, the instructor will look forwhere students are struggling.
Designing Instruction for the Cunningham Memorial Library 46Chapter 11 EVALUATIONIntroduction Shelving materials forms the backbone of student worker’s tasks. Knowing the basics ofshelving is a skill all student workers must master. To teach this skill, a lesson on the cognitivetask of identifying materials was created. In it, students were required to create word webs thatdescribed individual material types. The following is an assessment of the lesson, andsuggestions for how to improve it.Student Assessment Instructional materials The lesson consisted of a PowerPoint presentation (Appendix 9), and group work to teachstudents a mnemonic that would teach them to look closely at material for shelving clues. Withthe PowerPoint as the guide, students learned the mnemonic and how to apply it. After eachsection students looked at real examples of the principle and were encouraged to discuss otherexamples of that identifier. At the end of the presentation, students were divided into groups,given a number of examples from a certain material type, and asked to assign them to theappropriate area of the library. They were given ten minutes to work through the example andthen present their findings. Formative assessment Originally the concept was taught using word webs. The student was given an item typeand asked to describe it. In order to assess this instruction, it was administered to a test student.The subject had no shelving experience in either the Cunningham Memorial Library or any
Designing Instruction for the Cunningham Memorial Library 47other libraries. It was found that this method did not fully utilize the word web’s potential.Webs created did not have many branches and not all identifiers were equally represented oneach item. Additionally, there are so many item types that going through them individually tooktoo long. Instead, the focus was shifted off the item and onto the identifiers, and the abovelesson was created. Summative assessments In order to assess the students and the coursework a number of assessments were put intoplace. The first was a pre and post test. Students were given a packet of examples (Appendix 8)and asked to circle or otherwise marked identifiers. After instruction they were given the sameexamples and asked to mark the identifiers again. This method had a twofold purpose.Morrison, Ross, and Kemp (2001) note the following advantages. The first is “to assess thelearner’s preparation to study the course of topic” and “to determine which competencies for thecourse or topic the learner may have already mastered” (Morrison, Ross, and Kemp 2001 p 220).Secondly, pretests “measure the degree of improvement after instruction is completed”(Morrison, Ross, and Kemp 2001 p 220). While these tests will not be “graded” the pretest (orlack thereof) will be used to address where attention should be focused in the lesson. The postwill show where attention needs to be focused in subsequent classes. In order to assess the students during the lesson a problem solving approach was used.Smith and Ragan (2005) suggest that assessment of concept learning should involve explaining,categorizing, or producing (non)examples of a group (p 180). This was combined with Morrison,Ross, and Kemp’s (2001) suggestion that problem-solving questions are useful for “application,analysis, and synthesis” of higher level cognitive skills (p 239). Together a group problemsolving activity was created. Each group was given a material type (Books, CDs, DVDs, and
Designing Instruction for the Cunningham Memorial Library 48Reserves) and asked to assign them to a floor and explain their method. They would presenttheir rationale to the rest of the class for feedback.Student Feedback Lastly, in order to get the students’ opinion of the lesson, a survey was created usingGoogle Forms. The survey was created using the suggestions in Morrison, Ross, and Kemp(2001 p 258). Mainly that: Rating scales should have no more than 5 points Describe what the number means (1 = poor, 5 = excellent) Use points that do not overlap Use clear and concrete language Express only one ideaThe front end of the form can be found at the following websitehttps://docs.google.com/spreadsheet/viewform?formkey=dFh5Q05hSGdrWDh4aDR0blJmYlVQc0E6MQ#gid=0. Screenshots of the back end are included in appendix 10. The survey gave the students an anonymous outlet to express their opinions of the lesson.A pairing of open and closed questions allowed the students a chance to express their opinionmore completely, or on a topic that was not covered in the closed questions. Google Forms waschosen over other online forms because of its unlimited responses, customizability, and plug-ins(third party coding aimed at adding additional functions to the platform).Data Analysis Testing was conducted with five subjects. The subjects were between twenty and fortyyears of age, with three females and two males. Two of the subjects were very experienced withthe materials and shelving; two were experienced with most of the materials and shelving; thelast had never worked with the materials before. All subjects were Caucasian, spoke English as afirst language, and had no impairments that would hinder their sight, hearing, comprehension, or
Designing Instruction for the Cunningham Memorial Library 49motor skills. Each subject was given the pretest, then a paper copy of the PowerPoint presentation.The instructor went through the presentation with each subject and answered any questions.After the subject felt they understood the material they were given the post. Finally, the subjectswere given five books to analyze. Books were chosen for the test run, as they had the mostvariation in identifiers. The analysis took place as a discussion between the instructor and thesubject. Subjects were then directed to the survey to express their opinion of the instruction. Data from the pretests did show an improvement in the identification of clues onmaterial. In the pretest all subjects circled the call number as a whole, whereas in the postsubjects circled individual parts of the call number. As expected the two advanced studentsidentified the majority of the clues in the pretest, while the most inexperienced circled mainly thecall number. The inexperienced student did recognize that color was probably important duringthe pretest, but didn’t know why. The test was printed on a black and white computer and thecolor was added later with colored pencils. Most likely, the subject knew color was important asit was deliberately added. Problems occurred with all students on the stickers portion of themnemonic as it was difficult to determine what was a sticker and what was a part of the material. Data from the surveys found the following issues: Reserve had regular call number instead of the teacher/class Students thought the instruction was useful but wanted more examples Students were not motivated to follow instruction. (figure 11.1) Students felt the mnemonic was “silly” which most likely brought down the score on “appropriateness” of instruction (figure 11.2)
Designing Instruction for the Cunningham Memorial Library 50Figure 11:1 MotivationFigure 11:2 AppropriatenessFigure 11:3 Remember
Designing Instruction for the Cunningham Memorial Library 51Recommended Changes Through the analysis of the data it was found that the following issues should beaddressed. The first is the addition of other material types should be included in the pretest.Representations for thesis, microform/Government Documents, and periodicals are the mostimportant material to be added. Additionally the pretest needs to be reformatted from paper tothe physical object. The ability to draw on the material will be lost, but the switch will gainrealness which will hopefully add to comprehension. The next focus should be on improving motivation and “silly”ness. Four of the subjectswork at the library and were not excited to get another lesson on shelving. The comment onsilliness was most likely directed towards this. The mnemonic is a bit childish, and results fromthe survey (figure 3) suggest it may not stick. However, this test was conducted towards the endof the semester, after the students had been required to undergo many requests to improvetraining. Frustration, with the job may have lead to a disproportionate amount of resistance. Iffurther use of the strategy proves similar the focus will be shifted in another direction.
Designing Instruction for the Cunningham Memorial Library 52Appendix 1 PretestI have been working here for __________Choose 2 of the following scenarios and write your response and/or the steps youwould use to complete the task. Feel free to write on the back of the page: 1. “Can I give you my 991 to check these books out?” 2. “I have a charge on my account for a book. What do I do?” 3. “Could I get a private room?” 4. *The gate beeps when they leave the building*Match the staff member with their responsibilities. Lines can go to multipleplaces.Ali Supervises all staff Student Schedule ChangesAndrea Stacks MaintenanceCarey Searches ReservesHolli Remote StorageJosh Patron Accounts ILLKatherine HoldsKelly Hold for Class Graduate CarrelsPaula Government DocumentsSusan Browsing Media Issues BillingSheilaPlease mark your answer to the follow multiple choice questions.1. I need to verify if an item’s status 4. Questions about class reserves go to (checked in, piece count) when A. Ali A. They are checked in B. Carey B. I am preshelving C. Paula C. I check out the item D. Susan D. All of the above 5. Altering my task list is ok when2. It is ok to ignore a system message when A. I switch tasks with another worker A. They are over 2 years old B. I have an injury that would impede B. I am preshelving my work C. Never C. Paula/Staff on duty approves it D. I am really busy D. Never3. The Kurzweil room is 6. To make a resident card the patron A. Designated for those who ask for it should fill out the ___ form. directly A. Change of Address B. A group room reserved on the website B. Processing C. Located on the 3rd floor C. Social Security D. All of the above D. DonationChoose 1 scenario and describe how you would provide excellent customerservice. Feel free to present the information in the way that makes sense to you
Designing Instruction for the Cunningham Memorial Library 53(bulleted list, flow chart, paragraph, ect.). Responses should not exceed the spaceprovided. You are reshelving books when you notice a patron roaming the stacks. They are comparing the range markers against a piece of paper and look confused. While checking out dvds to a patron, they mention they couldn’t find the second season of their favorite series. A faculty member wants to renew a book. Renewing it would cause them to exceed their maximum of 3 renewals. You have served this professor before and know s/he is often very demanding. You are helping a patron, when another approaches you with a question.
Designing Instruction for the Cunningham Memorial Library 54Assessment Tool for Applicants Indiana State University Library uses the Library of Congress classification System forre-shelving library material. Read the instructions for each section of the test carefully and studythe examples before beginning the problem. To put “books” in order, start at the top of each callnumber and arrange them alphabetically and numerically. When you encounter a letter versus anumber, always put the letter first. Work as accurately and as rapidly as you can. Section I: Same or DifferentExamine each pair of call numbers. If both call numbers in the pair are the same, write “S” onthe line next to the pair. If they are different, write “D”.Example: HM HM S HF HF D 132 132 54.5 54.5 .G37 .G37 .F63 .F361. KFN KFN 6. RC RC 971.5 977.5 280 208 .M4 .M4 .B8E8 .B8E82. HQ HQ 7. TR TR 71 71 898 898 .J35 .K35 .B8 .B83. L L 8. M ML 13 13 1 1 .E37 .E37 .M7D4 .M7B44. E F 9. QA BLIND 442 442 3612 QA .B21 .B21 .H8 3612 .H85. QA QA 10. PN PN 76.6 76.6 1995 1995 .M352 .M352 .K2X426 .K2S426