IPT 530-EDubose evaluation methodology team project
Upcoming SlideShare
Loading in...5
×
 

IPT 530-EDubose evaluation methodology team project

on

  • 583 views

 

Statistics

Views

Total Views
583
Views on SlideShare
566
Embed Views
17

Actions

Likes
0
Downloads
4
Comments
0

2 Embeds 17

http://ipttoolkit.wordpress.com 15
http://www.slideshare.net 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

IPT 530-EDubose evaluation methodology team project IPT 530-EDubose evaluation methodology team project Document Transcript

  • Merchandise Unlimited UPS Truck Crew Process Evaluation Final ReportEvaluation Team: Eboni DuBose, Perri Kennedy, and Chester Stevenson IPT 530 Evaluation Methodology Boise State University Spring 2011
  • Table of ContentsPreliminaries ................................................................................................................................... 2 I. Executive Summary .................................................................................................................. 2 II. Preface ..................................................................................................................................... 4 III. Methodology .......................................................................................................................... 5Foundations .................................................................................................................................... 6 1. Background and Context ......................................................................................................... 6 2. Descriptions and Definitions ................................................................................................... 7 3. Stakeholders .......................................................................................................................... 12 4. Resources .............................................................................................................................. 12 5. Dimensions of Merit and Importance Weighting ................................................................. 14Sub-Evaluations............................................................................................................................. 15 6. Process Evaluations ............................................................................................................... 15 7. Exportability .......................................................................................................................... 17Conclusions ................................................................................................................................... 18 8. Overall Significance ............................................................................................................... 18 9. Recommendations ................................................................................................................ 21 Reporting ................................................................................................................................... 23References .................................................................................................................................... 24Appendix A: MU UPS Receiving Process Assessment Rubrics with Criteria Highlighting ............. 25Appendix B: Summary of Ratings with Rationale ......................................................................... 28Appendix C: Data Sources for Instrument Dimension .................................................................. 29Appendix D: Data Sources for Workspace Dimension .................................................................. 31Appendix E: Data Sources for Process Management Dimension ................................................. 34Appendix F: Sub-dimension Weighting of Importance by Stakeholders ...................................... 38Appendix G: Scoring Rubrics for MU UPS Receiving Process Evaluation ..................................... 39 1
  • PreliminariesI. Executive SummaryIntroductionMerchandise Unlimited (MU, pseudonym), a merchandise distribution company based in theMidwestern region of the United States, had not examined the processes and resources inplace to receive inbound shipments for at least a decade. The Director of Operations and theOperations Manager requested a formative evaluation to both establish how well the inboundreceiving process (hereinafter referred to as “evaluand.”)should function and discover how wellit actually does function.In the spring of 2011, three graduate students (hereinafter referred to as “evaluation team”)from Boise State University’s Instructional & Performance Technology program were asked toconduct an evaluation of the inbound receiving process. The stakeholders asked for theevaluation team to look specifically at the 10-person truck crew (hereinafter referred to as “UPSTruck Crew”) which offloads and sorts inbound shipments of used and returned merchandisewhich were delivered via the United Parcel Service (UPS). The evaluation team createdassessment rubrics based on industry best practices and the needs of MU, and then examinedthe practices and resources in place.MethodologyThe evaluation team conducted a formative goal-based evaluation structured on Scriven’s(2007) Key Evaluation Checklist. It was designed to examine the current state of the UPSreceiving process and how it could be improved. After discussions with the stakeholders, theevaluation team focused on three dimensions: Instruments Workspace Process ManagementEach dimension was weighted based on the stakeholders’ consensus determination of itsimportance (Important, Very Important, or Extremely Important). The evaluation team createdan assessment rubric based on warehousing best practices, and then compared the currentprocesses and resources in use by the UPS Truck Crew to the assessment rubric.ConclusionsUsing a 5-point scale (Poor, Mediocre, Acceptable, Very Good, and Excellent), the overallquality of the UPS receiving process is rated “Acceptable” for current organizational needs.However, the evaluation team concluded that the current set of processes and resources maynot be sufficient to handle future needs. A major concern is that Process Management,weighted as Extremely Important by the stakeholders, was rated as “Mediocre” when 2
  • compared to industry best practices. The overall evaluand rating and dimensions with ratingsand weighting are shown in Table 1. Rating Overall  Evaluand Dimension Weighting Instruments  Important Very Workspace  Important Process Extremely Management Important Poor Mediocre Acceptable Very Good ExcellentTable 1. MU UPS Truck Crew Receiving Process Evaluation ResultsStrengthsBased on the evidence gathered, the evaluation team noted the following strengths of the UPSreceiving process: Equipment in use is well maintained and sufficient to maintain current operation levels. Space is utilized as well as possible given its limitations. The supervisor and members of the UPS Truck Crew are capable, experienced, and work together as a cohesiveness unit.Opportunities for ImprovementBased on the evidence gathered, the evaluation team made the following recommendations tothe stakeholders: Develop and document a standardized operating procedure to ensure that tacit knowledge is not lost and that new hires receive consistent on-the-job training. Develop performance metrics to drive continual process improvements. Develop a standardized package labeling system for use by retail and individual customers. Examine all options for increasing floor space. Evaluate existing hardware for possible replacement or upgrades to meet future needs. Look into a possible redundancy in the initiation process, and consider the options for having initiation performed at the next stage of the receiving process. 3
  • II. PrefaceBackgroundMerchandise Unlimited, a merchandise distribution company based in the Midwestern regionof the United States, had substantially upgraded and modernized much of its warehouseoperations in recent years. However, the processes and resources in place to receive inboundshipments had not been updated or even examined for at least a decade.In January 2011, the Director of Operations and the Operations Manager of MerchandiseUnlimited requested an evaluation of the processes in use by the receiving department, whichhandles all inbound shipments delivered to the company’s warehouse. Although otherwarehouse processes had been assessed and improved, the receiving functions had not beenevaluated for many years.The stakeholders asked for the evaluation to look specifically at the receiving processconducted by the UPS Truck Crew, a 10-person workgroup which handles overstock and usedmerchandise shipped back, via UPS, to MU in individual non-palletized packages. The volume ofshipments handled by this crew ranges from 1,500 to 3,500 packages each working day.The Director of Operations and the Operations Manager stated that they planned to adopt anyrecommendations which, in their judgment, will be both feasible and cost-effective toimplement within the next two years.Evaluation FocusThis evaluation examined the merit of the UPS receiving process. The Director of Operationsand the Operations Manager wanted to assess the intrinsic quality of the UPS receiving processand see how it compared to industry best practices. The purpose of the evaluation was todevelop the definition of merit for the receiving process, as the company had never establishedany benchmarks or ideals of performance.Evaluation TypeThis project was developed as a formative evaluation, as the stakeholders wanted tounderstand the current level of performance and what changes to the process or environmentcould improve that performance. Formative evaluation is defined as one “typically conductedduring the development or improvement of a program… and it is conducted, often more thanonce, for the in-house staff of the program with the intent to improve” (Scriven, 1991, pp. 168-169).AudienceThis evaluation report was created for the Director of Operations, the Operations Manager, theInbound Manager, the UPS Truck Crew Supervisor, and the Vice President of Human Resources. 4
  • III. MethodologyEvaluation ApproachWhen the evaluation project was proposed, the stakeholders had a general goal of improvingefficiency in the UPS receiving process; they defined efficiency as the speed, accuracy, and cost-effectiveness of the overall process. However, no performance targets or ideals for thereceiving department had ever been defined nor was any system in place to measure speed,accuracy or cost-effectiveness. Without any existing metrics against which to compare currentperformance, it was impossible to determine efficiency based on the stakeholders’ definition.After discussing this with the stakeholders, the evaluation team agreed to instead develop anassessment rubric based on industry best practices and then evaluate how the UPS receivingprocess compared to those best practices.The evaluation framework was based on Scriven’s (2007) Key Evaluation Checklist (KEC), whichoutlined a systemic examination of both processes and outcomes. Due to the lack of existingmetrics, this evaluation was conducted strictly as a process evaluation which would examinethe procedures and resources currently in use by the UPS Truck Crew. The methodology usedwas a goal-based dimensional evaluation of the evaluand’s absolute merit, in that theevaluation team identified dimensions of merit for the evaluand and then examined how theprocedures and resources utilized in each dimension compared to the best practices defined inthe assessment rubric.To develop the assessment rubric, the evaluation team researched best practices forwarehouse receiving departments to determine ideal processes and identify key issues toexamine. The evaluation team also documented the current procedures and environment inplace to look for strengths and weaknesses in actual practice. The evaluation team developedits assessment rubric based on industry best practices compiled by Supply Chain Visions andsponsored by the Warehousing Education and Research Council (WERC). The evaluation teamthen hosted a focus group with the Operations Manager, Inbound Manager, and the UPS TruckCrew Supervisor to discuss the key elements of warehousing best practices. Some proposeddimensions were eliminated as not relevant or not of sufficient importance. The remainingdimensions were then shaped into a set of assessment rubrics which the evaluation team usedto identify the areas of greatest opportunity for improvement. The process assessment rubricsare included as Appendix A.Data Collection MethodsThe evaluation team used a combination of data collection methods including interviews,extant data, and observation. Specific methods used for each dimension of merit are identifiedin Appendix C (Instruments), Appendix D (Workspace), and Appendix E (Process Management). 5
  • Foundations1. Background and ContextMerchandise Unlimited (MU) is a merchandise distribution company based in the Midwesternregion of the United States. MU has been in business in different incarnations for about acentury, and has existed in its current form for nearly four decades. Its primary objective is tocollect and distribute new and used products and ship them to its customers at the lowest costpossible. MU’s single warehouse is in operation 24 hours a day, five days a week to receive,sort, and distribute more than 20 million pieces of merchandise each year to both individualconsumers and the company’s 2,500+ retail clients. For many years, the primary concern ofwarehouse management has been how fast products are shipped to the company’s customers,so almost all efforts at efficiency and modernization have been focused on this area. Thecurrent sorting and distribution areas of the warehouse make extensive use of moderntechnology, including robotics and 3-D scanning, to ensure that products are shipped accuratelyand rapidly.Management has recently turned its focus on how fast products are brought into the building, aprocess that has not been re-evaluated in over a decade. Out of all the products received in thewarehouse, approximately 75% are delivered by a single carrier, UPS. The operationsdepartment would like to evaluate the existing UPS receiving process to improve inefficienciesand reduce costs.The stakeholders asked the evaluation team to conduct the first evaluation by focusing on the10-person workgroup (the UPS Truck Crew) which handles offloading, sorting, and initiatingindividual packages of overstock and used merchandise shipped to the warehouse via UPS; thiscrew does not handle packages sent through other carriers, nor does it handle palletized loads.This merchandise includes both retail store overstock and returns from individual customers.The packages are classified by the UPS Truck Crew as large or small. Large packages: Shipments from retail customers Small packages: Shipments from individual customers, or internal mailConstraintsSeasonal FluctuationsDue to the nature of its business, MU experiences predictable and significant seasonalfluctuations in both shipping and receiving merchandise through its warehouse. During normal 6
  • times, the UPS Truck Crew may offload and sort 1,000-1,500 packages daily, most of which arepicked up by the receiving departments that same day. During seasonal peak times, the crewmay process as many as 3,500 packages daily; since the receiving departments cannot handlethis inbound volume immediately, the UPS Truck Crew must store the packages for later pickup.Warehouse SpaceThe UPS Truck Crew’s assigned space was sufficient a decade ago, when the average dailyvolume was approximately 800 packages. However, the average daily volume of packages hasnearly doubled since that time. No additional floor space is available for the crew’s use. Thestorage area’s shelves are permanent fixtures which cannot be reconfigured or removed.2. Descriptions and DefinitionsEvaluand DescriptionThe receiving department is broken into four sections: Truck crews, including the UPS Truck Crew, which unload the inbound shipments Receiving, which receives large packages from the truck crews Direct Receiving, which receives small packages from the truck crews Internal Mail, which receives small packages from the truck crewsThe UPS receiving process begins when the UPS truck driver unloads packages from his or hertrailer onto a mechanical roller, which rolls the packages into the building. As each packagecomes down the roller, a member of the UPS Truck Crew removes it from the roller andexamines its label. Large packages which are part of a shipment of three or more boxes aresorted directly onto pallets and then initiated. One- or two-box shipments of large packages areput on the express belts, initiated, and then sorted onto pallets. All small packages are put onthe express belts, initiated, and then sorted into bins. Large package pallets are picked up byReceiving, and small package bins are picked up by Direct Receiving or Internal Mail.During the initiation process, a crew member enters a package’s shipping information into thewarehouse inventory software using a laptop computer. Most package labels include UPSbarcodes, which are scanned using a USB device that feeds a limited amount of packageinformation into the warehouse inventory software; the crew member must initiate the rest ofthe information into the system. During times of normal inbound volume, the pallets and binsare picked up promptly by the destination department. 7
  • When inbound volume is very high, the UPS Truck Crew moves the pallets into the storage area, where they may remain for up to two weeks awaiting pickup. A visual representation of this process is shown in Figure 1. Packages unloaded by UPS employee from trailer onto mechanical roller Packages removed from mechanical roller by UPS Truck Crew Packages sorted by type and placed on wooden pallets (Receiving) or onto express belts (Direct Receiving and Internal Mail) Packages initiated into inventory system using laptop computer and handheld scanner Normal receiving volume: Peak receiving volume: Pallets (Receiving) and bins (Direct Pallets and bins stored on shelving forReceiving and Internal Mail) prepared for later pickup immediate pickup Pallets and bins picked up by Receiving, Direct Receiving, and Internal Mail employeesFigure 1. MU UPS Truck Crew Receiving Process. 8
  • Process Logic ModelThe MU UPS Truck Crew Process Logic Model (see Figure 2) describes the factors and crewactivities involved in the UPS receiving process, what activities the evaluation team performedin examining the process, and the intended results of the evaluation and subsequent changesto the process.Figure 2. MU UPS Truck Crew Receiving Process Logic Model 9
  • DefinitionsTable 2 defines terminology used in this report, including evaluation concepts and terms usedby MU for elements involved in the receiving departments.Absolute Merit The quality or value of something in absolute terms measured against a defined standard.Assessment Rubric A tool which provides an evaluative description of what performance looks like at each of two or more defined levels.Dimension of Merit Criteria of merit. These are aspects of an evaluand that define whether it is good or bad and whether it is valuable or not.Dimensional Evaluation An analytical evaluation in which the quality of the evaluand is determined by looking at its performance on multiple dimensions (criteria) of merit.Direct Receiving The division of the receiving department which opens small packages and sorts their contents for processing.Downstream Impactees Impactees that are not directly affected by the evaluand.Evaluand That which is being evaluated (e.g., program, policy, project, product, service, organization).Express Belt A 10-ft long mechanical roller with a computer and scanner at the end, used to initiate individual packages.Formative Evaluation An evaluation conducted for the purpose of improving a currently operational evaluand for the purpose of making improvements.Goal-based Evaluation A type of evaluation based on the goals of the evaluand.Inbound Shipments coming into the warehouse from outside vendors or customers.Initiating MU terminology for the process of entering package data into the warehouse inventory software.Internal Mail Packages received which are not return merchandise. These packages are given to the company’s internal mail facility, which delivers them to the appropriate recipients. 10
  • Mechanical Roller A non-electrical device consisting of a series of rollers which is used to transport boxes from one area to another.Merit Intrinsic value, quality or significance of the evaluand.Overstock Excess merchandise in a store’s inventory, which may be returned to the vendor.Package Refers to a box, carton, or other shipping container received. They are typically referred to at MU as small or large depending on their actual size.Palletize To load packages onto wooden pallets for transportation or storage.Primary Consumers Actual or potential users of the evaluand.Program Logic Model A diagram that illustrates the cause-and- effect relationships between resources and activities, and the outcomes and impacts of the program.Truck Crew A crew of MU employees which sorts merchandise as it is unloaded from a truck into the warehouse.UPS Receiving Process Refers to the operations involved in unloading, sorting, and initiating inbound packages.UPS Truck Crew The 10-person truck crew which processes shipments of overstocks and used merchandise delivered by UPS to the warehouse. The UPS Truck Crew is the primary consumer of this evaluand.Upstream Stakeholders Stakeholders who are involved in the development, implementation, or management of the evaluand.Weight The determination of which evaluation criteria are the most important.Table 2. Definitions of Evaluation and MU Terminology 11
  • 3. StakeholdersThis evaluation was concerned with three sets of stakeholders: upstream stakeholders, primaryconsumers, and downstream impactees. The stakeholders are listed in Table 3. Upstream Stakeholders Primary Consumers Downstream Impactees Director of Operations UPS Truck Crew (n = 10) Immediate Impact Direct Receiving staff Operations Manager Receiving staff Internal Mail staff Inbound Manager Secondary Impact UPS Truck Crew Supervisor Product stockers Product shippers Distant Impact External customers Clients of external customersTable 3. MU Stakeholders for the UPS Truck Crew Evaluation.4. ResourcesFinancial ResourcesThe UPS Truck Crew is funded by the organization as part of its Operations division. Theyreceive whatever funding necessary to perform their required tasks. The upstream stakeholdersstated that they are willing to spend money on changes which they deem likely to increaseefficiency and/or decrease operating costs. The stakeholders will not reduce costs through staffreductions; if increased efficiency allows for a smaller UPS Truck Crew, extra crew members willbe moved to other parts of Operations rather than be dismissed.Physical ResourcesSee Figure 3 for a simplified illustration of the physical layout. The storage area assigned to theUPS Truck Crew is capable of holding between 65,000 and 75,000 packages on its shelving units.The storage shelves are permanent fixtures which cannot be removed or reconfigured. 12
  • Figure 3. Layout of MU UPS Truck Crew Receiving SpaceKnowledge ResourcesThe supervisor of the UPS Truck Crew has held this position for 15 years, and five of the tencrew members have worked there for five or more years. Turnover is modest and notconsidered by management to be an issue; morale within the crew appeared to be excellent.New hires receive on-the-job training in sorting and initiating packages.Technological ResourcesThe UPS Truck Crew inputs data from each package label into a laptop computer runningproprietary inventory management software. The software has a command-line interface. Ittakes an experienced crew member approximately one minute to type in the data for eachindividual package. The laptops in use are old and slow. The shipping labels on some packagesinclude barcodes, which are scanned using devices which connect to the laptops via a USBcable. Data incorporated into the barcode is automatically entered into the inventorymanagement software, but additional data must still be entered manually. 13
  • 5. Dimensions of Merit and Importance WeightingThe evaluation team met several times with the Operations Manager, Inbound Manager, andUPS Truck Crew Supervisor to refine the focus of the evaluation. When the final dimensions ofmerit were selected, the evaluation team conducted a group interview with these threeupstream stakeholders so they could weigh the importance of each dimension using a 3-levelscale (extremely important, very important and important).The evaluation team then divided each dimension into sub-dimensions, around which it createdan assessment rubric. The dimensions, weighting, and sub-dimensions are shown in Table 4.Dimension of Merit Dimension Weighting Sub-dimension Hardware (computerized) Instruments Important Hardware (non-electronic) Software Pallet Layout Workspace Very Important Utilization of Floor Space Space Available Dock Management Package Labeling Process Management Extremely Important Documented Standard Operating Procedures MetricsTable 4. MU UPS Truck Crew Evaluation Dimensions, Importance Weighting, and Sub-dimensionsAll the dimensions evaluated were process dimensions. The stakeholders had initially requestedan evaluation of outcomes such as speed and accuracy, but it was determined that since noperformance standards had ever been developed or measurement methods implemented,there was no criteria against which to rate the UPS Truck Crew’s current level of performance.The stakeholders’ schedules did not allow for a second group interview, so they submitted theirsub-dimension weighting as individuals. The different professional priorities and viewpoints ofthe stakeholders were reflected in how differently they weighted some of the sub-dimensions.As the evaluation team was unable to obtain a consensus on the importance of the sub-dimensions, only the weighting for the dimensions was used to calculate the final ratings. Thesub-dimension weighting is included as Appendix F for reference. 14
  • Sub-Evaluations6. Process EvaluationsThe entire evaluation was conducted as a process evaluation, and examined three dimensionsof merit: Instruments Workspace Process ManagementInstrumentsDimension Evaluation Question: How do the instruments in use by the UPS Truck Crew ratewhen compared to industry best practices?Data Sources: Interviews, observation, extant dataOverall Rating: AcceptableThe instruments dimension examined the tools in use by the UPS Truck Crew in the course of itsoperations, and was divided into three sub-dimensions: 1. Hardware (Computerized) – laptop computers, handheld barcode scanners 2. Hardware (Non-Electronic) – forklifts, pallet jacks, conveyors, bins, and other equipment 3. Software – proprietary warehouse inventory system which runs on the laptops and accepts data from the barcode scannerAn assessment rubric was created to compare the current state of the three sub-dimensions toindustry best practices. See Appendix A for the Instruments Assessment Rubric; the highlightedtext indicates the criteria which best describe the current state of Instruments used by the UPSTruck Crew.WorkspaceDimension Evaluation Question: How does the physical workspace in use by the UPS TruckCrew rate when compared to industry best practices?Data Sources: Interviews, observation, extant dataOverall Rating: AcceptableThe workplace dimension examined the physical workspace in use by the UPS Truck Crew in thecourse of its operations, and was divided into three sub-dimensions: 15
  • 1. Pallet Layout – how pallets are placed on the floor space for efficient sorting 2. Utilization of Floor Space 3. Space AvailableAn assessment rubric was created to compare the current state of the three sub-dimensions toindustry best practices. See Appendix A for the Workspace Assessment Rubric; the highlightedtext indicates the criteria which best describe the current state of the Workspace as used by theUPS Truck Crew. Although Workspace is considered “Acceptable” under current conditions,there is no room for expansion under the current configuration should the volume of inboundpackages continue to increase.Process ManagementDimension Evaluation Question: How does the process management in use by the UPS TruckCrew rate when compared to industry best practices?Data Sources: Interviews, observation, extant dataOverall Rating: MediocreThe process management dimension examined the management of operating processes in useby the UPS Truck Crew in the course of its operations, and was divided into four sub-dimensions: 1. Dock Management – scheduling of deliveries and commitment to unload times 2. Package Labeling – external labeling on packages to indicate shipment type, package contents, and quantity of individual packages in shipment 3. Documentation of Standard Operating Procedures (DSOP) 4. Metrics – defined standards of performance against which actual performance can be measuredAn assessment rubric was created to compare the current state of the four sub-dimensions toindustry best practices. See Appendix A for the Process Management Assessment Rubric; thehighlighted text indicates the criteria which best describe the current state of the ProcessManagement in use by the UPS Truck Crew.Of the four sub-dimensions, Dock Management and Package Labeling were rated as “StandardPractices”. The other sub-dimensions, DSOP and Metrics, were rated as “Poor Practices” as nostandard operating procedure or performance metrics exist for this workgroup.See Appendix B for a summary of the sub-dimension ratings and the rationale for each. 16
  • 7. ExportabilityThe processes in the evaluand are specific to the workgroup. There are no innovative methodsin place which could be adapted for other MU departments. The processes in the evaluand arespecific to the workgroup, and the following aspects of the UPS Truck Crew and the evaluationare valuable to other similar context.The greatest strengths of the UPS Truck Crew are the camaraderie and cooperation among thecrew members and their loyalty to their long-time supervisor. This positive environment hascontributed to the low turnover in this unskilled position, thus maintaining a high level ofexperience among the crew. Although the team chemistry encouraged by the UPS Truck CrewSupervisor cannot itself be exported, the supervisor’s interactions with his team could bestudied to determine what elements of his supervisory style could help departmentsexperiencing high personnel turnover rates.The evaluation team developed performance rubrics for the UPS Truck Crew based ondocumented industry best practices. According to the Inbound Manager, no performancerubrics or productivity standards are in place elsewhere in the receiving department.Productivity is a major concern of not only MU but the warehousing industry in general. MUmust develop performance standards and measure performance outcomes to ensure that allparts of the receiving operation function at optimal levels, which should lead to improvedwarehouse productivity and continued revenue growth. To develop those standards, theoperations management at MU should consult industry associations such as WERC, theAssociation for Operations Management, and the Council for Supply Chain ManagementProfessionals for information on benchmarking and best practices. 17
  • Conclusions8. Overall SignificanceThe purpose of this evaluation was to understand the receiving process’s current level of meritand to identify what changes to the process or resources could improve its performance. Theresults obtained from each dimension were calculated and summarized in Appendix G. Theoverall results (shown in Table 5) of the evaluation revealed the receiving process’s currentoverall level of merit as “Acceptable” when measured on a 5-level scale (Poor - Mediocre -Acceptable - Very Good - Excellent). Rating Overall  Evaluand Dimension Weighting Instruments  Important Very Workspace  Important Process Extremely Management Important Poor Mediocre Acceptable Very Good ExcellentTable 5. Ratings for Overall Evaluand and DimensionsThe Instruments dimension was rated as "Acceptable" overall because the hardware andsoftware in use meet current needs, are properly maintained, and function correctly. All threeInstrument sub-dimensions were rated as “Standard Practices”; none are in critical need ofimprovement to meet current needs, nor do any exceed industry standards.The Workspace dimension was rated as “Acceptable” overall because under normal conditionsthe floor space and layout is sufficient for the crew members to navigate easily betweenlocations. All three Workspace sub-dimensions were rated as “Standard Practices”; none are incritical need of improvement to meet current needs, nor do any exceed industry standards. 18
  • The Process Management dimension was rated as “Mediocre” overall. The Dock Managementand Package Labeling sub-dimensions were both rated as “Standard Practices”. The DSOP andMetrics sub-dimensions were rated as “Poor Practices”; neither standard operating proceduresnor any performance metrics or measurements exist for this workgroup. Although currentneeds are being met by the processes in place, future needs will be negatively impacted by thelack of documentation and metrics.The dimensions’ strengths and weaknesses are shown in Table 6, in order of importance withineach dimension based on their potential level of impact on the evaluand. Dimension Strengths WeaknessesInstruments 1. Enough hardware to meet 1. Not enough equipment to(Important) current needs. meet future demands. 2. Have the right tools to do 2. Pallet jacks are past or near the job. the end of their life 3. Hardware is easy to use and expectancy. in good repair. 4. Software integrates with next step in receiving process.Workspace 1. Equipment is laid out to 1. No space to grow.(Very maximize available floor 2. Lack of space causes choke-Important) space. points. 2. Enough space to meet 3. Lack of space makes it current needs during periods difficult to walk during busy of normal inbound volume. periods. 4. During busy periods floor space is very limited at best, non-existent at times.Process 1. A package labeling system 1. No metrics exist to measureManagement exists and is used (although performance of the truck(Extremely not always followed) crew or their suppliers.Important) 2. UPS truck driver scheduled 2. No SOP exists to verify to arrive early enough for Standards, train new unloading to be completed employees, or pass on tacit in a timely manner. knowledge. 3. Truck Crew is at work before 3. UPS truck driver may be late, UPS Truck Driver shows up. with no negative consequences, impacting schedule of UPS Truck Crew and other receiving areas. 19
  • Table 6. Strengths and Weaknesses of Evaluand Dimensions 20
  • 9. RecommendationsOf the three dimensions evaluated in this project, Process Management was considered themost important by the stakeholders (see Table 5 on page 17), but earned only a “Mediocre”rating from the evaluation team. This dimension has the most opportunities for improvement inthe shortest amount of time, and at the lowest cost. Specific recommendations to improve thisdimension include: 1. Develop a standardized product labeling system. Technology exists to develop a barcode that stores all shipment information and could update the system once scanned. Customers could be required to print this barcode and attach it to the box or invoice before shipment. The evaluation team understands that this step is already under consideration by MU, and recommends that it be implemented if feasible. 2. Gather metrics to improve and manage the receiving process. By tracking metrics, the receiving department can take appropriate actions to drive continual process improvements. It is also good practice to report metrics to upper management and post results where they are accessible to the UPS Truck Crew. 3. Document operating procedures as a way to standardize processes and terminology. This will ensure that tacit knowledge is not lost as part of a succession management plan, and will help provide consistent on-the-job training for new hires.Workspace and Instruments both received an “Acceptable” rating. Although these dimensionsare less important than the process management dimension, both offer areas of opportunitywhich can significantly improve the receiving process. These recommendations will require agreater investment in monetary and time resources. 1. Within the Workspace dimension, the floor space should be of critical interest. The Truck Crew is currently using its workspace to its full capacity. Because of the size and amount of equipment required to perform the job, workspace is limited. The limited workspace reduces the possible maximization of pallet and floor space usage due to a lack of space to place needed pallets, and creates choke points between an Express Belt and the Mechanical Roller. MU must seriously consider increasing floor space in preparation for future demand or face decreased productivity and bottlenecks. 21
  • 2. Within the Instruments dimension, the organization would need to update its computerized and non-electronic equipment as well as its software to meet increasing demands. Some hardware in use is near end-of-life, and will need to be replaced soon. All hardware should be inspected and evaluated for possible replacement or upgrades. MU should consider using hand-held computers with scanners rather than bulky laptops. Small packages are initiated by the UPS Truck Crew, and then the same data is entered again by Direct Receiving. It would be more efficient to have Direct Receiving initiate small packages so that the UPS Truck Crew can focus on large packages. 22
  • ReportingThis evaluation report was created for the benefit of the evaluand stakeholders at MU: Director of Operations Operations Manager Inbound Manager UPS Truck Crew SupervisorEvaluation team member Chester Stevenson will e-mail a brief summary of the evaluationteam’s conclusions and recommendations, plus a copy of the full report, to the fourstakeholders and to the Vice President of Human Resources at MU. The stakeholders mayshare the information with other MU personnel as desired.Any questions about the evaluation may be directed to team member Chester Stevenson. 23
  • ReferencesBrinkerhoff, R. O. (2006). Telling training’s story: Evaluation made simple, credible, and effective. San Francisco: Berrett-Koehler.Davidson, E. J. (2005). Evaluation methodology basics: The nuts and bolts of sound evaluation. Thousand Oaks, CA: SAGE Publications.Scriven, M. (1991). Evaluation thesaurus (4th ed.). Newbury Park, CA: SAGE Publications.Scriven, M. (2007). Key evaluation checklist. Retrieved from http://www.wmich.edu/evalctr/checklists/kec_feb07.pdfVitasek, K., Harrity, C., ODonoghue, K., & Symmes, S. (2007). Warehousing & fulfillment process benchmark & best practices guide. Retrieved from http://www.werc.org/assets/1/workflow_staging/Publications/701.PDF 24
  • Appendix A: MU UPS Receiving Process Assessment Rubrics with Criteria Highlighting Dimension of Sub- Poor Practices Standard Practices Best Practices Merit Dimension Hardware is out of date and limited in Available hardware meets current needs. Available hardware meets current and futureINSTRUMENTS quantity or not in an adequate quantity. needs. Hardware Hardware is properly maintained and (Non-Electronic) - Available hardware does not meet current functional. Backup equipment is available to replace out WEIGHTING: includes forklifts, needs. of service equipment. pallets, shelving, Important conveyors, bins Hardware is out of repair or requires Hardware is efficient and does not add excessive service. additional steps to work processes. Hardware is not mobile. Hardware is mobile. Hardware is highly mobile and compatible with the latest software. Hardware is out of date and not Hardware is compatible with newer compatible with newer software. software. Available hardware meets current and future needs. Hardware is out of date and limited in Available hardware meets current needs. Hardware quantity or not in an adequate quantity. Backup equipment is available to replace out (Computerized) Hardware is properly maintained and of service equipment. Available hardware does not meet current functional. needs. Hardware is efficient and does not add additional steps to work processes. Hardware is out of repair or requires excessive service. Software is slow, not easy to understand, Software can be learned within an Software is easily understandable and easy to and not efficient. acceptable time period. use. Software does not interface with the next Software interfaces with - and can be Software is easily upgradeable and easily area in the receiving process. retrieved by other parts of the receiving integrates with other company software. Software process. Software provides metrics that can be Software can be upgraded when provided to customers. requirements change. Software is updated regularly to support best practices. 25
  • Dimension of Sub- Poor Practices Standard Practices Best Practices Merit Dimension Pallet placement is not formalized Pallets are laid out so that they do not hinder Product placement is clearly defined. WORKSPACE movement. Package placement is not clearly defined Sorting area is arranged to support demand Package placement is clearly defined. issues meeting current and future volume. WEIGHTING: Sorting area is not arranged to support Pallet LayoutVery Important demand Issues meeting current volume. Pallets are laid out to optimize movement, and restrict fatigue or injury. Mobility around pallet area is difficult. Floor space is poorly laid out and mobility Fixtures are placed so that they do not Floor space is well laid out to support current is difficult. hinder movement of product. and future demand. Utilization of Floor Space Poor layout in high traffic areas. Poor Areas are laid out well to support current Area is kept clean and clear of debris. housekeeping demand. Floor space does not meet current needs. Floor and storage space meets current Floor and storage space is adequate to needs. accommodate current and future demand. There is not enough room to unload an entire truck during peak times. There is adequate space to unload a truck Space Available and still allow for mobility. Storage space is not adequate to hold overstock during peak times. Storage space is adequate to hold overstock during peak times. 26
  • Dimension of Sub-Dimension Poor Practices Standard Practices Best Practices Merit Trailer deliveries not scheduled Shipper given specified delivery window Doc appointments are made and a Process commitment to unload every vehicle within Dock ManagementManagement Trailers not unloaded in a timely way Timely unloading to avoid two hours when the appointment is kept, detention/demurrage labor and dock space utilization is optimized No package labeling OR package Not all packages labeled by suppliers, but is Package is labeled by supplier, to Weighting: inconsistently or incorrectly labeled labeled upon receipt specification receipt so check-in can be Package Labeling made with bar-code scan into the system Extremely Important Documented Standard No process documentation exists OR Receiving processes are documented and Receiving processes documented, made Operating Procedure processes are written but not consistently available but tacit knowledge is not made readily available, and consistently followed (DSOP) followed or made available explicit Customer shipping errors are not tracked Ad hoc process to track inbound receiving Basic receiving requirements are outlined (e.g. % of packages errors or track internal functional metrics and shared with all customers received without a invoice or % of packages received mislabeled) Customer receiving errors are not shared Formal process to collect inbound receiving with customers metrics and data is shared with customers No established performance Basic receiving requirements are outlined and Internal performance metrics are collected metrics or SLAs shared with all customers and posted or shared with employees (service level Metrics agreements) Formal process to collect Inbound receiving established with metrics, but data is not shared with suppliers customers Internal performance indicators (e.g. dock to stock time) are not tracked 27
  • Appendix B: Summary of Ratings with Rationale Dimension of Sub-Dimension Rating Rationale Merit The criteria for non-electronic hardware all fall under the Standard Practices rating. Hardware (Non-Electronic) Standard Practices According to one stakeholder, some of the existing hardware will require replacement in the near future but all equipment is currently adequate. The criteria for computer hardware all fall under the Standard Practices rating. The Instruments Hardware (Computerized) Standard Practices laptops are not modern, but have sufficient capacity to run the necessary software. The criteria for software all fall under the Standard Practices rating. The software was Software Standard Practices created in-house and continues to meet the organization’s needs. The criteria for pallet layout all fall under the Standard Practices rating. There is no Pallet Layout Standard Practices consistent layout plan for pallets, but the crew members are successfully able to create the layout for daily conditions. The criteria for utilization of floor space all fall under the Standard Practices rating. Space Workspace Utilization of Floor Space Standard Practices is limited, but the crew reports no significant problems in navigating the space under normal volume conditions. The criteria for space available all fall under the Standard Practices rating. The space does Space Available Standard Practices accommodate current normal volume, but is less able to accommodate peak volume. There is no room for future expansion. The criteria for dock management all fall under the Standard Practices rating. The main Dock Management Standard Practices issue with shipment delivery is related to the reliability of UPS delivery employees, which MU cannot control. The criteria for package labeling all fall under the Standard Practices rating. Current Package Labeling Standard Practices package labeling is adequate, but MU is looking at ways to improve it. Process Management The criteria for DSOP fall under the Poor Practices rating. There is no documentation of Documented Standard Operating Poor Practices standard operating procedures. All knowledge is tacit and communicated verbally to new Procedure (DSOP) employees. The criteria for metrics all fall under the Poor Practices rating. No performance metrics Metrics Poor Practices exist, nor do any methods of measuring aspects of performance. 28
  • Appendix C: Data Sources for Instrument Dimension InterviewsSub-Dimension Data Sources Observation Extant Data Truck Crew Supervisors (1) Extant Data – Q11: Can you think of one thing that you use now Q: How long do the Pallet Equipment Inventory Review equipment (like a piece of equipment or a procedure) that Jacks typically last before includes: inventory. prevents you from working faster or more they need to be replaced? Review maintenance accurately? Forklift (n=2) records a. No -TC #1 With the amount of use they Pallet Jack (n=4) get during our receiving Express BeltHardware (non- (2) Interviews – b. Not really offhand. Space is the biggest hindrance. process, they typically have a w/computer (n=4) Management (n=3) TC #2 lifecycle of 3 – 4 years. Two Roller (n=1) electronic) and Crew (n=4). of the Pallet Jacks have been Pallets c. Some of the belts aren’t that good for rolling boxes. in use for 3 years, and two Bins (n=2) When these belts are extended, the rollers are too far have been in use for 6 – 7 apart and don’t roll boxes easily. TC #3 years. Truck Crew d. Nothing that gets in my way. The other guys get in Supervisor my way because of the limited space. TC #4 (1) Extant Data – Q13: How would you improve the layout of your Computerized Review equipment work area? Inventory includes: inventory. d. The racks are not very sturdy. TC #4 Review available Computer with Hardware hardware scanner (n=3) specifications.(Computerized) (2) Interviews – Management (n=3) and Crew (n=4). (1) Interviews - Q7: How long does it take you to do the data entry Q1: How long have you Management (n=3), for each package? How easy is it to use the software been using the current Crew (n=4), to do the data entry? How easy is it to make software? Programmers (n=1). corrections if you type something wrong? The software has been Software a. 15 sec - 1.5 min (depending on how many unchanged for the last 15 packages) Labels without bar codes = does not add years. significant extra time to initiation To edit data entry, search by invoice # and edit, very simple. Receiving cannot access this software - if they find an error, they bring it to the truck crew’s attention so the truck crew 29
  • can make the correction. Can track which truck crewmember made the error, but not known if accuracyproblems are tracked and taken into account forperformance reviews -TC #1b. 3-4 minutes at the longest. Very repetitive onceyou learn the process should be quick (his time islonger because he is new). Easy up to a point untilyou get to the 2nd to last screen. Once it gets to thatpoint it would have to be corrected by supervisor -TC#2c. Not long, I just scan the barcode and UPS code(these packages have standard UPS labels, I assume). Ihave to open the packages occasionally to get apacking list, but in most cases I just scan it. It’s easy tomake corrections. TC #3d. Maybe 10 seconds, 20 seconds - I’ve been here 10years so I can fly through it. The software is really easyto use, and I can barely send an e-mail! Software isnot complex. Everybody has their own log-in, but noteverybody has editing permission. Supervisor andunofficial assistant supervisor can make changes iferrors are found at the end of entry TC #4 30
  • Appendix D: Data Sources for Workspace Dimension InterviewsSub-Dimension Data Sources Observation Extant Data Truck Crew Supervisors (1) Interviews - Q6: How easy is it to sort the large packages so that Pallets are laid out in Management (n=3) all the packages from one sender are together? Are rows but package layout and Crew (n=4). the labels consistent or do you run into problems is not standardized. with labels having different names for one sender? (2) Observation - Observe receiving b. It’s fairly simple, my biggest problem was spending process to determine too much time trying to remember where I had put product placement things down, this became easier with practice. trends. Sometimes pallets are organized by #, other times by return/guide/etc - it depends on who sets up the pallets. TC#2 Q13: How would you improve the layout of your work area? a. The racks used to run the opposite way, but that reduced the loading area so we changed the rack Pallet Layout orientation to give us more space. Racks are almost to the ceiling now, at max height for forklifts -TC#1 b. A branching conveyor belt - pull off different categories-guides, returns, and others This would probably take up a lot of space, but would make it easier to sort and pallet. TC #2 c. More space. Not sure offhand what would help the layout. (moving student returns to another part of the building - would require a different address and would only eliminate one roller) TC#3 d. Just expand, honestly. TC #4 31
  • (1) Interviews - Q12: Is there anything about the physical layout of There is a choke point Management (n=3) your warehouse area that gets in the way during between the third and Crew (n=4). normal weeks? What about during high-volume Express Belt and the weeks? Roller. (2) Observation - a. High volumes SKUs that have no invoice (labeling), Observe receiving having to wade through many packets -TC#1 Every open space is process to determine used, with narrow paths Utilization of product placement b. The track area coming from the belt to the student to walk though. trends and floor space returns is a high-traffic area and is pretty tight for Floor Space usage. space. (area around the first computer - the rack is close to the conveyor, causes a bottleneck). Takes more time because you have to detour with the forklift. TC #2 c. In a normal week no, but yes during a busy week. TC #3 (1) Interviews - Q6: How easy is it to sort the large packages so that Every space is used. Management (n=3) all the packages from one sender are together? Are and Crew (n=4). the labels consistent or do you run into problems with labels having different names for one sender? (2) Observation - a. The larger the load the harder it is. Space makes it Observe receiving area hard (after you get so much you forget where you put to determine space it). Senders sometimes put the wrong information availability. (like name of individual rather than name of school) or no information (they don’t understand they are NOT the only vendor) -TC#1 Q8: During normal times, is it easy to move theSpace Available packages around to where they need to go? a. Pretty easy but space is limiting -TC#1 b. Yes - TC#2 c. Pretty Easy TC # 3 d. Normal times yes but space is also an issue during busy times. Freight (pallets) is bumping into other shipments. Q9: How does your work routine change during those weeks when the number of inbound packages is twice the usual volume? a. It’s a lot more fast pace. Where we’re rushed we are out of space quickly and it can be backed up for 5- 32
  • 6 weeks. When outbound is busy, inbound personnelare diverted there to help, which slows down theinbound processes. -TC#1b. Between freight coming off and coming in, spacebecomes very tight, hard to move around with theforklift because there’s stuff everywhere.-TC #2c. Space (paraphrase) TC #3Q10: Can you think of one thing that could be addedto your work area that would make your job easier? a. Space -TC#1b. More space, definitely more space TC#2c. When I scan, wondering why it can’t be done aboutthe people that are checking it in. Seems likeinitiation is redundant - why not just put studentreturns in the bins and have the direct receiving groupdo the check-in? Jerry and the rest of the crew are infavor of this. This would both remove duplication ofeffort and free up space that the student return rollerscurrently take up. TC#3 33
  • Appendix E: Data Sources for Process Management DimensionSub-dimension Data Sources Observation Extant Data Truck Crew Supervisors (1) Interviews - Q4: How much time is there between the time you The UPS Truck is Management (n=3) start work and the time the first UPS truck arrives? available before the and Crew (n=4). a. Anywhere from an hour to 1.5 hours, depending on truck crew arrives. if we have the usual UPS driver. We come in at 7, the However, it cannot be (2) Observation - drivers don’t come in until 8:30 -TC#1 unloaded until the Observe unloading driver is ready. process and record b. When they have the normal driver, he’s usually times to completion. here about 8:30-9. When he was changing his job, the communication over at UPS was off - the driver would show up late, we would have to call UPS to find out Dock where the truck was. -TC #2 Management c. Depends on driver (8:15-10am).. Sometimes there is packages left in receiving from the day before that they can unpack until the driver is there TC#3 The truck trailer gets there before we do (it’s driven over and left - the truck cab goes off on another job). d. The truck trailer gets there before we do (it’s driven over and left - the truck cab goes off on another job). TC#4. (1) Observation Q6: How easy is it to sort the large packages so that Weaknesses in receiving Labels are colored, No package labeling all the packages from one sender are together? Are process depending on the type documentation exists. (2) Interviews - the labels consistent or do you run into problems There are 7 to 9 different of delivery. Management (n=3), with labels having different names for one sender? variations for any given ISBN No documentation exists Crew (n=4), and a. Senders sometimes put the wrong information (book sorted by type of Customers do not to inform customers of Package Programmers (n=2). (like name of individual rather than name of school) edition and physical always properly label how to properly label a or no information (they don’t understand they are condition). Returns may not packages. package. Labeling NOT the only vendor) -TC#1 be classified correctly by the store/buyback, or returns Sometimes customers c. The count can sometimes be off but as long as the received may not have forget to put labels on labels are there. On the majority, the labels are there originated at MBS in the packages. -TC #3 first place. Returns come in from stores, buyback When customers ship 34
  • d. It’s pretty simple after doing it for so long. Store contractors, and individuals multiple packages, UPS numbers are written on the packages, so we get used - labeling not consistent, does not always deliver to looking for store numbers to match up shipments. may not be accurate. them in the same Labels are mostly consistent, but I have seen Guaranteed buybacks pre- shipment. packages with label but no information or that have labeled for accuracy. just the sender’s first name and not the last, or boxes Department would like to Barcodes on the with no labels (it’s a sticky tag that may accidentally look into more bar code package label only get peeled off during shipment) -TC #4 information. Right now the include the UPS receiving crew has to input Shipping number. much information manually. Inconsistency in how No customer education bookstores label returns, so process exists at any receiving crew won’t have level of the company accurate information (so to inform shippers of billing for returns may be proper procedures. inaccurate because returns from a given store may not be processed as a group).- Operations Manager Problem with Invoicing of shipments, improper labeling, and improper counts - weaknesses are on the sender side. If sender information is correct, receiving goes like clockwork. Biggest problem is that improper identification. Truck Crew Supervisor (1) Extant Data - Q3: When you first started, was there any part of New hires should be None Available Review receiving and the unloading and check-in process that took a little comfortable within a 2-3Document shipping process more time to learn? How long did it take for you to week time frame, by then documentation. feel pretty comfortable doing your job? we’ll know if they can cut it.Standard a. Took about a week to get comfortable with the They receive OJT. Just theOperating (2) Interviews - entire process (including the computer). TC #1 hands-on responsibility. Go Management (n=3) b. It was all kind of new to me, so I learned one thing through proper liftingProcedure and Crew (n=4). at a time - there’s a lot to learn at once, so I had to methods (safety training, I (DSOP) break it down. Had to learn the difference between assume) . This (receiving what’s a return and what’s a guide. I’m used to having process) is a new beast even written directions/ guidelines, but all instruction here if you have dock experience. 35
  • was verbal and each person had his own way of doing Can be overwhelming even things. It would have helped if I had something to with experience. The read that was standardized. Without that, I wasn’t sorting is the hardest part of able to learn something until I actually needed to do the learning curve. (labeling, it. I didn’t think any particular process was more categorizing). The guys difficult to learn than another. I started at the already do categorization on beginning of a buyback rush so I was thrown into it - floor. Ebb and flow of learned the motions, wasn’t able to get more in- volume would not really depth until rush was over. -TC# 2 make a color coded floor c. Took about a week to get comfortable with the labeling system feasible -- entire process (including the computer). TC#3 Truck Crew Supervisor d. The unloading part. I’m not really computer savvy, but it’s easy to pick up. With unloading, had to learn the different shipment types. Took about two weeks tops to feel comfortable. TC #4 Q14: If you were in charge, how would you improve the efficiency of the receiving process? a. I don’t know if there’s really any way to improve (the people). Nothing I can think of offhand. TC #1 b. When bringing someone in - I can pick up things quickly because of my training but it might be hard for others. I see it as a weeding out process to see who they want to keep, but it would help to have a truck crew-specific training manual and job aid. Documented SOP - there’s isn’t one now. For example, a standardized way of setting up the pallets. TC #2 c. Current method is good as long as team works together. TC#3 d. The number one thing is space, then the timing for UPS arrival - we get here at 7 but he doesn’t get here until later. TC #4 (1) Extant Data - Setting benchmarks for None Available Review receiving and future evaluations; shipping metric establishing regular re- documentation. assessments of receivingMetrics processes; defining ideal (2) Interviews - performance for individuals; Management (n=3) defining ideal performance and Crew (n=4). for the receiving team; exporting elements of the 36
  • receiving processes to other parts of the organization; and/or developing evaluation processes for other parts of the organization. -Operations Manager Would like to see more metrics [in receiving process} such as copies checked per hour, book shelved per hour no formal written documentation - Inbound Manager37
  • Appendix F: Sub-dimension Weighting of Importance by Stakeholders Sub-Dimensions Inbound Manager Operations Manager Truck Crew Manager Hardware Extremely Important Important Extremely Important (non-electronic) Hardware (Computerized) Extremely Important Important Extremely Important Software Very Important Very Important Very Important Pallet Layout Important Important Extremely Important Utilization of Extremely Important Extremely Important Extremely Important Floor Space Space Available Extremely Important Very Important Extremely Important Dock Management Extremely Important Important Extremely Important Package Labeling Extremely Important Very Important Extremely Important Documented Standard Operating Important Important Important Procedure (DSOP) Metrics Extremely Important Very Important Important 38
  • Appendix G: Scoring Rubrics for MU UPS Receiving Process Evaluation Rubric for Ranking Importance Rubric for Ranking Sub-Dimensions Standard Important Very Important Extremely Important Poor Practices Best Practices Practices 1 2 3 1 2 3 Sum of Weighted Dimensions Weighting Value Sub-Dimension Dimension Weighting Sub-Dimension SDR Value (R) IxR (I) Rating (SDR) 1 Hardware (non- Standard Practices 2 2 electronic) INSTRUMENTS Important Hardware (Computerized) Standard Practices 2 2 1 1 Software Standard Practices 2 2 Sum of Importance and Rating 6 2 Pallet Layout Standard Practices 2 4 Utilization of WORKSPACE Very Important 2 Floor Space Standard Practices 2 4 2 Space Available Standard Practices 2 4 Sum of Importance and Rating 12 3 Dock Management Standard Practices 2 6 3 Package Labeling Standard Practices 2 6 ExtremelyPROCESS MANAGEMENT Important Documented Standard 3 Operating Procedure (DSOP) Poor Practices 1 3 3 Metrics Poor Practices 1 3 39 Sum of Importance and Rating 18 Total Sum of Weighted Dimensions 36
  • Score Combinations for Dimension Rating Dimension Lower (L)Sub-Dimension Weighting Poor (P) Mediocre (M) Acceptable (A) Very Good (V) Excellent (E) Boundary (W) Most sub-dimensions Most sub-dimensions Most sub-dimensions Most sub-dimensions receive a 1, and one receive a 2, and one receive a 2, and one receive a 3, and one All sub-dimensions All sub-dimensions sub-dimension sub-dimension sub-dimension sub-dimension received a 1 receive a 3 receives a 2 for each receives a 1 for each receives a 3 for each receives a 2 for each Dimension Dimension Dimension Dimension Hardware (non- 1 1 1 1 2 2 3 electronic) Hardware 1 1 1 2 2 3 3 (Computerized) Software 1 1 2 2 3 3 3 Pallet Layout 2 2 1 1 2 2 3Utilization of Floor 2 2 1 2 2 3 3 Space Space Available 2 2 2 2 3 3 3Dock Management 3 3 1 1 2 2 3 Package Labeling 3 3 1 2 2 3 3 DSOP 3 3 1 2 2 3 3 Metrics 3 3 2 2 3 3 3 40
  • Matrix for Calculating the Lowest Possible Score for Each Rating Level Dimension Sub- Mediocre Acceptable Very Good Excellent Weighting Lower (L) Boundary Poor (P) Dimension (M) (A) (V) (E) (W) Highest Highest Highest Highest possible score possible score possible score possible score Highest possible score for Highest possible score for each weighted dimension for each for each for each for each each weighted dimension (W x L) weighted weighted weighted weighted (W x P) dimension dimension dimension dimension (W x M) (W x A) (W x V) (W x E)Hardware (non- 1 1 1 1 2 2 3 electronic) Hardware 1 1 1 2 2 3 3 (electronic) Software 1 1 2 2 3 3 3 Pallet Layout 2 2 2 2 4 4 6 Utilization of 2 2 2 4 4 6 6 Floor SpaceSpace Available 2 2 4 4 6 6 6 Dock 3 3 3 3 6 6 9 ManagementPackage Labeling 3 3 3 6 6 9 9 DSOP 3 3 3 6 6 9 9 Metrics 3 3 6 6 9 9 9 Sum of Sub-Dimensions 21 27 36 48 57 63 41
  • Synthesized Rubric Levels to draw Evaluative Conclusion about OVERALL Evaluand Rating Description Rationale Poor (21-26) Score of total weighted dimensions is between 21-26 Most or all are Poor PracticesMediocre (27-35) Score of total weighted dimensions is between 27-35 Within each dimension, only one sub-dimension met Standard Practices, and others are Poor PracticesAcceptable (36-47) Score of total weighted dimensions is between 36-47 Within each dimension, only one sub-dimension is Poor Practices and others are Standard PracticesVery Good (48-56) Score of total weighted dimensions is between 48-56 Within each dimension, at least one sub-dimension was Best Practices and others were Standard PracticesExcellent (57-63) Score of total weighted dimensions is between 57-63 Within each dimension, most or all sub-dimensions were Best Practices and one was Standard Practices 42
  • Instruments Scoring RubricSub-Dimension Dimension Lower (L) Poor (P) Mediocre (M) Acceptable (A) Very Good (V) Excellent (E) Weighting (W) Boundary Most sub-dimensions Most sub-dimensions Most sub-dimensions Most sub-dimensions receive a 1, and one receive a 2, and one receive a 2, and one receive a 3, and one All sub-dimensions All sub-dimensions sub-dimension sub-dimension sub-dimension sub-dimension received a 1 receive a 3 receives a 2 for each receives a 1 for each receives a 3 for each receives a 2 for each Dimension Dimension Dimension Dimension Hardware (non- 1 1 1 1 2 2 3 electronic) Hardware 1 1 1 2 2 3 3 (Computerized) Software 1 1 2 2 3 3 3 Sum of sub-dimensions 3 4 5 7 8 9 43
  • Synthesized Rubric Levels to draw Evaluative Conclusion about INSTRUMENTS Rating Description Rationale Most or all are Poor Practices Poor (3) Score of total weighted sub-dimensions is 3 Within each dimension, only one sub- Mediocre (4) Score of total weighted sub-dimensions is 4 dimension met Standard Practices, and others are Poor Practices Within each dimension, only one sub- Score of total weighted sub-dimensions is between 5Acceptable (5 or 6) dimension is Poor Practices and others are or 6 Standard Practices Within each dimension, at least one sub- Very Good (7) Score of total weighted sub-dimensions is 7 dimension was Best Practices and others were Standard Practices Within each dimension, most or all sub- Excellent (8 or 9) Score of total weighted sub-dimensions is 8 or 9 dimensions were Best Practices and one was Standard Practices 44
  • Workspace Scoring Rubric Sub- Dimension Lower (L) Poor (P) Mediocre (M) Acceptable (A) Very Good (V) Excellent (E) Dimension Weighting (W) Boundary Most sub-dimensions Most sub-dimensions Most sub-dimensions Most sub-dimensions receive a 1, and one receive a 2, and one receive a 2, and receive a 3, and one All sub-dimensions All sub-dimensions sub-dimension sub-dimension one sub-dimension sub-dimension received a 1 receive a 3 receives a 2 for each receives a 1 for each receives a 3 for each receives a 2 for each Dimension Dimension Dimension Dimension Pallet Layout 2 2 2 2 4 4 6Utilization of Floor 2 2 2 4 4 6 6 Space Space Available 2 2 4 4 6 6 6 Sum of sub-dimensions 6 8 10 14 16 18 45
  • Synthesized Rubric Levels to draw Evaluative Conclusion about WORKSPACE Rating Description Rationale Poor (6 or 7) Score of total weighted sub-dimensions is 6 or 7 Most or all sub-dimensions are Poor Practices Only one sub-dimension was Standard Practices, and Mediocre (8 or 9) Score of total weighted sub-dimensions is 8 or 9 others are Poor Practices Score of total weighted sub-dimensions is Only one sub-dimension is Poor Practices and othersAcceptable (10-13) between 10 and 13 were Standard Practices Score of total weighted sub-dimensions is 14 or At least one sub-dimension was Best Practices andVery Good (14 or 15) 15 others were Standard Practices Score of total weighted sub-dimensions between Most or all sub-dimensions were Best Practices and Excellent (16-18) 16 and 18 one was Standard Practices 46
  • Process Management Scoring Rubric Sub- Dimension Lower (L) Poor (P) Mediocre (M) Acceptable (A) Very Good (V) Excellent (E) Dimension Weighting (W) Boundary Most sub- Most sub-dimensions Most sub-dimensions Most sub-dimensions dimensions receive receive a 2, and one receive a 2, and receive a 3, and one All sub-dimensions a 1, and one sub- All sub-dimensions sub-dimension one sub-dimension sub-dimension received a 1 dimension receives receive a 3 receives a 1 for each receives a 3 for each receives a 2 for each a 2 for each Dimension Dimension Dimension Dimension Dock 3 3 3 3 6 6 9Management Package 3 3 3 6 6 9 9 Labeling DSOP 3 3 3 6 6 9 9 Metrics 3 3 6 6 9 9 9 Sum of sub-dimensions 12 15 21 27 33 45 47
  • Synthesized Rubric Levels to draw Evaluative Conclusion about PROCESS MANAGEMENT Rating Description Rationale Score of total weighted sub-dimensions is Most or all are Poor Practices Poor (12-14) between 12 and 14 Score of total weighted sub-dimensions is Within each dimension, only one sub-dimension metMediocre (15-20) between 15 and 20 Standard Practices, and others are Poor Practices Score of total weighted sub-dimensions is Within each dimension, only one sub-dimension isAcceptable (21-26) between 21 and 26 Poor Practices and others are Standard Practices Within each dimension, at least one sub-dimension Score of total weighted sub-dimensions isVery Good (27-32) was Best Practices and others were Standard between 27 and 32 Practices Score of total weighted sub-dimensions is Within each dimension, most or all sub-dimensionsExcellent (33-45) were Best Practices and one was Standard Practices between 33 and 45 48