This presentation summarizes processes in evaluation student workforce performance for the HelpDesk at the University at Albany's Information Technology Services-Client Support Services. I presented this at the 2013 SUNY Technology Conference and the 2014 SUNYLA Empire Collaborations conferences.
4. Hiring process
• Post on student job
site and notify by
e-mail
• Technical
qualifications
• Review applications
…
Experience using University IT
services:
• MyUAlbany portal
• UAlbany Mail and Office 365
• Blackboard LMS
• UAlbanyWiFi wireless service
• UNIX services for file storage
or personal web site
General knowledge of:
• Windows operating system
• Macintosh operating system
• Web browsers
(IE, Firefox, Safari, Chrome)
• Microsoft Office
(Word, Excel, PowerPoint,
Outlook)
5. Application review and scoring
First
name
Last
name
Recom. GradDate
(≥5/2015)
Exp
sem
GPA
(2+)
Previous
work
Specific,
relevant skills
Skill
Work
Rel
Assist
Q
Diff
Cust
Q
Why
ITS Q
Writ
Qual
Score
(sum)
Inter
view?
A Z 2016-May 6 2.5 Lifeguard Excel 2 3 1 3 2 18 Perhaps
B Y MNO 2015-May 4 1.7 2 3 2 3 2 16.4 No
C X 2015-May 4 2.8 2 3 2 2 3 15.6 Perhaps
D W 2014-Dec 3 4.0 Retail OS X, Office 3 1 2 2 1 14 NoSem
E V 2015-May 4 2.5 2 3 3 3 3 17 Perhaps
F U 2015-May 4 3.5 2 3 2 2 3 17 Perhaps
G T JKL 2016-May 6 2.0 3 3 3 2 2 23 Yes
H S 2016-May 6 3.6 2 2 2 3 2 20.1 Yes
I R 2015-May 4 3.4 Clerical Malware clean 3 3 3 3 2 19.8 Yes
J Q 2015-May 4 2.7 2 3 3 2 3 16.4 No
K P 2014-May 2 2.9 3 2 3 2 3 13.8 NoSem
L O ABC 2015-May 4 3.1 2 3 3 3 2 20.2 Yes
M N 2014-Dec 3 3.0 3 3 3 3 2 17 NoSem
N M 2015-May 4 3.7 2 2 3 2 3 17.4 Perhaps
O L 2015-May 4 3.9 2 3 3 3 3 19.8 Yes
P J DEF 2014-May 2 2.9 2 1 1 2 2 10.8 NoSem
Q K 2016-May 6 3.3 2 3 2 2 2 19.6 Yes
6. Interviews and training
• Student staff panel
• Professional staff meeting
• Discussion and consensus
12. Performance statistics
• Tickets/interactions recorded
• Net substitute shifts
• Incidence of late or no show
• Project work and KB articles
• Hours worked
• Tickets per hour
• Net subs per hour
13. Performance metric
Summed semester data Weekly collected data
8/3/2012
Fname Lname Metric Busy Slow Temp Perm Projects Late/MIA Busy Slow Temp Perm Projects Late/MIA
A Z -6.75 11 6 -46 -3 0 5 4 -6 0 0
B Y 16 38 3 -26 2 0 2 -5 0 0
C X 55.5 92 1 1 0 3 9 15 0 0 1 0
D W -1.75 3 10 -41 0 0 4 -6 0 0
E V 2 13 1 -16 0 0 2 0 0 0
F U 24.5 35 2 -19 0 3 7 4 0 0 3 0
G T 9.75 26 7 -9 2 1 20 -1 0 0
H S 15.75 45 3 -7 1 0 12 2 0 0 1
I R 18.25 26 0 1 0 1 0 4 1 0 0
J Q 50.25 34 0 39 0 5 2 0 0 2 0
K P 2.25 9 1 35 -3 0 12 4 3 0 0
L O 21.75 16 4 5 0 2 2 0 0 0
M N 16.75 35 0 -69 1 4 6 9 -1 0 1 1
N M 41.5 19 9 51 9 1 5 3 6 3 0
O L 22.75 21 13 6 -2 1 7 8 0 0 0
P J 5 9 13 -35 1 1 13 3 -1 2 1 0
Q K 18 43 5 -29 1 0 3 3 2 0 0
14. Performance highlights
Fname Lname Hours Incidents Inc/hour Net subs Subs/hour Late/MIA Metric
A Z 158.75 17 0.107 -44 -0.277 5.00 -5.50
B Y 221.75 41 0.185 -11 -0.050 2.00 19.25
C X 276.25 93 0.337 -1 -0.004 9.00 50.00
D W 178.75 13 0.073 -35 -0.196 3.00 0.50
E V 151.25 14 0.093 -16 -0.106 2.00 2.00
F U 112.25 37 0.330 -17 -0.151 7.00 10.00
G T 62.00 33 0.532 -4 -0.065 20.00 10.50
H S 197.75 48 0.243 -6 -0.030 10.00 17.25
I R 52.00 26 0.500 -2 -0.038 0.00 17.50
J Q 320.25 34 0.106 37 0.116 2.00 39.75
K P 129.00 10 0.078 29 0.225 12.00 1.50
L O 190.75 20 0.105 4 0.021 2.00 21.50
M N 215.75 35 0.162 -67 -0.311 5.00 12.75
N M 191.50 28 0.146 47 0.245 4.00 36.75
O L 260.75 34 0.130 3 0.012 7.00 22.50
P J 160.25 22 0.137 -34 -0.212 13.00 -1.50
Q K 247.75 48 0.194 -21 -0.085 2.00 20.50
By Mwtoews [CC-BY-2.5 (http://creativecommons.org/licenses/by/2.5)],
via Wikimedia Commons
16. Data sources and analysis
• Ticket tracking reports (FootPrints)
• Interactions recorded
• Scheduling notifications (ConPortal, Outlook)
• Substitute shifts (via e-mail)
• Late or no-show (via e-mail)
• Hours worked
• Excel for computation and summary
UAlbany statistics
>17,000 students
>6,000 employees
>155,000 alumni
ITS/HelpDesk services/staff
ITS ~130 employees, HD 4 FTE
Support University services (e-mail, wireless, network, Blackboard, portal, etc.) and some desktop
Me
10 years in IT
2 years as student staff
4 years managing students
Student staff/the job
~25 employees
Walk up support of ITS services
Cover remote locations, late hours
Public face of ITS
New technology and issues
IC support (training and troubleshooting)
Service point similarities
Multiple locations
Mostly or only students with some supervision
Late hours
Variety of services and questions
Track some work in addition to transactions, like checkouts
Posting
On portal’s job site and via e-mail
Qualifications are light
~75 applications for ~5-10 spots
Review/grade applicants
Criteria
Expected work time (grad. date)
GPA
Work experience
Relevant skills
Responses to application questions on customer service, why us, and overall writing quality
Interview and getting started
Students staff panel and manager meeting
Discover interest, attitude, personality, and some technical aptitude
Hire for responsible/personable > capable
You can teach skills, not personality
Discuss results for consensus
Set expectations
Shadowing
Probationary hire
Peer mentoring
Clearly defined expectations, rewards, and penalties
Handbook
Boot Camp
Correction
Instructive, flexible, applied, timely
Examples
Instruct responsibility to correct tardy
Strikes for repetition
Escalate for severe
Types and content
No surprises – should all be known already
Ongoing short notes of +/- (correct/reinforce
Periodic, detailed
Collect feedback
ITS staff
Customers
Library staff
Impressions/soft skills
Viewing interactions while on shift
Quality of submitted incidents/interactions
Templates for consistency
Responses to staff requests
List and describe each as needed
Performance metric
Weighted sum of statistics
.5xBusy + 1xSlow + .25xTemp + 1xPerm + 5xProjects - .75xLate/MIA
Share some data to create competition for prize
Highlight outliers via mean +/- std. dev.
General correlation with observed performance quality.
Significant? I hope so.
Summarize statistics and sources.
All of it is (almost) freely available for use by some method.