Insights from customer needs and satisfaction at UCF's FEEDS ...

  • 1,157 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,157
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
11
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Created as a result of Florida’s industry leadership requesting flexible engineering education to support working professionals. It was a strategy for Florida based technology companies to keep pace with new technologies, maintain adequate skill and eliminate the risk of loss of future business and job loss.

Transcript

  • 1. Insights from customer needs and satisfaction at UCF’s FEEDS program Results from the Voice of the Customer component of a Six Sigma methodology project. Roberto Champney 1 Alain Gaumier 2 Naomi Morris 1 1 University of Central Florida 2 Optimum Consulting, Inc.
  • 2. Objectives
    • Project Background and Description
    • Six-Sigma methodology overview
    • Voice of the Customer (VOC)
    • Results and Insights From VOC
    • Recommendations
  • 3. Project Background
    • FEEDS- Florida Engineering Education Delivery System.
    • Created in 1982 by the Florida State Legislature
    • The system has evolved from a tape system in 1982 to a CD and web-based education option in its current form.
    • The University of Central Florida (UCF) is one of 12 universities in the state that participates in the program, and holds the largest enrollment, 38% (2002-2003).
  • 4. Project Background
    • ASQ (American Society For Quality) Community Good Works Project.
      • Improve communities through the use of quality tools and technologies.
      • Provides skills transfer to non-traditional quality practitioners.
    • Project Goals
        • Document the fall 2003 version of the FEEDS system.
        • Identify Priority Process Improvements.
  • 5. Six-Sigma methodology overview DMAIC Define Improve Control Measure Analyze
  • 6. Classical View of Performance
    • Practical Meaning of “99% Good”:
      • 20,000 lost articles of mail per hour
      • 15 minutes of unsafe drinking water each day
      • 5,000 incorrect surgical operations per week
      • 2 short or long landings at most major airports each day
      • 200,000 wrong drug prescriptions each year
      • No electricity for almost 7 hours each month
    3 Sigma 4 Sigma 6 Sigma 93.32% 99.38% 99.99999% Historical Standard Current Standard New Standard Capability Long-Term Yield Standard
  • 7. Voice of the Customer (VOC)
    • Technically there are numerous ways to enhance the system, yet there has been little attempt to focus this enhancement to represent the needs of its users.
      • Identify customers and their needs.
      • Collect and analyze reactive data.
      • Consider proactive approaches.
      • Convert collected data into customer needs.
      • Sort out the most important attributes “ ctq ”.
      • Obtain specifications for the critical-to-quality characteristics.
  • 8.
    • Conducted a student focus group.
    • Conducted several interviews with FEEDS students to:
      • Capture major needs categories and issues.
      • Further understand FEEDS users.
    • Developed a survey concerning the satisfaction and needs of FEEDS student users.
    • End of Semester FEEDS Student Online Survey.
      • Set adequate performance levels.
      • Correlations to identify key drivers.
      • Align opportunities for improvement with FEEDS UCF priorities.
    Voice of the Customer (VOC)
  • 9. The Surveys
    • Two surveys over a two semester period
    • Interview comments were used to create the survey questions:
      • 30 and 39 questions
      • Some demographics.
        • gender, study program, program level.
      • Three types of questions (5-point likert scale and open ended).
        • Satisfaction with current attributes of the system.
        • Usefulness and Importance of particular attributes of the system (hypothetical and current).
        • Comments on particular issues with the system.
  • 10. The Survey (Sample Questions)
    • Overall how satisfied are you with the FEEDS system?   
    • Overall how effective are FEEDS courses compared to live courses?
    • How satisfied are you with the quality of the video stream (vision and audio) on your FEEDS courses?
      • Audio
      • Image
    • How satisfied are you with the communication with FEEDS professors?
  • 11. The Survey (Validating Sample)
    • Response Rate: 338 (1 st survey) and (149 2 nd Survey)
      • Mass email, and site banner.
    • Percentages represent actual FEEDS student body distribution
    100.00 100.00 Total 3.38 8.55 Other 13.51 12.09 Electrical Eng 6.08 7.96 Computer Science 12.84 10.32 Computer Eng 8.78 8.85 Civil Eng 21.62 18.29 Eng Technology 4.05 9.14 Mechanical Eng 20.95 17.70 Industrial Eng 4.05 3.24 Environmental Eng 4.73 3.83 Aerospace Eng SPRING '04 FALL '03 Percent (%)
  • 12. Insights From VOC 0.012 0.206 Notes Before Class satisfaction 0.247 0.258 How Consistent is Delivery of Course Material 0.187 0.334 Response Time From Professor / Satisfaction Correlations are significant at the 0.01 level (2-tailed). -0.080 -0.126 -0.166 -0.251 Response Time From Professor 0.353 0.330 0.434 0.439 Audio Quality Satisfaction 0.366 0.365 0.381 0.511 Image Quality Satisfaction 0.320 0.522 0.444 0.649 Communication With Professor / Satisfaction 0.539 0.690 FEEDS Satisfaction 0.539 0.690 FEEDS Effectiveness Spng '04 Fall '03 Spng '04 Fall '03 Effectiveness Satisfaction Survey Questions
  • 13. Insights From VOC Overall Satisfaction Comments 69% and 54% of respondents gave comments
    • First Survey
      • 30 % positive comments
      • 41 % negative comments
    • Second Survey
      • 39% positive comments
      • 19% negative comments
    Note: one respondent may have contributed to more than one category of comments.
  • 14. Insights From VOC
    • On Spring ’04 UCF FEEDS upgraded their stream format.
    1.32 3.58 Response time from professor, satisfaction 1.18 3.65 1.19 3.65 Communication with professor, satisfaction 1.18 3.49 Notes before class satisfaction 1.16 3.39 How Consistent is Delivery of Course Material 1.13 3.80 1.17 3.63 Sound Quality 1.15 3.61 1.26 3.45 Image Quality 0.85 4.14 1.05 3.97 FEEDS Satisfaction 1.03 4.01 1.11 3.80 FEEDS Effectiveness* SD Mean SD Mean Spring '04 Fall '03
  • 15. Insights From VOC
    • Prioritize future efforts using differences between ‘received’ and ‘desired’
      • See SERVQUAL survey method.
      • Delivery of Service Quality," Journal of Marketing, April 1988, pp. 35-48. Parasuraman , Zeithaml and Berry , " SERVQUAL : A Multiple-Item Scale for Measuring Customer Perceptions of Service Quality," Journal of Retailing, Spring 1988, pp. 12-40
    0.11 -0.09 Availability of Stream   0.13 -1.04 Notes Before Class 0.06 -0.07 Course Material Consistency 0.14 -0.98 Assignment Confirmation 0.11 -0.86 Video Image 0.11 -0.70 Video Audio SPRING '04 Std. Error Statistic   Mean   Difference        
  • 16. Insights From VOC
    • How often are you able to attend a live class?
      • 29% to 39% always (4 & 5 scores)
    • For what reasons are you not able to ALWAYS attend a live class?
    20 Work 23 Class Schedule 40 Other 44 Convenience 54 Distance 81 Personal Schedule Count  
  • 17. Insights From VOC
    • Please indicate all the mediums with which you communicate with FEEDS professors? (Check the two most frequent)
    2.0 Other 31.8 Phone 39.9 WebCt 49.3 Person 91.2 Email Percent
  • 18. Insights From VOC
    • How long does it takes on average to get a response from your professor?
    • How satisfied are you with how long it takes to get a response from your professor?
      • 55 % satisfied (4&5 score)
  • 19. Insights From VOC
    • Where do you regularly watch the
    • FEEDS stream or CD? (Check the
    • two most frequent)
      • Home 126
      • Work 47
      • Library 17
      • Computer Lab 18
      • Other 8
      • (Beach, Starbucks, Outdoors)
    In the event that you could have access to the class notes before the class session, what would be the best way to gain access to them? 2.6 Buy Bookstore 0.5 Other 29.4 Email 67.5 Download Percent Notes access preference
  • 20. Insights From VOC
    • How often have you experienced missing material from a FEEDS stream?
    • In the event that you have experienced missing material from a FEEDS stream, what part was missing?
  • 21. VOC Instrument Recommendations
    • ~5 interviews will identify most of the problems.
    • Don’t get carried away with questions, you’ll have to make sense of them later.
    • Everything is important or useful.
      • Be careful with these types of questions.
      • Ex. Teacher assistant meetings:
        • Usefulness score: 4.07
        • Attend score: 2.79
  • 22. VOC Instrument Recommendations
    • Open ended comments are most valuable, but very time consuming.
      • Use at least two persons for categorization.
    • Provide enough space for comments.
      • 255 characters is not enough
    • Use online surveys, don’t send attachment surveys
  • 23. Insight Summary
    • Identify Student Strata
      • 100% remote students.
        • Miss student interaction.
        • Miss out on after class talk.
        • Value recording of ‘pre-class’ and ‘break’ periods
      • Convenience students
        • Register for FEEDS but attend live classes.
        • Use FEEDS as backup or study aid.
      • Gender differences
        • Females more than males value the interaction with other students and professors (per comments).
      • Study Program differences
        • Teacher Assistant meetings, and problem solving groups.
  • 24. Insight Summary
    • User satisfaction and system effectiveness are correlated.
      • Effectiveness is driving satisfaction.
    • Communication with professors is critical to satisfaction and effectiveness.
    • ~1/3 of students regularly attend live classes
    • Email and in-person communication is used mostly for communication between FEEDS students and professors.
  • 25. Insight Summary
    • Provide access to class notes before class, when possible.
    • Consider scanning professor hand notes if possible.
    • Chalk/White Board is not appropriate for this medium.
    • Live students should see the same image as Remote students.
      • They’ll become the ‘directors’ of the class production.
  • 26. Insight Summary
    • Standardize equipment and training.
      • Encourage ( enforce ) professor training.
    • Consider standardizing method of delivery and procedures (i.e. all use WebCT, all email assignments).
      • Use one location for files, content, or materials access.
      • Need commitment from professors.
  • 27. Insight Summary
    • Computer demos are particularly difficult to follow (unable to read buttons and options).
    • Most of students’ problems with the ‘stream’ are with accessing it due to technical difficulties.
    • Provide content online, consider avoiding mailings (they consume resources).
  • 28. Insight Summary
    • Other online services or university sections confused as part of FEEDS (i.e. WebCT).
    • Apply VOC methodology with other stakeholders
      • Professors, Administrators, etc.
  • 29. Questions ?