Fitness for Use: Present or Future? Insights on Utility as a Primary Dimension of Quality in e-Learning

Fitness for Use: Present or Future? Insights on U3lity as a Primary Dimension of Quality in e‐Learning IASK - Teaching and Learning 2009 Conference – Porto, December 9, 2009 Rosário Cação António Dias de Figueiredo 1

Agenda 1. Research Ques3ons 2. Case Study 3. Conclusions 2

1. RESEARCH QUESTIONS 3

1. Research Ques;ons Starting points: •  In educational contexts, quality is related to the ability to satisfy assessed needs. •  Those needs reflect ends-in-view (Dewey, 1939): expected changes or uses. •  Quality is fitness for use (Juran, 1951). •  Utility reactions can be used as an indicator of the possibility of use (Ruona, Leimbach, Holton, & Bates, 2002). Research questions: •  Are perceptions of utility relevant dimensions of quality in e-learning courses? •  Are future uses more valued than immediate uses? 4

2. CASE STUDY 5

2. Case Study www.evolui.com A Portuguese provider of asynchronous e-learning for professional training, with ten years of experience in the consumer e-learning market. •  Has 50.000 clients from 26 countries. •  Offers more than 160 short-term courses. •  The courses take from 1 to 9 weeks to be completed. The subjects of the study were participants in an online training course that they had paid. 6

2. Case Study Figure: Three ways of mixing quan3ta3ve and qualita3ve data 7 Source: Creswell & Clark, 2007, p. 7

2. Case Study Qualita3ve study Quan3ta3ve studies •  Respondents: training professionals •  Pedagogical courses •  Researcher as trainer •  Participant observation and interviewing •  August 2007 – June 2008 •  Collected: 2398 messages posted by 210 trainees •  Used: 104 messages from 44 customers – the theoretical saturation point (Glaser & Strauss, 1967) 8

2. Case Study •  Learning transfer •  Increase of performance •  Return of investment •  Behavior changes •  Future utility •  Practical uses •  Professional utility •  Motivation for future learning •  Use deadline •  Motivation •  Trainees •  Knowledge •  Transfer of knowledge •  Organizational impact Node to be explored: U3lity •  Applicability •  Use Short‐term vs. long‐term u3lity •  Changes in behavior 9

2. Case Study Qualita3ve study Quan3ta3ve studies •  Satisfaction survey (in the end of the course) •  Single-item survey with 15 variables in a 1-10 Likert scale •  March 2008 – February 2009 •  2741 answers •  145 courses •  1085 unique customers •  Reliability (internal consistency) - Cronbach’s alpha: .963 (very good) 10

2. Case Study U;li;es Future vs. immediate Types of courses Gender diﬀerences Future utility is higher than immediate utility. Quality Future: Mo;va;on Mean: 8,39 Median: 9 Special case Immediate: Mean: 8,2 Median: 9 Wilcoxon Test on the Equality of the Means of the Utilities Ho: Immediate utility is equal to future utility H1: immediate utility is different from future utility With a 95% confidence future utility is perceived as higher than immediate utility (p value = 0). 11

2. Case Study U;li;es Future vs. immediate Types of courses Gender diﬀerences Quality Regular vocational Certification courses e-learning courses Mo;va;on Special case •  Both utilities were high in both kinds of courses. •  Future utility is higher in certification courses but immediate utility is lower (Mann-Whitney test). 12

2. Case Study U;li;es Future vs. immediate Types of courses Gender diﬀerences •  Women have higher perceptions of both utilities Quality •  Future utility is higher than immediate utility for both groups •  The difference between utilities is wider among male users Mo;va;on Special case Women Men p value = 0 13

2. Case Study U;li;es General percep;ons Regression Factor analysis •  Quality perceptions have a positive Kurtose, a Quality negative skewness and an average of 8.35 out of 10. Mo;va;on •  Female users have a higher perception of quality but an equal perception of quality-price relation. Special case •  Quality is highly correlated with global satisfaction (.8), immediate utility (.67), and future utility (.86). These variables must be relevant to explain quality, even in a linear regression model. 14

2. Case Study U;li;es General percep;ons Regression Factor analysis stepwise method, R2 = .85 Quality Y = a + b1 X1 + b2 X2 + b3 X3 + b4 X4 + b5 X5 + b6 X6 + b7 X7 + b8 X8 + b9 X9 + b10 X10 + b11 X11 Mo;va;on Y = Global quality X1 = Global satisfaction X2 = Future utility Special case X3 = Training contents X4 = Quality-price relation X5 = Competence, kindness and promptness of the staff X6 = Fulfillment of expectations X7 = The trainer's expertise X8 = The platform and its functions X9 = Initial motivation Excluded variables: X10 = Final motivation •  Fulfillment of training objectives X11 = Immediate utility •  The contribution of the forum for the learning process •  The dynamics and help provided by the tutor 15

2. Case Study U;li;es General percep;ons Regression Factor analysis Y = .3 + .125 X1 + .169 X2 + .152 X3 + .133 X4 + .83 X5 + .122 Quality X6 + .7 X7 + .057 X8 - .045 X9 + .069 X10 + .044 X11 Y = Global quality X1 = Global satisfaction Mo;va;on •  Alone, satisfaction explains X2 = Future utility X3 = Training contents 71.6% of the variability of quality X4 = Quality-price relation X5 = Competence, kindness and Special case promptness of the staff •  Future utility is the second X6 = Fulfillment of expectations most important contributor X7 = The trainer's expertise X8 = The platform and its functions X9 = Initial motivation X10 = Final motivation •  Immediate utility and final X11 = Immediate utility motivation provide low contributions to explain quality. •  Quality is a long-term attitude 16

2. Case Study U;li;es General percep;ons Regression Factor analysis Quality factors: Quality Mo;va;on Special case 17

2. Case Study U;li;es General interest Course dura;on Does quality depends on who pays the bill? Quality (Mann-Whitney tests) •  All variables have higher ratings when the Mo;va;on course is paid by the trainee. Special case •  Initial motivation has a wider distribution when the course is paid by the company where the trainee works. •  When the trainee does not pay anything to attend the course, she has a lower perception on quality- price relation, most probably due to a lower quality perception. 18

2. Case Study U;li;es General interest Course dura;on Why Look Deeper into Motivation? Quality If a person cannot foresee the consequences of his act (…), it is impossible for him to guide his act intelligently. (Dewey, 1916, p. 24) Mo;va;on •  Initial motivation helps to explain quality (regression). Special case •  Both initial and final motivation are considered in the attitude factor (factor analysis). •  Correlation initial-final motivation: .5 •  Final motivation helps to explain global satisfaction (fulfillment of expectations is the most contributor) (R2 = .873). 19

2. Case Study U;li;es General interest Course dura;on The evolution of final motivation Quality The final motivation of courses which length is two weeks is Mo;va;on different from the final motivation of shorter and longer courses. Special case Final motivation is optimized with two week courses. Those courses also maximize global satisfaction, immediate, and future utility … and quality perceptions. 20

2. Case Study The special case of the courses in initial U;li;es certification of trainers Promoter: EVOLUI.COM Quality p-value Decision Conclusion Factor 1: Training Process 8. The trainer’s expertise .137 Not Reject May be equal Mo;va;on 9. The contribution of the forum to the learning process .214 Not Reject May be equal 10. The dynamics and help provided by the trainer in the .103 Not Reject May be equal forum 11. Competence, kindness and promptness of the staff .000 Reject Are different Special case 6. Platform and its functions .774 Not Reject May be equal Factor 2: Training Attitudes 1. Global satisfaction .001 Reject Are different 2. Expectations fulfilment .000 Reject Are different 4. Final motivation .000 Reject Are different 5. Fulfilment of training objectives .041 Reject Are different Factor 3: Training Utility 12. Immediate utility .000 Reject Are different 13. Future utility .000 Reject Are different 21

CONCLUSIONS 22

Conclusions •  Future use is the second most important dimension of quality, following sa3sfac3on. –  Quality can be defined as fitness for future use. •  Whenever training reac3ons are studied, immediate and future u3lity percep3on and ini3al and final mo3va3ons should also be addressed, besides affec3ve reac3ons. •  The dura3on of e‐learning‐based professional training courses is an important factor to be taken into account. –  Two week courses op3mize the percep3ons of quality and final mo3va3on, in the case we studied. 23

Slides available at www.slideshare.net/rosariocacao Fitness for Use: Present or Future? Insights on U3lity as a Primary Dimension of Quality in e‐Learning Teaching and Learning 2009 – Oporto, December 9, 2009 Rosário Cação (mrac@dei.uc.pt) António Dias de Figueiredo (adf@dei.uc.pt) 24

Fitness for Use: Present or Future? Insights on Utility as a Primary Dimension of Quality in e-Learning

by Rosário Cação on Dec 09, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

Embed Views

Statistics

Fitness for Use: Present or Future? Insights on Utility as a Primary Dimension of Quality in e-Learning — Presentation Transcript

Get analytics right now with SlideShare PRO

An effective way to find out who is viewing your content

Remove ads from all your pages with SlideShare PRO

Your viewers will engage with your content, not ads