• Save
Fitness for Use: Present or Future? Insights on Utility as a Primary Dimension of Quality in e-Learning
Upcoming SlideShare
Loading in...5
×
 

Fitness for Use: Present or Future? Insights on Utility as a Primary Dimension of Quality in e-Learning

on

  • 2,877 views

 

Statistics

Views

Total Views
2,877
Views on SlideShare
2,874
Embed Views
3

Actions

Likes
3
Downloads
0
Comments
0

1 Embed 3

http://www.slideshare.net 3

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Fitness for Use: Present or Future? Insights on Utility as a Primary Dimension of Quality in e-Learning Fitness for Use: Present or Future? Insights on Utility as a Primary Dimension of Quality in e-Learning Presentation Transcript

  • Fitness for Use: Present or Future?  Insights on U3lity as a Primary Dimension of  Quality in e‐Learning  IASK - Teaching and Learning 2009 Conference – Porto, December 9, 2009 Rosário Cação  António Dias de Figueiredo  1 
  • Agenda  1. Research Ques3ons  2. Case Study  3. Conclusions  2 
  • 1. RESEARCH QUESTIONS  3 
  • 1. Research Ques;ons  Starting points: •  In educational contexts, quality is related to the ability to satisfy assessed needs. •  Those needs reflect ends-in-view (Dewey, 1939): expected changes or uses. •  Quality is fitness for use (Juran, 1951). •  Utility reactions can be used as an indicator of the possibility of use (Ruona, Leimbach, Holton, & Bates, 2002). Research questions: •  Are perceptions of utility relevant dimensions of quality in e-learning courses? •  Are future uses more valued than immediate uses? 4 
  • 2. CASE STUDY  5 
  • 2. Case Study  www.evolui.com A Portuguese provider of asynchronous e-learning for professional training, with ten years of experience in the consumer e-learning market. •  Has 50.000 clients from 26 countries. •  Offers more than 160 short-term courses. •  The courses take from 1 to 9 weeks to be completed. The subjects of the study were participants in an online training course that they had paid. 6 
  • 2. Case Study  Figure: Three ways of mixing quan3ta3ve and qualita3ve data  7  Source: Creswell & Clark, 2007, p. 7 
  • 2. Case Study  Qualita3ve study  Quan3ta3ve studies  •  Respondents: training professionals •  Pedagogical courses •  Researcher as trainer •  Participant observation and interviewing •  August 2007 – June 2008 •  Collected: 2398 messages posted by 210 trainees •  Used: 104 messages from 44 customers – the theoretical saturation point (Glaser & Strauss, 1967) 8 
  • 2. Case Study  •  Learning transfer •  Increase of performance •  Return of investment •  Behavior changes •  Future utility •  Practical uses •  Professional utility •  Motivation for future learning •  Use deadline •  Motivation •  Trainees •  Knowledge •  Transfer of knowledge •  Organizational impact Node to be explored: U3lity   •  Applicability •  Use Short‐term vs. long‐term u3lity  •  Changes in behavior 9 
  • 2. Case Study  Qualita3ve study  Quan3ta3ve studies  •  Satisfaction survey (in the end of the course) •  Single-item survey with 15 variables in a 1-10 Likert scale •  March 2008 – February 2009 •  2741 answers •  145 courses •  1085 unique customers •  Reliability (internal consistency) - Cronbach’s alpha: .963 (very good) 10 
  • 2. Case Study  U;li;es  Future vs. immediate  Types of courses  Gender differences  Future utility is higher than immediate utility. Quality  Future: Mo;va;on  Mean: 8,39 Median: 9 Special case  Immediate: Mean: 8,2 Median: 9 Wilcoxon Test on the Equality of the Means of the Utilities Ho: Immediate utility is equal to future utility H1: immediate utility is different from future utility With a 95% confidence future utility is perceived as higher than immediate utility (p value = 0). 11 
  • 2. Case Study  U;li;es  Future vs. immediate  Types of courses  Gender differences  Quality  Regular vocational Certification courses e-learning courses Mo;va;on  Special case  •  Both utilities were high in both kinds of courses. •  Future utility is higher in certification courses but immediate utility is lower (Mann-Whitney test). 12 
  • 2. Case Study  U;li;es  Future vs. immediate  Types of courses  Gender differences  •  Women have higher perceptions of both utilities Quality  •  Future utility is higher than immediate utility for both groups •  The difference between utilities is wider among male users Mo;va;on  Special case  Women  Men  p value = 0 13 
  • 2. Case Study  U;li;es  General percep;ons  Regression  Factor analysis  •  Quality perceptions have a positive Kurtose, a Quality  negative skewness and an average of 8.35 out of 10. Mo;va;on  •  Female users have a higher perception of quality but an equal perception of quality-price relation. Special case  •  Quality is highly correlated with global satisfaction (.8), immediate utility (.67), and future utility (.86). These variables must be relevant to explain quality, even in a linear regression model. 14 
  • 2. Case Study  U;li;es  General percep;ons  Regression  Factor analysis  stepwise method, R2 = .85 Quality  Y = a + b1 X1 + b2 X2 + b3 X3 + b4 X4 + b5 X5 + b6 X6 + b7 X7 + b8 X8 + b9 X9 + b10 X10 + b11 X11 Mo;va;on  Y = Global quality X1 = Global satisfaction X2 = Future utility Special case  X3 = Training contents X4 = Quality-price relation X5 = Competence, kindness and promptness of the staff X6 = Fulfillment of expectations X7 = The trainer's expertise X8 = The platform and its functions X9 = Initial motivation Excluded variables: X10 = Final motivation •  Fulfillment of training objectives X11 = Immediate utility •  The contribution of the forum for the learning process •  The dynamics and help provided by the tutor 15 
  • 2. Case Study  U;li;es  General percep;ons  Regression  Factor analysis  Y = .3 + .125 X1 + .169 X2 + .152 X3 + .133 X4 + .83 X5 + .122 Quality  X6 + .7 X7 + .057 X8 - .045 X9 + .069 X10 + .044 X11 Y = Global quality X1 = Global satisfaction Mo;va;on  •  Alone, satisfaction explains X2 = Future utility X3 = Training contents 71.6% of the variability of quality X4 = Quality-price relation X5 = Competence, kindness and Special case  promptness of the staff •  Future utility is the second X6 = Fulfillment of expectations most important contributor X7 = The trainer's expertise X8 = The platform and its functions X9 = Initial motivation X10 = Final motivation •  Immediate utility and final X11 = Immediate utility motivation provide low contributions to explain quality. •  Quality is a long-term attitude 16 
  • 2. Case Study  U;li;es  General percep;ons  Regression  Factor analysis  Quality factors: Quality  Mo;va;on  Special case  17 
  • 2. Case Study  U;li;es  General interest  Course dura;on  Does quality depends on who pays the bill? Quality  (Mann-Whitney tests) •  All variables have higher ratings when the Mo;va;on  course is paid by the trainee. Special case  •  Initial motivation has a wider distribution when the course is paid by the company where the trainee works. •  When the trainee does not pay anything to attend the course, she has a lower perception on quality- price relation, most probably due to a lower quality perception. 18 
  • 2. Case Study  U;li;es  General interest  Course dura;on  Why Look Deeper into Motivation? Quality  If a person cannot foresee the consequences of his act (…), it is impossible for him to guide his act intelligently. (Dewey, 1916, p. 24) Mo;va;on  •  Initial motivation helps to explain quality (regression). Special case  •  Both initial and final motivation are considered in the attitude factor (factor analysis). •  Correlation initial-final motivation: .5 •  Final motivation helps to explain global satisfaction (fulfillment of expectations is the most contributor) (R2 = .873). 19 
  • 2. Case Study  U;li;es  General interest  Course dura;on  The evolution of final motivation Quality  The final motivation of courses which length is two weeks is Mo;va;on  different from the final motivation of shorter and longer courses. Special case  Final motivation is optimized with two week courses. Those courses also maximize global satisfaction, immediate, and future utility … and quality perceptions. 20 
  • 2. Case Study  The special case of the courses in initial U;li;es  certification of trainers Promoter: EVOLUI.COM Quality  p-value Decision Conclusion Factor 1: Training Process 8. The trainer’s expertise .137 Not Reject May be equal Mo;va;on  9. The contribution of the forum to the learning process .214 Not Reject May be equal 10. The dynamics and help provided by the trainer in the .103 Not Reject May be equal forum 11. Competence, kindness and promptness of the staff .000 Reject Are different Special case  6. Platform and its functions .774 Not Reject May be equal Factor 2: Training Attitudes 1. Global satisfaction .001 Reject Are different 2. Expectations fulfilment .000 Reject Are different 4. Final motivation .000 Reject Are different 5. Fulfilment of training objectives .041 Reject Are different Factor 3: Training Utility 12. Immediate utility .000 Reject Are different 13. Future utility .000 Reject Are different 21 
  • CONCLUSIONS  22 
  • Conclusions  •  Future use is the second most important dimension of  quality, following sa3sfac3on.   –  Quality can be defined as fitness for future use.  •  Whenever training reac3ons are studied, immediate and  future u3lity percep3on and ini3al and final mo3va3ons  should also be addressed, besides affec3ve reac3ons.  •  The dura3on of e‐learning‐based professional training  courses is an important factor to be taken into account.  –  Two week courses op3mize the percep3ons of quality and final  mo3va3on, in the case we studied.  23 
  • Slides available at www.slideshare.net/rosariocacao Fitness for Use: Present or Future?  Insights on U3lity as a Primary Dimension of  Quality in e‐Learning  Teaching and Learning 2009 – Oporto, December 9, 2009 Rosário Cação (mrac@dei.uc.pt)  António Dias de Figueiredo (adf@dei.uc.pt)  24