Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Measuring worker productivity:  outcome or cost?  <ul><li>Dorcas Beaton, BScOT,  PhD </li></ul><ul><li>Health Policy Manag...
Perspective: OMERACT <ul><li>O utcome  Me asurement in  R heum a tological  C linical  T rials  (est. 1992) </li></ul><ul>...
Decision making – OMERACT Style <ul><li>Data driven decisions: </li></ul><ul><ul><li>OMERACT Filter  (Boers, J Rheum1998) ...
Challenges and advances- so far!  <ul><li>Challenges </li></ul><ul><ul><li>Defining our target  </li></ul></ul><ul><ul><li...
Challenge #1: Defining our target <ul><li>Perspective:  </li></ul><ul><li>Work place  productivity = productivity at level...
Interest in Work in arthritis research <ul><li>As an  outcome  state </li></ul><ul><ul><li>Off work / RTW (absenteeism) </...
Target: the impact of arthritis on work at the level of the worker Goal:  to have solid indicators of each of these cells....
Challenge #2 Contextual nature of work <ul><li>Difficulty experienced with work is, in part, defined by the job.  </li></u...
Context of work <ul><li>At an individual level  we can sort out context </li></ul><ul><li>At a group level, numbers from o...
Contextual factors to be considered  <ul><li>Environment </li></ul><ul><ul><li>Nature of job - Physical, control over task...
Solutions?  <ul><li>I suggest we agree on four cells, whether we wish to move towards a goal of same indicators across out...
Challenges    Advances
Advance #1 Available Measures <ul><li>OMERACT 9 </li></ul><ul><ul><li>Absenteeism > days absent from work  </li></ul></ul>...
Absenteeism <ul><li>94% OMERACTer’s agreed:  </li></ul><ul><ul><li>Days off sick due to arthritis </li></ul></ul><ul><ul><...
At-work productivity loss  <ul><li>21 different scales  (Escorpizo, 2007; Beaton, 2009) </li></ul><ul><li>5 direct compari...
Head to head comparison <ul><li>Five measures in 250 persons with arthritis followed over one year period (OA and IA)  </l...
Direct comparison: conceptual differences <ul><li>Different concepts, different strengths </li></ul><ul><ul><li>WALS – Wor...
Patient/consumer view Patient votes on preferred scale:  Percent of sample √ √ EWPS  SPS6 WLQ WIS  WALS
Measurement properties  Discrimination between those hindered (46%) or not hindered (54%) at their work Reliability  Valid...
Picking up change over 1 year Mean change/SD change Change in productivity  Mean change/SD change Change in difficulty doi...
The winner is….??? √ √ √ WIS √ Validity  √ ~ √ WALS  √ √ WLQ ~ ~ SPS √ EWPS Responsiveness - productivity Responsiveness –...
Potential candidates for At work productivity – OMERACT Vote – not quite there <ul><ul><li>WALS </li></ul></ul><ul><ul><li...
Advance #2: blending absenteeism and presenteeism  <ul><li>Currently: measured separately.  </li></ul><ul><ul><li>At-work ...
Issues <ul><li>People vary in course </li></ul><ul><ul><li>Absent-present-absent </li></ul></ul><ul><li>Modelling longitud...
Distribution of Work Status and At-Work Disability
Sample Descriptive Statistics and Individual Trajectories
Approach 1:  2 part Mixed Model <ul><li>Advantages of two-part mixed model </li></ul><ul><ul><li>Able to jointly model the...
Approach 2:  2 part mixture models <ul><li>Part 1 manages work status and missing due to drop out </li></ul><ul><li>Part 2...
Two-part Mixture Model: Result
Two-part Mixture Model: Result Note: The top blue line represents those sustained off-work having higher disability score ...
Two-part Mixture Model: Result Summary <ul><li>Dropout is non-ignorable missing. The likelihood of dropout, the level of p...
Structural equation modeling  At work  difficulty At work productivity loss  Absent from work  (WALS) (WLQ) Person-job ins...
final thoughts… <ul><li>Which way to go with instrumentations </li></ul><ul><ul><li>Measures are not equivalent  </li></ul...
Loss in productivity Full time status Part time Modified <ul><li>Absenteeism </li></ul><ul><li>Sick </li></ul><ul><li>Vaca...
Upcoming SlideShare
Loading in …5
×

Beaton Can Workshop Worker Productivity

494 views

Published on

Published in: Technology, Business
  • Be the first to comment

Beaton Can Workshop Worker Productivity

  1. 1. Measuring worker productivity: outcome or cost? <ul><li>Dorcas Beaton, BScOT, PhD </li></ul><ul><li>Health Policy Management & Evaluation, University of Toronto, </li></ul><ul><li>Mobility Program Clinical Research Unit, St Michael’s Hospital, </li></ul><ul><li>Institute for Work & Health. </li></ul>CAN Workshop, November 2009
  2. 2. Perspective: OMERACT <ul><li>O utcome Me asurement in R heum a tological C linical T rials (est. 1992) </li></ul><ul><li>Goal: to improve outcomes used in clinical trials </li></ul><ul><ul><li>Principle of consensus </li></ul></ul><ul><ul><li>What to measure, and how </li></ul></ul><ul><ul><li>Allow better communication across trials, across stakeholders. </li></ul></ul><ul><ul><ul><li>Clinicians, consumers, researchers, regulators, industry (pharma) </li></ul></ul></ul><ul><li>Meets bi-annually informal networking meeting with work in between </li></ul><ul><ul><li>Work began being addressed at OMERACT 8 (2006) </li></ul></ul>
  3. 3. Decision making – OMERACT Style <ul><li>Data driven decisions: </li></ul><ul><ul><li>OMERACT Filter (Boers, J Rheum1998) </li></ul></ul><ul><ul><ul><li>Truth – validity (content, validity) </li></ul></ul></ul><ul><ul><ul><li>Feasibility – costs, ease </li></ul></ul></ul><ul><ul><ul><li>Discrimination – between treatment and control arms in a trial (reliability, responsiveness) </li></ul></ul></ul><ul><li>Evidence gathered, recommendation made </li></ul><ul><li>Consensus vote for endorsement </li></ul>
  4. 4. Challenges and advances- so far! <ul><li>Challenges </li></ul><ul><ul><li>Defining our target </li></ul></ul><ul><ul><li>Contextual nature of work </li></ul></ul><ul><li>Advances </li></ul><ul><ul><li>Available measures </li></ul></ul><ul><ul><li>Methods to blend absenteeism and presenteeism </li></ul></ul>
  5. 5. Challenge #1: Defining our target <ul><li>Perspective: </li></ul><ul><li>Work place productivity = productivity at level of workplace </li></ul><ul><ul><li>Number of cars off line </li></ul></ul><ul><li>Work er productivity = experience/contribution of the individual worker </li></ul><ul><ul><li>Our target for arthritis research </li></ul></ul><ul><li>Defining worker productivity </li></ul><ul><li>Absenteeism = time not at work (but employed) </li></ul><ul><li>Presenteeism = at-work productivity loss = loss efficiency or effectiveness while on the job. </li></ul><ul><li>Both important to arthritis research </li></ul>
  6. 6. Interest in Work in arthritis research <ul><li>As an outcome state </li></ul><ul><ul><li>Off work / RTW (absenteeism) </li></ul></ul><ul><ul><li>Amount of difficulty at work (at-work productivity loss or presenteeism) </li></ul></ul><ul><li>As an indicator of cost </li></ul><ul><ul><li>Cost of days off work +/- replacement workers (absenteeism costs) </li></ul></ul><ul><ul><li>Cost of at-work productivity loss (presenteeism costs) </li></ul></ul>
  7. 7. Target: the impact of arthritis on work at the level of the worker Goal: to have solid indicators of each of these cells. Solid in terms of breadth & depth, measurement properties Possibly same measure for Cost and Outcome goals! YES YES Cost estimate YES YES Outcome State Presenteeism/ At-work productivity loss Absenteeism
  8. 8. Challenge #2 Contextual nature of work <ul><li>Difficulty experienced with work is, in part, defined by the job. </li></ul><ul><ul><li>Physical, mental, time demands, where work is done </li></ul></ul><ul><li>Case scenario </li></ul><ul><ul><li>XX was having a lot of difficulty at work, working what he felt was ~ 75% productivity, so had to leave ideal job and get a less demanding role where he could function without difficulty </li></ul></ul><ul><ul><li>Numbers say 75%  100% productivity. Is he “better”? </li></ul></ul>
  9. 9. Context of work <ul><li>At an individual level we can sort out context </li></ul><ul><li>At a group level, numbers from our scales are on their own. </li></ul><ul><ul><li>Can we validly add together numbers for a group mean with meaning? </li></ul></ul><ul><ul><li>Can we interpret change? </li></ul></ul>
  10. 10. Contextual factors to be considered <ul><li>Environment </li></ul><ul><ul><li>Nature of job - Physical, control over tasks and pace </li></ul></ul><ul><ul><li>Flexibility of workplace to accommodate </li></ul></ul><ul><ul><li>Workplaces organizational practices around work </li></ul></ul><ul><li>Individual factors </li></ul><ul><ul><li>adaptations and coping behaviours used to stay at work (Gignac, 2005) </li></ul></ul><ul><ul><ul><li>Adjust time (inside and outside of work), use vacation, </li></ul></ul></ul><ul><ul><ul><li>Get help from others, </li></ul></ul></ul><ul><ul><ul><li>Modify activities to conserve energy, make things easier </li></ul></ul></ul><ul><ul><ul><li>Anticipate and avoid problems (organize, pace, take breaks). </li></ul></ul></ul>
  11. 11. Solutions? <ul><li>I suggest we agree on four cells, whether we wish to move towards a goal of same indicators across outcome & economic studies </li></ul><ul><li>Contextual factors need to be sorted out </li></ul><ul><ul><li>Funded for qualitative study – decision making around absenteeism/presenteeism </li></ul></ul>
  12. 12. Challenges  Advances
  13. 13. Advance #1 Available Measures <ul><li>OMERACT 9 </li></ul><ul><ul><li>Absenteeism > days absent from work </li></ul></ul><ul><ul><li>What about half days? Vacation days? </li></ul></ul><ul><li>Break out groups, consensus votes </li></ul><ul><ul><li>What should be included in “absenteeism” </li></ul></ul>
  14. 14. Absenteeism <ul><li>94% OMERACTer’s agreed: </li></ul><ul><ul><li>Days off sick due to arthritis </li></ul></ul><ul><ul><li>Vacation days taken because of arthritis </li></ul></ul><ul><ul><li>Part days/hours missed because of arthritis </li></ul></ul><ul><ul><li>Change in number of hours worked per week </li></ul></ul><ul><ul><li>Temporary work cessation (work disability/sick leave) </li></ul></ul><ul><ul><li>Permanent work cessation due to arthritis </li></ul></ul><ul><li>Consider further: </li></ul><ul><ul><ul><li>All cause work absence </li></ul></ul></ul><ul><ul><ul><li>Cessation of work due to choice or retirement </li></ul></ul></ul>OMERACT 9, 2008
  15. 15. At-work productivity loss <ul><li>21 different scales (Escorpizo, 2007; Beaton, 2009) </li></ul><ul><li>5 direct comparisons </li></ul><ul><ul><li>Different levels of work limitations (Lavigne, 2003) </li></ul></ul><ul><ul><li>Low correlations between measures </li></ul></ul><ul><ul><li>Little information in arthritis </li></ul></ul>HLQ, Osterhaus (Lavigne, 2003) WLQ25, SPS13 (Turpin, 2004) WLQ25, WPSI (Ozminkowski, 2004) QQ, HLQ (Meerding, 2005) Osterhaus, QQ (Brouwer, 1999)
  16. 16. Head to head comparison <ul><li>Five measures in 250 persons with arthritis followed over one year period (OA and IA) </li></ul><ul><ul><li>Validity and responsiveness of individual instruments demonstrated (Beaton, in press) </li></ul></ul><ul><ul><li>Predictive validity – predictive of work transitions (change job, hours, duties because of arthritis) presented yesterday (Tang, ACR Meeting 2009) </li></ul></ul><ul><ul><li>Correlations between scales were low to moderate </li></ul></ul><ul><li>Cost estimates – varied up to 7-fold depending on indicator of presenteeism (Zhang, under review) </li></ul><ul><li>clear differences between scales in quantifying work </li></ul>
  17. 17. Direct comparison: conceptual differences <ul><li>Different concepts, different strengths </li></ul><ul><ul><li>WALS – Work Activity Limitations Questionnaire (Gignac, 2004) </li></ul></ul><ul><ul><ul><li>Difficulties in tasks of work. Based on HAQ type questions aimed at work </li></ul></ul></ul><ul><ul><li>WIS – Work Instability Scale (Gilworth, 2003) </li></ul></ul><ul><ul><ul><li>Designed for RA, seems okay in OA, workers. </li></ul></ul></ul><ul><ul><ul><li>Mismatch between person and job, tipping point to going off work. </li></ul></ul></ul><ul><ul><li>WLQ – Work Limitations Questionnaire (Lerner, 2002) </li></ul></ul><ul><ul><ul><li>Proportion of time experiencing any level of work limitation. </li></ul></ul></ul><ul><ul><ul><ul><li>Convertible to dollar cost estimate </li></ul></ul></ul></ul>
  18. 18. Patient/consumer view Patient votes on preferred scale: Percent of sample √ √ EWPS SPS6 WLQ WIS WALS
  19. 19. Measurement properties Discrimination between those hindered (46%) or not hindered (54%) at their work Reliability Validity √ √
  20. 20. Picking up change over 1 year Mean change/SD change Change in productivity Mean change/SD change Change in difficulty doing job Instruments are sensitive to change in their intended target √
  21. 21. The winner is….??? √ √ √ WIS √ Validity √ ~ √ WALS √ √ WLQ ~ ~ SPS √ EWPS Responsiveness - productivity Responsiveness – difficulty Reliability Preference
  22. 22. Potential candidates for At work productivity – OMERACT Vote – not quite there <ul><ul><li>WALS </li></ul></ul><ul><ul><li>RA-WIS </li></ul></ul><ul><ul><li>WLQ </li></ul></ul><ul><ul><li>WPAI </li></ul></ul><ul><ul><li>WPS-RA </li></ul></ul><ul><ul><li>HPQ </li></ul></ul><ul><ul><li>Other </li></ul></ul><ul><ul><li>No opinion </li></ul></ul>OMERACT Endorsement to continue Percent endorsement to continue to work with this measure based on filter evidence
  23. 23. Advance #2: blending absenteeism and presenteeism <ul><li>Currently: measured separately. </li></ul><ul><ul><li>At-work productivity = missing data for those absent from work in study </li></ul></ul><ul><ul><li>Losing important information – they are definitely at one end of at-work spectrum </li></ul></ul><ul><li>Statistical solutions </li></ul><ul><ul><li>Hierarchical modelling </li></ul></ul><ul><ul><ul><li>Working yes / no, if yes then level of at-work productivity </li></ul></ul></ul><ul><ul><li>Structural equation models / path analyses </li></ul></ul>
  24. 24. Issues <ul><li>People vary in course </li></ul><ul><ul><li>Absent-present-absent </li></ul></ul><ul><li>Modelling longitudinal course </li></ul><ul><ul><li>ML will impute data if we use presenteeism scores BUT this missing is MNAR </li></ul></ul><ul><ul><li>Clearly absenteeism has meaning </li></ul></ul><ul><li>Methods to blend dichotomous variable with continuous </li></ul><ul><ul><li>Depeng Jiang’s idea… </li></ul></ul>
  25. 25. Distribution of Work Status and At-Work Disability
  26. 26. Sample Descriptive Statistics and Individual Trajectories
  27. 27. Approach 1: 2 part Mixed Model <ul><li>Advantages of two-part mixed model </li></ul><ul><ul><li>Able to jointly model the two outcomes (absenteeism and at-work disability) simultaneously. </li></ul></ul><ul><ul><li>Able to find the interesting associations between the trajectory of two outcomes. </li></ul></ul><ul><li>Problems of two-part mixed model </li></ul><ul><ul><li>Unable to model the missing mechanism. </li></ul></ul><ul><ul><li>Those estimates from the two-part mixed model might be biased if the dropout is not MAR . </li></ul></ul><ul><ul><li>Unable to model the diversity of trajectories of absenteeism and at-work disability. </li></ul></ul>
  28. 28. Approach 2: 2 part mixture models <ul><li>Part 1 manages work status and missing due to drop out </li></ul><ul><li>Part 2 allows cluster analysis </li></ul><ul><ul><li>Subgroups defined by intraindividual differences in course!! </li></ul></ul><ul><ul><li>More meaningful results </li></ul></ul>
  29. 29. Two-part Mixture Model: Result
  30. 30. Two-part Mixture Model: Result Note: The top blue line represents those sustained off-work having higher disability score (censored above the maximum possible score)
  31. 31. Two-part Mixture Model: Result Summary <ul><li>Dropout is non-ignorable missing. The likelihood of dropout, the level of presenteeism and level of at-work disability were significantly associated. </li></ul>6 5 4 3 2 1 <.001 Increased over 60% in 3 months <10%, <10%, <15% Initially at work, later off (19.2%) <.001 Decreased 80.5% over 12 months 46%, 60%, 76% Initially off work with RTW (11.8%) <.001 Decreased 44.3% over 12 months <10%, <10%, <15% Working at most time points (8.7%) NA Can not evaluate (no WLQ measures) <10%, <10%, <15% Sustained off work (25.9%) 0.92 No significant change 54%, 75%,86% High presenteeism (9.6%) 0.006 Decreased 20.7% over 12 months <10%, <10%, <15% Sustained at work (24.9%) P-value for change in at-work disability At-work disability Dropout rate at 3-, 6-, 12M Class
  32. 32. Structural equation modeling At work difficulty At work productivity loss Absent from work (WALS) (WLQ) Person-job instability (WIS) Potential to build more complex models to explain the relationship between different measures
  33. 33. final thoughts… <ul><li>Which way to go with instrumentations </li></ul><ul><ul><li>Measures are not equivalent </li></ul></ul><ul><li>Still need to conquer contextual factors. </li></ul><ul><ul><li>Which are priority </li></ul></ul><ul><li>New modelling methods can help us to blend absenteeism and presenteeism into one “ruler” for studies </li></ul><ul><ul><li>What do we do for smaller studies, clinical practice </li></ul></ul>
  34. 34. Loss in productivity Full time status Part time Modified <ul><li>Absenteeism </li></ul><ul><li>Sick </li></ul><ul><li>Vacation </li></ul><ul><li>Hours off … </li></ul><ul><li>Presenteeism </li></ul><ul><li>Ability to work </li></ul><ul><li>Effective hours lost </li></ul>Job context Unemployed Retired, age

×