Beaton Can Workshop Worker Productivity


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • The perspective I am bringing to this panel reflect our work with OMERACT – an informal international group that has been meeting since 1992 with the goal of improving outcomes used in arthritis research. They look for consensus on what to measure and how to do so in order to allow better communication across many stakeholders: clinicians, consumers, researchers, regulators and industry. OMERACT meets every other year with lots of work taking place in between. The measurement of work productivity began being addressed by a subgroup of us at OMERACT 8 (in 2006).
  • Decisions on outcomes at OMERACT are based on whether the instruments have passed the “OMERACT Filter” of Truth (that is content and construct validity), Feasibility (is it practical to use from a cost and ease of use perspective) and Discrimination (is it sensitive to difference between treatment and control arms in clinical trials – either change scores or responder analyses). Evidence must be gathered from the literature or new studies and presented at OMERACT meetings for discussion and a vote. The decisions guide the instruments that researchers will use in trials and this will help with communication between trials, and their application in clinical practice.
  • Today I would like to give you a sense of our progress to date by summarizing some of our challenges and advances through the OMERACT initiative.
  • There are many terms being used in measuring work productivity. And the first things we needed to do was define our target. First we considered the perspective. Workplace productivity measures from the level of the workplace, cars off the line and is important for societal level analyses. But it is affected by many different factors. The perspective we chose was workER productivity – the experience or contribution of the individual worker. Within that, arthritis could impact the worker productivity by causing the person to be absent from their job. But also by presenteeism or at-work productivity loss the loss of efficiency or effectiveness on the job. Both are important outcomes in arthritis research
  • But beyond that it became apparent that some of us needed an outcome state measure – perhaps to measure the impact of an ergonomics program on absenteeism or at-work productivity. And others had an equal interest in measuring measuring worker productivity as a cost indicator to quantify the costs associated with either absenteeism or at-work productivity loss for economic analyses
  • we decided we needed to be sensitive to these four domains: absenteeism and presenteeism or at-work productivity loss and measures of outcome state and cost estimate. *But further, we also know that people transition between absenteeism and being at-work with some difficulties, and ideally we would want a way to measure that could measure both in one metric.
  • This leads to the second challenge we encountered: the contextual nature of work. The difficulty experienced with work is, in part, defined by the job that person is at – the physical, mental and time demands and even things like where the work needs to be done. Our outcome measures don’t know this so they will give us what they can and we get a number that tells us about amount of difficulty. So a person having difficulty at work, say functioning at 75% of maximum productivity in his ideal job changes jobs to a less demanding one where he can function well. His number say he has gone from 75% to 100% productivity. But does that mean he is better at work? Work productivity is always a balance of person ability and job demands, but in this case we have totally changed jobs, perhaps to a less satisfying one. The numbers cannot catch that.
  • There are many contextual factors that may influence the numbers we get on our outcome instruments. Some are more environmental – job as we have discussed, but also the flexibility of the workplace to be able to accommodate – some small workplaces can not accommodate lighter duties. Also the organizational practices around disabilities – availability of ergonomists, therapy or different equipment at work. But there are also individual factors as our colleague Monique Gignac and others have described as the many adaptations and coping behaviours people with arthritis make to stay at work – adjusting their time at work, using vacation for appointments, getting help from coworkers, modifying tasks to make them less demanding, and anticipating and avoiding problems.
  • Recognizing the challenges of this measurement, I want now to focus on our advances. And there have been advances.
  • At OMERACT 9 we agreed that absenteeism was more than days absent from work. That we had to consider half days off, or use of vacation instead of sick time. We decided what should be considered in absenteeism.
  • We voted that arthritis research should consider a broader set of indicators of absenteeism that includes not only sick days off due to arthritis, but also vacation days taken to manage arthritis, part days off, change in number of hours worked (reduced hours), temporary cessation of work on disability or prolonged sick leave and permanent cessation (leaving work force). We were asked to consider further, perhaps with sensitivity analyses, the impact of using absence and cessation of work due to any cause, including by choice.
  • In terms of at-work productivity, we found 21 different scales that included the measurement of at-work productivity loss. In the literature there were only five head to head comparisons of their measurement properties, with concerning results – the scales described different levels of work limitations in the same patients. Lavigne found one instrument said 53% of the sample had work limitations, and another said 10%. There were low correlations between measures and little information on arthritis.
  • To help with gathering evidence, we conducted a head to head comparison of five measures of productivity in a cohort of 250 persons with either OA or inflammatory arthritis. We were able to show some evidence of validity, responsiveness and predictive validity of the individual scales – but the correlations between the scales were only moderate at best. From a cost perspective, Wei Zhang compared cost estimations and found them to vary up to seven fold depending on the instrument you chose. We compared a wider array of measures and continued to find clinically significant differences in the way they would quantify the impact of arthritis on work.
  • We believe one of the likely reasons is that the instruments are targetting different concepts so they have different strengths. These are three instruments we have been looking at, the first two developed for arthritis. The WALS focuses on difficulties at work based on HAQ type questions. The WIS or work instability scale focuses on the worry about mismatch between job and the person’s abilities and an estimates that risk or concern about loss of work due to arthritis. The WLQ by contrast quantifies the proportion of time a person is having any level of difficulty with their work, and is easily converted into hours of productivity lost, and from their to costs.
  • At OMERACT 9 we reviewed this evidence believed there were six measures that were gathering evidence towards the filter and we encourage people to focus on these and try them. There was no clear winner yet, as shown by the percent endorsing each questionnaire, but a clear direction to focus on these 6 of the 21 for now. We asked people to work with some of these measures and come back with more evidence about their relative strengths as outcome measures.
  • The other advance worth mentioning is in the blending of absenteeism and presenteeism into one metric. Currently we measure absenteeism or at-work difficulties separately. And at-work issues are assigned "missing data status" if a person is absent. Losing important information, as they are clearly at one end of the at-work productivity spectrum. We need to find ways of integrating them into one measure. The application of newer statistical approaches can allow us to blend information on both into one outcome. Hierarchical modelling can handle absenteeism without assigning it to missing or zero. SEM allow absenteeism and presenteeism to be blended into a single indicator or latent trait.
  • Sustained at work (24.9%): consistently at-work and low dropout rate; at-work productivity improved significantly ( p = .006); on average at-work disability decreased 20.7% after 12 months. High level of presenteeism but more likely dropout (9.6%): consistently at-work if not dropout; no significant change in the level of at-work disability ( p = .92). Sustained absenteeism (25.9%): consistently off-work; unable to evaluate its productivity loss or improvement over time. Usually at work (8.7%): working at most of the time points and returned to work at 12-month assessment; at-work productivity improved significantly ( p <.001); on average at-work disability decreased 44.3% after 12 months. Return to work (11.8%): initial off-work but return to work; at-work productivity improved significantly ( p <.001); on average at-work disability decreased 80.5% after 12 months. Working then off (19.2%): initial at-work but off at follow-up. At-work disability increased dramatically and significantly over time. Dropout is non-ignorable missing (not MAR or MCAR) Those sustained at work and with less change of at-work disability over time were more likely dropout of the study. Group dropout rates were 54%, 75%, and 86% at 3-, 6-, and 12-month follow-up, respectively. Those initial off-work then return to work at follow-up (with decreased at-work disability) were more likely dropout. Group dropout rates were 46%, 60%, and 76% at 3-, 6-, and 12-month follow-up, respectively. Conversely, those sustained absenteeism and those initial at-work then off-work at follow-up were less likely dropout. Group dropout rates were less than 15%.
  • SEM can also allow the potential to build more complex models to explain the differences we are seeing in our tools. Perhaps at work difficulties picked up in the WALS will lead to worker productivity loss which might lead to absence. And work instability might pick up when work productivity loss transitions to absenteeism. But these models need large sample sizes, and we also need solutions applicable to smaller studies and to clinical practice. We also need solutions for these situations.
  • I would conclude by saying that work IS an important outcome in persons with arthritis not only for financial reasons but self-esteem and social roles. We have many outcomes available for absenteeism and at-work productivity loss, but they are not equivalent and likely should not be compared across studies. The OMERACT type approach tries to achieve a consensus across stakeholders based on evidence of “truth, feasibility and discrimination” – key measurement properties for choosing the best way to measure work.
  • Beaton Can Workshop Worker Productivity

    1. 1. Measuring worker productivity: outcome or cost? <ul><li>Dorcas Beaton, BScOT, PhD </li></ul><ul><li>Health Policy Management & Evaluation, University of Toronto, </li></ul><ul><li>Mobility Program Clinical Research Unit, St Michael’s Hospital, </li></ul><ul><li>Institute for Work & Health. </li></ul>CAN Workshop, November 2009
    2. 2. Perspective: OMERACT <ul><li>O utcome Me asurement in R heum a tological C linical T rials (est. 1992) </li></ul><ul><li>Goal: to improve outcomes used in clinical trials </li></ul><ul><ul><li>Principle of consensus </li></ul></ul><ul><ul><li>What to measure, and how </li></ul></ul><ul><ul><li>Allow better communication across trials, across stakeholders. </li></ul></ul><ul><ul><ul><li>Clinicians, consumers, researchers, regulators, industry (pharma) </li></ul></ul></ul><ul><li>Meets bi-annually informal networking meeting with work in between </li></ul><ul><ul><li>Work began being addressed at OMERACT 8 (2006) </li></ul></ul>
    3. 3. Decision making – OMERACT Style <ul><li>Data driven decisions: </li></ul><ul><ul><li>OMERACT Filter (Boers, J Rheum1998) </li></ul></ul><ul><ul><ul><li>Truth – validity (content, validity) </li></ul></ul></ul><ul><ul><ul><li>Feasibility – costs, ease </li></ul></ul></ul><ul><ul><ul><li>Discrimination – between treatment and control arms in a trial (reliability, responsiveness) </li></ul></ul></ul><ul><li>Evidence gathered, recommendation made </li></ul><ul><li>Consensus vote for endorsement </li></ul>
    4. 4. Challenges and advances- so far! <ul><li>Challenges </li></ul><ul><ul><li>Defining our target </li></ul></ul><ul><ul><li>Contextual nature of work </li></ul></ul><ul><li>Advances </li></ul><ul><ul><li>Available measures </li></ul></ul><ul><ul><li>Methods to blend absenteeism and presenteeism </li></ul></ul>
    5. 5. Challenge #1: Defining our target <ul><li>Perspective: </li></ul><ul><li>Work place productivity = productivity at level of workplace </li></ul><ul><ul><li>Number of cars off line </li></ul></ul><ul><li>Work er productivity = experience/contribution of the individual worker </li></ul><ul><ul><li>Our target for arthritis research </li></ul></ul><ul><li>Defining worker productivity </li></ul><ul><li>Absenteeism = time not at work (but employed) </li></ul><ul><li>Presenteeism = at-work productivity loss = loss efficiency or effectiveness while on the job. </li></ul><ul><li>Both important to arthritis research </li></ul>
    6. 6. Interest in Work in arthritis research <ul><li>As an outcome state </li></ul><ul><ul><li>Off work / RTW (absenteeism) </li></ul></ul><ul><ul><li>Amount of difficulty at work (at-work productivity loss or presenteeism) </li></ul></ul><ul><li>As an indicator of cost </li></ul><ul><ul><li>Cost of days off work +/- replacement workers (absenteeism costs) </li></ul></ul><ul><ul><li>Cost of at-work productivity loss (presenteeism costs) </li></ul></ul>
    7. 7. Target: the impact of arthritis on work at the level of the worker Goal: to have solid indicators of each of these cells. Solid in terms of breadth & depth, measurement properties Possibly same measure for Cost and Outcome goals! YES YES Cost estimate YES YES Outcome State Presenteeism/ At-work productivity loss Absenteeism
    8. 8. Challenge #2 Contextual nature of work <ul><li>Difficulty experienced with work is, in part, defined by the job. </li></ul><ul><ul><li>Physical, mental, time demands, where work is done </li></ul></ul><ul><li>Case scenario </li></ul><ul><ul><li>XX was having a lot of difficulty at work, working what he felt was ~ 75% productivity, so had to leave ideal job and get a less demanding role where he could function without difficulty </li></ul></ul><ul><ul><li>Numbers say 75%  100% productivity. Is he “better”? </li></ul></ul>
    9. 9. Context of work <ul><li>At an individual level we can sort out context </li></ul><ul><li>At a group level, numbers from our scales are on their own. </li></ul><ul><ul><li>Can we validly add together numbers for a group mean with meaning? </li></ul></ul><ul><ul><li>Can we interpret change? </li></ul></ul>
    10. 10. Contextual factors to be considered <ul><li>Environment </li></ul><ul><ul><li>Nature of job - Physical, control over tasks and pace </li></ul></ul><ul><ul><li>Flexibility of workplace to accommodate </li></ul></ul><ul><ul><li>Workplaces organizational practices around work </li></ul></ul><ul><li>Individual factors </li></ul><ul><ul><li>adaptations and coping behaviours used to stay at work (Gignac, 2005) </li></ul></ul><ul><ul><ul><li>Adjust time (inside and outside of work), use vacation, </li></ul></ul></ul><ul><ul><ul><li>Get help from others, </li></ul></ul></ul><ul><ul><ul><li>Modify activities to conserve energy, make things easier </li></ul></ul></ul><ul><ul><ul><li>Anticipate and avoid problems (organize, pace, take breaks). </li></ul></ul></ul>
    11. 11. Solutions? <ul><li>I suggest we agree on four cells, whether we wish to move towards a goal of same indicators across outcome & economic studies </li></ul><ul><li>Contextual factors need to be sorted out </li></ul><ul><ul><li>Funded for qualitative study – decision making around absenteeism/presenteeism </li></ul></ul>
    12. 12. Challenges  Advances
    13. 13. Advance #1 Available Measures <ul><li>OMERACT 9 </li></ul><ul><ul><li>Absenteeism > days absent from work </li></ul></ul><ul><ul><li>What about half days? Vacation days? </li></ul></ul><ul><li>Break out groups, consensus votes </li></ul><ul><ul><li>What should be included in “absenteeism” </li></ul></ul>
    14. 14. Absenteeism <ul><li>94% OMERACTer’s agreed: </li></ul><ul><ul><li>Days off sick due to arthritis </li></ul></ul><ul><ul><li>Vacation days taken because of arthritis </li></ul></ul><ul><ul><li>Part days/hours missed because of arthritis </li></ul></ul><ul><ul><li>Change in number of hours worked per week </li></ul></ul><ul><ul><li>Temporary work cessation (work disability/sick leave) </li></ul></ul><ul><ul><li>Permanent work cessation due to arthritis </li></ul></ul><ul><li>Consider further: </li></ul><ul><ul><ul><li>All cause work absence </li></ul></ul></ul><ul><ul><ul><li>Cessation of work due to choice or retirement </li></ul></ul></ul>OMERACT 9, 2008
    15. 15. At-work productivity loss <ul><li>21 different scales (Escorpizo, 2007; Beaton, 2009) </li></ul><ul><li>5 direct comparisons </li></ul><ul><ul><li>Different levels of work limitations (Lavigne, 2003) </li></ul></ul><ul><ul><li>Low correlations between measures </li></ul></ul><ul><ul><li>Little information in arthritis </li></ul></ul>HLQ, Osterhaus (Lavigne, 2003) WLQ25, SPS13 (Turpin, 2004) WLQ25, WPSI (Ozminkowski, 2004) QQ, HLQ (Meerding, 2005) Osterhaus, QQ (Brouwer, 1999)
    16. 16. Head to head comparison <ul><li>Five measures in 250 persons with arthritis followed over one year period (OA and IA) </li></ul><ul><ul><li>Validity and responsiveness of individual instruments demonstrated (Beaton, in press) </li></ul></ul><ul><ul><li>Predictive validity – predictive of work transitions (change job, hours, duties because of arthritis) presented yesterday (Tang, ACR Meeting 2009) </li></ul></ul><ul><ul><li>Correlations between scales were low to moderate </li></ul></ul><ul><li>Cost estimates – varied up to 7-fold depending on indicator of presenteeism (Zhang, under review) </li></ul><ul><li>clear differences between scales in quantifying work </li></ul>
    17. 17. Direct comparison: conceptual differences <ul><li>Different concepts, different strengths </li></ul><ul><ul><li>WALS – Work Activity Limitations Questionnaire (Gignac, 2004) </li></ul></ul><ul><ul><ul><li>Difficulties in tasks of work. Based on HAQ type questions aimed at work </li></ul></ul></ul><ul><ul><li>WIS – Work Instability Scale (Gilworth, 2003) </li></ul></ul><ul><ul><ul><li>Designed for RA, seems okay in OA, workers. </li></ul></ul></ul><ul><ul><ul><li>Mismatch between person and job, tipping point to going off work. </li></ul></ul></ul><ul><ul><li>WLQ – Work Limitations Questionnaire (Lerner, 2002) </li></ul></ul><ul><ul><ul><li>Proportion of time experiencing any level of work limitation. </li></ul></ul></ul><ul><ul><ul><ul><li>Convertible to dollar cost estimate </li></ul></ul></ul></ul>
    18. 18. Patient/consumer view Patient votes on preferred scale: Percent of sample √ √ EWPS SPS6 WLQ WIS WALS
    19. 19. Measurement properties Discrimination between those hindered (46%) or not hindered (54%) at their work Reliability Validity √ √
    20. 20. Picking up change over 1 year Mean change/SD change Change in productivity Mean change/SD change Change in difficulty doing job Instruments are sensitive to change in their intended target √
    21. 21. The winner is….??? √ √ √ WIS √ Validity √ ~ √ WALS √ √ WLQ ~ ~ SPS √ EWPS Responsiveness - productivity Responsiveness – difficulty Reliability Preference
    22. 22. Potential candidates for At work productivity – OMERACT Vote – not quite there <ul><ul><li>WALS </li></ul></ul><ul><ul><li>RA-WIS </li></ul></ul><ul><ul><li>WLQ </li></ul></ul><ul><ul><li>WPAI </li></ul></ul><ul><ul><li>WPS-RA </li></ul></ul><ul><ul><li>HPQ </li></ul></ul><ul><ul><li>Other </li></ul></ul><ul><ul><li>No opinion </li></ul></ul>OMERACT Endorsement to continue Percent endorsement to continue to work with this measure based on filter evidence
    23. 23. Advance #2: blending absenteeism and presenteeism <ul><li>Currently: measured separately. </li></ul><ul><ul><li>At-work productivity = missing data for those absent from work in study </li></ul></ul><ul><ul><li>Losing important information – they are definitely at one end of at-work spectrum </li></ul></ul><ul><li>Statistical solutions </li></ul><ul><ul><li>Hierarchical modelling </li></ul></ul><ul><ul><ul><li>Working yes / no, if yes then level of at-work productivity </li></ul></ul></ul><ul><ul><li>Structural equation models / path analyses </li></ul></ul>
    24. 24. Issues <ul><li>People vary in course </li></ul><ul><ul><li>Absent-present-absent </li></ul></ul><ul><li>Modelling longitudinal course </li></ul><ul><ul><li>ML will impute data if we use presenteeism scores BUT this missing is MNAR </li></ul></ul><ul><ul><li>Clearly absenteeism has meaning </li></ul></ul><ul><li>Methods to blend dichotomous variable with continuous </li></ul><ul><ul><li>Depeng Jiang’s idea… </li></ul></ul>
    25. 25. Distribution of Work Status and At-Work Disability
    26. 26. Sample Descriptive Statistics and Individual Trajectories
    27. 27. Approach 1: 2 part Mixed Model <ul><li>Advantages of two-part mixed model </li></ul><ul><ul><li>Able to jointly model the two outcomes (absenteeism and at-work disability) simultaneously. </li></ul></ul><ul><ul><li>Able to find the interesting associations between the trajectory of two outcomes. </li></ul></ul><ul><li>Problems of two-part mixed model </li></ul><ul><ul><li>Unable to model the missing mechanism. </li></ul></ul><ul><ul><li>Those estimates from the two-part mixed model might be biased if the dropout is not MAR . </li></ul></ul><ul><ul><li>Unable to model the diversity of trajectories of absenteeism and at-work disability. </li></ul></ul>
    28. 28. Approach 2: 2 part mixture models <ul><li>Part 1 manages work status and missing due to drop out </li></ul><ul><li>Part 2 allows cluster analysis </li></ul><ul><ul><li>Subgroups defined by intraindividual differences in course!! </li></ul></ul><ul><ul><li>More meaningful results </li></ul></ul>
    29. 29. Two-part Mixture Model: Result
    30. 30. Two-part Mixture Model: Result Note: The top blue line represents those sustained off-work having higher disability score (censored above the maximum possible score)
    31. 31. Two-part Mixture Model: Result Summary <ul><li>Dropout is non-ignorable missing. The likelihood of dropout, the level of presenteeism and level of at-work disability were significantly associated. </li></ul>6 5 4 3 2 1 <.001 Increased over 60% in 3 months <10%, <10%, <15% Initially at work, later off (19.2%) <.001 Decreased 80.5% over 12 months 46%, 60%, 76% Initially off work with RTW (11.8%) <.001 Decreased 44.3% over 12 months <10%, <10%, <15% Working at most time points (8.7%) NA Can not evaluate (no WLQ measures) <10%, <10%, <15% Sustained off work (25.9%) 0.92 No significant change 54%, 75%,86% High presenteeism (9.6%) 0.006 Decreased 20.7% over 12 months <10%, <10%, <15% Sustained at work (24.9%) P-value for change in at-work disability At-work disability Dropout rate at 3-, 6-, 12M Class
    32. 32. Structural equation modeling At work difficulty At work productivity loss Absent from work (WALS) (WLQ) Person-job instability (WIS) Potential to build more complex models to explain the relationship between different measures
    33. 33. final thoughts… <ul><li>Which way to go with instrumentations </li></ul><ul><ul><li>Measures are not equivalent </li></ul></ul><ul><li>Still need to conquer contextual factors. </li></ul><ul><ul><li>Which are priority </li></ul></ul><ul><li>New modelling methods can help us to blend absenteeism and presenteeism into one “ruler” for studies </li></ul><ul><ul><li>What do we do for smaller studies, clinical practice </li></ul></ul>
    34. 34. Loss in productivity Full time status Part time Modified <ul><li>Absenteeism </li></ul><ul><li>Sick </li></ul><ul><li>Vacation </li></ul><ul><li>Hours off … </li></ul><ul><li>Presenteeism </li></ul><ul><li>Ability to work </li></ul><ul><li>Effective hours lost </li></ul>Job context Unemployed Retired, age
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.