Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Participation and Environmental Factors Measures For TBI Rehabilitation






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • Some overlaps to ICF but more direct application to categories people with disabilities framed as most influencing their participation choice and control; item sets developed for each.Systems IS THE SAME; Natural and Built is the sameSocial Environment overlaps with support and relationships and attitudesEcon is newTechnology is split in 2
  • Did transportation get binned with something else…?

Participation and Environmental Factors Measures For TBI Rehabilitation Participation and Environmental Factors Measures For TBI Rehabilitation Presentation Transcript

  • Participation and Environmental Factors Measures For TTBI Rehabilitation
    Susan Magasi
    Allen Heinemann
    Noelle Carlozzi
  • Presentation Overview
    Developing a measure of community participation after rehabilitation
    Evaluating the feasibility of computer-adapted administration of a participation measure with discharged rehabilitation patients
    Measuring the Impact of the Environment on Home, Community and Work Participation for People with Cognitive Disabilities
  • Developing a Measure of Community Participation after Rehabilitation
    Susan Magasi
  • Presentation Objectives
    Describe ways to assess participation
    Evaluate the psychometric properties of community enfranchisement items and develop an item bank that is suitable for computer adaptive testing (CAT)
    Explore the validity of a composite participation score and measures of community enfranchisement
  • 5
    International Classification of Functioning Disability and Health
    Environmental Factors
    Body Functions
  • Project Task Overview
    Conduct literature review
    Conduct focus groups with consumers, caregivers, providers, payers, policy makers
    Develop items and rating scales to operationalize participation
    Conduct cognitive interviews with consumers and general public
    Revise items, rating scales
    Pilot test participation instrument
    Refine instrument
    Collect population data from persons with and without disabilities
    Refine instrument
    Collect and analyze population data from persons with and without disabilities (Behavioral Risk Factor Surveillance System)
    Refine instrument
    Collect 3rd round of data, augmented with panel sample
    Evaluate CPI as part of routine post-discharge follow-up assessment - underway
  • Construct Development: Stakeholder Focus Groups
    Consumers (5)
    Chicago, IL
    Englewood, CO
    Caregivers (6)
    Chicago, IL
    Englewood, CO
    Rehabilitation professionals (4)
    Chicago, IL
    Englewood, CO
    Payers (1)
    Insurance Research Group
    Policy Makers (2)
    Federal: Washington, DC
    State: Chicago, Illinois
    Total Participants
    18 groups
    138 participants
  • Focus Group Questions
    What does the word “participation” mean to you?
    What does it mean “to participate”
    Why is participation important to you?
    What areas of everyday life are most important to you?
    If taken away, what areas would you miss most/would be the biggest loss to you?
    What issues or barriers most affect your participation?
    What are your biggest supports to participation?
    What recommendations do you have for improving participation opportunities for people with disabilities?
  • Qualitative Analytic Procedures
    • Analyses and interpretation
    • Open coding of data by 2 researchers, including team members with disabilities
    • Topical coding to identify key emerging themes
    • Detailed coding of
    • individual data within groups
    • comparative analysis across individuals
    • across stakeholder groups
    • Codes represent
    • Content themes
    • Narrative stories & examples
    • Areas of agreement and differences on specific issues
    • End product
    • Participation item pool that reflects the themes and concerns & priorities of multiple stakeholders
  • 10
    Focus Group Input
  • 11
    What We Learned: Participation Values
    Respect & Dignity
    • Hammel J, Magasi S, Heinemann AW, Whiteneck G, Bogner J, Rodriguez E. What does participation mean? An insider perspective from people with disabilities. Disability and Rehabilitation, 30:19,1445-1460.
    • Magasi S, Hammel J, Heinemann AW, Whiteneck G, Bogner J. Participation: A comparative analysis of multiple rehabilitation stakeholders’ perspectives. Journal of Rehabilitation Medicine, 41, 936-944, 2009.
  • Community Participation IndicatorsVersion 1
    • Source Material
    • Focus group thematic codes
    • Comprehensive literature review
    • Content Domains
    • Productive roles (student, worker, and homemaker)
    • Social roles (spouse or partner, family member, caregiver, caregiver to children or aging parents, relationships with adults and friends
    • Community roles (volunteer, civic, political, organization, or community group member)
  • CPI Version 1: Content Organization
    • Engagement (frequency, duration of activities)
    • Economic / productive (5 items)
    • Social (5)
    • Community (11)
    • Evaluation (agreement, satisfaction)
    • Economic / productive (4)
    • Social (5)
    • Community (1)
    • Enfranchisement
    • 19 items
    • 4-point rating scale
  • Rating Scale Analysis of Enfranchisement Items
    • BRFSS addendum N = 912
    • Self-identified disability = 51%
    • Person separation reliability = .77 (marginal)
    • Rating scale
    • Too easy to endorse items
    • Item targeting on sample
    • Mistargeted
    • Ceiling effect
    • Item statistics
    • No misfitting items
    • No evidence of differential item functioning
    • Disability vs. nondisability samples
    • Assessing only frequency of activities ignores appraisal of importance
    • Revision Decision
    • Expand enfranchisement items
    • Assess activity importance, evaluation
  • Conclusions: Participation enfranchisement constitutes a new, previously unmeasured aspect of participation - one that addresses subjective perceptions rather than objective performance -with items that are clearly distinct from more generalized satisfaction with participation. The 19 enfranchisement items describe aspects of participation that may prove useful in characterizing longer-term rehabilitation outcomes.
  • Community Participation IndicatorsVersion 2
    • 20 engagement items
    • Frequency of activity
    • Importance of activity
    • Evaluation of activity frequency
    • 48 enfranchisement items
    • Higher “ceiling” sought
    • Revised rating scale – from 4 to 5 agreement categories
  • 17
  • 18
  • 19
  • Enfranchisement Items
    • Retained 8 thematic domains
    • Expanded from 19 to 48 items
    • Cognitive testing with consumers
    • New rating scale
    • All the time
    • Frequently
    • Sometimes
    • Seldom
    • Almost never
  • 21
  • 22
    Round 3 Data: Sample Characteristics
    N = 1163
    Panel survey: 51%
    Community sites: 31%
    Former inpatients: 18%
    Median age = 53 yr (+ 17)
    Women = 49%
    Caucasian 72%
    African-American 12%
    Hispanic 7%
    Self-report 72%
    Mobility 63%
    Mental 23%
    Hearing 14%
    Vision 13%
    Learning 11%
    Communication 9%
    Disability Severity
  • Results: Activity Patterns
    • Activity Categories
    • Social activities
    • Productive activities
    • Low frequency activities
    • Activity Importance
    • Yes / No
  • Social Activities
  • ‘Productive’ Activities
  • Low Frequency Activities
  • 27
    Percent of Important Activities Performed Often Enough
    Mean = 55%
    SD = 28
    N = 1149
  • Scoring DecisionActivities and Activity Importance
    • Avoid creating a “busy-ness” index
    • Personal preferences, opportunities, environmental factors influence activity patterns
    • Report descriptive information about activity patterns
    • Evaluate utility of “percent of important activities performed often enough”
  • Rating Scale Analysis of Enfranchisement Items
    • Began with all 48 items
    • Used exploratory factor analysis to identify number of subdomains in half the sample
    • Used confirmatory factor analysis to demonstrate reproducibility in the second half of the sample
    • Used rating scale analysis with subdomains to evaluate unidimensionality and reliability
    Involvement in life situations
    Control over participation
  • Involvement in Life SituationsSummary Statistics
    • Person Information
    • How well do items differentiate the people?
    • Very well
    • Reliability .87
    • Item Information
    • How well do the people differentiate the items?
    • Exceptionally well
    • Reliability 1.00
  • Involvement in Life SituationsItem Statistics (13 retained items)
  • Control over ParticipationSummary Statistics
    • Person Information
    • How well do items differentiate the people?
    • Very well
    • Reliability .87
    • Item Information
    • How well do the people differentiate the items?
    • Exceptionally well
    • Reliability 1.00
    • Item Fit
    • I speak up for myself (Infit Mean Square=1.41)
  • Control over ParticipationItem Statistics (15 retained items)
  • People with More Severe Disabilities Report Less Involvement in Life Situations
    Known Groups Validity Evidence
  • People with More Severe Disabilities Report Less Control over Participation
    Known Groups Validity Evidence
  • Conclusions
    Participation as measured by activity frequency, evaluation and enfranchisement items is not a unidimensional construct
    • Don’t model activity frequency, importance and evaluation in a composite measure
    • Individuals’ preferences and opportunities vary greatly
    • Personal preferences determine individual’s participation profile
    Involvement in and control over participation are distinct constructs that can be measured reliably
    • Expanded set of community enfranchisement items contains 2 distinct item sets that are suitable for item banking and computer adaptive administration
    Construct validity of involvement in and control over participation is promising
  • Feasibility of Measuring Community Participation after Rehabilitation Discharge
    Allen Heinemann
  • Presentation Objectives
    Evaluate the feasibility of measuring community participation after inpatient rehabilitation
    Compare the efficacy of telephone vs. web-based data collection
    Explore the relationship between functional status and the three aspects of community participation
    Explore demographic, impairment, and rehabilitation variables as they relate to community participation
  • Method
    • Sample Recruitment
    • Outcomes Management Systems and Analysis staff invited discharged adult inpatients to complete study instruments after they completed a 1-month post-discharge telephone satisfaction survey.
    • Survey administration
    • Patients chose to complete the survey on a secure web site or using an interactive voice response system.
    • Questions were administered using a computer-adaptive testing (CAT) algorithm based on calibration of the items relating to involvement in life situations and control over participation
    • Data analysis
    • Participation data from the CAT were matched to de-identified inpatient data and analyzed using SPSS software.
  • Results
    • Of the 457 patients who agreed to participate in the study, 124 (27%) completed the survey.
    • Of the 124 patients who completed the survey, 66% did so by phone and 34% by internet.
    • Completion rates were similar for both modes of completion.
    • Subjects who agreed to participate were younger than those who did not agree to participate.
    • Subject who completed the survey by phone were older than those who completed the survey by internet.
  • Response Rate by Telephone and Internet
  • Days to Complete the Instrument
  • Average Age: Phone vs. Internet
  • Results Highlights, continued
    • Subjects who completed the CAT by phone completed the survey sooner than those who completed the survey by the internet.
    • Older adults reported doing enough of their important activities than did younger adults.
    • Functional status as measured by the FIM was unrelated to involvement, control, or important activities performed often enough.
  • Conclusions
    CAT administration of participation items by phone and internet is feasible and cost-effective for collection of post-rehabilitation outcome data.
    Older subjects prefer the telephone to the internet.
    The three measures of community participation are not related to FIM scores, indicating that they are measuring a unique aspect of a patients’ experience.
    Further studies are needed to evaluate the validity and reliability of participation scores.
  • Acknowledgements
    • Outcomes Management
    • Tom Snyder and OMSA Staff
    • Brian Arnold
  • 47
    For More Information
  • Developing Environmental Factor Measures for Persons with TBI, Stroke and SCI
    Noelle Carlozzi
  • Presentation Objectives
    Understand environmental factors and how they influence participation
    Describe efforts to develop measures of environmental factors that influence participation
    Discuss preliminary findings using these measures in TBI
  • ICF Environmental Factors Taxonomy
  • State-of-the-Art Measurement
    Many ways to assess environmental factors
    Recent measurement advances are not captured by the current environmental factors measures
    Future work is needed to develop “smart tests” that can assess environmental factors using state-of-the-art measurement development techniques (IRT and CAT)
  • RRTC Project to Develop Environmental Factor Measures (TBI, Stroke, & SCI)
    • Allen W. Heinemann, PhD
    • Rehabilitation Institute of Chicago
    • David Tulsky, PhD
    • University of Michigan
    • Noelle Carlozzi, PhD
    • University of Michigan
    • Pam Kisala, MA
    • University of Michigan
    • Sofia Garcia, PhD
    • Northwestern University
    • Beth Hahn, MA
    • Northwestern University
    • Susan Magasi, PhD
    • Northwestern University
    • Jin-Shei Lai, PhD
    • Northwestern University
    • David Gray, PhD
    • Washington University
    • Holly Hollingsworth, PhD
    • Washington University
    • Joy Hammel, PhD
    • University of Illinois at Chicago
  • RRTC Project Progress
    Year 1
    Analyzed existing focus group data
    Reviewed environmental factors literature and current measures
    Developed conceptual framework
    Pooled, binned, winnowed, wrote items
    Year 2
    Completed several rounds of cognitive testing
    Completed pilot data collection
    Data analysis in progress
  • 54
    Conceptual Framework
    Environmental Factors
  • Binned and Winnowed Items
  • Economic Quality of Life
    Financial influence on:
    • Satisfaction with income level
    • Comfortable living situation
    • Adequate and affordable health services
    • Adequate and affordable food
    • Affordable community recreational activities
    • Family and friend financial assistance
  • Economic Quality of Life
    Sample Items:
    • I can afford to eat out when I want.
    • I can afford to pay my bills.
    • I have delayed getting health care because I couldn’t pay for it.
    • I have access to extra money in case of an emergency.
  • Systems, Services, and Policies
    Affect on participation:
    • Home
    • Community
    • Work
    Five sub-sets:
    • Managing health
    • Living situation
    • Community participation
    • Work
    • Transportation
  • Systems, Services, and Policies
    Sample Items:
    My community offers support groups I can use.
    My community offers information on low or no cost activities and things to do.
    Disability accommodation services and supports are available at my work or school.
    Public transportation in my area is reliable.
  • Built and Natural Environment
    Features of:
    • Home
    • Buildings
    • Outdoors
    How much difficulty do you have...?
    • Environmental ?
    • Non-environmental ?
  • Built and Natural Environment
    Sample Items:
    • How much difficulty would you have feeling safe in your home during an emergency?
    • The difficulty I have feeling safe is due to problems getting out or getting help in an emergency. (yes – no)
    • How much difficulty do you have hearing sounds such as voices and music in buildings in your community?
    • The difficulty I have hearing sounds is due to background noise. (yes – no)
  • Assistive Technology
    Four sub-sets:
    • Mobility
    • Personal care
    • Vision/Hearing
    • Communication
    • Availability
    • Quality
    • Impact
  • Assistive Technology
    Sample Items:
    • My device is easy to use.
    • My device is reliable.
    • My device gives me more control over my daily activities.
    • My device allows me to participate in activities that I enjoy.
  • Access to Information & Technology
    Devices and technology to transmit and receive information
    cell phone, land lines, computer, email, and internet services
    Usability of this information/technology
    ability to access and understand information, literacy, transparency, information finding
  • Sample Items
    • I have easy access to the internet if I want to use it.
    • My telephone is easy to use.
    • Information about community resources is easily available to me if I need it.
    • In case of a health emergency I can get the information I need easily.
    Access to Information & Technology
  • Social Environment
    Social Support:
    • Companionship
    • Emotional Support
    • Informational Support
    • Instrumental Support
    Perceived Positive & Negative Disability-related Attitudes & Behaviors:
    • Family/Friends
    • Community
    • Society / Public
    • Acceptance
    • Stigma
    • Marginalization
  • Social Environment
    Sample Items:
    • The people in my life are willing to accommodate my disability.
    • People are patient when I take extra time to do things because of my disability.
    • Criminals see people with disabilities as easy targets.
    • Because of my disability, my family complains that I am too needy.
    • Society respects the need for disability accommodations.
    • Society limits my opportunities because of my disability.
    • People with disabilities are encouraged to participate in my community.
  • Cognitive Interviewing
    • 5 people per impairment group
    • Interviews were recorded for transcription purposes
    • Follow-up questions were asked intentionally after each question
    • Comprehension
    • Opinions
    • Recommendations
  • 69
    Cognitive Interviewing
  • 70
    Cognitive Interviewing
  • 71
    Cognitive Interviewing
  • Cognitive Interviewing
  • Cognitive Interviewing
  • Cognitive Interviewing
  • Preliminary Pilot Test Results
  • Preliminary Pilot Test Analyses, cont.
  • Preliminary Pilot Test Analyses, cont.
  • Relevance to TBI
  • 80
    Next Steps
    Complete Data Analysis
    Explore dimensionality of domains
    Explore differential item functioning by impairment group
    Field Testing
    Recruit 200 participants with TBI, 200 with Stroke, and 200 with SCI (N=600)
    Administer EF items banks, legacy measures, newly developed social participation measures (TBI-QOL) and traditional neuropsychological measures
  • Acknowledgments
    Rehabilitation Research and Training Center on Improving Measurement of Medical Rehabilitation Outcomes (H133B090024)
    Rehabilitation Research and Training Center on Measuring Rehabilitation Outcomes and Effectiveness (H133B040032)