SlideShare a Scribd company logo
1 of 7
Download to read offline
SIS

                                              Supports Intensity Scale



                                              Psychometric properties

                                              Similarities and differences
                                              between original US version and
                                              Dutch, Italian, French and Spanish
                                              translations



                                              Wil Buntinx, PhD / FAAIDD


                                               Handout? Go to
       AAIDD post conference workshop          www.buntinx.org
       St. Paul (MN) – June 9, 2011




Central question:
Is the SIS a reliable and valid psychometric tool across countries and
cultures?


Translating is not enough   validity and reliability should be examined in
                            representatieve groups for each language and culture.
Standards for Educational and Psychological Testing , Washington: APA 1999




Is the SIS a valid and reliable instrument to assess supports intensity needs of
persons with ID in other countries than the US?

Translations considered:     Dutch (Flanders-Belgium / the netherlands)
                             Italian
                             Spanish (Catalan)
                             French (Canadian)




                                                                                    1
SIS relilability
                       Internal consistency (α) – split-half – SEM


                              Dutch Standardization         Flemish standardization       USA 1    Italy     Spain


                 number                          split-                         split-
 SIS subscale    of items
                                   α     SEM                      α     SEM                 α           α      α
                                                  half                           half




 Section 1

     A              8          .95        .7      .97         .93        .8      .90       .95     0.96       .93

     B              8          .96        .6      .96         .95        .7      .91       .95      .97       .97

     C              9          .97        .5      .96         .95        .7      .84       .97      .97       .96

     D              8          .96        .6      .92         .94        .7      .86       .97      .96       .95

     E              8          .95        .7      .94         .92        .8      .86       .94      .95       .93

     F              8          .97        .5      .97         .94        .7      .87       .96      .97       .95

 SIS Index         49          .99        1,5     .98         .98        2,1     .91       .99      .99       .99




                                                SIS reliability
             Test-retest - Inter-interviewer - Inter-respondent - client-staff

                                                                                              Inter-        Client-
                        Test-retest                       Inter-interviewer                                  staff
                                                                                           respondent
                USA 1       Canad      Spain    USA 1     USA 2       Canad    Spain     USA 2    Canad     Dutch
    SIS
  subscale
 section 1
     A           .87         .85        .98      .90        .80        .92      .86       .73      .88       .74
     B           .74         .77        .94      .68        .89        .82      .86       .91      .87       .58
     C           .75         .75        .93      .55        .88        .85      .71       .75      .87       .44
     D           .83         .75        .96      .55        .77        .90      .62       .93      .87       .31
     E           .86         .81        .90      .72        .96        .79      .71       .91      .91       .77
     F           .94         .68        .98      .60        .74        .79      .66       .87      .85       .51
 SIS Index       .82         .84        .94      .59        .88        .91      .80       .87      .82       .80
                  3     3     3                  <1         <1         <1       <1        <1       <1
 Interval       weeks weeks weeks               week       week       week     week      week     week
 N              106          19        143      106         40         39      143         40      72        29

Effect of TRAINING interviewers !                                                      Client – staff respondent
no training versus training                                                            correspondence




                                                                                                                      2
SIS psychometric characteristics
• Reliability summary

      Cronbach α (int. consistency): .99
      (all subscales α > .93)
      Test-retest: .82 - .94
      Inter-respondent: .82 - .87
      Inter-interviewers: [.59] .80 - .91

      Special case: correlation between client
      response and professional staff / parent responses
          when interviewing the client, always use more
       information sources in order to get a reliable estimate of
      intensity of supports need.




       SIS psychometric characteristics

•   Validity

      Content validity

      content validity of translated SIS = similar to original
      (‘universal construct’ of ‘support need’)




      Criterion validity = independent judgment of intensity of needed supports
      On a 5 point Likert scale per domain and full section 1 scale




                                                                                  3
Criterion validity                      correlations between SIS scores and
                                             independent judgment of support need intensity
                                             by direct support staff


                SIS               Independant        Independent     Independent           Concurrent
                section 1            judgment           judgment       Judgment               validity
                subscale                 staff              staff           staff             ELIDA
                                                                                                 scale


                                          USA 1       Dutch pilot         Spanish        Flemish stand.
                                                           study            study                group



                 A                           .66              .79              .74                 .86
                 B                           .51              .73              .71                 .69
                 C                           .46              .81              .48                 .55
                 D                           .59              .76              .68                 .55
                 E                           .53              .81              .69                 .69
                 F                           .62              .75              .69                 .58
                SIS Index                    .62              .83              .75                 .74
                N                            106             101              352                4,211




       Construct validity                                   Subscale intercorrelations

SIS
Section 1   A               B                  C                 D                   E                    F


A


B           .84/.84/.50


C           .57/.77/.43     .66/.85/.45


D           .59/.73/.30     .62/.77/.34       .85/.82/.54


E           .86/.85/.56     .84/.86/.52       .70/.85/.40       .71/.79/.48


F           .70/.76/.46     .77/.85/.51       .77/.84/.51       .77/.82/.46       .82/.85/.51

SIS Index   .86/.90/.60     .89/.94/.59       .86/.93/.62       .86/.89/.59       .93/.94/.62         .91/.93/.59



                                          Dutch / Flemish study / American original study




                                                                                                                    4
Construct validity                                        Correlation SIS – VABS / ICAP




                      SIS                     VABS total       VABS total        ICAP
                      section 1                   score            score         Score

                                                  USA 1            NL/VL         Spain


                      A                             -.61             -.84         -.71
                      B                             -.57             -.76         -.72
                      C                             -.45             -.41         -.42
                      D                             -.48             -.42         -.61
                      E                             -.52             -.81         -.69
                      F                             -.49             -.65         -.69
                      SIS Index                     -.59             -.77         -.73
                      N                             178               75          352




VABS: Vineland Adaptive Behavior Scale
ICAP: Inventory for Client and Agency Planning




       Construct validity



         Correlations between SIS Section 1 subscales and age & gender (Spain)

                                                    SUBSCALES

                                                                                   Health
                      Home          Community         Lifelong                                Social
                                                                    Employment      And
                      living          living          learning                               activities
                                                                                   safety
     AGE              0.08             0.08            0.05            0.10         0.16         0.07
     GENDER           -0.06            0.00            -0.06           -0.06        0.01         -0.03




     Correlations between SIS Section 1 subscales and gender in Dutch version is
                                   .00 (N=4,211)




                                                                                                          5
Construct validity
                                     Distribution of SIS Index scores
                                     In 4.211 individuals with ID Dutch – Flemish standardization




                             12.00



                                                                                                    Group mean
                 SIS Index
                             10.00




                              8.00




                              6.00




                                        mild        moderate severe         profound

                                           Level of Intellectual Functioning


    Are traditional ‘levels of functioning’ a good indicator of support needs ????




   Factor validity


Spanish and Dutch factor analysis studies confirm 6 factor structure
of section 1


Exploratory factor analysis in Section 1 also shows
a solide 6 factor structure that corresponds with subscales A-F


Extra finding: when section 2 is included in the factor analysis
a 7 factor SIS is a also reliably demonstrated!


Extra finding: in resource allocation tasks, a short form of the SIS (22 items)
showed to be valid, reliable and useful

(Bossaert et al., 2009; Kuppens et al., 2010; also                              Gomiero et al. 2011)




                                                                                                                 6
Conclusions

The SIS fully maintains its (very) high internal consistency throughout all
translations.

Inter-interviewer / inter-respondent correspondences are ‘good’.

Construct validity = OK! / subscale intercorrelations are higher in translated
                    versions (i.e. Dutch & Spanish studies) as compared to US.

Small differences in averages and SD of SIS raw scores related to
standardization sample characteristics?   difficult to tell... A special
international study would be welcome to see how these differences can be
explained.

So far, no independent (re)constructions of SIS in other cultures have been
made. However, some test accreditation councils do stress this appraoch.

A short form of SIS is likely to maintain all psychometric properties of full scale.




            Wil H.E. Buntinx
            Maastricht University
            The Netherlands



            Contact: w.buntinx@maastrichtuniversity.nl



            For international SIS basic training and SIS train-the-trainer courses,
            see: www.buntinx.org




                                                                                       7

More Related Content

Viewers also liked

Sse Courtney Group4
Sse Courtney Group4Sse Courtney Group4
Sse Courtney Group4Liang Wang
 
Kruskal
KruskalKruskal
Kruskalzovche
 
An Introduction to Usability - Chris Burke: for Barcamp Orlando 2010
An Introduction to Usability - Chris Burke: for Barcamp Orlando 2010An Introduction to Usability - Chris Burke: for Barcamp Orlando 2010
An Introduction to Usability - Chris Burke: for Barcamp Orlando 2010Chris Burke
 
SSE_Wumart_Group4
SSE_Wumart_Group4SSE_Wumart_Group4
SSE_Wumart_Group4Liang Wang
 
Ultima Practica
Ultima PracticaUltima Practica
Ultima PracticaKaren Cach
 
D Correa Resume Technical Recruiter V20111024
D Correa Resume Technical Recruiter V20111024D Correa Resume Technical Recruiter V20111024
D Correa Resume Technical Recruiter V20111024dalepcorrea
 
Quality Assessment and Quality Improvement in Support Services for Persons wi...
Quality Assessment and Quality Improvement in Support Services for Persons wi...Quality Assessment and Quality Improvement in Support Services for Persons wi...
Quality Assessment and Quality Improvement in Support Services for Persons wi...Buntinx Training & Consultancy
 
Islamic smoking presentation
Islamic smoking presentation Islamic smoking presentation
Islamic smoking presentation Qatar Academy
 

Viewers also liked (12)

Digital Citizenship
Digital CitizenshipDigital Citizenship
Digital Citizenship
 
Sse Courtney Group4
Sse Courtney Group4Sse Courtney Group4
Sse Courtney Group4
 
Kwaliteitonderzoek met de OKE-LZ methode
Kwaliteitonderzoek met de OKE-LZ methodeKwaliteitonderzoek met de OKE-LZ methode
Kwaliteitonderzoek met de OKE-LZ methode
 
Kruskal
KruskalKruskal
Kruskal
 
EurCongres11dec14 Context.pptx
EurCongres11dec14 Context.pptxEurCongres11dec14 Context.pptx
EurCongres11dec14 Context.pptx
 
Eur congres11dec14 context.pptx
Eur congres11dec14 context.pptxEur congres11dec14 context.pptx
Eur congres11dec14 context.pptx
 
An Introduction to Usability - Chris Burke: for Barcamp Orlando 2010
An Introduction to Usability - Chris Burke: for Barcamp Orlando 2010An Introduction to Usability - Chris Burke: for Barcamp Orlando 2010
An Introduction to Usability - Chris Burke: for Barcamp Orlando 2010
 
SSE_Wumart_Group4
SSE_Wumart_Group4SSE_Wumart_Group4
SSE_Wumart_Group4
 
Ultima Practica
Ultima PracticaUltima Practica
Ultima Practica
 
D Correa Resume Technical Recruiter V20111024
D Correa Resume Technical Recruiter V20111024D Correa Resume Technical Recruiter V20111024
D Correa Resume Technical Recruiter V20111024
 
Quality Assessment and Quality Improvement in Support Services for Persons wi...
Quality Assessment and Quality Improvement in Support Services for Persons wi...Quality Assessment and Quality Improvement in Support Services for Persons wi...
Quality Assessment and Quality Improvement in Support Services for Persons wi...
 
Islamic smoking presentation
Islamic smoking presentation Islamic smoking presentation
Islamic smoking presentation
 

Recently uploaded

Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)lakshayb543
 
Man or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptx
Man or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptxMan or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptx
Man or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptxDhatriParmar
 
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnvESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnvRicaMaeCastro1
 
ROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptxROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptxVanesaIglesias10
 
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...DhatriParmar
 
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptx
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptxDIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptx
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptxMichelleTuguinay1
 
Mental Health Awareness - a toolkit for supporting young minds
Mental Health Awareness - a toolkit for supporting young mindsMental Health Awareness - a toolkit for supporting young minds
Mental Health Awareness - a toolkit for supporting young mindsPooky Knightsmith
 
How to Fix XML SyntaxError in Odoo the 17
How to Fix XML SyntaxError in Odoo the 17How to Fix XML SyntaxError in Odoo the 17
How to Fix XML SyntaxError in Odoo the 17Celine George
 
Unraveling Hypertext_ Analyzing Postmodern Elements in Literature.pptx
Unraveling Hypertext_ Analyzing  Postmodern Elements in  Literature.pptxUnraveling Hypertext_ Analyzing  Postmodern Elements in  Literature.pptx
Unraveling Hypertext_ Analyzing Postmodern Elements in Literature.pptxDhatriParmar
 
Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4JOYLYNSAMANIEGO
 
Multi Domain Alias In the Odoo 17 ERP Module
Multi Domain Alias In the Odoo 17 ERP ModuleMulti Domain Alias In the Odoo 17 ERP Module
Multi Domain Alias In the Odoo 17 ERP ModuleCeline George
 
How to Make a Duplicate of Your Odoo 17 Database
How to Make a Duplicate of Your Odoo 17 DatabaseHow to Make a Duplicate of Your Odoo 17 Database
How to Make a Duplicate of Your Odoo 17 DatabaseCeline George
 
Transaction Management in Database Management System
Transaction Management in Database Management SystemTransaction Management in Database Management System
Transaction Management in Database Management SystemChristalin Nelson
 
31 ĐỀ THI THỬ VÀO LỚP 10 - TIẾNG ANH - FORM MỚI 2025 - 40 CÂU HỎI - BÙI VĂN V...
31 ĐỀ THI THỬ VÀO LỚP 10 - TIẾNG ANH - FORM MỚI 2025 - 40 CÂU HỎI - BÙI VĂN V...31 ĐỀ THI THỬ VÀO LỚP 10 - TIẾNG ANH - FORM MỚI 2025 - 40 CÂU HỎI - BÙI VĂN V...
31 ĐỀ THI THỬ VÀO LỚP 10 - TIẾNG ANH - FORM MỚI 2025 - 40 CÂU HỎI - BÙI VĂN V...Nguyen Thanh Tu Collection
 
Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1GloryAnnCastre1
 
4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptxmary850239
 
Active Learning Strategies (in short ALS).pdf
Active Learning Strategies (in short ALS).pdfActive Learning Strategies (in short ALS).pdf
Active Learning Strategies (in short ALS).pdfPatidar M
 
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...Team Lead Succeed – Helping you and your team achieve high-performance teamwo...
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...Association for Project Management
 
Q-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITWQ-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITWQuiz Club NITW
 

Recently uploaded (20)

Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
 
Man or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptx
Man or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptxMan or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptx
Man or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptx
 
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnvESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
 
ROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptxROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptx
 
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
 
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptx
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptxDIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptx
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptx
 
Mental Health Awareness - a toolkit for supporting young minds
Mental Health Awareness - a toolkit for supporting young mindsMental Health Awareness - a toolkit for supporting young minds
Mental Health Awareness - a toolkit for supporting young minds
 
How to Fix XML SyntaxError in Odoo the 17
How to Fix XML SyntaxError in Odoo the 17How to Fix XML SyntaxError in Odoo the 17
How to Fix XML SyntaxError in Odoo the 17
 
Unraveling Hypertext_ Analyzing Postmodern Elements in Literature.pptx
Unraveling Hypertext_ Analyzing  Postmodern Elements in  Literature.pptxUnraveling Hypertext_ Analyzing  Postmodern Elements in  Literature.pptx
Unraveling Hypertext_ Analyzing Postmodern Elements in Literature.pptx
 
Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4
 
Multi Domain Alias In the Odoo 17 ERP Module
Multi Domain Alias In the Odoo 17 ERP ModuleMulti Domain Alias In the Odoo 17 ERP Module
Multi Domain Alias In the Odoo 17 ERP Module
 
How to Make a Duplicate of Your Odoo 17 Database
How to Make a Duplicate of Your Odoo 17 DatabaseHow to Make a Duplicate of Your Odoo 17 Database
How to Make a Duplicate of Your Odoo 17 Database
 
Transaction Management in Database Management System
Transaction Management in Database Management SystemTransaction Management in Database Management System
Transaction Management in Database Management System
 
31 ĐỀ THI THỬ VÀO LỚP 10 - TIẾNG ANH - FORM MỚI 2025 - 40 CÂU HỎI - BÙI VĂN V...
31 ĐỀ THI THỬ VÀO LỚP 10 - TIẾNG ANH - FORM MỚI 2025 - 40 CÂU HỎI - BÙI VĂN V...31 ĐỀ THI THỬ VÀO LỚP 10 - TIẾNG ANH - FORM MỚI 2025 - 40 CÂU HỎI - BÙI VĂN V...
31 ĐỀ THI THỬ VÀO LỚP 10 - TIẾNG ANH - FORM MỚI 2025 - 40 CÂU HỎI - BÙI VĂN V...
 
Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1
 
4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx
 
Active Learning Strategies (in short ALS).pdf
Active Learning Strategies (in short ALS).pdfActive Learning Strategies (in short ALS).pdf
Active Learning Strategies (in short ALS).pdf
 
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...Team Lead Succeed – Helping you and your team achieve high-performance teamwo...
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...
 
Q-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITWQ-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITW
 
Paradigm shift in nursing research by RS MEHTA
Paradigm shift in nursing research by RS MEHTAParadigm shift in nursing research by RS MEHTA
Paradigm shift in nursing research by RS MEHTA
 

SIS Psychometric Properties Across Countries

  • 1. SIS Supports Intensity Scale Psychometric properties Similarities and differences between original US version and Dutch, Italian, French and Spanish translations Wil Buntinx, PhD / FAAIDD Handout? Go to AAIDD post conference workshop www.buntinx.org St. Paul (MN) – June 9, 2011 Central question: Is the SIS a reliable and valid psychometric tool across countries and cultures? Translating is not enough validity and reliability should be examined in representatieve groups for each language and culture. Standards for Educational and Psychological Testing , Washington: APA 1999 Is the SIS a valid and reliable instrument to assess supports intensity needs of persons with ID in other countries than the US? Translations considered: Dutch (Flanders-Belgium / the netherlands) Italian Spanish (Catalan) French (Canadian) 1
  • 2. SIS relilability Internal consistency (α) – split-half – SEM Dutch Standardization Flemish standardization USA 1 Italy Spain number split- split- SIS subscale of items α SEM α SEM α α α half half Section 1 A 8 .95 .7 .97 .93 .8 .90 .95 0.96 .93 B 8 .96 .6 .96 .95 .7 .91 .95 .97 .97 C 9 .97 .5 .96 .95 .7 .84 .97 .97 .96 D 8 .96 .6 .92 .94 .7 .86 .97 .96 .95 E 8 .95 .7 .94 .92 .8 .86 .94 .95 .93 F 8 .97 .5 .97 .94 .7 .87 .96 .97 .95 SIS Index 49 .99 1,5 .98 .98 2,1 .91 .99 .99 .99 SIS reliability Test-retest - Inter-interviewer - Inter-respondent - client-staff Inter- Client- Test-retest Inter-interviewer staff respondent USA 1 Canad Spain USA 1 USA 2 Canad Spain USA 2 Canad Dutch SIS subscale section 1 A .87 .85 .98 .90 .80 .92 .86 .73 .88 .74 B .74 .77 .94 .68 .89 .82 .86 .91 .87 .58 C .75 .75 .93 .55 .88 .85 .71 .75 .87 .44 D .83 .75 .96 .55 .77 .90 .62 .93 .87 .31 E .86 .81 .90 .72 .96 .79 .71 .91 .91 .77 F .94 .68 .98 .60 .74 .79 .66 .87 .85 .51 SIS Index .82 .84 .94 .59 .88 .91 .80 .87 .82 .80 3 3 3 <1 <1 <1 <1 <1 <1 Interval weeks weeks weeks week week week week week week N 106 19 143 106 40 39 143 40 72 29 Effect of TRAINING interviewers ! Client – staff respondent no training versus training correspondence 2
  • 3. SIS psychometric characteristics • Reliability summary Cronbach α (int. consistency): .99 (all subscales α > .93) Test-retest: .82 - .94 Inter-respondent: .82 - .87 Inter-interviewers: [.59] .80 - .91 Special case: correlation between client response and professional staff / parent responses when interviewing the client, always use more information sources in order to get a reliable estimate of intensity of supports need. SIS psychometric characteristics • Validity Content validity content validity of translated SIS = similar to original (‘universal construct’ of ‘support need’) Criterion validity = independent judgment of intensity of needed supports On a 5 point Likert scale per domain and full section 1 scale 3
  • 4. Criterion validity correlations between SIS scores and independent judgment of support need intensity by direct support staff SIS Independant Independent Independent Concurrent section 1 judgment judgment Judgment validity subscale staff staff staff ELIDA scale USA 1 Dutch pilot Spanish Flemish stand. study study group A .66 .79 .74 .86 B .51 .73 .71 .69 C .46 .81 .48 .55 D .59 .76 .68 .55 E .53 .81 .69 .69 F .62 .75 .69 .58 SIS Index .62 .83 .75 .74 N 106 101 352 4,211 Construct validity Subscale intercorrelations SIS Section 1 A B C D E F A B .84/.84/.50 C .57/.77/.43 .66/.85/.45 D .59/.73/.30 .62/.77/.34 .85/.82/.54 E .86/.85/.56 .84/.86/.52 .70/.85/.40 .71/.79/.48 F .70/.76/.46 .77/.85/.51 .77/.84/.51 .77/.82/.46 .82/.85/.51 SIS Index .86/.90/.60 .89/.94/.59 .86/.93/.62 .86/.89/.59 .93/.94/.62 .91/.93/.59 Dutch / Flemish study / American original study 4
  • 5. Construct validity Correlation SIS – VABS / ICAP SIS VABS total VABS total ICAP section 1 score score Score USA 1 NL/VL Spain A -.61 -.84 -.71 B -.57 -.76 -.72 C -.45 -.41 -.42 D -.48 -.42 -.61 E -.52 -.81 -.69 F -.49 -.65 -.69 SIS Index -.59 -.77 -.73 N 178 75 352 VABS: Vineland Adaptive Behavior Scale ICAP: Inventory for Client and Agency Planning Construct validity Correlations between SIS Section 1 subscales and age & gender (Spain) SUBSCALES Health Home Community Lifelong Social Employment And living living learning activities safety AGE 0.08 0.08 0.05 0.10 0.16 0.07 GENDER -0.06 0.00 -0.06 -0.06 0.01 -0.03 Correlations between SIS Section 1 subscales and gender in Dutch version is .00 (N=4,211) 5
  • 6. Construct validity Distribution of SIS Index scores In 4.211 individuals with ID Dutch – Flemish standardization 12.00 Group mean SIS Index 10.00 8.00 6.00 mild moderate severe profound Level of Intellectual Functioning Are traditional ‘levels of functioning’ a good indicator of support needs ???? Factor validity Spanish and Dutch factor analysis studies confirm 6 factor structure of section 1 Exploratory factor analysis in Section 1 also shows a solide 6 factor structure that corresponds with subscales A-F Extra finding: when section 2 is included in the factor analysis a 7 factor SIS is a also reliably demonstrated! Extra finding: in resource allocation tasks, a short form of the SIS (22 items) showed to be valid, reliable and useful (Bossaert et al., 2009; Kuppens et al., 2010; also Gomiero et al. 2011) 6
  • 7. Conclusions The SIS fully maintains its (very) high internal consistency throughout all translations. Inter-interviewer / inter-respondent correspondences are ‘good’. Construct validity = OK! / subscale intercorrelations are higher in translated versions (i.e. Dutch & Spanish studies) as compared to US. Small differences in averages and SD of SIS raw scores related to standardization sample characteristics? difficult to tell... A special international study would be welcome to see how these differences can be explained. So far, no independent (re)constructions of SIS in other cultures have been made. However, some test accreditation councils do stress this appraoch. A short form of SIS is likely to maintain all psychometric properties of full scale. Wil H.E. Buntinx Maastricht University The Netherlands Contact: w.buntinx@maastrichtuniversity.nl For international SIS basic training and SIS train-the-trainer courses, see: www.buntinx.org 7