SlideShare a Scribd company logo
Experimental Design and Analysis CE: 540
Y.Doc.Dr. M. TOLGA GOGUS
Present By : Mustafa Mohammed Rashid
EXAMPLE
A comparison of the reliability of measurements from two
therapists was performed. Data from real time ultrasound
imaging of a muscle in 10 participants, one reading per therapist,
are recorded in columns 2 and 3 in Table. Do the two therapists
produce 'reliable' readings?
 Two techniques exploring the variability of the data to gauge
reliability are demonstrated; interclass correlation coefficient
(ICC) and Bland & Altman plot.
 There are various forms of ICC and they are discussed in the
paper, along with their associated labels and formulae for
calculation, although the worksheet uses SPSS for their
calculations.
 The Bland & Altman plot is illustrated in MS Excel.
 An ICC is measured on a scale of 0 to 1; 1 represents perfect
reliability with no measurement error, whereas 0 indicates no
reliability.
Our estimated reliability between therapists is 0.92, with 95% CI (0.72,
0.98), which is quite 'wide'.
Comments
The Rankin & Stokes (1998) paper gives much more detailed
discussion around measures of reliability. In particular they give
references for the following comments:
• Pearson’s correlation coefficient is an inappropriate measure of
reliability because the strength of linear association, and not
agreement, is measured (it is possible to have a high degree of
correlation when agreement is poor.
• A paired t-test assesses whether there is any evidence that two
sets of measurements agree on average. However, it is the
difference between within-subjects scores that is of interest
(taking the mean score of all subjects has potential to provide
misleading estimates).
• A high scatter of individual differences can result in the difference
between the means being non-significant.
• It is no longer considered to be appropriate (in most cases) to use
the coefficient of variation (CV) to calculate reliability.
Item total statistic
Solving by minitab

More Related Content

What's hot

Model Blending
Model BlendingModel Blending
Model Blending
Amir Amini
 
Measures of central tendency
Measures of central tendencyMeasures of central tendency
Measures of central tendency
renukamorani143
 
Review comments nt
Review comments ntReview comments nt
Review comments nt
tuanhsm
 
Why we run cronbach’s alpha
Why we run cronbach’s alphaWhy we run cronbach’s alpha
Why we run cronbach’s alpha
Aiden Yeh
 
Resume
ResumeResume
0.0 Notes
0.0 Notes0.0 Notes
0.0 Notes
Jacob Cedarbaum
 
Morse et al 2012
Morse et al 2012Morse et al 2012
Morse et al 2012
Brendan Morse
 
Regression and Classification Analysis
Regression and Classification AnalysisRegression and Classification Analysis
Regression and Classification Analysis
YashIyengar
 
Assess data reliability from a set of criteria using the theory of belief fun...
Assess data reliability from a set of criteria using the theory of belief fun...Assess data reliability from a set of criteria using the theory of belief fun...
Assess data reliability from a set of criteria using the theory of belief fun...
IAEME Publication
 
Qualitative Analysis of a Discrete SIR Epidemic Model
Qualitative Analysis of a Discrete SIR Epidemic ModelQualitative Analysis of a Discrete SIR Epidemic Model
Qualitative Analysis of a Discrete SIR Epidemic Model
ijceronline
 
Data Analysis and Statistics
Data Analysis and StatisticsData Analysis and Statistics
Data Analysis and Statistics
T.S. Lim
 
Unit 9 introduction to ih stat
Unit 9 introduction to ih statUnit 9 introduction to ih stat
Factor of reliability, fragility and probability of
Factor of reliability, fragility and probability ofFactor of reliability, fragility and probability of
Factor of reliability, fragility and probability of
Mrinmoy Majumder
 
Regression, Multiple regression in statistics
Regression, Multiple regression in statistics Regression, Multiple regression in statistics
Regression, Multiple regression in statistics
Rashna Asif
 
Use of excel
Use of excelUse of excel
Use of excel
Hasnain Sarwar
 
11.forecast analysis of food price inflation in pakistan
11.forecast analysis of food price inflation in pakistan11.forecast analysis of food price inflation in pakistan
11.forecast analysis of food price inflation in pakistan
Alexander Decker
 
Forecast analysis of food price inflation in pakistan
Forecast analysis of food price inflation in pakistanForecast analysis of food price inflation in pakistan
Forecast analysis of food price inflation in pakistan
Alexander Decker
 
Rationality analysis for arima forecasts
Rationality analysis for arima forecastsRationality analysis for arima forecasts
Rationality analysis for arima forecasts
Alexander Decker
 

What's hot (18)

Model Blending
Model BlendingModel Blending
Model Blending
 
Measures of central tendency
Measures of central tendencyMeasures of central tendency
Measures of central tendency
 
Review comments nt
Review comments ntReview comments nt
Review comments nt
 
Why we run cronbach’s alpha
Why we run cronbach’s alphaWhy we run cronbach’s alpha
Why we run cronbach’s alpha
 
Resume
ResumeResume
Resume
 
0.0 Notes
0.0 Notes0.0 Notes
0.0 Notes
 
Morse et al 2012
Morse et al 2012Morse et al 2012
Morse et al 2012
 
Regression and Classification Analysis
Regression and Classification AnalysisRegression and Classification Analysis
Regression and Classification Analysis
 
Assess data reliability from a set of criteria using the theory of belief fun...
Assess data reliability from a set of criteria using the theory of belief fun...Assess data reliability from a set of criteria using the theory of belief fun...
Assess data reliability from a set of criteria using the theory of belief fun...
 
Qualitative Analysis of a Discrete SIR Epidemic Model
Qualitative Analysis of a Discrete SIR Epidemic ModelQualitative Analysis of a Discrete SIR Epidemic Model
Qualitative Analysis of a Discrete SIR Epidemic Model
 
Data Analysis and Statistics
Data Analysis and StatisticsData Analysis and Statistics
Data Analysis and Statistics
 
Unit 9 introduction to ih stat
Unit 9 introduction to ih statUnit 9 introduction to ih stat
Unit 9 introduction to ih stat
 
Factor of reliability, fragility and probability of
Factor of reliability, fragility and probability ofFactor of reliability, fragility and probability of
Factor of reliability, fragility and probability of
 
Regression, Multiple regression in statistics
Regression, Multiple regression in statistics Regression, Multiple regression in statistics
Regression, Multiple regression in statistics
 
Use of excel
Use of excelUse of excel
Use of excel
 
11.forecast analysis of food price inflation in pakistan
11.forecast analysis of food price inflation in pakistan11.forecast analysis of food price inflation in pakistan
11.forecast analysis of food price inflation in pakistan
 
Forecast analysis of food price inflation in pakistan
Forecast analysis of food price inflation in pakistanForecast analysis of food price inflation in pakistan
Forecast analysis of food price inflation in pakistan
 
Rationality analysis for arima forecasts
Rationality analysis for arima forecastsRationality analysis for arima forecasts
Rationality analysis for arima forecasts
 

Similar to my reliability

How to assess the reliability of measurements in rehabilitation
How to assess the reliability of measurements in rehabilitationHow to assess the reliability of measurements in rehabilitation
How to assess the reliability of measurements in rehabilitation
analisedecurvas
 
statistical estimation
statistical estimationstatistical estimation
statistical estimation
Amish Akbar
 
A Statistical Analysis Of Summarization Evaluation Metrics Using Resampling M...
A Statistical Analysis Of Summarization Evaluation Metrics Using Resampling M...A Statistical Analysis Of Summarization Evaluation Metrics Using Resampling M...
A Statistical Analysis Of Summarization Evaluation Metrics Using Resampling M...
Lisa Cain
 
ProjectWriteupforClass (3)
ProjectWriteupforClass (3)ProjectWriteupforClass (3)
ProjectWriteupforClass (3)
Jeff Lail
 
Chapter 8
Chapter 8Chapter 8
Chapter 8
MaryWall14
 
Measurement theory
Measurement theoryMeasurement theory
Measurement theory
GauriSShrestha
 
What is Reliability and its Types?
What is Reliability and its Types? What is Reliability and its Types?
What is Reliability and its Types?
Dr. Amjad Ali Arain
 
Lab 7 Template1. Using the data you collected for the Week 5 .docx
Lab 7 Template1.  Using the data you collected for the Week 5 .docxLab 7 Template1.  Using the data you collected for the Week 5 .docx
Lab 7 Template1. Using the data you collected for the Week 5 .docx
pauline234567
 
Representing and generating uncertainty effectively presentatıon
Representing and generating uncertainty effectively presentatıonRepresenting and generating uncertainty effectively presentatıon
Representing and generating uncertainty effectively presentatıon
Azdeen Najah
 
cannonicalpresentation-110505114327-phpapp01.pdf
cannonicalpresentation-110505114327-phpapp01.pdfcannonicalpresentation-110505114327-phpapp01.pdf
cannonicalpresentation-110505114327-phpapp01.pdf
JermaeDizon2
 
Lecture note 2
Lecture note 2Lecture note 2
Lecture note 2
sreenu t
 
Agreement analysis
Agreement analysisAgreement analysis
Agreement analysis
Dhritiman Chakrabarti
 
Exploratory Data Analysis and Machine Learning.pptx
Exploratory Data Analysis and Machine Learning.pptxExploratory Data Analysis and Machine Learning.pptx
Exploratory Data Analysis and Machine Learning.pptx
AraniNavaratnarajah2
 
Accuracy precision and significant figures
Accuracy precision and significant figuresAccuracy precision and significant figures
Accuracy precision and significant figures
nehla313
 
Statistical inference: Estimation
Statistical inference: EstimationStatistical inference: Estimation
Statistical inference: Estimation
Parag Shah
 
Confidence Intervals in the Life Sciences PresentationNamesS.docx
Confidence Intervals in the Life Sciences PresentationNamesS.docxConfidence Intervals in the Life Sciences PresentationNamesS.docx
Confidence Intervals in the Life Sciences PresentationNamesS.docx
maxinesmith73660
 
STRUCTURAL EQUATION MODEL (SEM)
STRUCTURAL EQUATION MODEL (SEM)STRUCTURAL EQUATION MODEL (SEM)
STRUCTURAL EQUATION MODEL (SEM)
AJHSSR Journal
 
Practical Methods To Overcome Sample Size Challenges
Practical Methods To Overcome Sample Size ChallengesPractical Methods To Overcome Sample Size Challenges
Practical Methods To Overcome Sample Size Challenges
nQuery
 
IRJET- An Empirical Study on the Relationship Between Meditation, Emotion...
IRJET-  	  An Empirical Study on the Relationship Between Meditation, Emotion...IRJET-  	  An Empirical Study on the Relationship Between Meditation, Emotion...
IRJET- An Empirical Study on the Relationship Between Meditation, Emotion...
IRJET Journal
 
Evidential reasoning based decision system to select health care location
Evidential reasoning based decision system to select health care locationEvidential reasoning based decision system to select health care location
Evidential reasoning based decision system to select health care location
IJAAS Team
 

Similar to my reliability (20)

How to assess the reliability of measurements in rehabilitation
How to assess the reliability of measurements in rehabilitationHow to assess the reliability of measurements in rehabilitation
How to assess the reliability of measurements in rehabilitation
 
statistical estimation
statistical estimationstatistical estimation
statistical estimation
 
A Statistical Analysis Of Summarization Evaluation Metrics Using Resampling M...
A Statistical Analysis Of Summarization Evaluation Metrics Using Resampling M...A Statistical Analysis Of Summarization Evaluation Metrics Using Resampling M...
A Statistical Analysis Of Summarization Evaluation Metrics Using Resampling M...
 
ProjectWriteupforClass (3)
ProjectWriteupforClass (3)ProjectWriteupforClass (3)
ProjectWriteupforClass (3)
 
Chapter 8
Chapter 8Chapter 8
Chapter 8
 
Measurement theory
Measurement theoryMeasurement theory
Measurement theory
 
What is Reliability and its Types?
What is Reliability and its Types? What is Reliability and its Types?
What is Reliability and its Types?
 
Lab 7 Template1. Using the data you collected for the Week 5 .docx
Lab 7 Template1.  Using the data you collected for the Week 5 .docxLab 7 Template1.  Using the data you collected for the Week 5 .docx
Lab 7 Template1. Using the data you collected for the Week 5 .docx
 
Representing and generating uncertainty effectively presentatıon
Representing and generating uncertainty effectively presentatıonRepresenting and generating uncertainty effectively presentatıon
Representing and generating uncertainty effectively presentatıon
 
cannonicalpresentation-110505114327-phpapp01.pdf
cannonicalpresentation-110505114327-phpapp01.pdfcannonicalpresentation-110505114327-phpapp01.pdf
cannonicalpresentation-110505114327-phpapp01.pdf
 
Lecture note 2
Lecture note 2Lecture note 2
Lecture note 2
 
Agreement analysis
Agreement analysisAgreement analysis
Agreement analysis
 
Exploratory Data Analysis and Machine Learning.pptx
Exploratory Data Analysis and Machine Learning.pptxExploratory Data Analysis and Machine Learning.pptx
Exploratory Data Analysis and Machine Learning.pptx
 
Accuracy precision and significant figures
Accuracy precision and significant figuresAccuracy precision and significant figures
Accuracy precision and significant figures
 
Statistical inference: Estimation
Statistical inference: EstimationStatistical inference: Estimation
Statistical inference: Estimation
 
Confidence Intervals in the Life Sciences PresentationNamesS.docx
Confidence Intervals in the Life Sciences PresentationNamesS.docxConfidence Intervals in the Life Sciences PresentationNamesS.docx
Confidence Intervals in the Life Sciences PresentationNamesS.docx
 
STRUCTURAL EQUATION MODEL (SEM)
STRUCTURAL EQUATION MODEL (SEM)STRUCTURAL EQUATION MODEL (SEM)
STRUCTURAL EQUATION MODEL (SEM)
 
Practical Methods To Overcome Sample Size Challenges
Practical Methods To Overcome Sample Size ChallengesPractical Methods To Overcome Sample Size Challenges
Practical Methods To Overcome Sample Size Challenges
 
IRJET- An Empirical Study on the Relationship Between Meditation, Emotion...
IRJET-  	  An Empirical Study on the Relationship Between Meditation, Emotion...IRJET-  	  An Empirical Study on the Relationship Between Meditation, Emotion...
IRJET- An Empirical Study on the Relationship Between Meditation, Emotion...
 
Evidential reasoning based decision system to select health care location
Evidential reasoning based decision system to select health care locationEvidential reasoning based decision system to select health care location
Evidential reasoning based decision system to select health care location
 

my reliability

  • 1. Experimental Design and Analysis CE: 540 Y.Doc.Dr. M. TOLGA GOGUS Present By : Mustafa Mohammed Rashid
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19. EXAMPLE A comparison of the reliability of measurements from two therapists was performed. Data from real time ultrasound imaging of a muscle in 10 participants, one reading per therapist, are recorded in columns 2 and 3 in Table. Do the two therapists produce 'reliable' readings?
  • 20.  Two techniques exploring the variability of the data to gauge reliability are demonstrated; interclass correlation coefficient (ICC) and Bland & Altman plot.  There are various forms of ICC and they are discussed in the paper, along with their associated labels and formulae for calculation, although the worksheet uses SPSS for their calculations.  The Bland & Altman plot is illustrated in MS Excel.  An ICC is measured on a scale of 0 to 1; 1 represents perfect reliability with no measurement error, whereas 0 indicates no reliability.
  • 21.
  • 22. Our estimated reliability between therapists is 0.92, with 95% CI (0.72, 0.98), which is quite 'wide'.
  • 23. Comments The Rankin & Stokes (1998) paper gives much more detailed discussion around measures of reliability. In particular they give references for the following comments: • Pearson’s correlation coefficient is an inappropriate measure of reliability because the strength of linear association, and not agreement, is measured (it is possible to have a high degree of correlation when agreement is poor. • A paired t-test assesses whether there is any evidence that two sets of measurements agree on average. However, it is the difference between within-subjects scores that is of interest (taking the mean score of all subjects has potential to provide misleading estimates). • A high scatter of individual differences can result in the difference between the means being non-significant. • It is no longer considered to be appropriate (in most cases) to use the coefficient of variation (CV) to calculate reliability.