Chapter 6 validity & reliability


Published on

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Validity and Reliability are two important characteristics of both the research process and research output.
  • Diligently, they researched whether people liked the new flavor, performing taste tests and giving out questionnaires. People loved the new flavor, so Coca-Cola rushed New Coke into production, where it was a titanic flop. The mistake that Coke made was that they forgot about criterion validity, and omitted one important question from the survey.   People were not asked if they preferred the new flavor to the old, a failure to establish concurrent validity. The Old Coke, known to be popular, was the perfect benchmark, but it was never used. A simple blind taste test, asking people which flavor they preferred out of the two, would have saved Coca Cola millions of dollars.
  • For example, a researcher designs a questionnaire to find out about college students' dissatisfaction with a particular textbook. Analyzing the internal consistency of the survey items dealing with dissatisfaction will reveal the extent to which items on the questionnaire focus on the notion of dissatisfaction.
  • Two or more researchers are observing a high school classroom. The class is discussing a movie that they have just viewed as a group. The researchers have a sliding rating scale (1 being most positive, 5 being most negative) with which they are rating the student's oral responses. Interrater reliability assesses the consistency of how the rating system is implemented. For example, if one researcher gives a "1" to a student response, while another researcher gives a "5," obviously the interrater reliability would be inconsistent. Interrater reliability is dependent upon the ability of two or more individuals to be consistent. Training, education and monitoring skills can enhance interrater reliability.
  • Scientists frequently use  statistics to analyze  their results.  Why do researchers use statistics?  Statistics can help understand a phenomenon by confirming or rejecting a hypothesis. It is vital to how we acquire knowledge to most scientific theories.
  •   Statistical treatment of data also involves describing the data. The best way to do this is through the measures of central tendencies like mean, median and mode. These help the researcher explain in short how the data are concentrated. Range, uncertainty and standard deviation help to understand the distribution of the data. Therefore two distributions with the same mean can have wildly different standard deviation, which shows how well the data points are concentrated around the mean.
  • Note: Variance: Rotweiller = 600 – 394 = 206
  • So, the Variance is  21,704 . And the Standard Deviation is just the square root of Variance, so:
  • Mean = 4 +7+9+6+4 = 30/5 = 6 Variance = Not satisfied : 4-6 = -2 Mod.satisfied : 7-6 = 1 Satisfied : 9-6 = 3 Very Satisfied = 6-6 = 0 Extremely satisfied = 4-6 = -2
  • When reports were written on typewriters, the names of publications were underlined because most typewriters had no way to print italics. If you write a bibliography by hand, you should still underline the names of publications. But, if you use a computer, then publication names should be in italics as they are below. Always check with your instructor regarding their preference of using italics or underlining
  • Format: Books : Author's last name, first initial. (Publication date). Book title . Additional information. City of publication: Publishing company. Encyclopedia : Author's last name, first initial. (Date). Title of Article. Title of Encyclopedia (Volume, pages). City of publication: Publishing company. Examples:
  • Note: Do not enclose the title in quotation marks. Put a period after the title. If a periodical includes a volume number, italicize it and then give the page range (in regular type) without "pp." If the periodical does not use volume numbers, as in newspapers, use p . or pp . for page numbers. Note: Unlike other periodicals, p. or pp. precedes page numbers for a newspaper reference in APA style.
  • Note: When citing Internet sources, refer to the specific website document. If a document is undated, use "n.d." (for no date) immediately after the document title. Break a lengthy URL that goes to another line after a slash or before a period. Continually check your references to online documents. There is no period following a URL. Note: If you cannot find some of this information, cite what is available.
  • Chapter 6 validity & reliability

    2. 2. VALIDITYValidity refers to the degree ofappropriateness, correctness, truthfulnessand accuracy of the study. In other words,the procedure shall measure what isintended to measure.
    3. 3. Types of Validity• Content Validity - pertains to the degree to which the instrument fully assesses or measures the construct of interest. For example, an educational test with strong content validity will represent the subjects actually taught to students, rather than asking unrelated questions.
    4. 4. • Face Validity - is a component of content validity and is established when an individual reviewing the instrument concludes that it measures the characteristic or trait of interest. It requires a personal judgment, such as asking participants whether they thought that a test was well constructed and useful.
    5. 5. • Criterion Validity - assesses whether a test reflects a certain set of abilities. To measure the criterion validity of a test, researchers must calibrate it against a known standard or against itself. Example: One famous example is when Coca-Cola decided to change the flavor of their trademark drink.
    6. 6. RELIABILITYThe degree of consistency between two measures of the same thing.The measure of how stable, dependable, trustworthy, and consistent a test is in measuring the same thing each timeReliability is the extent to which an experiment, test, or any measuring procedure yields the same result on repeated trials.
    7. 7. 4 TYPES OF RELIABILITY1. Equivalency Reliability - is the extent to which two items measure identical concepts at an identical level of difficulty. Also called alternate forms reliability, this type of reliability is used when there is an equivalent test (or another form of the same test) available. Both tests are administered and a correlation between the two is calculated.
    8. 8. 2. Stability Reliability - sometimes called test, re-test reliability) is the agreement of measuring instruments over time.this method requires two administrations of the same test, separated by some time delay (a few days to a few weeks). The scores between the two tests are then correlated.
    9. 9. 3. Internal consistency is the extent to which tests or procedures assess the same characteristic, skill or quality.Measures how well one part of a single test correlates to another part of the same test.
    10. 10. • For example, a researcher designs a questionnaire to find out about college students dissatisfaction with a particular textbook. Analyzing the internal consistency of the survey items dealing with dissatisfaction will reveal the extent to which items on the questionnaire focus on the notion of dissatisfaction.
    11. 11. 4.Interrater reliability is the extent to which two or more individuals agree. Interrater reliability addresses the consistency of the implementation of a rating system
    12. 12. Statistical Treatment of Data
    13. 13. Basic Statistics1. MEAN - The average of a set of n data x M = ∑x n
    14. 14. Standard DeviationDeviation just means how far from the normal.• Its symbol is σ (the greek letter sigma)The formula is easy: it is the square root of the Variance. So now you ask, "What is the Variance?"
    15. 15. What is a Variance?• The average of the squared differences from the Mean.
    16. 16. You and your friends have just measured the heights ofyour dogs (in millimeters): The heights (at the shoulders) are: 600mm, 470mm, 170mm, 430mm and 300mm. Find out the Mean, the Variance, and the Standard Deviation.
    17. 17. • Step 1: Mean = 600 + 470 + 170 + 430 + 300 = 1970 = 394 5 5 so the mean (average) height is 394 mm.
    18. 18. Step 2: o calculate the Variance, take each difference,square it, and then average the result:
    19. 19. Standard Deviation: square root of variance 21,704= 147.32... = 147 (to the nearest mm)
    20. 20. And the good thing about the Standard Deviation is that it is useful. Now wecan show which heights are within one Standard Deviation (147mm) of theMean:So, using the Standard Deviation we have a "standard" way of knowing whatis normal, and what is extra large or extra small.Rottweilers are tall dogs. And Dachshunds are a bit short ..
    21. 21. Let’s try:Compute the Standard Deviation of the survey conducted by the 4th yr. researchers on the level of customer satisfaction of a ABC Restaurant:Out 30 respondents, this is what they gathered: Not satisfied = 4 Moderately satisfied = 7 Satisfied = 9 Very Satisfied = 6 Extremely Satisfied = 4
    22. 22. Mean = 6Variance = (-2)2 + (1)2 + (3)2 + (0)2 + (-2)2 =4+1+9+0+4 = 18Standard Deviation = 18 = 9Majority of the respondents are “satisfied” in the customer service of ABC Restaurant.
    23. 23. Summary, Conclusion andRECOMMENDATION
    24. 24. Results of SummaryYou should keep this section brief andidentify the result with a general statementparagraph which it then followed byanother paragraph that supports theevidence collected. You should avoidinterpretation here and thus be objectiveabout the results.
    25. 25. Discussion of ResultsYou should discuss the meaning of theresults here, in brief, and highlight anyimportant areas that you have identified.You should also look at the different thingsthat the study means and how this isevaluated to the overall understanding inyour thesis.
    26. 26. Recommendations• These could be to your employer or to the academic community. You will want to keep this section brief and maybe to one paragraph or two, and explain what, from the research that has been conducted, there will be recommendations to the organizations or, if you are presenting to academia, then what further research should be conducted in the future
    28. 28. Bibliography APA Style• References. Alphabetize the entries in your list by the authors last name, using the letter-by-letter system (ignore spaces and other punctuation.) Only the initials of the first and middle names are given. If the authors name is unknown, alphabetize by the title, ignoring any A, An, or The.Example:• Allen, T. (1974). Vanishing Wildlife of North America. Washington, D.C.: National Geographic Society.• Boorstin, D. (1992). The creators: A history of the heroes of the imagination. New York: Random House.• Nicol, A. M., & Pexman, P. M. (1999). Presenting your findings: A practical guide for creating tables. Washington, DC: American Psychological Association.
    29. 29. • For dates, spell out the names of months in the text of your paper, but abbreviate them in the list of works cited, except for May, June, and July.Example : (22 July 1999) or the month-day-year style (July 22, 1999)
    30. 30. Underlining or Italics?Typewriter : Publications were underlined(typewriters have no italics) Hand : Name of publication were underlined Computer : Publications should be written initalics
    31. 31. Hanging IndentationAll APA citations should use hanging indents,that is, the first line of an entry should be flushleft, and the second and subsequent linesshould be indented 1/2".
    32. 32. Capitalization, Abbreviation, and Punctuation• Capitalize only the first word of a title and subtitle. The exceptions to this rule would be periodical titles and proper names in a title which should still be capitalized. The periodical title is run in title case, and is followed by the volume number which, with the title, is also italicized.• If there is more than one author, use an ampersand (&) before the name of the last author. If there are more than six authors, list only the first one and use et al. for the rest.• Place the date of publication in parentheses immediately after the name of the author. Place a period after the closing parenthesis. Do not italicize, underline, or put quotes around the titles of shorter works within longer works
    33. 33. FORMAT EXAMPLE:Books :Authors last name, first initial. (Publication date). Book title. Additional information. City of publication: Publishing company.Example:Allen, T. (1974). Vanishing wildlife of North America. Washington, D.C.: National Geographic Society.Boorstin, D. (1992). The creators: A history of the heroes of the imagination. New York: Random House
    34. 34. ENCYCLOPEDIA & DICTIONARY:Format: Authors last name, first initial. (Date). Title of Article. Title of Encyclopedia (Volume, pages). City of publication: Publishing company.Example:Bergmann, P. G. (1993). Relativity. In The new encyclopedia britannica (Vol. 26, pp. 501-508). Chicago: Encyclopedia Britannica.Merriam-Websters collegiate dictionary (10th ed.). (1993). Springfield, MA: Merriam-Webster.
    35. 35. Magazine & Newspaper ArticlesFormat: Authors last name, first initial. (Publication date). Article title. Periodical title, volume number(issue number if available), inclusive pages.Example:Harlow, H. F. (1983). Fundamentals for preparing psychology journal articles. Journal of Comparative and Physiological Psychology, 55, 893-896.Henry, W. A., III. (1990, April 9). Making the grade in todays schools. Time, 135, 28-31.
    36. 36. Website or WebpageFormat: Online periodical: Authors name. (Date of publication). Title of article. Title of Periodical, volume number, Retrieved month day, year, from full URLOnline document: Authors name. (Date of publication). Title of work. Retrieved month day, year, from full URLEx.Devitt, T. (2001, August 2). Lightning injures four at music festival. The Why? Files. Retrieved January 23, 2002, from
    37. 37. ReferencesBattery. (1990). Encyclopedia britannica. (pp. 100-101). Chicago:Encyclopedia Britannica.Best batteries. (December 1994). Consumer Reports Magazine, 32, 71-72.Booth, Steven A. (January 1999). High-Drain Alkaline AA-Batteries. PopularElectronics, 62, 58.Brain, Marshall. How batteries work. howstuffworks. Retrieved August 1,2006, from and batteries. (1993). The DK science encyclopedia. New York: DKPublishing.Dell, R. M., and D. A. J. Rand. (2001). Understanding batteries. Cambridge,UK: The Royal Society of Chemistry.Learning center. Energizer. Eveready Battery Company, Inc. RetrievedAugust 1, 2006, from centre. Duracell. The Gillette Company. Retrieved July 31, 2006,from