Standardized test analysis project


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Standardized test analysis project

  1. 1. Primary and Intermediate Measures of Music Audiation<br />Carlee Francisco & Morgan Soja<br />
  2. 2. Bibliographic Information<br />PMMA<br /><ul><li>Edwin Gordon, GIA Publications Inc., 1979, 1986.
  3. 3. $100 for the complete kit
  4. 4.</li></ul>IMMA<br />Edwin Gordon, GIA Publications Inc., 1979, 1986.<br />$100 for the complete kit<br /><br />Other products available<br />
  5. 5. Purpose of Tests<br />Designed to act as an objective aid <br />Teachers and parents <br />Help each child make the best use of his or her music aptitudes <br />Provide appropriate opportunities and instruction<br />PMMA and IMMA:<br />
  6. 6. Purpose of Tests cont.<br />Used periodically to evaluate tonal and rhythm aptitudes. Comparing scores helps inform instruction for child’s individual music needs.<br />Used periodically to identify high achieving children who can profit from additional music activities. (PMMA is not designed to exclude).<br />Used periodically to provide normative analysis (comparing each child with other children) to inform instruction and further the child’s individual music needs. <br />
  7. 7. What is audiation?<br />Gordon states: “Audiation takes places when one hears and feels music through recall or creativity, the sound not being physically present except when one is audiating while also aurally perceiving music that is being performed by others or that one is performing himself.”<br />PMMA and IMMA = immediate reaction<br />No memory involved<br />
  8. 8. Individualized instruction requires understanding of aptitude<br />Developmental music aptitude for young children seems directly concerned with same and different<br />Stabilized music aptitude for older children and adults seems indirectly concerned with same and different<br />Children and adults audiate differently thus, learn music differently, thus must be taught differently<br />
  9. 9. Level<br />PMMA<br />K-Grade 3<br />Teaching 4-12, not necessarily appropriate<br />Test audiation, identify students for remediation<br />IMMA<br />Grades 1-6 <br />Developmental aptitude, AND stabilized aptitude<br />Students with high achievement (80%+) on PMMA <br />Changes in aptitude cannot be made by comparing scores on PMMA and IMMA.<br />5-6th grade chorus<br />
  10. 10. Validity<br />Content<br />Concurrent<br />Inverse Concurrent<br />Congruent<br />Longitudinal Predictive<br />
  11. 11. Content Validity<br />Gordon states: “unless the teacher who uses a test is satisfied that the test content fairly represents factors which should be measured, he will have no confidence in the results of the tests, even if the objective validity of the test is overwhelming.”<br />Review of the manual, the purpose, the studies that provide the basis for the tests.<br />
  12. 12. Intercorrelation of Test Items<br />Intercorrelation was significant at the 1% level if the coefficient was .23 or higher.<br />Only pairs of items that were intercorrelated at ±.30 or higher were used on the tests.<br />
  13. 13. Intercorrelation of Test Items cont.<br />26% (422) of tonal pattern intercorrelations≤ .30 were positive, 8% negative (34% or 544 total)<br />17% (277) of rhythm pattern intercorrelations ≤ .30 were positive, 9% (147) negative (26% or 424 total)<br />Lower rhythm is likely due to lower reliability on rhythm test items and test.<br />
  14. 14. Concurrent Validity <br />Gordon notes that correlation of test scores with teacher given grades is not the best measure of validity for a test of music aptitude. <br />Constant variation of developmental music aptitude<br />
  15. 15. Concurrent Validity cont.<br />Chesnut Hill Academy – Philadelphia, PA<br />Private boys school<br /> Music teacher asked to rate children on 5-point scale for overall music aptitude.<br />
  16. 16. Chesnut Hill Academy cont.<br /> Music teacher ratings were correlated with IMMA scores<br />* Note low rhythm correlations – emphasis of need to use test to diagnose musical strengths and weaknesses in all children<br />
  17. 17. Concurrent Validity cont.<br />Jenkintown, PA - Elementary School<br /> Band director (4th Grade) asked to rate children on a five-point scale for overall music aptitude<br /> 32 students in fourth grade – 21 in instrumental class (group who received ratings)<br /> IMMA was administered to children after ratings given<br />
  18. 18. Jenkintown, PA cont.<br /> Music teacher ratings were correlated with IMMA scores<br />Follow up study was conducted<br />
  19. 19. Jenkintown, PA cont.<br /> 11 students that were not involved in instrumental class<br />No real differences in the means of groups<br />*Highest scoring student was in non-instrumental group<br />
  20. 20. Concurrent Validity cont.<br />West Irondequoit – Kindergarten 1st, 2nd , 3rd<br /> Means and standard deviations for PMMA given approximately two weeks apart show less than 1.3 points gained or lost when students were re-tested<br /> Practice effects do not effect validity of the PMMA or IMMA<br />
  21. 21. Concurrent Validity cont.<br />PMMA Test-Retest Means and Standard Deviations<br />
  22. 22. Inverse Concurrent Validity<br />PMMA and IMMA should have little in common with tests that are designed for purposes other than measuring music aptitude<br />PMMA scores (tonal, rhythm, composite) were correlated with scores on academic achievement and intelligence tests.<br />
  23. 23. Inverse Concurrent Validity cont.<br />Kindergarten<br />
  24. 24. Inverse Concurrent Validity cont.<br />Grade One<br />
  25. 25. Inverse Concurrent Validity cont.<br />Grade Two<br />
  26. 26. Inverse Concurrent Validity cont.<br />Grade Three<br />
  27. 27. Inverse Concurrent Validity cont.<br />Grade Three<br />
  28. 28. Inverse Concurrent Validity cont.<br /> Correlations between PMMA and academic/intelligence tests are low<br /> Correlations are similar to that of MAP and academic/intelligence tests<br /> Valid music aptitude tests should have no more than 15% of their variance in common with academic tests.<br /> What is in common is likely due to behaviors that are similar.<br />
  29. 29. Congruent Validity<br /> Correlation of two tests that are designed for the same purposes<br /> Examination of PMMA, IMMA and MAP<br />
  30. 30. Congruent Validity cont.<br />Great Valley, PA<br /> PMMA and IMMA administered approximately one month apart<br />
  31. 31. Congruent Validity cont.<br />Correlations of PMMA and IMMA<br />
  32. 32. Congruent Validity cont.<br /> Correlations of PMMA and IMMA are almost as high as split halves reliabilities for IMMA<br /> The two tests share much in common<br /> High scores on PMMA suggest need to administer IMMA<br />
  33. 33. Congruent Validity cont.<br />Troy, NY – IMMA and MAP<br /> -Only Tonal Imagery and Rhythm Imagery sections of MAP were used<br />
  34. 34. Congruent Validity cont.<br /> IMMA has as much in common with the MAP as it has with the PMMA<br /> Valid for developmental and stabilized music aptitude<br />
  35. 35. Congruent Validity cont.<br />West Irondequoit, NY – 4th Grade<br />PMMA and MAP administered to 227 children in 4th grade over 3 week period<br />
  36. 36. Congruent Validity cont.<br /> High scores for 4th graders<br /> Ceiling effect<br /> Split halves reliabilities:<br /> Coefficients were corrected<br /> PMMA has more in common with IMMA than MAP<br />
  37. 37. Longitudinal Predictive Validity<br />Plymouth Meeting, PA – Parochial School<br />Students required to take violin for one year<br /> Just after receiving violins and/or playing for no longer than a year, the PMMA was administered to 26 students who were 7 or 8 years old<br /> At the end of one year of violin instruction, students performed on violin and were rated in three categories by two judges<br />
  38. 38. Longitudinal Predictive Validity cont.<br />PMMA and Instrumental Performance Achievement<br /> One year predictive validity coefficients for PMMA are comparable to three year predictive validity coefficients for MAP<br />
  39. 39. Longitudinal Predictive Validity cont.<br /> Violin teacher rated students (on a five point scale) in overall music aptitude after teaching them for one year<br />Validity coefficients are lower than longitudinal predictive validity coefficients.<br />-Teacher’s knowledge of music aptitude is less accurate than what PMMA can determine even before instruction<br />
  40. 40. Longitudinal Predictive Validity cont.<br />Hunter College Elementary – NYC, NY<br /> Five year old children from two classes (n=13, and n=10)<br /> PMMA administered before 12 week group violin lessons were taught by one instructor<br /> Children performed by rote two musical phrases with rhythmic variation<br />Children’s performances were reviewed by two judges, whose ratings were correlated with PMMA scores<br />
  41. 41. Longitudinal Predictive Validity cont.<br />Hunter College Elementary – NYC, NY<br /> PMMA Longitudinal Predictive Validity Coefficients using Instrumental Performance Achievement<br />
  42. 42. Longitudinal Predictive Validity cont.<br />Chesnut Hill, PA<br />33 students, aged nine ( 4th grade) studied violin and recorder for one semester each<br /> IMMA was administered before instrumental instruction<br /> After instructional period, students sang two songs that they then performed on their instrument<br /> Two judges evaluated melodic and rhythmic accuracy and musical expression<br />
  43. 43. Longitudinal Predictive Validity cont.<br />Chesnut Hill, PA<br />IMMA Longitudinal Predictive Validity Coefficients using Ratings of Instrumental Performance Achievement<br />
  44. 44. Longitudinal Predictive Validity cont.<br />Chesnut Hill, PA<br />Coefficients are similar to predictive validity coefficients for the PMMA and the MAP<br /> To study practice and training effects, on the IMMA tests were given at the end of the instructional period<br /> Negligible differences <br /> IMMA is a valid test of developmental and stabilized music aptitude, and practice and training have no effect on a valid test of stabilized music aptitude<br />
  45. 45. Longitudinal Predictive Validity cont.<br />IMMA Pre and Post Instruction Means and SD<br />IMMA Pre-and Post Instruction Correlations<br />
  46. 46. Reliability<br /><ul><li>Split-Halves Reliability
  47. 47. Test-Retest Reliability
  48. 48. Standard Error of Measurement</li></li></ul><li>Reliability<br />PMMA Reliability and SE<br />
  49. 49. Reliability<br />IMMA Reliability and SE<br />
  50. 50. Reliability<br />Tonal test reliabilities were higher<br />Could be due to difficulty with audiation of rhythm patterns for young children<br />IMMA includes criterion scores used for indentifying students with exceptionally high music aptitude<br />Children who achieved the criterion score or higher received a 2, and those who did not were awarded a 1.<br />Test-Retest reliabilities for criterion scores are higher than the split-halves reliabilities and lower than the test-retest reliabilities of the raw scores<br />Odd/Even scores were not used for IMMA<br />Abnormal distribution of item difficulties<br />
  51. 51. Test Construction<br />2 subtests – Tonal, Rhythm<br />Items presented on CD (although directions refer to cassette tape)<br />First example is announced then played, second example is announced, then played<br />Children circle two happy faces for “same”, one happy, one sad face for “different”<br />Directions are spoken (female voice), examples are played on synthesizer<br />
  52. 52. Standardization Process<br />Instructions are clear<br />Audio device where children can hear<br />Answer sheets on desk when students come in, or given afterwards<br />K – students may sit on floor and spread out<br />Extra pencils for students<br />Student’s name on answer sheet (clearly)<br />Become familiar with recording before administering<br />Make sure children understand<br />Teacher may start and stop recording at any time<br />Clearly written directions in manual<br />
  53. 53. Manual<br />Face validity <br />Attractiveness <br />Orderly and understandable <br />Necessary information provided <br />Usefulness to test administrator <br />
  54. 54. Manual<br />Face validity <br />Attractiveness <br />Orderly and understandable <br />Necessary information provided <br />Usefulness to test administrator <br />
  55. 55. Audio Recording<br />Quality of directions on recording is good<br />Quality of tonal examples is good<br />Rhythmic examples are difficult to hear at times<br />Presentation is currently in CD form<br />(previously cassette tape) <br />
  56. 56. Scoring & Answer Sheet<br />Scoring masks provided = Easy, quick scoring<br />Accurate scoring with no machine<br />CONVERSION OF SCORES?<br />Answer sheet is clear and easily understandable<br />Use of pictures instead of numbers<br />
  57. 57. Norms<br />
  58. 58. Norms cont.<br />
  59. 59. Item Discrimination<br />
  60. 60. Item Difficulty<br />
  61. 61. Uses<br />
  62. 62. General Opinion<br />