1. The association between youth
participation in the El Sistema
orchestra program at
the Rainey Institute and academic
outcomes
Allison Silverman, MPH Candidate
August 4th, 2016

2. Introduction
CEOMPH Class of 2016
- Home Institution: Cleveland State University
Preceptors
- Faculty: Dr. Joanna DeMarco
- Program Director, Cleveland State University
- Community: Mr. Lee Lazar
- Executive Director of The Rainey Institute

3. Outline
- Background of study
- Methods
- Results
- Analysis
- Discussion and Recommendations
- Core Courses & Electives

4. Background of Study

5. Literature Review
- Statistical significant relationship between almost daily
after-school activities and academic skills
- Students in quality instructional programs performed
better on standardized tests
- Students participating in instrumental music received
higher test scores compared to non-instrumental
students at high and low-SES levels

6. Background on El Sistema
Orchestra Program
- Intensive orchestra program offered at the Rainey
Institute
- Three hours per day and five days per week
- Most popular program at the Rainey Institute
- Monthly performances

7. Purpose of Study
- Preliminary study looking at association between
participation in arts-based after-school program and
academic achievement
- Hypothesis: Standardized tests are positively affected by
participation in the El Sistema Orchestra Program
- Goal is to show a cause-and-effect relationship
- Need a more rigorous evaluation

8. SMART Objectives
1) Standardized test scores will be obtained for 85% of the
El Sistema orchestra participants by April 1st.
2) By May 1st, it will be determined that at least 75% of the
El Sistema participants have at least met the minimum
standards for the OAA or PARCC standardized tests.
3) By April 1st, El Sistema attendance will be obtained for
60% of the El Sistema study participants.

9. Methods

10. Design of Study
- Secondary, retrospective data (records review)
- Cross-sectional study
- Variables for El Sistema subjects
- Standardized test results for math and reading
- Grade
- Gender
- Comparison data of Cleveland Metropolitan School
District (CMSD) students
- From Department of Education’s Ohio State Report Card

11. Model for the study: Social
Cognitive Theory
- Three concepts to theory
- Reinforcement
- Behavior
- Cognition
- Theory cannot prove causation
- Mutually-influenced elements

12. Sample : Targeted Sample
- Targeted Sample: El Sistema Orchestra Participants at
the Rainey Institute
- Sample estimate: n = 65 participants
- Actual sample: n = 13 participants
- Further Breakdown of Actual Targeted Sample
- Represented only one CMSD school (total of 96 schools)
- Stratified by grade (4th, 5th, and 6th grade)
- Stratified by type of standardized test (Math and English/Reading)

15. Sample : Comparison Sample
- 1st Comparison Sample: All CMSD students
- Total Sample: n = 15,480
- Total number of schools: n = 96 schools
- 2nd Comparison Sample: All schools in the State of Ohio
- Further Breakdown of Comparison Sample
- Stratified by grade (4th, 5th, and 6th grade)
- Stratified by type of standardized test (Math and English/Reading)

18. Results

19. Measures: Descriptive Statistics
- Cluster Bar Charts: Percentage for Proficient or Above
Test Scores
- Two charts for each type of standardized test (Math and
Reading/English)
- Stratified by Sample (Targeted and Two Comparison Samples)
- Cluster Bar Chart: Weighted Averages
- Combing all grade levels (4th, 5th, and 6th Grade)
- Stratified by type of standardized test (Math and Reading/English)
- Stratified by sample (subjects in El Sistema Program at Rainey and
all CMSD subjects)

20. Measures: Descriptive Statistics
- Cluster Bar Charts: Attendance Rate Data
- Stratified by Grade (4th, 5th, and 6th Grade)
- Stratified by sample (subjects in El Sistema Program at Rainey and
all CMSD subjects)

25. Analysis
- Small sample size of target population
- Response rate for standardized tests
- 20% for math
- 14% for reading/english
- Difficult to detect statistically significant differences
- Lack of feasibility to conduct inferential statistics
- Difference in passing rate of standardized tests and attendance rate
between El Sistema Orchestra and all CMSD subjects
- Passing rate was almost double for both standardized tests

26. Analysis
- Objective: standardized test scores will be obtained for
85% of El Sistema Orchestra Participants
- Obtained 20% or less of standardized test scores for both reading
and math tests (DID NOT MEET OBJECTIVE)
- Objective: attendance data will be obtained for 60% of
the El Sistema study participants
- Obtained attendance data for 100% of study participants (MET
OBJECTIVE)

27. Analysis
- Objective: at least 75% of El Sistema subjects should
meet state standards
- 69.2% met state standard for math test (DID NOT MEET
OBJECTIVE)
- 88.9% met state standard for reading/english test (MET
OBJECTIVE)

28. Discussion and
Recommendations

29. Discussion: explanation of results
- Could not detect significant difference in passing rate
- Sample size was too small to use chi-square test
- Sample size was too small to create a generalized linear model
(GLM)
- GLM and Chi-Square test are sensitive to sample size
- Chi-Square Test: need at least five variables in each cell
- GLM: difficult to detect specific statistical differences between
groups

30. Discussion: Limitations
- Non-response bias
- Larger than 5% margin of error
- Impacted reliability of the study
- Could not use inferential statistics

31. Discussion: deficiency of
research protocols
- Arts-based research is lacking certain research protocols
- Small number of longitudinal studies
- Limiting cause and effect studies
- Use of Cross-sectional studies
- Most arts-based research studies used a cross-sectional approach
- Quicker and more affordable study than longitudinal
- Both are observational studies

32. Discussion: Comparison of study with
previous research
- Babo, 2004 multiple regression analysis study
- Looked at association between academic achievement and
instrumental music participation
- Controlled for socioeconomic status (SES) and gender
- Showed significant results for reading standardized test scores
- Large enough sample size

33. Discussion: Comparison of study with
previous research
- Thornton, 2013 statewide standardized assessment
comparison study
- Between music and non-music students in Pennsylvania
- Low attrition rate: 11 out of 187 school districts participated

34. Discussion: Comparison of study with
previous research
- Meta-analysis from the SERVE Center at The University
of North Carolina
- Associations for participation in an after-school program include
- Improved school attendance
- Better grades/achievement
- More positive attitudes towards school
- Better work and interpersonal skills

35. Discussion: protocols for a future
research study
- Goal is to establish a more causal relationship
- Problem: Using only standardized test scores to measure
academic achievement
- Future: use a mixed-method approach

36. Discussion: protocols for a future
research study
- Identify and control for extraneous factors
- Data sharing agreement between Rainey Institute and
CMSD
- Creation and distribution of socio-emotional
questionnaire
- Use of a growth model for measuring academic
achievement

37. Overall Conclusions
- Analyzed a Rainey Institute program using statistical analyses
- El Sistema Orchestra participants may have a greater likelihood of
passing standardized tests for math and reading compared to all
non-participating CMSD students
- Limited feasibility of study due to small sample size and deficiency
of data
- Future studies should attempt to show more concrete statistical
conclusions

38. Core Courses and Electives

39. Public Health Concepts
- Fall, 2014
- Instructor: Amy Lee, MD, MPH, MBA
- Proposal: Improving Food Insecurity in Franklin County
- Competencies: F1,2,4-7; J1-11; K1,2,9; L1-4,6,8-11;
M1,3-5,7,8

40. Biostatistics in Public Health
- Fall, 2014
- Instructor: G. Andy Chang, PhD
- Assignment: A Comparison of Sugary Beverage
Consumption and Gender
- Competencies: A1-10; F1,2,5-10; H4-6; M4

41. Epidemiology in Public Health
- Spring, 2015
- Instructor: Alexander Sergeev, MD, PhD, MPH
- Assignment: Osteoporosis Poster Presentation
- Competencies: A1, 4; C1-10; F1,5,7,8; G6; H4, 6; L11

42. Social and Behavioral Sciences in
Public Health
- Spring, 2015
- Instructor: Don Allensworth-Davies, PhD, MSc
- Assignment: Addressing the Abuse of Prescription
Medications by the Adolescent Population of Ohio
- Competencies: A5; D3-4, 9-10; E1-10; F,2-5, 7; G9-10;
H2-8; J1-3, 5-6, 8-9; K1-6, 9-10; L1, 4-5, 7-9, 11; M1, 3-5,
8

43. Environmental Health Sciences
- Fall, 2015
- Instructor: Felicia Armstrong & Mark Adams
- Assignment: Human Trafficking and the Environment
- Competencies: B1-8; D3; H4, 6; J1, 5; L2-3, 5, 8, 10-11

44. Health Services Administration
- Fall, 2015
- Instructor: Lynn Clough, Ph.D
- Assignment: Safety on the Streets: An Intervention
Addressing Gun Violence through Education
- Competencies: A9; D1-2, 4-10;E1-6, 8-10; F7; G1-7,
9-10; H1-9; I2; J1-5, 7-10; K2-5, 7-10; L1-2, 6-9, 11;
M3-7

45. Public Health Practice and Issues
- Summer, 2015
- Instructors: Julie Aultman, PhD, Neil Casey, MS, Amy Lee,
MD, MPH, MBA, and Keisha Robinson, DrPH, MPH
- Assignment: Cleveland Police and Youth: How to make a
good relationship
- Competencies: A10; B1-2, 7; C1-4, 6, 8-10; D2-4; E9;
F1-10; G1-10; H1-2, 4, 6-9; I1-10; J2-11; K1-2, 7, 10;
L1-8, 11; M1-4, 5, 7

46. Grant Writing
- Spring, 2016
- Instructor: Rachael J. Pohle-Krauza, PhD, RD/LD
- Assignment: Mock Grant Proposal- Expansion of an
exercise program for low-income older adults
- Competencies: A5, 9; C1, 3, 9-10; D5-6; E3-8; G1, 7; H4,
6-9; J3; K1-8, 10; L1, 8-9, 11; M3, 5-6, 8

47. Capstone
- Fall, 2016
- Instructors: Amy Lee, MD, MPH, MBA & Kim Trowbridge
- Competencies: A1-10; C1, 5, 9-10; E1-2, 4, 6, 8; F7-10;
G1, 5; H2-4, 6; J3, 5, 9-10; K3, 5, 7, 10; L6, 8, 11; M3, 5

48. Electives
- Introduction to Health Policy
- Northeast Ohio Medical University
- Michelle Chyatte, DrPH, MPH
- Healthcare Law & Ethics, HCA 640
- Cleveland State University
- Dr. Meehan

49. Electives
- Culture & Health Care, HSC 505
- Cleveland State University
- Kate Elizabeth Masley
- Measurement and Evaluation, HED 553
- Cleveland State University
- Manuella Crawley