TIMELESS LearningPolicy & Practice
JD HoyePresident National Academy Foundation
Theory of  ChangeStudent OutcomesAcademyStudentsLocal Partners & Advisory BoardCommunity BenefitsIncreased Capacity to Implement NAF ModelNAF SupportsAcademy OutcomesLocal Education PartnersNAF Funders & Partners
Research, Data and EvaluationPANEL DISCUSSION
Moving Toward a Performance Measurement System
Performance Measurement Where We Are Going:Clarifying What it Means to Be a NAF Academy
Streamline Data Collection and Reporting
Self-Assessment Tool revised based on feedback from Academies and additional input from experts
Increase Opportunities For Teachers, Administrators, Parents, and StudentsWhere We Have Been:Creating Standards for High Fidelity
Essential Data to Collect
Including all Members of the Network in the Discussion
Rich Dialogue Around Performance MeasurementPerformance Measurement SystemClearly Articulated Standards & BenchmarksSystem for Collecting Data For Those Indicators & BenchmarksSystem for Analyzing Reporting, Process for Using Indicator Data to improve student performance
Components of the NAF Performance Measurement SystemSelf-Assessment & Planning ToolStudent-level Data (ConnectEDU)Stakeholders SurveysCreating a Culture of Inquiry
Context for Performance Measurement System
Key Findings From Previous ResearchMDRC Random Assignment Study Most DefinitiveNo Measurable EffectsAcademic AchievementGraduation RatesPost-Secondary Education OutcomesPositive Effects OnStudent EngagementCareer Related/Work-Based Learning ExperiencesLong Term Earnings (Substantial  Effects)
Key Developments in Research and EvaluationIncreased Premium on Rigorous Evaluation and Reliable Estimates of Program EffectsIncreased Attention to Career and College Readiness and Overlap Between the TwoEarly Indicators of High School SuccessEmphasis on Data-Driven Decision-Making
Need For Additional ResearchImpacts on Academic Outcomes, Post-Secondary EducationKey Drivers of Positive ImpactsUnderstanding Cross-Site Variation in ImpactsNAF Specific Impacts
What does it mean to be a NAF Academy?What Experiences Should Students Have?What Outcomes Should Fully Implemented NAF Academies Achieve?What Data Are RequiredTo Evaluate NAF? To Monitor Implementation?To Maximize program Effectiveness? Key Research & Evaluation Questions for NAF
Components of the NAF Performance Measurement SystemSelf-Assessment & Planning ToolStudent-level Data (ConnectEDU)Stakeholders SurveysCreating a Culture of Inquiry
Self-Assessment and Planning Tool OverviewDevelopment Process:Conducted three tier process:
280 academies reviewed
Variance review, 5 in each region
Visit 15 sites
Site visits conducted by a team of experts including MDRC, National Career Academy Coalition, and members of the Data Tracking Steering Committee to the 15 Academies that scored in the Leader/ Distinguished category or are AOEs
Self-Assessment Tool revised based on feedback from Academies and additional input from expertsGuiding Principles:Clearly articulate a tangible rubric on what defines a NAF Academy
Demonstrate emerging and best practice along the rubric
Develop a tool that allows Academies to reflect on their performance against the rubric
Encourage and inform an exchange of best practices and lessons learned among Academies and between NAF and the AcademiesThe Self-Assessment Tool has 4 key sectionsEach Section outlines the core concepts in the rubric……and drives each concept to a level of tangible detailAcademy DevelopmentAdvisory BoardCurriculum & InstructionWork-Based Learning & InternshipsRecruitment
Cohort scheduling ensures that academy students experience NAF academy courses plus a minimum of two core subject at each grade level.
Cohort scheduling ensures that students receive instruction in a personalized environment    where every child is known by a caring adult.

Research panel

  • 1.
  • 2.
    JD HoyePresident NationalAcademy Foundation
  • 3.
    Theory of ChangeStudent OutcomesAcademyStudentsLocal Partners & Advisory BoardCommunity BenefitsIncreased Capacity to Implement NAF ModelNAF SupportsAcademy OutcomesLocal Education PartnersNAF Funders & Partners
  • 4.
    Research, Data andEvaluationPANEL DISCUSSION
  • 5.
    Moving Toward aPerformance Measurement System
  • 6.
    Performance Measurement WhereWe Are Going:Clarifying What it Means to Be a NAF Academy
  • 7.
  • 8.
    Self-Assessment Tool revisedbased on feedback from Academies and additional input from experts
  • 9.
    Increase Opportunities ForTeachers, Administrators, Parents, and StudentsWhere We Have Been:Creating Standards for High Fidelity
  • 10.
  • 11.
    Including all Membersof the Network in the Discussion
  • 12.
    Rich Dialogue AroundPerformance MeasurementPerformance Measurement SystemClearly Articulated Standards & BenchmarksSystem for Collecting Data For Those Indicators & BenchmarksSystem for Analyzing Reporting, Process for Using Indicator Data to improve student performance
  • 13.
    Components of theNAF Performance Measurement SystemSelf-Assessment & Planning ToolStudent-level Data (ConnectEDU)Stakeholders SurveysCreating a Culture of Inquiry
  • 14.
    Context for PerformanceMeasurement System
  • 15.
    Key Findings FromPrevious ResearchMDRC Random Assignment Study Most DefinitiveNo Measurable EffectsAcademic AchievementGraduation RatesPost-Secondary Education OutcomesPositive Effects OnStudent EngagementCareer Related/Work-Based Learning ExperiencesLong Term Earnings (Substantial Effects)
  • 16.
    Key Developments inResearch and EvaluationIncreased Premium on Rigorous Evaluation and Reliable Estimates of Program EffectsIncreased Attention to Career and College Readiness and Overlap Between the TwoEarly Indicators of High School SuccessEmphasis on Data-Driven Decision-Making
  • 17.
    Need For AdditionalResearchImpacts on Academic Outcomes, Post-Secondary EducationKey Drivers of Positive ImpactsUnderstanding Cross-Site Variation in ImpactsNAF Specific Impacts
  • 18.
    What does itmean to be a NAF Academy?What Experiences Should Students Have?What Outcomes Should Fully Implemented NAF Academies Achieve?What Data Are RequiredTo Evaluate NAF? To Monitor Implementation?To Maximize program Effectiveness? Key Research & Evaluation Questions for NAF
  • 19.
    Components of theNAF Performance Measurement SystemSelf-Assessment & Planning ToolStudent-level Data (ConnectEDU)Stakeholders SurveysCreating a Culture of Inquiry
  • 20.
    Self-Assessment and PlanningTool OverviewDevelopment Process:Conducted three tier process:
  • 21.
  • 22.
    Variance review, 5in each region
  • 23.
  • 24.
    Site visits conductedby a team of experts including MDRC, National Career Academy Coalition, and members of the Data Tracking Steering Committee to the 15 Academies that scored in the Leader/ Distinguished category or are AOEs
  • 25.
    Self-Assessment Tool revisedbased on feedback from Academies and additional input from expertsGuiding Principles:Clearly articulate a tangible rubric on what defines a NAF Academy
  • 26.
    Demonstrate emerging andbest practice along the rubric
  • 27.
    Develop a toolthat allows Academies to reflect on their performance against the rubric
  • 28.
    Encourage and informan exchange of best practices and lessons learned among Academies and between NAF and the AcademiesThe Self-Assessment Tool has 4 key sectionsEach Section outlines the core concepts in the rubric……and drives each concept to a level of tangible detailAcademy DevelopmentAdvisory BoardCurriculum & InstructionWork-Based Learning & InternshipsRecruitment
  • 29.
    Cohort scheduling ensuresthat academy students experience NAF academy courses plus a minimum of two core subject at each grade level.
  • 30.
    Cohort scheduling ensuresthat students receive instruction in a personalized environment where every child is known by a caring adult.

Editor's Notes

  • #7  in your commentary, you will explain that we moved from phases of practice beginning, emerging ,etc to a more standards driven model.Thanks we are going to make sure all the presentations flow in illustatins. I am waiting for otehrs, thanks very mcuh. Right on track
  • #8 To respond to national trends and expectationsTo answer specific questions about the value of the NAF academy at my high school
  • #9 Self Assessment and Planning Tool- academy team review of capacity to meet NAF standards and planning processConnectEDU-Student level data-college and career ready dataPerception Survey- to establish how key stakeholders perceive academy performanceScoring process to determine placement on the NAF development Stages
  • #15 Self Assessment and Planning Tool- academy team review of capacity to meet NAF standards and planning processConnectEDU-Student level data-college and career ready dataPerception Survey- to establish how key stakeholders perceive academy performanceScoring process to determine placement on the NAF development Stages
  • #16  in your commentary, you will explain that we moved from phases of practice beginning, emerging ,etc to a more standards driven model.Thanks we are going to make sure all the presentations flow in illustatins. I am waiting for otehrs, thanks very mcuh. Right on track
  • #18 Self Assessment and Planning Tool- academy team review of capacity to meet NAF standards and planning processConnectEDU-Student level data-college and career ready dataPerception Survey- to establish how key stakeholders perceive academy performanceScoring process to determine placement on the NAF development Stages
  • #19 Again, where does data center fit in, it is participation data, self reported- Perhaps the focus here is that since the data center is self reported it is not as reliable as data directly form the district.Mary is addressing the qualitative piece with the survey, wonder if you should stick to the necessity of student level and at risk data and the importance of comparison data that will be available.
  • #20 Maybe divide CE slides into collection and reporting – what you will know as a reslut ofthis partnership and how it will benefit you.
  • #23 Self Assessment and Planning Tool- academy team review of capacity to meet NAF standards and planning processConnectEDU-Student level data-college and career ready dataPerception Survey- to establish how key stakeholders perceive academy performanceScoring process to determine placement on the NAF development Stages
  • #24 Mary, this is the most important issue for you to discuss. The addition of survey information will enable a more comprehensive review of placement on the network stages.
  • #27 Self Assessment and Planning Tool- academy team review of capacity to meet NAF standards and planning processConnectEDU-Student level data-college and career ready dataPerception Survey- to establish how key stakeholders perceive academy performanceScoring process to determine placement on the NAF development Stages
  • #31 Self Assessment and Planning Tool- academy team review of capacity to meet NAF standards and planning processConnectEDU-Student level data-college and career ready dataPerception Survey- to establish how key stakeholders perceive academy performanceScoring process to determine placement on the NAF development Stages