Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

JPI Conference Dublin - Edvard Beem - Evaluation and Monitoring Framework


Published on

JPI Conference Dublin - Edvard Beem - Evaluation and Monitoring Framework

  • Login to see the comments

  • Be the first to like this

JPI Conference Dublin - Edvard Beem - Evaluation and Monitoring Framework

  1. 1. EU Joint Programme - Neurodegenerative Diseases Research (JPND) Developing a framework for monitoring and evaluationJoint Programming Conference 2013Dublin, 28 February & 1 March 2013, Parallel Session 8Edvard Beem, Co-director ZonMw, member of the Executive Board of JPND
  2. 2. This presentation• Rationales and objectives of JPND• Development of a monitoring and evaluation framework • Logical Framework Analysis as a systemic tool • Overview of the defined set of indicators of performance• Execution of monitoring to date• Key issues, challenges and lessons learned
  3. 3. Alzheimer’s Disease in Europe 40 Number of Societal costs (€) cases 30 Dementia 2010 6,000,000 72,000,000,000 Cases / 100 20 2040 12,000,000 144,000,000,000 Stroke 10 Parkinsons 0 A major societal challenge 60 70 80 90 100 Age (years) for the coming yearsFrom Rocca W et al, Eurodem
  4. 4. A major societal« Grand Challenge » Energy Climate Change Food & Health Neurodegeneration
  5. 5. Scope of JPND Scientific Scientific • Animal models Medical Medical Animal models • Biobanks • Biobanks • Early diagnosis Early diagnosis Focus on • Prevention • Cohorts/registries • Cohorts/registries • PreventionThree Domains • Disease pathology • Disease pathology • Clinical trials • Clinical trials Social Social • Health care delivery Health care delivery • Home automation • Home automation • Health economics • Health economics • Ethics • EthicsDiseases Targeted Alzheimer’s Disease and other Parkinson’s Disease & related disorders dementias Huntington’s Disease Motor Neurone Disease Spinocerebellar Ataxia (SCA) Prion Disease Spinal Muscular atrophy (SMA)
  6. 6. What are the JPND goals?• Increase coordinated investment in ND research aimed at finding causes of disease, developing cures, and identifying appropriate ways to care for those with ND• Our immediate goals: • Launch activities that add value to national research efforts • Leverage additional resources through partnerships • Link and support national plans with relevance for ND
  7. 7. Why JPI?• Awareness • common challenge, common vision• Mapping • Inventory running national programmes (database) • Overlaps and gaps, opportunities for joint actions • Baseline for monitoring• SRA (>10 years) • Roadmap for EU-wide investment• Implementation plan phase 1 2012-14 • Joint actions: calls, enabling activities, action groups
  8. 8. CALLSTotal commitment 2011-2014• Pilot Joint Transnational Call 2011 = €15M• CoEN program call 2011 (€ 4M) and calls 2012-2014 (€12M)• Joint Transnational Call 2012 • “identification of genetic, epigenetic and environmental risk and protective factors” = €19M • “evaluation of health care policy, strategies and interventions = € 10M• Estimate commitment 2013 = €25M• Estimate commitment 2014 = €25M• Total projected investment: over €100M in cash contribution (no in-kind resources included)
  9. 9. Enabling activitiesJPND will execute the SRA within the next ten years through a combinationof: • Knowing our research • Capacity capability building • Opportunities for • Education & infrastructure and platforms training • Working in partnership with industry • Connection to • Working with the regulators policy makers • Global partnership (third • Communication country collaboration) and outreach
  10. 10. Action groups• Action Groups to determine transnational research needs and opportunities in specific scientific areas • Longitudinal Cohort Studies (disease- & population-based) • Animal and Cell Models • Assisted Living Technologies• Action Groups to promote engagement, commitment and partnerships • Engagement and Partnership with Industry • Engagement and Partnership with the EC and beyond • User and Public Involvement in ND Research • Alignment of National Plans and activities
  11. 11. Why monitoring and evaluation?Critical questions• Does a co-ordinated EU-wide approach makes a difference achieving the outcomes?• Will the outcomes of a concerted approach be more than the sum of the outcomes of national approaches?• Can we reduce fragmentation?• Can we recognize and learn from successes and failures?
  12. 12. Monitoring and evaluation framework• Analysis of the links between (societal) challenges, objectives and activities of a programme or initiative related to the expected effects and impacts• By mapping the intervention logic it will be easier to assess whether the objectives have been met and what the success criteria are• However it is “not as easy as it looks” • Different objectives and effects • Changing objectives • Unexpected effects
  13. 13. Logical Framework Analysis (LFA)The logic:• A public action is undertaken for a reason (rationale)• It has objectives which address (societal) needs• It provides inputs which lead to activities• These activities create (direct) outputs• Which on their lead to outcomes/results and finally to (long-term) impacts
  14. 14. Intervention logic Mission Long-term Impacts Specific Result Outcomes or goals chain results Programme logic Operational Outputs goalsInterventio n logic Inputs Activities Rationale or reasons (Societal) challenges
  15. 15. Logical Framework of JPND
  16. 16. Indicators of performance• Based on the Logical Framework Analyse, indicators of performance have been developed: • Type A indicators monitor the effect of JPND on (European) research programming, research policy and funding (the concept of Joint Programming) • Type B indicators monitor the scientific and societal impact of research on neurodegenerative diseases conducted in the context of JPND• Distinction is made between: • Input indicators • Output indicators • Outcome indicators • Impact indicators
  17. 17. Development of indicators
  18. 18. Examples of type A indicators• Input indicators • Attitude towards JPND goals and objectives • The entry or drop out of countries• Output indicators • The national research (funding) priorities based on JPND’s Research Strategy• Outcome indicators • The amount of JPND common research funding for neurodegenerative diseases as share of total EU research funding• Impact indicators • The number of publications in high impact journals of European researchers in neurodegenerative diseases research
  19. 19. Examples of type B indicators• Input indicators • The number of collaborative research projects funded through JPND • The number of non-project funded activities• Output indicators • The number of Europe-wide population-based studies with contribution of JPND• Outcome indicators • Regular interactions between JPND and stakeholder groups• Impact indicators • Exchange of practices across different types of research (basic, clinical and healthcare)
  20. 20. Execution of monitoring to date1. Monitoring reports based on indicators • Quantitative indicators based on desk study, information made available by other work packages • Qualitative indicators obtained through a questionnaire investigating the attitudes towards JPND: • Expectations regarding JPND • Opinions on current JPND activities • Views on the future of JPND1. External review of experts • Review focusing on JPND’s governance and and processes of strategic policy programming (strategic direction) that have led to the development of the Research Strategy (SRA) and the implementation plan
  21. 21. Key issues, challenges and lessonslearned• Special attention should be paid to monitoring and evaluation as an important element to show progress and results of the initiative to various audiences• Development of monitoring and evaluation framework should be aligned with the development of the Research Strategy• Monitoring and evaluation is NOT just listing a set of indicators and start measuring, but takes some time and effort to develop according to intended objectives and achievements of the initiative• The systematic use of a Logical Framework Analysis approach is considered to be very helpful in (re)defining the objectives of JPND
  22. 22. More information• The full version of the monitoring and evaluation framework to be found on the website of JPND: http://www.jpnd.euThe team responsible for monitoring and evaluation of JPND:• Edvard Beem• Bastian Mostert