F&I Laura Hillier - Business Intelligence II: Research Impact

•Download as PPTX, PDF•

0 likes•290 views

CASRAI

Research portfolios in support of societal challenges: concepts and quantitative tools.

Science

Research Impact @ the
Canada Foundation for
Innovation
CASRAI ReConnect 2104
November 20, 2014

CFI Mandate
Created by the Government of Canada in
1997, the Canada Foundation for
Innovation (CFI) has a legislated mandate
to build our nation’s capacity to undertake
world-class research and technology
development that benefits Canadians.

Capturing progress & results
Organizational level
• corporate performance metrics
Project level
• application data, progress reports, financial
reports
Institutional level
• Outcome measurement studies (OMS)
Beyond an Institutional level
• Platform outcome measurement studies
(POMS)
• Special studies & evaluations
HOW?

Performance Data
Data
typically
originates
at the
most
granular
level &
gets
combined
to allow
for
system
level
use

Collaboration
WHY?
• Because we need to rely on our funded
institutions and their researchers for
critical information
• Because we want to reduce the reporting
burden and maximize the use of
important performance information
• Because we want a system perspective or
lens

Opportunities
• Share Project level data and/or analyses
with Institutions and Provincial partners
• Seeking collaborative approach for New
Outcome Measurement Study, for
Platform Outcome Measurement Studies
and for Socioeconomic Impact Analyses
• Funded Institutions
• Other funders (Federal &
Provincial)
•WWHHAYT??

Project level data
CFI’s Annual
Project
Progress
Report
- projects report
for a period of 5
years, using an
online reporting
system

Challenges
Measurement of organizational
performance
Measurement of research
performance
Individual System

How do we……
Answer individual and system level questions?
• ensure useful analyses
Improve our ability to ‘share’ data and
information?
• avoid burden
• avoid double counting
• ensure meaningful data
• ensure appropriate use?

Thank you!
Laura Hillier
Director, Evaluation & Outcome Assessment
laura.hillier@innovation.ca

What's hot

IDEC 2014 Winter Leadership Seminar, Developing Strategic Direction

MarcieCraigPost

Evaluating Global and Regional Partnership Programs (2011 Evaluation Week)

Independent Evaluation Group /World Bank Group

Self-Evaluation of EQAR: Process & Key Results

Colin Tück

Using Data to Improve the Delivery of Civil Legal Aid Services

Legal Services Corporation

Innovation Programme Meeting Evaluation

Andy Dyson

Overview of iesd slides meet the funder 2013

Department of Health Voluntary Sector Grants Hub

Monitoring, Evaluation, and Learning in adaptive programmes

Arnaldo Pellini

Ces 2013 - Doing Developmental Evaluation at the System Level

CesToronto

Using case-based methods to assess scalability and sustainability: Lessons fr...

Barb Knittel

The session will provide details on: the tool developed by the LEG for monitoring and evaluating progress, effectiveness and gaps (PEG M&E tool) and its application in the process to formulate and implement NAPs; and the best practices for developing monitoring and evaluation (M&E) systems for adaptation at the national level. It will also look at the experiences of countries in developing and applying M&E systems at their national levels.

PEG M&E tool: a tool for monitoring and reviewing Progress, Effectiveness & G...

Tariq A. Deen

Lisa Lipe Resume Feb 2016

Lisa Lipe

Presented by: Cecilia Silva Bernardo 8.3 Monitoring and evaluation The session will provide details on: the tool developed by the LEG for monitoring and evaluating progress, effectiveness and gaps (PEG M&E tool) and its application in the process to formulate and implement NAPs; and the best practices for developing monitoring and evaluation (M&E) systems for adaptation at the national level. It will also look at the experiences of countries in developing and applying M&E systems at their national levels.

Monitoring and Evaluation System for Adaptation Projects

NAP Events

CRP on Policies, Institutions and Markets (PIM) independent evaluation : Brie...

Independent Evaluation Arrangement of CGIAR

Elements of implementing innovation platform

CGIAR Research Program on Dryland Systems

SC CTSI Perspective on Planning, Evaluation and Tracking (PET) in an Academic...

SC CTSI at USC and CHLA

Afid project monitoring and evaluation practices and lessons

Rosemirta Birungi

Understanding effectiveness of training & development PIP (Pursuing Independent Paths) is a charity supporting adults with learning disabilities to achieve their potential. PiP had begun to collect lots data on the effectiveness of training & development interventions, however they did not have an easy way to aggregate and analyse the data to understand trends and overall effectiveness. This was necessary to provide assurance internally, for external funding bodies and to provide feedback to students.

Understanding effectiveness of training & development

The OR Society

What's hot (17)

IDEC 2014 Winter Leadership Seminar, Developing Strategic Direction

Evaluating Global and Regional Partnership Programs (2011 Evaluation Week)

Self-Evaluation of EQAR: Process & Key Results

Using Data to Improve the Delivery of Civil Legal Aid Services

Innovation Programme Meeting Evaluation

Overview of iesd slides meet the funder 2013

Monitoring, Evaluation, and Learning in adaptive programmes

Ces 2013 - Doing Developmental Evaluation at the System Level

Using case-based methods to assess scalability and sustainability: Lessons fr...

PEG M&E tool: a tool for monitoring and reviewing Progress, Effectiveness & G...

Lisa Lipe Resume Feb 2016

Monitoring and Evaluation System for Adaptation Projects

CRP on Policies, Institutions and Markets (PIM) independent evaluation : Brie...

Elements of implementing innovation platform

SC CTSI Perspective on Planning, Evaluation and Tracking (PET) in an Academic...

Afid project monitoring and evaluation practices and lessons

Understanding effectiveness of training & development

Viewers also liked

F&I Thorsten Hoellrigl - Exchange Realities

CASRAI

RDC: Kathleen Shearer - Joint Session Project ARC & Federated DMP Pilot

CASRAI

Snowball Metrics aim to become the international standard that is endorsed by research-intensive universities so that they can build and monitor the most effective strategies. It is a “bottom-up”, or sector-led, initiative in which universities themselves agree a single method to calculate metrics about their own performance so that they can compare themselves against each other in an apples-to-apples way. This enables them to benchmark and understand their strengths and weaknesses to help to decide in which areas to invest, and in which to divest.

Snowball Metrics as Standard Information Agreements - Anna Clements and Peter...

CASRAI

When we say “Research Impact” many things come to mind and the reasons for why we are concerned with it vary. The underlying concepts are complex and often require expert knowledge, and there is also no one single interpretation or answer. Stakeholders are diverse and so are the means of communication. Therefore across Canada, we continue to seek more consistent and harmonized ways of telling the “Impact Story.” The panel will reflect on harmonization efforts across provinces.

Provincial Perspectives on Research Impacts: Eddy Nason, Renata Osika, Krista...

CASRAI

RDC Jane Fry, Chantal Ripp - Data Interoperability I

CASRAI

Dragon slides_David Baker

CASRAI

Workshop humphrey watkins-boyko-dmp workshop

CASRAI

The Canadian Clinical Trials Asset Map (CCTAM) – Signaling Canada’s clinical ...

CASRAI

RDC - Benoit Pierenne: Data Interoperability I

CASRAI

Update: David Baker - About CASRAI

CASRAI

Project Credit: Laure Haak - Contributor Role Pilot

CASRAI

Closing the Loop - Technology Implementations

CASRAI

L&P Eric Celeste - SHARE

CASRAI

NW ANP meeting 01.12.14

North West ANP Network

Somebody worked hard to tailor a new standard, but how well does it fit when we try it on a real system? CRediT (an emerging standard for identifying scholarly manuscript contribution) can potentially address chronic problems in research workflow, but what are the practical implications of implementing such a standard in software? It should be “easy” to add a list of options – right? In this presentation, Richard Wynne will address the practical software issues arising from integrating CRediT into Editorial Manager®, an online submission and peer review system used by more than 6,500 scholarly journals to process millions of submissions per year. This presentation will give you a clear picture of the benefits of CRediT and what issues to consider during its adoption.

CRediT – Honey, do I look good wearing this new standard? - Richard Wynne.

CASRAI

Admin Burden in Canada: ABC Example: Money makes the world go round - Judith ...

CASRAI

COAR: AN INTRODUCTION TO SHARE

CASRAI

RDC - Chuck Humphrey and Susan Babcock - Research Ethics Boards and Data Mana...

CASRAI

Viewers also liked (18)

F&I Thorsten Hoellrigl - Exchange Realities

RDC: Kathleen Shearer - Joint Session Project ARC & Federated DMP Pilot

Snowball Metrics as Standard Information Agreements - Anna Clements and Peter...

Provincial Perspectives on Research Impacts: Eddy Nason, Renata Osika, Krista...

RDC Jane Fry, Chantal Ripp - Data Interoperability I

Dragon slides_David Baker

Workshop humphrey watkins-boyko-dmp workshop

The Canadian Clinical Trials Asset Map (CCTAM) – Signaling Canada’s clinical ...

RDC - Benoit Pierenne: Data Interoperability I

Update: David Baker - About CASRAI

Project Credit: Laure Haak - Contributor Role Pilot

Closing the Loop - Technology Implementations

L&P Eric Celeste - SHARE

NW ANP meeting 01.12.14

CRediT – Honey, do I look good wearing this new standard? - Richard Wynne.

Admin Burden in Canada: ABC Example: Money makes the world go round - Judith ...

COAR: AN INTRODUCTION TO SHARE

RDC - Chuck Humphrey and Susan Babcock - Research Ethics Boards and Data Mana...

Similar to F&I Laura Hillier - Business Intelligence II: Research Impact

[Merit trac webinar] - it is assessments that cause improvement

MeritTracSvc

Benchmarking for hrd professionals

Sorab Sadri

Measuring_HR_ROI-1.pdf

HarmanSingh510326

Performance Measurement: Ensuring Accuracy, Transparency and Relevance

Massachusetts Department of Higher Education

Educause_2013_OIT+Virtual+PMO+Presentation+1032013.pptx

ssuserf016f4

Planning for the Future of Your Agency

Jodi Rudick

Evolving How We Measure Digital Success in Higher Ed

Mediacurrent

FINAL_OracleGuident-PPT

Carolyn Reid

2016 NAEP Annual Meeting presentation by Dan Traub of FinVantage Solutions LLC and Sandy Hicks of the University of Colorado. Delivered in San Antonio, TX. In this update to the 2015 presentation, we explore the rollout and impact of the departmental scorecard program as it was introduced to over 215 financial managers across the University of Colorado System. Learn how the University of Colorado Procurement Service Center launched an innovative program to gain valuable insights into its performance using the latest data visualization tools. Utilizing and analyzing data from multiple sources, CU has built a 360-degree platform to explore outcomes from procurement, payables, travel management, payment card, customer outreach, and other functions. The transition to a data-driven mindset will be examined, including a few surprises they learned along the way. The team will demonstrate how these metrics are driving new levels of collaboration with campus stakeholders, thanks to the popular departmental scorecards and executive dashboards.

Delivering Aha Moments through Procurement Performance Analytics (part II)

Dan Traub

In the mission-oriented nonprofit world, it can sometimes be difficult to get your team to focus on the relationship between money coming in and the organization’s ability to effectively deliver programs and services to its constituents. While top-line measures are exciting they can also be distracting and counterproductive if they are not properly accounted for and effectively presented. Peeling back the covers on your finance organization can help your team, executive director, staff and board members truly understand your financial situation and see how finances directly impact programs. Financial operations reviews deliver insight. Reviews help organizations move beyond budgets and expenses into staffing, policies and processes. You will learn… • The 5 “must ask” questions • The 3 reports you should study • The secrets of financial management • How to determine financial viability of new programs and link money with mission

Measuring Performance: Linking Money To Mission

4Good.org

Evidence On Trial: weighing the value of evidence in academic enquiry, policy...

CCAFS | CGIAR Research Program on Climate Change, Agriculture and Food Security

This presentation will take you through TSB Bank’s journey from first establishing the audit function through to developing a data analytics strategy as the organization gets ready to move to a new, state-of-the-art online banking platform. SLIDESHARE: www.slideshare.net/CaseWare_Analytics WEBSITE: www.casewareanalytics.com BLOG: www.casewareanalytics.com/blog TWITTER: www.twitter.com/CW_Analytic

How to build a data analytics strategy in a digital world

CaseWare IDEA

Consulting Brochure / The Performance Institute

Nicole Cathcart

When implementing SharePoint you might think that you are adding just another technology to your heterogeneous IT environment. But after a while you realize, that users are doing things with SharePoint that you did not expect them to do. This talk dives into the dynamics of adopting SharePoint as a platform, and shows you a how to setup an operational governance practice using a structured governance framework.

Why Are We Still Talking About SharePoint Governance? presented by Anders Skj...

European SharePoint Conference

Data Quality Assurance: An Impetus in Improving Partner(s) Data Management an...

National Organization of Peer Educators (NOPE Kenya)

Web Maturity Models

Vera Rhoads

Program Dashboards and Scorecards_CLEAN

Jen Braly, PgMP, PMP

Importance of M&E

clearsateam

In the past decade, the HR function has undergone a significant transformation. It has evolved from being a support function to a strategic business driver. Modern day HR’s can leverage plethora of data that to manage Employee Engagement. This presentation describes about BRIDGEi2i’s offering on Employee Engagement Analytics and how HR’s can leverage the data eco system to get granular insights for improving Employee Engagement

A quick Introduction to Employee Engagement Analytics Suite – EmPOWER

BRIDGEi2i Analytics Solutions

Pyramid Service Profile

Sherif Farghal

Similar to F&I Laura Hillier - Business Intelligence II: Research Impact (20)

[Merit trac webinar] - it is assessments that cause improvement

Benchmarking for hrd professionals

Measuring_HR_ROI-1.pdf

Performance Measurement: Ensuring Accuracy, Transparency and Relevance

Educause_2013_OIT+Virtual+PMO+Presentation+1032013.pptx

Planning for the Future of Your Agency

Evolving How We Measure Digital Success in Higher Ed

FINAL_OracleGuident-PPT

Delivering Aha Moments through Procurement Performance Analytics (part II)

Measuring Performance: Linking Money To Mission

Evidence On Trial: weighing the value of evidence in academic enquiry, policy...

How to build a data analytics strategy in a digital world

Consulting Brochure / The Performance Institute

Why Are We Still Talking About SharePoint Governance? presented by Anders Skj...

Data Quality Assurance: An Impetus in Improving Partner(s) Data Management an...

Web Maturity Models

Program Dashboards and Scorecards_CLEAN

Importance of M&E

A quick Introduction to Employee Engagement Analytics Suite – EmPOWER

Pyramid Service Profile

More from CASRAI

Optimising benefits from Canadian Research - Jim Woodgett

CASRAI

Optimising benefits from Canadian Research - Janet Halliwell

CASRAI

Keynote: Repaving the Road to Hell with the Common CV and CIHR Reforms

CASRAI

Admin Burden in Canada (ABC) - Janet Halliwell Board Chair CASRAI

CASRAI

Admin Burden in Canada (ABC) Introductory Panel Discussion (CA, UK and US ove...

CASRAI

Admin Burden in Canada (ABC) Introductory Panel Discussion (CA, UK and US ove...

CASRAI

Admin Burden in Canada (ABC) Introductory Panel Discussion (CA, UK and US ove...

CASRAI

ABC Project 1 - Piloting Auto-upload of Standardized Funding Award Results - ...

CASRAI

ABC Project 2 - Launching an ORCID Consortia in Canada - Clare Appavoo & Geof...

CASRAI

Introduction to the Federal Demonstration Partnership (FDP) of the US - David...

CASRAI

Keynote: Today's Data Grow Tomorrow's Citizens - Tracey P. Lauriault

CASRAI

Canada’s Research Data Management (RDM) Ecosystem - Mark Leggott

CASRAI

Tutorial: CASRAI Standards Development (for a non-technology audience) - Davi...

CASRAI

Tutorial: the new Portage Research Data Management Planning Tool - Chuck Hump...

CASRAI

Building a Canadian National Research Data Management Framework - Mark Leggott

CASRAI

How Do I Know Thee? Let Me Count the Ways: Panel 2: Jeffrey Alexander & Patri...

CASRAI

All R&D organizations classify their research activities, either implicitly (e.g., by laboratory or department) or explicitly (e.g., by creating taxonomies to define and map research disciplines and domains). However the lack of clear standards for doing so impedes the sharing and aggregation of data on R&D activities. In this workshop, Jeff Alexander and Patrick Lambe will provide an overview of the organizational needs driving the development of a classification of R&D activities, use cases for such a classification, and the potential advantages of international coordination across such classifications. The workshop, based heavily on a study they conducted for the National Center for Science & Engineering Statistics at the U.S. National Science Foundation, will review alternate approaches to both developing R&D classifications, and streamlining the process of classifying research programs and projects. Topics to be covered include examples of international R&D classifications and their development (such as the Australia-New Zealand Standard Research Classification), design principles for R&D classifications, and new automated and semi-automated classification techniques using semantic analysis and machine learning.

Classifying R&D: Why and How Organizations Develop Taxonomies for Research Fi...

CASRAI

Classification of research plays an integral role in the functioning of research funding organizations. As such it is important to have a classification system for efficient research data collection, use, analysis and reporting. Hear about lessons learned as well as key limitations and challenges for the implementation of a standard approach to classification through the analyses of different international standards currently in use with respect to their governance, development, implementation and maintenance

How Do I Know Thee? Let Me Count the Ways: Sarah Moreault, Monica Valsangkar-...

CASRAI

Over the past 10 years, research systems have evolved from systems that focused on how to structure and record information on research, to systems capable of allowing significant insights to be derived based upon years of high quality information. In 2015, the maturity of the information now collected within many Current Research Information Systems, and the insights that this can provide is of equal or greater value than the insights that could be gleaned from established externally provided research metrics platforms alone. The ability to intersect these external and internal worlds provides new levels of strategic insight not previously available. With the addition of platforms that track altmetrics, and their ability to connect university publications data with a constant flow of real time attention level metrics, an image of a dynamic network of systems emerges, connected together by ever turning ‘cogs’ pushing and translating information. Add to this, the success of ORCID as pervasive researcher identifier infrastructure, and CASRAI as the emerging social contract for information exchange, and it becomes possible to extend this network back from the systems that track and record research information, through to the platforms through which research knowledge is created. The ‘Mechanics’ of this network of systems is more than just getting the ‘plumbing’ right. As research information moves through the network, its audience and purpose changes, the requirements for contextual metadata can also change. This presentation will explore the lived experience of Research Data Mechanics at Digital Science though illustrating how connections between Figshare, Altmetric, Symplectic Elements, and Dimensions can both enhance research system capability and reduce the burden on researchers, and research administration.

Research Metadata Mechanics - Simon Porter

CASRAI

Lightning Reports on 2015 CASRAI Standards Work: Data Management Plan

CASRAI

More from CASRAI (20)