Presentation by Melanie Funes, PhD, Director, Strategic Implementation, Associate Director, Research Development program at the Southern California Clinical and Translational Science Institute (SC CTSI).

1. SC CTSI
Planning, Evaluation and Tracking (PET)
Melanie Funes, PhD
Director, Strategic Implementation
Associate Director, Research Development program

2. TEAM
PET
 Melanie Funes, PhD
Director, Strategic Implementation
(0.5 FTE)
 Eun Evans, PhD
Project Manager
(1.0 FTE)
 April Bagaporo, BS
Program Coordinator
(0.5 FTE)
 Annie Hong, BS
Program Coordinator
(1.0 FTE)
eHome: Technology development partners
 Katja Reuter, PhD
Director, Digital Strategy and eHome
 Praveen Angyan, MS
Programmer
 John Burgos, MS
Project Manager

3. PURPOSE
 Demonstrate SC CTSI’s value in a concrete way to stakeholders
 Introduce approaches to create a strategic organization
 Increase alignment between program and SC CTSI priorities
 Provide support to develop goals, target and metrics of success
 Assess progress toward goals and targets
 Develop strategies and approaches to improve performance
 Build tools and best practices to achieve excellence
 Enable data-driven decision making

4. REMINDERS:
FROM THE RFA AND IOM
RFA: Tracking, Assessment and Evaluation
 Success = increased efficiency and decreased costs
 Timely tracking of service use, quality and cost
 Assess and adjust activities in response
 Processes, metrics and milestones for ongoing improvement
IOM
 Necessary for accountability, transparency, informed decision making, and
communication about outcomes and the value of an investment
 Formalize and standardize evaluation processes
 Lead in developing evaluation methodologies and metrics

5. WORK IN CONTEXT:
EXAMPLES OF WHAT WE DO
Planning
 Annual program planning
 Goals, metrics and targets
 Balanced Scorecard development
Tracking
 Annual progress and outcomes survey
 Service provision tracking
 Metrics tracking
Evaluation
 Turning numbers into knowledge
 Data, data and more data
 How do we use the data?
Reporting
 Annual progress report (NIH)
 Annual report (SC CTSI)
 Quarterly report (by School)
 Service provision reports
Technology Development
 Automated data aggregation
 Automated data distribution
 Integrated tracking
Research
 Pilot Funding ROI
 Community engagement metrics

6. Performance Management
Challenges and Lessons Learned from Balanced Scorecards
or too little

7. Connecting the Dots
The Aggregation and Distribution Challenge

8. CONNECTING THE DATA DOTS
The right data at the right time is crucial
 Actionable insight
 Informs course correcting decisions
 Adjust service provision (maximize resources – minimize costs)
We need critical data connections
 Activities + Individuals + Outcomes
Data Nexus, a technical solution
 Automated data aggregation from multiple sources
 Subsequent data distribution across multiple outputs
Create a data ecosystem for tracking use, cost, quality, and outcomes

9. Service Provision Proof-of-Concept
Biostatistics Consultation Service

10. BIOSTATISTICS CONSULTATION SERVICE PROOF-OF-CONCEPT:
CREATING AN INTEGRATED DATA ECOSYSTEM
Find and Access Services Online
Collect Investigator
Information
Track Service Provision
SC CTSI Website
Formstack: Online Form Builder
HelpSpot: Service Tracking System
Service Invoicing
Zoho: Business Invoicing Application
Performance Management
ClearPoint Strategy:
Balanced Scorecard Software
Service Reports
HR Database Service Agreement

11. DASHBOARDS AND REPORTS:
ENABLING DATA-DRIVEN DECISION MAKING
Month, Quarter, Annual
• Investigators served (#)
• Projects supported (#)
• Consultation hours (#)
• Consultation activities (type)
• Consultation charge categories (type)
• Billable hours (#)
• Non-billable hours (#)
• FTE grant hours (#)
• Cost recovery recharge ($)
• Cost recovery FTE ($)
• Consulting salary + benefits offset by cost recovery (%)
• True cost of service provision

12. Data Nexus
Creating an Integrated Data Ecosystem
(Adding Outcomes and Impact to Service Provision…and more)

13. DATA NEXUS:
CREATING AN INTEGRATED DATA ECOSYSTEM
Find and Access Services Online
Collect Investigator
Information
Track Service ProvisionSC CTSI Website
Formstack: Online Form Builder
HelpSpot: Service Tracking System
INFORMATION INPUT SYSTEMS
(Investigators, Services, Outcomes)
BBR
External:
PubMed
eRA
iEdison
News
Databases
Internal:
C&G
Kuali
iStar
OTT
HR Datafeed
Validation
Service
Agreement
Progress
Reporting
Survey System
Satisfaction
Survey
Progress Report
Demographics
eRA Commons
Publications
Patents
Licenses
Funding
APR YR1, YR2, YR3, YR4
CE
CTU
ECDE
RD
+
+
e‐GATE
e‐GATE
+ ~ 8 databases/site
Service Invoicing
Zoho: Business Invoicing Application
Service Reports
Performance Management
ClearPoint Strategy:
Balanced Scorecard Software
Communication
Real‐time updates
Web syndication
INFORMATION OUTPUT SYSTEMS

14. A DATA ECOSYSTEM IS CRUCIAL BUT NOT ENOUGH…
…for a long-lasting culture-shift in decision-making

15. OF EQUAL (IF NOT MORE) IMPORTANCE…
New approaches + Training + Trust

16. PET CONTACT INFORMATION
Questions?
pet@sc-ctsi.org | 323.442.1251