This document discusses strengthening collaborations through communicating timely evidence of impact. It describes different levels of collaboration from networking to unifying. It presents Kim Norris' organization, the University of Maryland Extension FSNE, and its collaborations mapped on the collaboration scale. Norris discusses the need to evaluate collaborations and share results. She introduces the RED system for inputting process and outcome data, narratives, and generating reports to strengthen collaborations through establishment, buy-in, participation, and program improvements.

1. Strengthening Collaborations –
Communicating Timely Evidence of Impact
Kim Norris, University of Maryland Extension, FSNE
Strengthening Collaborations – Communicating Timely Evidence of Impact
Kim Norris, University of Maryland Extension FSNE
Priester Health Conference, April 2012, Washington, D.C.
knorris1@umd.edu

3. Collaboration:
Two or more
people/ organizations
working together to
achieve a common goal

4. Today we are going to….
1. Identify and assess key aspects of our
collaborations
2. Define appropriate collaboration levels for our
shared efforts using two measures
3. See how information collection and sharing occur
for one organization (UME FSNE)
4. Determine kinds of evaluation information we
need to share with collaborators and how to best
obtain and share it

5. Levels of Collaboration
1.Networking
2.Cooperating
3.Partnering
4.Merging
5.Unifying

6. Collaboration: two or more people/ organizations working together to achieve a common goal.
Decision-making/
Level of Integration Shared Purpose Communication Actions/ Strategies
Adapted form Strategic Alliance Formative Assessment Rubric, Gajda 2000 Leadership
Networking Loose/ no structure; roles
Little interpersonal communication Non-hierarchical, flexible; minimal
Explore interests between organizations not
or conflict group decision-making
defined; few or no defined tasks
1
Cooperating Work together to ensure tasks are
Minor personal commitment/ Non-hierarchical; group makes low- Member links advisory only;
done; leverage or raise money
investment; clear but informal stakes decisions only; leaders minimal joint structure; some
together; identify mutual needs, but
communication is typical facilitate/ voluntary strategies/ tasks defined
2 maintain separate identities
Share some resources to address Autonomous leadership; alliance
Partnering common issues; organizations Communication channels members share decision-making Strategies, tasks developed and
autonomous but support something formalized; conflicts and problem- equally in low to medium-stakes maintained; central body of
new in order to reach mutual short- solving occur decisions; decision-making people with specific tasks
3
term goals together mechanisms are in place
Combine significant resources support Clear, formal structure to support
Merging Strong, visible leadership; sharing/
something new; extract $ from existing High level of commitment/ strategies and tasks; specific and
delegation of roles/
systems/ members; long-term investment; communication complex strategies and tasks
responsibilities; combine diversity
commitment to achieve long-term channels clear, frequent, prioritized defined; committees/
4 of organizational strengths
outcomes subcommittees formed
Unifying Clear and frequent communication Formal, legal, complex structure;
Create single structure from previously
both formally and informally; high Central structure; hierarchical permanent reorganization of
separate; relinquish autonomy to
level of interpersonal conflict and leadership strategies and tasks toward new
support new/ surviving organization
5 shared problem solving organization

7. University of Maryland FSNE
Selected Collaborator Alignments
Unifying
using SAFAR Grid
UME Extension
Merging
Partnering
Cooperating
Food Banks
Boys and Girls’ Schools
Networking Clubs Senior Centers
Farmers’ Markets
Gajda, R. 2002

8. Collaborator Shared Goal(s) Perceived Level of
Collaboration
UME Improve F&V
Extension consumption and Merging to Unifying
physical activity
Food Banks Increase F&V Cooperating
consumption
Schools Increase F&V
consumption & Cooperating
physical activity
Farmers’ Increase fresh F&V Networking
Markets consumption
Strengthening Collaborations – Communicating Timely Evidence of Impact;
Kim Norris, University of Maryland Extension FSNE; knorris1@umd.edu
Priester Health Conference, April 2012, Washington, D.C.

9. Share with Partners

10. Your Organization’s Collaborations and
Their Alignment on the Scale
Unifying
Merging
Partnering
Cooperating
Networking
Strengthening Collaborations – Communicating Timely Evidence of Impact;
Kim Norris, University of Maryland Extension FSNE; knorris1@umd.edu
Priester Health Conference, April 2012, Washington, D.C.

11. Collaborator Shared Goal(s) Perceived Level of
Collaboration
Strengthening Collaborations – Communicating Timely Evidence of Impact;
Kim Norris, University of Maryland Extension FSNE; knorris1@umd.edu
Priester Health Conference, April 2012, Washington, D.C.

12. IS THIS WHERE WE
WANT TO BE WITH
OUR
COLLABORATORS?

13. University of Maryland FSNE
Selected Collaborator Alignments
Unifying
using SAFAR Grid
UME Extension
Merging
Partnering
Cooperating
Food Banks
Boys and Girls’ Schools
Networking Clubs Senior Centers
Farmers’ Markets
Where we want to be
Lower than want

14. University of Maryland FSNE
Selected Collaborator Alignments
Unifying
WHERE WE WANT TO BE
UME Extension
Merging
Partnering
Cooperating
Food Banks
Boys and Girls’ Schools
Networking Clubs Senior Centers
Farmers’ Markets
From Gajda, 2000,
Strategic Alliance
Formative Assessment Rubric

15. Program/ Collaboration Lifecycle
What about Evaluation?
Discuss
Evaluate SHARED Decide
PURPOSE
Act
Strengthening Collaborations – Communicating Timely Evidence of Impact;
Kim Norris, University of Maryland Extension FSNE; knorris1@umd.edu
Priester Health Conference, April 2012, Washington, D.C.

16. Level of
SHARED PURPOSE DISCUSS DECIDE ACT EVALUATE
Adapted form Strategic Alliance Formative Assessment Rubric, Gajda 2000 Integration
Networking Non-hierarchical, flexible; Loose/ no structure; roles Separate evaluation goals,
Little interpersonal
Explore interests minimal group decision- between organizations not collection, reporting ; often
communication or conflict
making defined; few or no defined tasks rely on anecdotes
1
A few tasks shared; usually
Work together to ensure tasks
Cooperating Minor personal Non-hierarchical; group process/ outputs data only;
are done; leverage or raise Member links advisory only;
commitment/ investment; makes low-stakes decisions outcomes/ impacts
money together; identify minimal joint structure; some
clear but informal only; leaders facilitate/ measured and reported
mutual needs, but maintain strategies/ tasks defined
2 communication is typical voluntary separately; anecdotes
separate identities
sometimes shared
Share some resources to Autonomous leadership; Some shared responsibility
Partnering address common issues; alliance members share for collecting outcomes as
Communication channels Strategies, tasks developed and
organizations autonomous but decision-making equally in well as process data;
formalized; conflicts and maintained; central body of
support something new in low to medium-stakes separate analysis and
problem-solving occur people with specific tasks
3 order to reach mutual short- decisions; decision-making reporting, though some data
term goals together mechanisms are in place may be shared
Combine significant resources Shared responsibility
Strong, visible leadership; Clear, formal structure to
Merging support something new; High level of commitment/ regarding short-term
sharing/ delegation of roles/ support strategies and tasks;
extract $ from existing investment; communication outcomes measured, and
responsibilities; combine specific and complex strategies
systems/ members; long-term channels clear, frequent, collected; shared data from
diversity of organizational and tasks defined; committees/
4 commitment to achieve long- prioritized analyses for shared and
strengths subcommittees formed
term outcomes separate reports
Clear and frequent
Shared responsibility
Unifying Create single structure from communication both Formal, legal, complex
determining short and long-
previously separate; relinquish formally and informally; high Central structure; structure; permanent
term outcomes, measuring
autonomy to support new/ level of interpersonal hierarchical leadership reorganization of strategies and
all forms of data, analyzing
5 surviving organization conflict and shared problem tasks toward new organization
and reporting
solving
Strengthening Collaborations – Communicating Timely Evidence of Impact; Kim Norris, University of Maryland Extension ,FSNE; knorris1@umd.edu;
Priester Health Conference, April 2012, Washington, D.C.

17. Collaborator Shared Goal(s) Evaluation Efforts Perceived Level
Shared of Collaboration
UME Extension Improve F&V consumption Some common Partnering (vs
and physical activity indicators; Merging/
separate analysis/ Unifying)
reporting
Food Banks Increase F&V consumption None (we do if Networking (vs
any) cooperating)
Schools Increase F&V consumption & Common Partnering
physical activity indicators; (vs just
teachers share in cooperating)
data collection;
separate reports
Farmers’ Markets Increase fresh F&V None (we do) Networking
consumption
Strengthening Collaborations – Communicating Timely Evidence of Impact;
Kim Norris, University of Maryland Extension FSNE; knorris1@umd.edu
Priester Health Conference, April 2012, Washington, D.C.

18. University of Maryland FSNE
Selected Collaborator Alignments
using adapted SAFAR Grid to Unifying
Include Evaluation Efforts
Merging
Partnering
Schools UME Extension
Cooperating
Boys and Girls’ Senior Centers
Clubs
Networking
Food Banks Farmers’ Markets
Increases strength
Decreases strength

19. IS THIS WHERE WE
WANT TO BE WITH
OUR
COLLABORATORS?
(include Evaluation)

20. University of Maryland FSNE
Selected Collaborator Alignments
using adapted SAFAR Grid to Unifying
Include Evaluation Efforts
Merging
Partnering
Schools
Cooperating
Boys and Girls’ Senior Centers UME Extension
Clubs
Networking
Food Banks Farmers’ Markets
Where we want to be
Lower than want

21. Share with Partners

22. Newly
Evaluation Perceived Level
Collaborator Shared Goal(s) Efforts Shared of
Collaboration
Strengthening Collaborations – Communicating Timely Evidence of Impact;
Kim Norris, University of Maryland Extension FSNE; knorris1@umd.edu
Priester Health Conference, April 2012, Washington, D.C.

23. Your Collaborators and
Their Alignment using
adapted SAFAR Scale to Unifying
include Evaluation Efforts
Merging
Partnering
Cooperating
Networking
Strengthening Collaborations – Communicating Timely Evidence of Impact;
Kim Norris, University of Maryland Extension FSNE; knorris1@umd.edu
Priester Health Conference, April 2012, Washington, D.C.

24. Common Goals
Evaluate Impact/ Share Results

25. How do we do this?
Evaluate Impact/ Share Results
RED System
(Reporting, Evaluation, and Data System)

26. RED System
Educators input small amount of
Process Data*
Outcomes Data*
Narrative/ quotes*
* Some of data pre-popualted

27. RED System
Collaborator Reports Include
Collaborator-Specific
•Process data
•Outcomes data
•Narratives/ quotes

28. Process Data INPUT looks like this

29. …and this…

30. Pre-then-Post survey for outcomes data

31. Survey INPUT looks like this

32. Narrative INPUT looks like this

33. Collaborator Reports look like this –
Summarize

34. And this… sessions/ participants

35. …and include participants’
•Gender
•Age group
•Race/ Ethnicity

36. …and include Pre/Post Survey Data
Mean Reports
in Matched Pre Post
Read for Health Diff p Val T Val sample n Mean Mean
teach nutrition
1.24 < .01 6.1 63 2.93 4.17
with students?
provide
opportunities for
1.25 < .01 8.6 120 2.91 4.16
students to taste
F&V in class?
model healthy
eating behaviors
1.24 < .01 9.2 139 2.94 4.18
for your
students?

37. …and include associated Narratives

38. Collaborator Reports USES for
strengthening collaborations
•Collaborator Establishment
•Collaborator Buy-In
•Collaborator Participation
•Program Improvements with
Collaborators

39. Collaborator Establishment
Bring a New Judy Center on board

40. Select Outcomes Data
Read for Health Mean Matched Pre Post
p Val T Val
(avg=5 lessons) Diff sample n Mean Mean
teach nutrition
1.24 < .01 6.1 63 2.93 4.17
with students?
provide
opportunities for
1.25 < .01 8.6 120 2.91 4.16
students to taste
F&V in class?
model healthy
eating behaviors
1.24 < .01 9.2 139 2.94 4.18
for your
students?

41. SORT and SELECT Narratives
Kindergarten students, "Yesterday I
ate all my broccoli 'cuz it's good for
Behavioral Change
me." "I tried a sweet pepper that I
never ate before."
First grade students: "I exercise 'cuz it
Behavioral Change gets our body moving." "Now I
exercise with my mom."
"Our Judy Center kids have said that
they have asked their moms to buy
System Level Change foods they tasted here. We now
serve only healthy snacks to reinforce
what children learn."

42. Collaborator Buy-In
New School District to add Credit-
Bearing Nutrition Education Course

43. Show School District other Districts
County Youth
Baltimore 1,221
Frederick 305
Prince George's 2,375
Anne Arundel 480
Wicomico 102

44. Show School District Outcomes
Mean Matched Post
Integrating Nutrition into the Curriculum P Val T Val Pre Mean
Diff sample n Mean
I incorporate nutrition into one or more
subject areas, such as language arts or 1.27 <0.01 6.54* 67 2.95 4.22
math.
I incorporate physical activity into
1.27 <0.01 7.11* 80 2.96 4.23
lessons with my students.
I provide opportunities for students to
1.26 <0.01 6.48* 68 2.95 4.21
taste fruits or vegetables in class.
I make sure students have frequent
1.25 <0.01 6.43* 68 2.96 4.21
physical activity breaks during class.
I have students share examples of their
healthy eating and/ or physical activities 1.24 0.02 3.91* 26 2.97 4.21
at home and/ or at school.

45. Collaborator Participation In Evaluation
Missing Valuable Data

46. Collaborator Participation In Evaluation
Missing Valuable Data
e.g., Have survey results for one
of three curricula used (e.g.
F.U.N.)
Can we add surveys for other
curricula/ collaborators

47. Program Improvements – Compare Outcomes
Match
Pre Post Mean
Nutrition to Go – Stretch Food $ Mean Mean Diff
P Val sample
n
0.000
Plan meals before food shopping? 3 4.17 1.17 32
58
Turn off TV/ electronics when eating 0.000
3 5 2 11
meals? 71
Use a grocery list when grocery 0.002
2.5 4.67 2.17 22
shopping? 95
Make your own single servings from 0.080
2.67 4.67 2 13
bulk foods? 59
Buy generic brands instead of 0.434
3.67 4 0.33 16
national brands? 81
Compare prices before you buy
4.67 4.67 0 0.995 21
foods?
Eat meals together as a family? 5 5 0 1 11

48. What do you want to do with your
collaborators?
•Establish New
•Gain Buy-In
•Increase Investment
•Improve Current Programs
•Other….

49. WHAT you share and HOW you
share it is between you and your
collaborators

50. What do you need to share to help
strengthen collaboration goals?
Information to Share Data Type
Who reached (#, age groups, gender groups,
Process Data
race/ ethnicity groups)
How often, how long (dosage) Process Data
What was done together Formative/ Process Data
What participants/collaborators thought of
Narratives/ Focus Groups
process
Matched Pre-then-Post
What participants learned? What didn’t change?
Survey
Matched Pre-then-Post
What participants plan to do/ do differently
Survey
Long-term impacts of program Follow-Up survey
Strengthening Collaborations – Communicating Timely Evidence of Impact;
Kim Norris, University of Maryland Extension FSNE; knorris1@umd.edu
Priester Health Conference, April 2012, Washington, D.C.

51. Next Step for Your
Organization

52. Collaborator Information to share How to collect, collate, Best roles for
access collaborators / us
Strengthening Collaborations – Communicating Timely Evidence of Impact;
Kim Norris, University of Maryland Extension FSNE; knorris1@umd.edu
Priester Health Conference, April 2012, Washington, D.C.

53. How can you AND YOUR
COLLABORATORS work together to
OBTAIN and SHARE this
information?

55. Tips for Strengthening Collaborations
(HINT – Do these steps with collaborators
1. Identify shared goals
2. Identify what you currently do with collaborators
3. Use adapted SAFAR (with evaluation column) and tables to
• Identify current collaborative level
• See if this matches with desired level
• Identify ways you wish to strengthen the collaboration
• Determine data needed to strengthen collaboration
• Determine how data can be obtained, reviewed, and shared
• Determine role you and collaborators can best play
4. Evaluate strength each year
5. Modify actions as needed