Presentation slides for a webinar produced by the Leadership Learning Community. Full audio is available on their site, at http://www.leadershiplearning.org/blog/eleanor-cooney/2012-12-17/2013-webinar-network-analysis-snaona-methods-assessment-measurement

1. Network Analysis Methods for
Assessment & Measurement
January 14, 2012
Patti Anklam
With June Holley and Claire Reinelt

2. Webinar Goals
Share current thinking about how
network analysis is used in designing
and evaluating nonprofit programs
Provide examples of network analysis
used in assessment and measurement
contexts
Stimulate thinking about correlating
network analysis with measurement
and evaluation outputs and outcomes
2

3. Network Thinking & Non-Profits

4. The Evolution of Network Thinking
4

5. What is Network Analysis?
• Social network analysis (SNA) is a collection
of techniques, tools, and methods to map
and measure the relationships among
people and organizations
• Organizational network analysis (ONA)
often refers to the use of SNA methods in
the context of organization dynamics and
development
• In practice, we use these tools to map
connections among people and ideas,
issues, and other entities as well as the
social and organizational connections
5

6. Network Analysis: The Method in a Nutshell
Step Activities/Tools
Design Identify boundaries
Clarify and design questions
Collect Data Surveys
Interviews
Facebook, LinkedIn
Email logs
Analyze data to generate (Netdraw/UCINET, NodeXL, Gephi …
maps and metrics many others)
Review data Validate; look for questions
Prepare evaluation Match network results with context
and stories
Move into action Weaving & other interventions
6

7. Survey Example
7

8. Survey Example – Demographic Component
8

9. Survey Example – Affiliation Component
9

10. Survey Example – Network Questions
10

11. Network Questions Probe Relationships
11

12. Analyses Outputs: Map Patterns
Multi-Hub Hub and Spoke
Stove-piped (Siloed) Core/Periphery
12

13. Quick View: What an Analysis Can Tell
• Overall very well connected
• One region distinctly
clustered with few
connects to other
regions
• Staff are highly
central
• Identification of
key connectors
13

14. Reasons for a Network Analysis: Examples
1. Assessment, Planning, &
Weaving
2. Measure changes over time
3. Sense-making & story-
finding
4. Positioning and working with
individuals in the network
14

15. Assessment, Planning, & Weaving
Strategic Purpose
• Assess the network’s capacity for collaboration, information
transfer, innovation
• Identify key individuals
• Establish goals for enhancing connectivity
• Create an action plan
15

16. Assessment: Capacity for Collaboration
Current Funder Interaction Network Future Funder Interaction Network
When funders indicate with whom they would like to work in the near future, the network becomes more robust.
Funders are saying they want to work more together.
Source: Transcending Boundaries: Strengthening Impact. The Full Potential of a Justice Network (Research & Network-Building Project Report,
April 2011, Criminal Justice Funders Network). Courtesy of June Holley.
16

17. Assessment: Affiliation Network
Strategic Purpose
• Identify potential
relationships among
people based on
shared events,
meetings, ideas, or
areas of expertise
• Nonprofits use this to
see which
organizations “attach”
to different ideas
• Forms the basis for
network weaving
17

18. Drill Down Into Affiliation Network
• Identify people with
common interest –
basis for building
communities of
practice
• See which people
share interest in
multiple issues or
topics
• A way for the network
to reveal itself and
have rich
conversations
18

19. Measuring Changes Over Time
Very Well
Boston Green & Healthy Building Network Well
Somewhat
2005 2007
24 9 32
1 24 9 32
1
31
3 31
18
7 3 18
6 7
6
25
25
33 28 4
33 28 4
26
26
8 29
17 8 29
2 17
2
11
12 11
12
23 16
27 23 16
27
10 19
10 19
21
21
15
15
13
13
34 14
34 14
30
30
5 20 22
5 20 22
Source: Boston Green & Healthy Building Network, Beth Tener and Al Nierenberg, January 2008 Maps copyright © 2012 New Directions Collaborative
19

20. Analyses Outputs: Metrics
Overall network metrics Individual position metrics
• Look at the whole network • Look at positions of
and its components: individuals in the network:
– Overall cohesion – # of connections
– Degrees of separation – Favorability of position
• Good for comparing • Good for identifying
groups within networks or people who are well
for comparing changes in a positioned to influence the
network over time network or to move
information around
20

21. How the Metrics Enhance the Maps
2011
Year # Density Avg #
ties
2009 55 2.2% 1.2
2010 90 2.7% 2.4
2011 85 5.3% 4.5
2012 82 8% 6.88
2010
2009
2012
21

22. Sense-Making & Emergence
• Barr Foundation Fellows Program
– See changes over time, but really to see how the network has supported
emergence
– Work to shift Barr staff from the center
Pat Brandes
Source: Networking a City, Marianne Hughes & Didi Goldenhar, Stanford Social Innovation Review, Summer 2012
22

23. Sense-Making: New School Development in Boston
• An intentional network may “This person has helped me accomplish
work-related tasks.”
have no other purpose than to
enable emergence
• Maps that show the evolving
relationships within a network
help to identify powerful
network stories
Source: Networking a City, Stanford Social Innovation Review, Summer 2012
23

24. Positioning: The Individual View
Node Betweenness Indegree OutDegree
62 792.67 26 30
80 660.48 17 32
• Centrality metrics 64
23
530.61
333.36
20
20
33
14
identify people with 71 321.42 21 20
56 316.42 20 18
the most ties (in-
degree and out-
degree)
• Those positioned to
move information
around in the
network or be in the
know (betweenness)
• Can identify people to
lead task teams, to
provide resources to,
or to train as weavers
24

25. Tracking Individuals’ Changes
I learned something from this person that made me a better leader. – 2009
2005
2007
2008
2009
2010
25

26. Tracking Individuals’ Changes
I learned something from this person that made me a better leader. – 2011
26

27. Network Analysis & Measuring Outcomes

28. Summary – What We Know
What We Can Measure and Show in an Analysis:
• Measure the cohesion of the network overall:
– High-level structure (stove-piped, core/periphery, highly clustered)
– Average degree of separation
– Average number of connections each person has
• Identify individuals by their centrality to the network:
– Core or periphery? How do you bring people in from the outside?
– Broker? Connector? Facilitator? Bottleneck?
– Number and diversity of connections
• See changes over time
28

29. Things We Can Do With What We Know
Ways to change patterns in Practices from the KM/OD Repertoire
networks
Weaving. Create intentional Convene. Make introductions through meetings and webinars, face-to-
connections face events
Increase the flow of knowledge Establish collaborative workspaces, install instant messaging systems,
make existing knowledge bases more accessible and usable;
implement social software or social network software
Create awareness Provide expertise directories
Connect disconnected clusters Weave: establish knowledge brokering roles; expand communication
channels
Create more trusted relationships Assign people to work on projects together
Alter the behavior of individual nodes Create awareness of the impact of an individual’s place in a network;
foster network literacy
Increase diversity Add nodes; connect and create networks; encourage people to bring
knowledge in from their networks in the world
29

30. Measurement Challenges
• Maps area snapshot in time
• Targets and thresholds
– How much cohesion is “enough?”
Is there a point at which
increasing the number of ties
makes the network less efficient?
– Is it reasonable to set a target for
the cohesion metric?
• Tying Network Metrics to
Outcomes
– We have to think of the metrics as
indicators and as correlates of
Source: Dave Snowden, Cynefin Advanced Practitioner’s Course December 2012
other survey questions
30

31. Questions?
31

32. • patti@pattianklam.com
http://www.pattianklam.com
•claire@leadershiplearning.org
http://www.leadershiplearning.org/
•June@networkweaving.com
http://www.networkweaving.com
Thank you.
Question