Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Coordinators Bethany Laursen
Kylie Hutchinson
Meg Hargreaves
Jan Noga
Erin Whalen
Ginger Fitzhugh
SETIG Leadership
Registered)
Registered)
Registration was much higher than attendance.
Possibly because it was free.
Registered
51#
80#
127...
Most participants attended for one day only.
A"ended&1&day&
67%&
A"ended&2&
days&
24%&
A"ended&
&all&3&days&
9%&
Participants found the break-out sessions the “most useful”.
(N=13)
0%#
7%#
14%#
79%#
Cha$ng'with'other'par/cipants'in'the...
People participated as they felt comfortable.
(N=14)
7%#
21%#
29%#
43%#
I"didn't"par+cipate"in"a"breakout"
session"
I"list...
Yes$
85%$
No$
15%$
Yes$
100%$
The unconference met most people’s expectations.
Day 1
(N=33)
Day 2
(N=24)
Day 3
(N=19)
Yes$...
4.8$
4.8$
3.8$
5.0$
4.4$
4.0$
4.4$
4.4$
3.6$
3.4$
3.3$
3.0$
0.0# 1.0# 2.0# 3.0# 4.0# 5.0#
Approaches for Capturing High-le...
People were generally satisfied with most aspects of the
unconference.
With the exception of becoming more connected to col...
• software used (MIT’s Unhangout)
• having participants visible
• ability to chat and share resources in sidebar
• facilit...
“I am so impressed with this experience!
What a low cost, low resource way to meet
with like-minded colleagues, and have d...
“This was a great experience - well
organized, technology worked perfectly
and the conversation produced was very
fruitful...
• tech problems
• accessing Unhangout
• accessing breakout room
• lack of testing and preparation
• no video or mic capabi...
would attend another
unconference by SETIG
95%
would recommend
a similar-style event
to a friend or colleague
79%
Breakout facilitators similarly enjoyed the experience.
And would do it again if asked.
8%#
18%#
17%#
9%#
33%#
42%#
50%#
2...
“It was a great idea to do this. I hope
you’ll do it again!”
“I enjoyed the opportunity a lot.”
“Great modelling of risk t...
Next Time
• no last minute registrations
• mandatory orientation session
• test connection ahead of time
• make participan...
• designate beginner vs. more intermediate
sessions
• one facilitator and one note-taker per session
• share discussion pr...
3 things you learned
2things you want to learn more about
1thing you’ll use going forward
Smaller more regular unconferences that:
• focus on one specific question or area
• focus on one problem from a systems len...
2015 Systems in Evaluation Unconference Results
2015 Systems in Evaluation Unconference Results
2015 Systems in Evaluation Unconference Results
2015 Systems in Evaluation Unconference Results
2015 Systems in Evaluation Unconference Results
2015 Systems in Evaluation Unconference Results
Upcoming SlideShare
Loading in …5
×

2015 Systems in Evaluation Unconference Results

283 views

Published on

A summary of feedback received on the AEA Systems in Evaluation Topical Interest Group's 2015 Digital Unconference.

Published in: Science
  • Be the first to comment

  • Be the first to like this

2015 Systems in Evaluation Unconference Results

  1. 1. Coordinators Bethany Laursen Kylie Hutchinson Meg Hargreaves Jan Noga Erin Whalen Ginger Fitzhugh SETIG Leadership
  2. 2. Registered) Registered) Registration was much higher than attendance. Possibly because it was free. Registered 51# 80# 127# 188# 220# 249# Day#3# Day#2# Day#1# A/ended# Registered
  3. 3. Most participants attended for one day only. A"ended&1&day& 67%& A"ended&2& days& 24%& A"ended& &all&3&days& 9%&
  4. 4. Participants found the break-out sessions the “most useful”. (N=13) 0%# 7%# 14%# 79%# Cha$ng'with'other'par/cipants'in'the' main'lobby' Watching'the'live'broadcast'panel' discussion'in'the'lobby' Other' Par/cipa/ng'in'the'small'break=out' session'I'a?ended' 0%
  5. 5. People participated as they felt comfortable. (N=14) 7%# 21%# 29%# 43%# I"didn't"par+cipate"in"a"breakout" session" I"listened" I"mostly"listened,"but"occasionally" spoke" I"listened"and"spoke"
  6. 6. Yes$ 85%$ No$ 15%$ Yes$ 100%$ The unconference met most people’s expectations. Day 1 (N=33) Day 2 (N=24) Day 3 (N=19) Yes$ 63%$ No$ 37%$
  7. 7. 4.8$ 4.8$ 3.8$ 5.0$ 4.4$ 4.0$ 4.4$ 4.4$ 3.6$ 3.4$ 3.3$ 3.0$ 0.0# 1.0# 2.0# 3.0# 4.0# 5.0# Approaches for Capturing High-level Systems Patterns and Outcomes Support for a Systems Approach: Strategies for Promoting Buy-in Using a Developmental Evaluation Approach: Real-life Examples Role of Accountability in Systems Evaluation Defining and Distinguishing Systems Evaluation: Where are the Boundaries? Using a Complex Adaptive Systems Approach: Real-life Examples Identifying System Leverage Points: Helpful Approaches Link between Culturally Responsive Evaluation & Systems-Oriented Evaluation Using a Soft Systems Approach: Real-life Examples Defining System Boundaries: Building Stakeholder Consensus Visualizing Complex Systems: Tools and Strategies Incorporating a Systems Approach: Challenges and Strategies Participants rated some sessions higher on average than others. Over all three days on a scale of 1=poor and 5=excellent.
  8. 8. People were generally satisfied with most aspects of the unconference. With the exception of becoming more connected to colleagues. 9%# 10%# 5%# 9%# 9%# 11%# 15%# 5%# 10%# 37%# 10%# 5%# 36%# 41%# 50%# 65%# 32%# 50%# 40%# 59%# 50%# 32%# 15%# 16%# 25%# 55%# The$registra+on$process$was$straigh1orward.$ I$was$provided$with$the$informa+on$I$needed$prior$to$ the$unconference.$ I$experienced$no,$or$rela+vely$few$problems,$using$the$ MIT$Unhangout$technology.$ I$increased$my$knowledge$of$this$topic.$ I$feel$more$connected$to$my$colleagues$aAer$ par+cipa+ng$in$the$unconference.$ The$unconference$provided$me$with$good$value$for$the$ +me$spent.$ I$would$aBend$another$online$unconference$hosted$by$ the$Systems$in$Evalua+on$TIG.$ Strongly$disagree$ Disagree$ Neutral$ Agree$ Strongly$Agree$
  9. 9. • software used (MIT’s Unhangout) • having participants visible • ability to chat and share resources in sidebar • facilitators for sessions
  10. 10. “I am so impressed with this experience! What a low cost, low resource way to meet with like-minded colleagues, and have deep discussions about relevant things! I am totally jazzed up to use what I learned in my work and to connect with people I’ve met in breakout chats.”
  11. 11. “This was a great experience - well organized, technology worked perfectly and the conversation produced was very fruitful.” “This was an excellent event. Thought the format was just perfect for fitting into my life, but also brought deep and meaningful conversation about an important topic.”
  12. 12. • tech problems • accessing Unhangout • accessing breakout room • lack of testing and preparation • no video or mic capability • the usual technical glitches • low attendance in some sessions • low participation in some sessions • expectations for a traditional webinar Challenges
  13. 13. would attend another unconference by SETIG 95%
  14. 14. would recommend a similar-style event to a friend or colleague 79%
  15. 15. Breakout facilitators similarly enjoyed the experience. And would do it again if asked. 8%# 18%# 17%# 9%# 33%# 42%# 50%# 27%# 36%# 25%# 50%$ 50%$ 50%$ 55%$ 55%$ 75%$ Enjoyed$the$experience$of$facilita7ng$ Few$technical$problems$while$facilita7ng$ Training$session$for$facilitators$was$helpful$ Able$to$upload$summary$points$easily$ Would$volunteer$again$if$asked$ My$role$and$expecta7ons$were$made$clear$ Disagree$ Neutral$ Agree$ Strongly$Agree$
  16. 16. “It was a great idea to do this. I hope you’ll do it again!” “I enjoyed the opportunity a lot.” “Great modelling of risk taking with grace.”
  17. 17. Next Time • no last minute registrations • mandatory orientation session • test connection ahead of time • make participant expectations and facilitator role more explicit • shorten unconference to 2 days • fewer sessions to fill breakout rooms • crowd source session topics ahead of time • shorten sessions to 45-50 minutes • more extended descriptions of sessions
  18. 18. • designate beginner vs. more intermediate sessions • one facilitator and one note-taker per session • share discussion prompts with participants ahead of time • shrink Participants Guide to one page cheat sheet • strongly recommend use of headset with mic • display session title while in breakout room • ask for 3 key take-aways from each session • accept that some will always want to lurk Next Time (cont’d)
  19. 19. 3 things you learned 2things you want to learn more about 1thing you’ll use going forward
  20. 20. Smaller more regular unconferences that: • focus on one specific question or area • focus on one problem from a systems lens Other SETIG Event Ideas

×