Encouraging Active Participant Engagement in the Evaluation of Online Conferencing
Encouraging Active Participant Engagement in theEvaluation of Online Conferencing Angela Murphy Amy Antonio Shirley Reushle
Online conferences offer convenient professional development opportunities without the travel and expense of face to face. Most conference evaluations focus on participant satisfaction and reactions and do not evaluate the learning that actually occurs (Anderson & Anderson, 2010). This study adapted the newlearning methodology (Chapman et al., 2007) to evaluate the impact of an online conference holistically and interactively.
The Follow the Sun Online Learning Futures Festival ran non-stop for 48 hours, over six shifts of eight hours each, with consecutive handovers between Australia (University of Southern Queensland), United Kingdom (Leicester University) and Canada (Athabasca University).
The aim of the conference was to bring together university staff and students from a range of disciplines across the world to share ideas and explore knowledge development. 750 participants from over 35 countries registered for the conference.
The non-stop nature of the event aimed to mirror a 24-hour digital society and the 21st century learner who wants to be engaged with other learners any time, anywhere.Image source: http://www.rgbstock.com/bigphoto/niXkXUu/World+Time+1
Objectives & Approach:1. Identify participation trends during conference sessions. Analytics2. Explore networking and interactions trends. Twitter, chat and social media3. Real-time insights into participant Real time evaluation questions perceptions and new learnings.4. Identify and track new learning and intention to action. Summative survey evaluation5. Presenter experiences and future support requirements. Presenters’ survey6. Inform the conduct of future events. Focus group with organisers
“Please tell us about any new learning you have experienced or any new questions that have occurred to you as a result of this conference?” (Chapman et al., 2007) Image source: http://www.soil-net.com/
Festival analytics 3% 4% 81% 6 7 9 5% 5 4 7% 3 22% Attended two sessionsMost online conferenceparticipants attend only 51% one or two sessions Attended only one session
Twitter and social media conversations Conversational Tweets that included an observation or expressed an opinion “Interesting conversation“Nominal Group Technique about the future of theSession @ #fts12. Join us!” world” Promotional Informative Tweets that pointed to Tweets that promoted an resources with an upcoming presentation accompanying link “Recordings from #fts12 now available @ http://t.co/smWYn413” Instructional #FTS12 Tweets could Tweets with a directive to perform an activity be categorised “Click latecomers link to join! into four groups http://t.co/5iTLCwN”
Real-time evaluation questions Most found that the sessions inspired ideas for dailyPlease indicate your agreement with the sessions practice that they intend tofacilitated new ideas and learning : Total Sample (n=60) action
Summative survey evaluation for participants I was actively listening, asking questions and/or 38% communicating I listened closely to the sessions but did not ask 28% questions I listened occasionally while working or multi- 21% tasking I dropped in and out of sessions 6% I tried to listen and multitask but didnt manage 3% it very well Online conference None of these (please specify) 4% participants listen and participate activelyQ8. Please indicate which of the following are the closest to the way in which youparticipated in the sessions. (Select one) (n=116) during sessions
Summative survey evaluation for presenters “Its more difficult to keep participants engaged because you cantrely on eye contact, body language or movement around the room to help maintain interest. You also have to keep talking (most of the time) as the participant isnt sure what is happening during a pause. You also have to keep faith that the technology will work for both yourself & the participants!” “It was just hard to keep going with the presentation and keep a tab on what was going on with the social media at the same time.”“It was the first time I had talked at participants for 30 min, rather than seeking feedback periodically.Although I thought it went well, I did find that it was Presenters found hard to present for that long without seeking some kind of feedback along the way..” online presentations to be challenging
Focus group for conference organisers Using the same old tech advertising didn’t reach new audiences within disciplines Risk Collaboration Time consuming and technical to set up ~ difficult within otherReputational impact Global responsibilitiesand reach much reachgreater than couldbe achieved throughother means
Immediate Feedback XII Limited Responses Identify issues as they occur III Time consuming Address issues Create promptly Difficult to opportunities organisefor engagement
Difficulties in setting aside dedicated time above work commitments is the primary barrier to attending online conferencesWhat barriers have you experienced in attempting to attend online events ingeneral? Select those that are the most applicable to you) (n=172)
The combination of synchronous and asynchronous features is considered to be the ideal online conference format An event with synchronous and asynchronous features, e.g. pre-recorded video accompanied by questions in real time 34% Pre-recorded mini webinars that can be accessed at any time 17%Mini webinars of one or two hours that include options to post questions and communicate with other delegates 16% Shorter sessions, e g , 30 minutes 12% Full online conference over two or more days 7% Full online conference over a day 6%Mini webinars of no longer than half a day that include options to post questions and communicate with other delegates 5% Other preferences 3%
Learning, technologies and futures are the topics of greatest interest to most participants