Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Real Time Evaluation Handout


Published on

This handout accompanies the slides on real time evaluation (

Published in: Data & Analytics
  • Be the first to comment

Real Time Evaluation Handout

  1. 1. 1 Real Time Evaluation: Tips, Tricks, and Tools of the Trade Yuqi Wang Wednesday,November11,2015 AEA 2015, Chicago,IL @yuqi_2w6 Survey Creating real time surveys  The short methodof surveyingin real time is to use software withthe abilitytosurveypeople inreal time,andanalyze the results instantly.Oftentimes youcanembed the software in webinarsand PowerPointpresentations. Some commonlyusedonline software are: o o o  The longmethod of surveyinginreal time isto use hard copiesof surveystocollectreal time data, anduse Excel for data entryand analysis.Fora stepbystepprocessof how to do this,go to thislink: Tips and resources  An Excel resource fordoingthe longmethod of surveyinginreal time  Attentiontodetail andorganizationisimportant,especiallyforthe long method.Make sure all your formulasfunctioncorrectlybefore dataentryday.  Make sure the surveyyouare usinghasmostly,if notall,closed-endedquestions,like aLikert Scale.Closed-endedquestionshave ashortanalysistime,andthe analysiswill be easyfor both Excel andthe software todo.  An AEA 2015 sessionon the topicyoucan go to is “GettingReal with Real Time Data Collection” Thursday,November12, 3:00-4:30PM, Rooseveltroom, Ignite 31. Danielle Toussaint,Ruthie Chang,andKristie Glatze of Hatchuel TabernikandAssociateswill reviewthe evaluationof a2014 international techconference, andshow how the use of mixedmethods helpedtoenhance andproduce memorable results. H-form Creating real time H-forms Start withwritingthe questionortopicyouwantparticipantstoprovide theirfeedbackonat the topof a blankpiece of paper.Draw a horizontal line inthe middle of the paper,andlabel itwiththe spectrum of opinionsyouwantparticipantstoprovide feedbackon(e.g.stronglydisagree,disagree,neutral, agree,andstronglydisagree).Leave enoughspace onthe leftandrightside of the paperto write “Why do youdisagree?”and“Why doyou agree”,respectively. Once everyone hasgottenachance to provide their feedback,the groupcanbegintodiscusswhatthey shared,andstart to reach a group understandingaboutthe questionortopic.
  2. 2. 2 Real Time Evaluation: Tips, Tricks, and Tools of the Trade Yuqi Wang Wednesday,November11,2015 AEA 2015, Chicago,IL @yuqi_2w6 Tips and resources  Provide participantswithdotstickerstheycanplace anywhereonthe spectrumof opinions.You can alsoprovide post-itnotes forparticipantsso theycanjotdowntheirreasoningbehindtheir opinion.  An AEA 2015 session onthe topicyoucan go to is“Supercharge Your Focus Groups!UsingAdult FacilitationTechniquesto Capture rich Data and Ensuring Your Participants Have Fun inthe Process!” Wednesday,November11,6:15-7:00PM, Grand Suite 2A/B. Lynne Franco of EnCompassLLC will introduce atechnique andprocessthatinvolvesthe use of alarge stickywall and index cards – toolsthat can draw out perspectivesandfostergroupprocessingdata. Timeline Creating real time timelines Timelines canbe a good forparticipantstothinkabout whatworkedandwhat didn’t workfora particulareventsotheyare able to applywhattheylearnedtofuture events.Butbefore creatinga timeline, it’simportanttomake sure participantsare clearon whytheyare creatinga timeline,and whatit isgoingto be usedfor. Once you have a goodunderstandingof the timeline’s purposeandutility,one waytocreate a timeline isto use blue painterstape,andlarge butcher-block paper.Use tape tocreate a single line onthe paper – thiswill representthe pastand presentsectionsof the timeline.Use smallerpiecesof tape tocreate a dottedline representing future.Have participantswriteonpost-itnotesmilestonesthathave occurred withinthe spanof the timeline,andstickiton to the appropriate sections.Youcanalsohave participantschoose afewmilestonestheylistedtoexplore the milestones’conditionsof successand challenges. Tips and resources  You can begina timelinewhenaprojectstarts.Thisiscalleddoinga“pre-mortem”.Harvard BusinessReviewhasagreat article on pre-mortems:http://bit.l y/1EMTn7a  Angie Ficek wrote agreat AEA365 blog on creatingtimelineswithline charts:  Color-codingpastandpresent eventsorgroupsof pastand present of events onyourtimeline will be helpfulforyoutoorganize whathappenedinthe pastand present,and will be helpfulfor youto spot anypatterns withinyourtimeline.  An AEA 2015 sessiononthe topicyoucan go to is “PuttingData in Context:Timeliningfor Evaluators” Friday, November13, 7:00-7:45AM, Burnham room. Kat Athanasiades,Smriti Bajracharya,and Katherine Haughof InnovationNetworkwilldemonstrate how toincorporate a timelineintoareport,use a timelinetotrack progressinternally,andutilize datavisualization principlestocreate avisual timeline.