M&M Project Dec12 Mobile Audience Response Apps in Medical Education


Published on

Presentation about a project undertaken by James Petersen and Dr. Tod Aeby of the UHM School of Medicine to enhance engagement and participation by residents in the weekly Morbidity and Mortality Conference. The use of mobile audience response apps was added to the weekly conference with positive results

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • After the remainde of the presentation, responses changed appropriately
  • After the remainde of the presentation, responses changed appropriately
  • Now discussing case 2
  • Response to the question: Who thinks that every patient admitted with PID should have an ultrasound? True=yes, False=No
  • Here, they showed that they already knew the material pretty well so we could move quickly along to a new topic
  • M&M Project Dec12 Mobile Audience Response Apps in Medical Education

    1. 1. M&M -M&M - Final Mobile Learning ProjectFinal Mobile Learning ProjectThe Use of MobileThe Use of MobileAudience ResponseAudience ResponseApps in Post-GraduateApps in Post-GraduateMedical EducationMedical EducationETEC 647 - Fall12ETEC 647 - Fall12Tod AebyTod AebyJim PetersenJim PetersenSean MoroneySean Moroney
    2. 2. Please Join Us
    3. 3. Overview• Review the purpose of the project• The M&M Concerns• What happened on November 16, 2012• What we learned
    4. 4. Were you paying attention?• Please enter what you you rememberabout what we were trying to improvewith mobile learning
    5. 5. Were you paying attention?• Please enter what you you rememberabout what we were trying to improvewith mobile learningarner Engagement!!
    6. 6. Mobile Audience Response: enhanced engagement forlearningThere is substantial evidence in the educational literatureThere is substantial evidence in the educational literatureindicating that increased engagement, participation, andindicating that increased engagement, participation, andinteraction in classroom and other educational settings caninteraction in classroom and other educational settings canpositively impact the learning of studentspositively impact the learning of studentsThe purpose of this project is to introduce the use of a free mobileThe purpose of this project is to introduce the use of a free mobileaudience response application with the goal of increasingaudience response application with the goal of increasingengagement and participation of the residents who do not usuallyengagement and participation of the residents who do not usuallyrespond or participate in the conference. The assumption is thatrespond or participate in the conference. The assumption is thatincreased engagement and participation will enhance the value ofincreased engagement and participation will enhance the value ofthe M&M Conference as an important learning tool in the residentsthe M&M Conference as an important learning tool in the residents’’medical education.medical education.Recall:Recall:
    7. 7. Review of Literature•Note: the literature concerning Audience Response Systems refers almost entirely to the more traditional “Clicker” systems•Audience response systems (ARS) were first introduced at Stanford and Cornell Universities in the mid 1960s and becamecommercially available in 1992. (Kay & LeSage, 2009)•Since the early 1990s, the use of audience response systems in higher education, professional education, and business has grownsubstantially.•In the past two decades, there have been numerous studies concerning the use of audience response systems that have indicatedthat this strategy has benefits in the teaching and learning process.Some of these are:•“students are more engaged in class”, students participate with peers more...to solve problems”, and “students are morefocused in class”.•There is also evidence that the use of these devices improves formative assessment and contingent teaching. (Kay & LeSage,2009)•There is evidence that the use of ARS provides a more positive impact on student participation than do more traditionalaudience response strategies such as surveys and note cards. (Stowell & Nelson, 2007)
    8. 8. Review of LiteratureInvestigators have reported that the use ARS can be efficacious in medical education; specifically in the process ofdelivering medical information to postgraduate medical residents.In one study of Family Medicine Residents, the investigators conducted a controlled crossover study and founda significant difference in the retention of information between groups who received traditional didactic lectures,lectures with an interactive component, and interactive lectures incorporating ARS. The investigators concludedthat “improved retention occurs with active participation in the lecture process.” (Schackow, Chavez, Loya, &Friedman, 2004)Another study compared the efficacy of traditional lecture style delivery of information to Obstetrics andGynecology Residents with interactive lecture style delivery supplemented with an audience response system. Theresults of this randomized controlled trial indicated that Residents who received the ARS interactive lecturetreatment showed a pre/post test improvement approximately ten times that of the group receiving the traditionallecture treatment. (Pradhan, Sparano, & Ananth, 2005)A review of the literature concerning the efficacy of ARS predominantly indicates the value of interactive devices inpromoting learning, retention, and engagement. (Banks, 2006) (Schmidt, 2011) (Guse & Zobitz, 2011)There is also evidence that the use of ARS to replace traditional didactic lecture delivery enhances motivation and positiveaffective outcomes. (Cain, Black, & Rohr, 2009)*! - important*! - important
    9. 9. Clients costcost(50 seats)(50 seats)ease of deploymentease of deployment1-5 stars1-5 starsease of useease of use1-5 stars1-5 starstypes of responsesavailableother costsadd-onsdeveloper link Platforms CommentseClicker Audience Free to student*** ****yes/noagee/dismuliple choiceNo short answerPresenter App is $15.00 http://www.bignerdranch.com/apps/eclicker-audienceiOS, html Well established andstable company.MeridaWebKeyPadGet quote*** ****yes/nomultiple choiceNO short answerHard to know! http://www.meridiaars.com/audiencepollingsoftware/Android, iOS,BlackBerry or WindowsPhone, htmlLots of polling options. Have to email for aprice quote (soundsexpensive)Socrative free (50 seats)***** *****yes/nomultiple choiceshort answergamesexit ticketNone http://www.socrative.com Android, iOS,BlackBerry orWindows Phone, htmlGreat Blogeasy to useSolero Respond $5.00 Client**** ****yes/nomuliple choiceshort answer$10-30 per monthsubscriptionhttp://solaro.com/otherproducts IOSSRN Client free to student** **yes/nomuliple choiceshort answer$125 (50 clients) http://studentresponsenetwork.comIOS, html personal server-basedTurning TechnologiesResponseWare$19/yrper student ** ****yes/nomultiple choiceshort answerPresenter costs http://www.turningtechnologies.com/response-solutionsProprietary clickers,Android, iOS,BlackBerry or WindowsPhoneCan be used in mixedenvironment (phones,various clickers, etc)Via Response free (10 seat trial)** **extensive (if you have to ask, youcan’t afford it - fee notopenly posted)http://www.viaresponse.com/Whatisit.htmlIOS, html (low ratings on AppStore)Poll Everywhere(SMS based)available free on smartdevice *** ***yes/nomultiple choice$15-1400 per monthsubscriptionhttp://www.polleverywhere.complatform independent(smartphone)Selection of Mobile AudienceSelection of Mobile AudienceResponse AppResponse App
    10. 10. Three strategies• Test general knowledge on a subject• What do they know and are they paying attention?• Force commitment to test clinicaljudgement and reasoning• What would they do and are they paying attention?• Assess the current mental model• Retest to confirm mastery of the concepts• Did they follow the conversation and construct the appropriate mental model
    11. 11. The Conference The project team arrived early to setup the equipment
    12. 12. The ConferenceThe audience customarily seats itself ina “culturally” determined manner, by rolegroup
    13. 13. The Conference The Moderator was positioned sothat he could observe the presenter,audience, and both screens
    14. 14. The ConferenceAudience questions were clearly delineated by aspecial slide style
    15. 15. The ConferenceUsing Socrative on the iPad to presentresponses from the audience
    16. 16. The ConferenceAudience responses are then projected to thesmaller screen
    17. 17. The ConferenceUnexpected Consequence -Formative Assessment - immediate feedbackto moderator allowing instructional adjustment
    18. 18. The Residents present theThe Residents present thecases...cases...
    19. 19. Other information of importance tomedical practice is also presented
    20. 20. 73% Laparoscopy 65% Laparotomy
    21. 21. 73% Laparoscopy 65% Laparotomy
    22. 22. Evidence-basedTreatment of URI• Zinc lozenges• High dose vitamin C• Symptom relief
    23. 23. Vote or New List
    24. 24. Text
    25. 25. Text
    26. 26. Text
    27. 27. Text
    28. 28. Text
    29. 29. Text
    30. 30. Average 6.4 (2-10)• 30% “Totally Got It”, 68% “Pretty Well”• “Need bigger screen”, “Slow internet”,“Awesome, dude”,”Kinda fun”
    31. 31. Staff ObserversStaff ObserversThree staff members served asvolunteer observersThey were provided a link to aGoogle Form to record theirobservations
    32. 32. Observers• Audience size estimate at start, 39-50• Mobile devices out/running @ 5 minutes 40 -95%• “Socrative taking a while”, “May need printeddirections for late comers”, “Can you mask theentering answers so you don’t influence thelater ones”, Faculty and private docs hadmobile devices but not using Socrative”, “Mostresidence [sic] seemed engaged”
    33. 33. Observers• The observers, some known to the audience(hospital staff) and some unknown (UH ETECstudents), unobtrusively made periodic countsof the audience to assess involvement.• At any time throughout the session, about two-thirds of the audience members had a mobiledevice on.• Perhaps, not all of these were linked toSocrative.
    34. 34. Audience EngagementObserver7:45 8:00 8:15 8:30 8:451 9 9 9 8 92 10 10 9 9 93 4 7 9 8 8
    35. 35. Important Logistical Points• Late arrivals may have had difficulty logging in.Some form of written instructions on this wouldprobably be handy.• Not revealing the answers given to a questionuntil the question time is closed can give aclearer picture of the thinking of the audiencemembers.• Network speed can be an issue in thebriskness of the interactions.
    36. 36. JimJim’’ssReflectionReflection• Process• This was a fascinating project on a number of different levels• The process was challenging since all of the team members are over-employed adultprofessionals residing in geographically disparate locations and with vastly differingschedules.• Thus, online collaboration was absolutely necessary. Extensive use was made ofGoogle Docs, Dropbox, and BlackBoard.• It became apparent during the course of the project that organization andcommunication of information needs to be very different in an onlinecollaboration setting.
    37. 37. JimJim’’ssReflection 2Reflection 2• Content• One of the big realizations for me was how the app enabled the moderator to makeon-the-fly formative assessments and adjust instruction• This project has, for me, wide extensibility in a variety of educational settings.Several teachers in my school are now using the tool in classroom settings• It was mentioned in the research literature and observed in the trial that there is astrong affective component when mobile devices are used in teaching and learning.• I learned valuable lessons about the physical layout necessary to integrate themobile element and I have some suggestions I will forward to the app developer tomake it a more effective tool. (e.g. ability to hide responses until all recorded)
    38. 38. Sean’s Reflection
    39. 39. Tod’s ReflectionHow did all the old white-hairs all end up in the same group?A dry run really helpsImproved participation,will it last?Too many devices in the kitchen spoils the soup-Have to have broad band-What in the world am I going to do on Maui???This has been a wonderful, eye-opening educational experienceThanks Dr. Grace!