Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Research challenges


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

Research challenges

  1. 1. What are the main challenges inundertaking Educational Research and Training Evaluation and how can these be overcome? Matt McGovern 20th August 2o12
  2. 2. Key challenges Three challenges  Aims and central questions: Lacking clarity  Methods: appropriate to answer questions  Data analysis and presentation: errors and communication
  3. 3. 1) Aims and central questions Unclear or conflicting project aims and questions  Expectations of different stakeholders  How achievable is the project ? How observable are results/impacts? The Managed Solution  A question that can be answered: limited and specific  Aims widely discussed and agreed upon  Time, methods and resources available
  4. 4. 2) Research Methods Methods inappropriate for research aims  Methods used are those most comfortable with  Methods not appropriate for the topic under study and capturing relevant data The managed solution  Careful consideration of methods suitable for the question posed  Imagination, creativity and flexibility  Practical considerations of participation
  5. 5. 3) Data analysis and presentation  Analysis and presentation challenges  using overly complex analysis, repurposing data or too much data  Findings do not communicate to intended audience  The managed solution  Re focus analysis upon agreed aims  Flexible approach to dissemination
  6. 6. Conclusion Clear aims, matched with appropriate research methods Practical and focussed approach to methods , including consideration of what participation involves Analysis continually centred on research questions, with findings presented to communicate
  7. 7. Thank You
  8. 8. Challenges Structured and focussed but flexible Quality, capture experiences of course or actually definitely state somethinghonestly might be a tick box exercise or instructed tobe positive. Get feedback into the report Methods matched to answer the question, qualitative methods to explore relationships but must start with a specific question.
  9. 9. Purpose of educational research/ training evaluationAnswering a questionEvaluation: determine the level of meritShould always be a way to capture experiences, views, incidents
  10. 10. TimeResources Time of your research team Time of your research participants Quick questionnaire
  11. 11. Classic project managmentIdentify all stakeholders and their objectivesDefining questions/ the scope of the studyIndentify the methods that will actually answerquestions/meet the objectives of stakeholdersPlan, consider risks and alternate plansImplement and review
  12. 12. Quant challengesAsking what you don’t know you didn’t knowBiased by participantion, online, email, not comfortable.Include at end of course but less time for reflection.Pressure to go too complicated go into inferentialstatistics
  13. 13. Quanlitative challengesParticipantion,: needs times investmentValidity Sum of your questions actually captures
  14. 14. ChallengesWhat you don’t know your don’t knowParticipants: biases Those who are really passionateeither positively or negatively. Difficult to research those who are indifferent, didn’t complete the course, those who found it a waste of time
  15. 15. OvercomeWhat you don’t know you didn’t know: hard, conreast andblack and white, not exploring relationships are valuable, How many people completed the course? Participant observation: see group dynamics, actual courseNot intensely personal, use a focus group/ group interview, one on one interviews personal and biographicalLooking for effects: longitudinal aspect
  16. 16. Overcome: focussed but flexible Initial interviews with key informants to inform Questionnaire: short and focussed answering simple questions Focus group/ group interviews about the course experience allowing to group stimulate discussion
  17. 17. Altering experiencesParticipants produced stronger views than theyotherwise would haveTell interviwer what they want to hear
  18. 18. ParticipationToo busy to take partHow is your sample biased?Those who had a bad experience at a training may notwant to take part in any research about it waste oftime. I stopped attendingThose who had a good or terrible , mildly mediocreviewpoint gets lost
  19. 19. QuantQuestionnaire too longWhat you don’t know you didn’t know, missing theToo focussed upon narrow indicators
  20. 20. Disentagning effectGraduates from a training programme may go on toachieve an objectControl group which is comparable
  21. 21. Overcome, small and specficSpecific about what you are evaluatingMore than just did they like the courseFleixlble to overcomeInitial qualitative interviews with key informers, questionnaire togain hard data, even from those who so you are getting abroad, inclusive picture, practical questions such as did youcomplete the course or ansewre in a black and white way suchas did you contineu with the study afterwards, use questionairefindings to do deeper and more specific, wider but more awarequestions than before, with particpants, with key informers(those who designed and deliverd the course)
  22. 22. Key challenges Three challenges  Aims and central questions: clear, specific and achievable  Methods: appropriate participation and data capture  Data analysis and presentation: errors and communication