The Key Challenge in Behavioural Research
•Download as PPTX, PDF•
2 likes•1,211 views
In this webinar, Dr. Esther Tippmann will explain the Common Methods Bias - a well-documented phenomenon observed in research based on self-reported measures. We often use surveys in which respondents are asked about their perceptions, feelings, attitudes or beliefs. Under certain circumstances such self-reported data can suffer from severe quality issues. It is the aim of this webinar to introduce some of these and discuss effective survey design remedies.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (15)
Similar to The Key Challenge in Behavioural Research
Similar to The Key Challenge in Behavioural Research (20)
More from Qualtrics
More from Qualtrics (20)
Recently uploaded
Recently uploaded (20)
The Key Challenge in Behavioural Research
- 1. The Key Challenge in Behavioural Research Understand the Issue and Uncover the Solution Welcome
- 2. Housekeeping • The recording and slides for today’s presentation will be made available to you through email after the webinar commences • Use the chat bar to ask questions • Join the conversation on Twitter by tweeting @Qualtrics
- 3. Agenda • Introduction • Common Method Bias - Explained • Common Method Bias - Sources • Remedies to Solve Common Method Bias
- 4. Dr. Esther Tippmann Lecturer at the Department of Management University College Dublin Elena Sugrue Academic Account Executive Qualtrics
- 5. We often use self-report surveys Respondent reads the question and selects a response without researcher interference Involves asking about respondent’s behaviours, feelings, attitudes, beliefs, etc. Self-report surveys often have data quality issues
- 6. Common method bias as one data quality issue Bias attributable to measurement method: Respondents have a tendency to answer questions in a consistent manner Can cause systematic measurement error Concern most severe if: Data on both the outcome variable (dependent variable such as performance indicators) and explanatory variable (such as activities, behaviours, beliefs, attitudes) at the same time from the same respondent (Source: Podsakoff, MacKenzie, Lee & Podsakoff (2003) Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5): 879-903)
- 7. Common method bias may inflate covariation among variables Relationship in data collected from a single respondent Concern of common method bias Relationship in data collected from two respondents (One respondent reports outcome variable, another explanatory variable) (For example: Podsakoff, MacKenzie & Podsakoff (2012) Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63(8): 539-569)
- 8. Research design remedies to solve common method bias • Collect information for key constructs at different points in time • Questionnaire design • Using multiple respondents (sources) to collect information for key constructs 1st respondent reports on outcome variable 2nd respondent reports on explanatory variables
- 9. Multiple respondents and pairing surveys
- 10. Multiple respondents and pairing surveys
- 11. Reducing Bias in Survey Responses: The Role of Randomisation Problem: • Selecting the first reasonable option is one of the strategies employed by respondents that are not engaging in the optimal process of responding to a question Solution: 1. Answer choice randomisation (fixed display and subsets) 2. Block randomisation 3. Question randomisation
- 12. Label Anchors Different question types Different ways question types Different anchor points Order of labels
- 13. Q&A
- 14. Thank You!
Editor's Notes
- Hello and thank you for joining us for today’s webinar titled ‘The Key Challenge in Behavioural Research – Understand the Issue and Uncover the Solution’
- Just before we begin the webinar, I would like to go over some housekeeping items First off: Our webinar today will be recorded and sent to all of our attendees within the next 24 hours and you will also get a copy of the presentation slides. You also have the option to participate today and ask questions using the chat feature and if time allows, we will address the questions at the end of the webinar. If we do not have time to answer all questions, we will of course follow up afterwards by email. Finally, please don’t forget to join the conversation on Twitter by tweeting @Qualtrics That’s it for housekeeping and I am pleased to move on to today’s agenda
- For the agenda of today’s webinar ‘The Key Challenge in Behavioural Research – Understand the Issue and Uncover the Solution’] I will first start off with a brief introduction to myself and our guest speaker ‘Dr. Esther Tippmann’. Esther will walk us through what ‘Common Method Bias’ is and the main sources of this issue. We will then present some remedies. And lastly we will open up the session for Q&A so make sure to chat us your questions!
- Now, I would like to take the opportunity to introduce myself and and welcome our guest speaker. My name is Elena Sugrue and I am one of the founding members of Qualtrics’ European Academic Team. I currently work with universities, schools and other academic institutions across the Middle East and France to help roll out centralized research solutions for their students and staff to conduct everything from experimental research to student projects, market research surveys, behavioral studies, 360 evaluations, course feedback, alumni relations and much more. I have also worked with universities in other regions including the UK and Ireland and have also been invited as a guest speaker at some of these universities. I am joined today by Dr. Esther Tippmann. Esther is a Lecturer in the Department of Management at University College Dublin, Ireland. Her research and teaching interests revolve around the strategic challenges of internationally operating organizations. She has worked closely with several multinational corporations in Ireland, France, the U.K. and the U.S. on case studies and research projects. Esther is a regular presenter at international conferences and her work has been published in field-leading journals and received several honours, including awards from the Academy of Management and Academy of International Business. She also held a Marie-Curie Fellowship, funded by the European Commission and Irish Research Council for her Post-doctoral studies. Esther has taught in the areas of International Business, Strategic Management and Research Methods on internationally accredited programs in Ireland, Germany and France. We are absolutely delighted to have you join us today Esther and I would now like to hand it over to you.
- Thank you Esther for a great oververview of common method bias. I will now speak briefly about ways to help oververcome this when designing surveys in Qualtrics. So, when responding to a survey often times instead of reading all of the options in a question and selecting the most appropriate answer many survey respondents will select the first reasonable alternative and then quickly move on. Fortunately there is a relatively simple solution: Randomising the order of the list presented to each respondent. So randomising the answer options. This is just 1 of the levels of randomisation that exist in Qualtrics. There are 2 other levels of randomisation exist within qualtrics. The first one is to radomising the order of blocks or sets or groups of questions And the second is to randomise the order of the questions themselves One other thing you may wish to consider is that you can view the order in which your questions were displayed or randomised when downloading your data in Qualtrics. So while randomising doesn’t guarantee your respondents will engage with the question, it generally turns what would have been a bias in your data into variance.
- Another way to look at way of reducing common method bias would be taking care in the design of the individual questions while paying attention to labels and anchor points. Firstly, looking at varying the order in which you present the order of the labels seen in the first example on the right by switching around the labels from strongly disagree to strongly agree. What we are trying to achieve here is a mix throughout the survey as respondents tend to rush through surveys and they need to “stop and think” and engage with the survey more. You can also break out of consistency by changing up the question types for example by using a graphic slider with 5 anchor points instead of a multiple choice question. Finally you can also look at varying the balanced scales between 5 and 7 anchor points or even looking at removing the middle point by using a 6 point scale.