Data collection and analysis
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Data collection and analysis

on

  • 329 views

Week 4 lecture for im2044 2012-2013

Week 4 lecture for im2044 2012-2013

Statistics

Views

Total Views
329
Views on SlideShare
329
Embed Views
0

Actions

Likes
0
Downloads
14
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Analyzing video and data logs can be time-consuming <br />

Data collection and analysis Presentation Transcript

  • 1. IM2044 – Week 4: Lecture Dr. Andres Baravalle 1
  • 2. Interaction design • The next slides are based on the companion slides for the textbook • By the end of this week, you should have studied all chapters of the textbook up to chapter 8 2
  • 3. Overview • Data gathering – Interviews – Questionnaires – Ethnographic observations and observations in a controlled environment • Qualitative and quantitative analysis 3
  • 4. Key issues in data gathering • Setting goals – Decide how to analyze data once collected • Identifying participants – Decide who to gather data from • Relationship with participants – Clear and professional – Informed consent when appropriate • Triangulation – Look at data from more than one perspective • Pilot studies – Small trial of main study 4
  • 5. Data recording • Typically supported by a combination of: – Notes – Audio recording – Video recording – Photography 5
  • 6. Interviews • Unstructured - not directed by a script. Rich but not replicable. • Structured - tightly scripted, often like a questionnaire. Replicable but may lack richness. • Semi-structured - guided by a script but interesting issues can be explored in more depth. Can provide a good balance between richness and replicability. 6
  • 7. Closed vs. open questions • ‘Closed questions’ have a predetermined answer format, e.g., ‘yes’ or ‘no’ – Easier to analyse • ‘Open questions’ do not have a predetermined format – Can allow to better explore research topics 7
  • 8. Questions to avoid • Long questions • Compound sentences - split them into two • Jargon and language that the interviewee may not understand • Leading questions that make assumptions – e.g., why do you like …? Or – Asking a question that the respondent is not qualified to answer • Unconscious biases, e.g. gender stereotypes 8
  • 9. Running the interview • Introduction – introduce yourself, explain the goals of the interview, reassure about the ethical issues, ask to record, present any informed consent form. • Warm-up – make first questions easy and nonthreatening. • Main body – present questions in a logical order • A cool-off period – include a few easy questions to defuse tension at the end • Closure – thank interviewee and signal the end, e.g. switch recorder off. 9
  • 10. Enriching the interview process • Props - devices for prompting interviewee, e.g., a prototype, scenario 10
  • 11. Questionnaires • Questions can be closed or open – Closed questions are easier to analyze, and may be done by computer • Can be administered to large populations – Paper, email and the web used for dissemination • Sampling can be a problem when the size of a population is unknown as is common online 11
  • 12. Questionnaire design • Provide clear instructions on how to complete the questionnaire. • Decide on whether phrases will all be positive, all negative or mixed. • Different versions of the questionnaire might be needed for different populations. • The impact of a question can be influenced by question order. • Strike a balance between using white space and keeping the questionnaire compact. 12
  • 13. Question and response format • ‘Yes’ and ‘No’ checkboxes • Checkboxes that offer many options • Rating scales – Likert scales – Semantic scales – 3, 5, 7 or more points • Open-ended responses 13
  • 14. Encouraging a good response • Make sure purpose of study is clear • Ensure questionnaire is well designed – Consider offering a short version for those who do not have time to complete a long questionnaire • • • • Promise anonymity Follow-up with emails, phone calls, letters Provide an incentive 40% response rate is high, 20% is often acceptable 14
  • 15. On-line questionnaires • Responses are usually received quickly • No copying and/or postage costs • Data can be easily collected in database for analysis • Time required for data analysis is reduced • Errors can be corrected easily 15
  • 16. Problems with online questionnaires • Sampling is problematic if population size is unknown • Preventing individuals from responding more than once 16
  • 17. Observation • Direct observation in the field – Structuring frameworks – Degree of participation (insider or outsider) – Ethnography • Direct observation in controlled environments • Indirect observation: tracking users’ activities – Diaries – Interaction logging 17
  • 18. Structuring frameworks to guide observation • The Goetz and LeCompte (1984) framework: - Who is present? - What is their role? - What is happening? - When does the activity occur? - Where is it happening? - Why is it happening? - How is the activity organized? 18
  • 19. Ethnographic observations (1) • Ethnography is a is a qualitative research method aimed to understand human behaviour – Techniques used include participant observation, interviews and questionnaires • Ethnographers immerse themselves in the culture that they study – Collections of comments, incidents, and artifacts are made – A researcher’s degree of participation can vary along a scale from ‘outside’ to ‘inside’ 19
  • 20. Ethnographic observations (2) • Co-operation of people being observed is required – Informants are central – Questions get refined as understanding grows • Data analysis is continuous and typically time consuming • Reports usually contain examples 20
  • 21. Online ethnography • On-line interaction differs from face-toface • Virtual worlds have a persistence that physical worlds do not have • Ethical considerations and presentation issues are different 21
  • 22. Observation in a controlled environment • Direct observation: – Think-aloud technique • Indirect observation: – Diaries – Interaction logs – Web analytics 22
  • 23. Choosing and combining data gathering techniques • Depends on: – The focus of the study – The participants involved – The nature of the techniques selected – The resources available 23
  • 24. Quantitative and qualitative data • Quantitative data – expressed as numbers – Quantitative analysis – numerical methods to ascertain size, magnitude, amount • Qualitative data – difficult to measure sensibly as numbers e.g. dissatisfaction – Qualitative analysis – expresses the nature of elements and is represented as themes, patterns, stories • Be very careful on how you manipulate data and numbers! 24
  • 25. Simple quantitative analysis • Averages – Mean: add up values and divide by number of data points – Median: middle value of data when ranked – Mode: figure that appears most often in the data • Percentages • Graphical representations give overview of data Number of errors made Internet use 10 < once a day 8 once a day 6 4 once a week 2 2 or 3 times a week 0 0 5 10 15 20 once a month User 25 Number of errors made Number of errors made Number of errors made 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 1 3 5 7 9 User 11 13 15 17
  • 26. Visualising log data Interaction profiles of players in online game Log of web page activity 26
  • 27. Visualising log data • Web analytics 27
  • 28. Simple qualitative analysis • Recurring patterns or themes – Emergent from data, dependent on observation framework if used • Categorizing data – Categorization scheme may be emergent or pre-specified • Looking for critical incidents – Helps to focus in on key events 28
  • 29. Tools to support data analysis • Spreadsheet – simple to use, basic graphs • Statistical packages - e.g. SPSS • Specialist qualitative data analysis tools – Categorization and theme-based analysis, e.g. N6 – Quantitative analysis of text-based data 29
  • 30. Theoretical frameworks for qualitative analysis • Basing data analysis around theoretical frameworks provides further insight • Three such frameworks are: – Grounded Theory – Distributed Cognition – Activity Theory 30
  • 31. Grounded Theory • Aims to derive theory from systematic analysis of data • Based on categorization approach (called here ‘coding’) • Three levels of ‘coding’ – Open: identify categories – Axial: flesh out and link to subcategories – Selective: form theoretical scheme • Researchers are encouraged to draw on own theoretical backgrounds to inform analysis 31
  • 32. Distributed Cognition • The people, environment & artefacts are regarded as one cognitive system • Used for analyzing collaborative work • Focuses on information propagation & transformation 32
  • 33. Activity Theory • Explains human behavior in terms of our practical activity with the world • Provides a framework that focuses analysis around the concept of an ‘activity’ and helps to identify tensions between the different elements of the system • Two key models: one outlines what constitutes an ‘activity’; one models the mediating role of artifacts 33
  • 34. Presenting the findings • Only make claims that your data can support • The best way to present your findings depends on the audience, the purpose, and the data gathering and analysis undertaken • Graphical representations may be appropriate for presentation • Other techniques are: – Rigorous notations, e.g. UML – Using stories, e.g. to create scenarios 34