Surveys are easy to do – but harder to do well. In this interactive workshop - delivered to the UX Insight Festival 2020 - I take you through using Total Survey Error as a way of balancing the issues and good practice in survey design to get the best results from your survey.
The session also covered my 7-step survey process, starting with Goals and thinking about Sampling, Questions, Questionnaires, Fieldwork, Responses and Reports. Plus we tackle some of the questions I'm most often asked about creating surveys that work.
Surveys that Work 2020: training course for HMRC user researchers 2020Caroline Jarrett
Slides from a training course on effective surveys, delivered to usability researchers at HMRC. The course took place at HMRC's Longbenton, Newcastle, offices, on January 30, 2020. Survey examples submitted by participants for review have been removed from this presentation.
Surveys that work: a webinar for FocusVision 2021Caroline Jarrett
Creating surveys that work for participants and deliver high quality insight is no mean feat. This is because the survey process is complex, with multiple considerations at every step in the journey.
In this webinar for FocusVision, I introduce the Survey Octopus, my friendly way of talking about the many issues that make surveys one of the most challenging research methods. I also explain how the Survey Octopus maps into the Total Survey Error concept that underpins the work of many survey methodologists.
The Survey Octopus will help you design better surveys by thoughtfully considering:
• What you want to ask about
• Who you want to ask
• The number of people you need to ask
Surveys that work: an introduction to the Survey Octopus and Total Survey ErrorCaroline Jarrett
A presentation for Harvard University's User Research Community on some of the key issues in creating effective surveys, including: why run a survey, writing good questions, statistical significance and how to avoid errors.
Many of us receive multiple requests to complete surveys every day. Some of us find that colleagues or clients think of ‘doing a survey’ as the same as ‘doing some research’ – which may explain why organizations send out so many survey requests.
In this webinar, you’ll meet the Survey Octopus, Caroline Jarrett’s friendly way of talking about the many issues that make surveys one of the most challenging research methods.
The Survey Octopus will help you to:
Explain to colleagues that a survey may not be the first research method to try
Help to justify a choice to work with a “non significant” number of responses
Think about the steps that go into delivering a survey that works
As a bonus, Caroline will also explain how her Survey Octopus maps into the Total Survey Error concept that underpins the work of many survey methodologists.
How to get better results from a survey: Meet the Survey OctopusCaroline Jarrett
The Survey Octopus is a friendly creature who will help you to think about all the crucial issues in crafting a survey.
Presentation by Caroline Jarrett @cjforms for the 2014 Content Strategy Summit #CSSummit
Slides from a workshop introduction to survey methods. The workshop was prepared for staff of the European Bioinformatics Institute in Cambridge, February 2017
Introduction to survey methods at LibDesign2016. A workshop led by Caroline Jarrett for people working in the library service and public sector in the Czech Republic. Caroline Jarrett led this workshop in Prague in September 2016 as part of the LibDesign 2016 conference.
Surveys that work: using questionnaires to gather useful data, November 2010Caroline Jarrett
This presentation to the 22nd Australasian Computer-Human Interaction Conference, OZCHI 2010, compares survey processes and looks at some of the detail of designing surveys – including how to avoid survey error.
Surveys that Work 2020: training course for HMRC user researchers 2020Caroline Jarrett
Slides from a training course on effective surveys, delivered to usability researchers at HMRC. The course took place at HMRC's Longbenton, Newcastle, offices, on January 30, 2020. Survey examples submitted by participants for review have been removed from this presentation.
Surveys that work: a webinar for FocusVision 2021Caroline Jarrett
Creating surveys that work for participants and deliver high quality insight is no mean feat. This is because the survey process is complex, with multiple considerations at every step in the journey.
In this webinar for FocusVision, I introduce the Survey Octopus, my friendly way of talking about the many issues that make surveys one of the most challenging research methods. I also explain how the Survey Octopus maps into the Total Survey Error concept that underpins the work of many survey methodologists.
The Survey Octopus will help you design better surveys by thoughtfully considering:
• What you want to ask about
• Who you want to ask
• The number of people you need to ask
Surveys that work: an introduction to the Survey Octopus and Total Survey ErrorCaroline Jarrett
A presentation for Harvard University's User Research Community on some of the key issues in creating effective surveys, including: why run a survey, writing good questions, statistical significance and how to avoid errors.
Many of us receive multiple requests to complete surveys every day. Some of us find that colleagues or clients think of ‘doing a survey’ as the same as ‘doing some research’ – which may explain why organizations send out so many survey requests.
In this webinar, you’ll meet the Survey Octopus, Caroline Jarrett’s friendly way of talking about the many issues that make surveys one of the most challenging research methods.
The Survey Octopus will help you to:
Explain to colleagues that a survey may not be the first research method to try
Help to justify a choice to work with a “non significant” number of responses
Think about the steps that go into delivering a survey that works
As a bonus, Caroline will also explain how her Survey Octopus maps into the Total Survey Error concept that underpins the work of many survey methodologists.
How to get better results from a survey: Meet the Survey OctopusCaroline Jarrett
The Survey Octopus is a friendly creature who will help you to think about all the crucial issues in crafting a survey.
Presentation by Caroline Jarrett @cjforms for the 2014 Content Strategy Summit #CSSummit
Slides from a workshop introduction to survey methods. The workshop was prepared for staff of the European Bioinformatics Institute in Cambridge, February 2017
Introduction to survey methods at LibDesign2016. A workshop led by Caroline Jarrett for people working in the library service and public sector in the Czech Republic. Caroline Jarrett led this workshop in Prague in September 2016 as part of the LibDesign 2016 conference.
Surveys that work: using questionnaires to gather useful data, November 2010Caroline Jarrett
This presentation to the 22nd Australasian Computer-Human Interaction Conference, OZCHI 2010, compares survey processes and looks at some of the detail of designing surveys – including how to avoid survey error.
Surveys that work: training course for Rosenfeld Media, day 3 Caroline Jarrett
Surveys seem easy: anyone can throw together a few questions, send them out, and hope that they are rewarded with a decent response. But we’ve all seen examples of poorly conceived surveys that couldn’t possibly deliver real insights for the organisation that sponsored them.
This highly participative three-session training - arranged by Rosenfeld Media as part of its Virtual Training with UX Industry Leaders programme - takes you through the whole process of creating an effective survey, from defining a goal through analysis of data and creating a presentation.
These slides come from day 3 of the course: responses and reports.
Forms – the only non-optional part of most user experiences, but often the part that gets the least attention. This session at the 2016 Industry Conf in Newcastle was an opportunity to lead the audience through the design of typical forms and look at the problems and potential ways to improve them.
Surveys that work: training course for Rosenfeld media, day 2Caroline Jarrett
Surveys seem easy: anyone can throw together a few questions, send them out, and hope that they are rewarded with a decent response. But we’ve all seen examples of poorly conceived surveys that couldn’t possibly deliver real insights for the organisation that sponsored them.
This highly participative three-session training - arranged by Rosenfeld Media as part of its Virtual Training with UX Industry Leaders programme - takes you through the whole process of creating an effective survey, from defining a goal through analysis of data and creating a presentation.
These slides come from day 2 of the course: questions, questionnaire and fieldwork
Getting valid results from surveys: meet the Survey Octopus.
Surveys are a powerful research method, but not easy to get right. The Survey Octopus is a way of thinking through the issues that will ensure that you'll get solid results from your survey that you can use to make decisions. Presentation from the UX New Zealand conference 2015 #uxnz2015
Surveys that work:training course for Rosenfeld Media, day 1Caroline Jarrett
Surveys seem easy: anyone can throw together a few questions, send them out, and hope that they are rewarded with a decent response. But we’ve all seen examples of poorly conceived surveys that couldn’t possibly deliver real insights for the organisation that sponsored them.
This highly participative three-session training - arranged by Rosenfeld Media as part of its Virtual Training with UX Industry Leaders programme - takes you through the whole process of creating an effective survey, from defining a goal through analysis of data and creating a presentation.
These slides come from day 1 of the course: goals and sample.
The Survey Octopus - getting valid data from surveys, presentation for UX in ...Caroline Jarrett
Getting valid results from surveys: meet the Survey Octopus.
Surveys are a powerful research method, but not easy to get right. The Survey Octopus is a way of thinking through the issues that will ensure that you'll get solid results from your survey that you can use to make decisions. Presentation from the UX in the City conference, Oxford, March 2016
Total Survey Error for non-specialists: creating better conversations. A presentation of the Survey Octopus at the TSE2015 conference in Baltimore, September 2015.
Effective Use of Surveys in UX | Triangle UXPA WorkshopAmanda Stockwell
On a scale of 1-10, how much do you love this workshop?
Ok, hopefully that is an obviously bad question, both because it hasn't happened yet and because it has some bias baked right in. But take a quick look around all the surveys floating out in the world, and they often don't seem much better. Surveys can be a powerful tool for a UX researcher, but many of us haven't learned how to get the most out of them. In this workshop we'll cover:
Best use cases for surveys (and when to avoid them)
An overview of question types
Guidelines for writing effective, unbiased survey questions
Tips to increase overall engagement and participation
Hands on practice crafting surveys
Basic survey analysis
A presentation on Label placement in forms, at the Technical Communication Summit, the 56th Annual Conference of the Society for Technical Communication, Dallas, US, May 2010. Amongst the time-consuming controversies we look at are left and right alignment, labels above and below fields, how to handle required fields, colons, and sentence case.
Speaker: Caroline Jarrett
To help us get the best out of this tricky research method, Caroline will describe the Survey Octopus, a friendly creature that helps her to tackle all the issues that may lie between 'What we want to ask, and who we want to ask', and a solid, reliable number that can be used to make decisions.
Along the way, we'll encounter the key concept in survey methodology, Total Survey Error, and the various types of error that can affect your survey.
Forms workshop for ConCon Manchester 2016 by @cjformsCaroline Jarrett
How to think about and write for forms, starting with 'what is a form' and then working through how people read forms and how that affects how we write for them.
Ten tips for surveys: on questions, process, and testing your survey.
Books mentioned are listed here: http://rosenfeldmedia.com/uxzeitgeist/lists/cjforms/10-tips-for-a-better-survey-stc2011
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Ideas for extracting the maximum value from a survey that is going to happen anyway.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Working with complex forms such as insurance applications, medical claims, government transactions? This workshop at UXPA2013 has tips for improving them.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Two ways to improve your surveys: the Most Crucial Question and the Burning I...Caroline Jarrett
In this webinar for product managers, Caroline introduces two key concepts from her book on surveys: identifying the most crucial question as part of getting clear on your goals, and allowing respondents to tell you the things that they want to - their burning issue. The webinar was organised by Productboard and held on March 30, 2023.
In this half day workshop for ~WebExpo2023 Caroline Jarrett shares four ways to improve your survey so that you get plenty of useful responses.
Goals: Ruthlessly focus your survey on an immediate decision.
Sample: Write an invitation that makes people want to answer.
Questions: Ditch the rating scales.
Responses: Lose your fear of open answers.
Some thoughts on good survey design delivered to students at Olin College of Engineering. Caroline's talk covers her survey process, survey goals and focusing on a specific decision, sample and sampling error, ditching rating scales, and losing fear of open answers.
Some thoughts on surveys: Boye and Company member conference callCaroline Jarrett
Slides from a short presentation on creating effective surveys. The event was a conference call for members of a community network organised by Janus Boye of Boye & Company.
Surveys are still really popular as a research method with colleagues (if not with service designers).
These slides are from a workshop at the 2021 Service Design in Government conference (@sdingov21) on 'how to improve the survey that is going to happen whether you like it or not'.
In the workshop we looked at a 7-step process for a survey and considered ways of encouraging colleagues to combine surveys with other research methods.
We also practiced techniques for looking at – and improving - a questionnaire.
Surveys that work: training course for Rosenfeld Media, day 3 Caroline Jarrett
Surveys seem easy: anyone can throw together a few questions, send them out, and hope that they are rewarded with a decent response. But we’ve all seen examples of poorly conceived surveys that couldn’t possibly deliver real insights for the organisation that sponsored them.
This highly participative three-session training - arranged by Rosenfeld Media as part of its Virtual Training with UX Industry Leaders programme - takes you through the whole process of creating an effective survey, from defining a goal through analysis of data and creating a presentation.
These slides come from day 3 of the course: responses and reports.
Forms – the only non-optional part of most user experiences, but often the part that gets the least attention. This session at the 2016 Industry Conf in Newcastle was an opportunity to lead the audience through the design of typical forms and look at the problems and potential ways to improve them.
Surveys that work: training course for Rosenfeld media, day 2Caroline Jarrett
Surveys seem easy: anyone can throw together a few questions, send them out, and hope that they are rewarded with a decent response. But we’ve all seen examples of poorly conceived surveys that couldn’t possibly deliver real insights for the organisation that sponsored them.
This highly participative three-session training - arranged by Rosenfeld Media as part of its Virtual Training with UX Industry Leaders programme - takes you through the whole process of creating an effective survey, from defining a goal through analysis of data and creating a presentation.
These slides come from day 2 of the course: questions, questionnaire and fieldwork
Getting valid results from surveys: meet the Survey Octopus.
Surveys are a powerful research method, but not easy to get right. The Survey Octopus is a way of thinking through the issues that will ensure that you'll get solid results from your survey that you can use to make decisions. Presentation from the UX New Zealand conference 2015 #uxnz2015
Surveys that work:training course for Rosenfeld Media, day 1Caroline Jarrett
Surveys seem easy: anyone can throw together a few questions, send them out, and hope that they are rewarded with a decent response. But we’ve all seen examples of poorly conceived surveys that couldn’t possibly deliver real insights for the organisation that sponsored them.
This highly participative three-session training - arranged by Rosenfeld Media as part of its Virtual Training with UX Industry Leaders programme - takes you through the whole process of creating an effective survey, from defining a goal through analysis of data and creating a presentation.
These slides come from day 1 of the course: goals and sample.
The Survey Octopus - getting valid data from surveys, presentation for UX in ...Caroline Jarrett
Getting valid results from surveys: meet the Survey Octopus.
Surveys are a powerful research method, but not easy to get right. The Survey Octopus is a way of thinking through the issues that will ensure that you'll get solid results from your survey that you can use to make decisions. Presentation from the UX in the City conference, Oxford, March 2016
Total Survey Error for non-specialists: creating better conversations. A presentation of the Survey Octopus at the TSE2015 conference in Baltimore, September 2015.
Effective Use of Surveys in UX | Triangle UXPA WorkshopAmanda Stockwell
On a scale of 1-10, how much do you love this workshop?
Ok, hopefully that is an obviously bad question, both because it hasn't happened yet and because it has some bias baked right in. But take a quick look around all the surveys floating out in the world, and they often don't seem much better. Surveys can be a powerful tool for a UX researcher, but many of us haven't learned how to get the most out of them. In this workshop we'll cover:
Best use cases for surveys (and when to avoid them)
An overview of question types
Guidelines for writing effective, unbiased survey questions
Tips to increase overall engagement and participation
Hands on practice crafting surveys
Basic survey analysis
A presentation on Label placement in forms, at the Technical Communication Summit, the 56th Annual Conference of the Society for Technical Communication, Dallas, US, May 2010. Amongst the time-consuming controversies we look at are left and right alignment, labels above and below fields, how to handle required fields, colons, and sentence case.
Speaker: Caroline Jarrett
To help us get the best out of this tricky research method, Caroline will describe the Survey Octopus, a friendly creature that helps her to tackle all the issues that may lie between 'What we want to ask, and who we want to ask', and a solid, reliable number that can be used to make decisions.
Along the way, we'll encounter the key concept in survey methodology, Total Survey Error, and the various types of error that can affect your survey.
Forms workshop for ConCon Manchester 2016 by @cjformsCaroline Jarrett
How to think about and write for forms, starting with 'what is a form' and then working through how people read forms and how that affects how we write for them.
Ten tips for surveys: on questions, process, and testing your survey.
Books mentioned are listed here: http://rosenfeldmedia.com/uxzeitgeist/lists/cjforms/10-tips-for-a-better-survey-stc2011
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Ideas for extracting the maximum value from a survey that is going to happen anyway.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Working with complex forms such as insurance applications, medical claims, government transactions? This workshop at UXPA2013 has tips for improving them.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Two ways to improve your surveys: the Most Crucial Question and the Burning I...Caroline Jarrett
In this webinar for product managers, Caroline introduces two key concepts from her book on surveys: identifying the most crucial question as part of getting clear on your goals, and allowing respondents to tell you the things that they want to - their burning issue. The webinar was organised by Productboard and held on March 30, 2023.
In this half day workshop for ~WebExpo2023 Caroline Jarrett shares four ways to improve your survey so that you get plenty of useful responses.
Goals: Ruthlessly focus your survey on an immediate decision.
Sample: Write an invitation that makes people want to answer.
Questions: Ditch the rating scales.
Responses: Lose your fear of open answers.
Some thoughts on good survey design delivered to students at Olin College of Engineering. Caroline's talk covers her survey process, survey goals and focusing on a specific decision, sample and sampling error, ditching rating scales, and losing fear of open answers.
Some thoughts on surveys: Boye and Company member conference callCaroline Jarrett
Slides from a short presentation on creating effective surveys. The event was a conference call for members of a community network organised by Janus Boye of Boye & Company.
Surveys are still really popular as a research method with colleagues (if not with service designers).
These slides are from a workshop at the 2021 Service Design in Government conference (@sdingov21) on 'how to improve the survey that is going to happen whether you like it or not'.
In the workshop we looked at a 7-step process for a survey and considered ways of encouraging colleagues to combine surveys with other research methods.
We also practiced techniques for looking at – and improving - a questionnaire.
Plain language to improve your survey houston 2022Caroline Jarrett
Plain language skills are vital for surveys - and especially to writing good questions and creating them for your survey audience. This presentation was prepared for the University of Houston's 8th Biannual Forum on Plain English, 24 February 2022.
Two ways to improve your survey, webinar for Delib 2023.pptxCaroline Jarrett
In this webinar for Delib, Caroline shows you how to get better results from shorter, more frequent surveys - with a special emphasis on local government and the requirement to run statutory consultations. Understanding and identifying the Most Crucial Question and making space for the Burning Issue are both helpful techniques for creating shorter more focused surveys.
A presentation for the the Content Wrangler's coffee and content session on how to design and run surveys and gain actionable insights from the survey data.
Did you love the form that you filled in most recently? Or did you hit some problems? Most of us find all sorts of small or major problems with lots of the forms we are forced to use.
In this talk for #WebExpo2023, Caroline turns that around. She points out the ways in which not fixing your forms is costing your organisation a lot of money. She then goes on to share plenty of practical tips for making improvements that will enable people to successfully complete your forms.
In this member call for Boye & Co Caroline takes participants through her process for expert reviews of forms. She also shares some of her top tips for making them easier to use and more effective.
Social Query is a new and efficient way to get
answers on the social networks. However, the popular method of sharing public questions could be optimized by directing the question to an expert, a process called query routing. In this work, we propose a Social Query System for query routing on Twitter, currently, one of the most popular social networks. The Social Query Systems analyzes the information about the questioner’s followers and recommends the most suitable users to answer the questions. The use of the system changes the usual process, working apart of Twitter and allowing questioner and responder exceed the limit of 140 characters. Through a qualitative evaluation, we showed promising results and ideas for improving the system and the recommendation algorithm.
Feedback & Surveys - How to use the Constant Contact Toolkit Part 2Frithjof Petscheleit
Take Marketing To the Next Level with the Constant Contact Toolkit
Finally, with a single login you can engage and grow your audience in all the places that matter: the inbox, mobile, social media, and the web. The Constant Contact Toolkit has beautiful, customizable templates to create your campaign fast. Integrated contact management and real-time reporting insights help you see results with each campaign.
This webinar series introduces all the awesome new Constant Contact tools. With one click you can sign up and take part in all free sessions.
Newsletters and Announcements
Surveys and Feedback
Event Promo & Registration
Deals and Promotions
Auto responders
How to ask better questions and how to assess UX using surveys.
This workshop at UXLX 2014 in Lisbon was a deep dive into two important topics in survey design for user research.
We used the four-step model of how people answer questions to work on better questions, then we focused on two special uses of questionnaires in user research: the post-test assessment of satisfaction, and then how to gather information from users for redesign.
Thanks to all the attendees for making this workshop a lot of fun.
Caroline Jarrett @cjforms
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Forms that work: Understanding forms to improve their design by @cjformsCaroline Jarrett
A day-long workshop on forms design, focusing on why businesses need forms and how people interact with them.
Accessibility note: I've tried to make this version of the presentation accessible. If you find that it's not working for you, please let me know and I'll try my best to solve the problems.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
How to Conduct a Survey gf form to anylyzedenjrodrigo
Expect to conduct a survey to analyze and make a solution to the challenges and problems faced of students in highschool and college. This can help students to make or conduct a survey easily. Expect to conduct a survey to analyze and make a solution to the challenges and problems faced of students in highschool and college. This can help students to make or conduct a survey easily. Expect to conduct a survey to analyze and make a solution to the challenges and problems faced of students in highschool and college. This can help students to make or conduct a survey easily. Expect to conduct a survey to analyze and make a solution to the challenges and problems faced of students in highschool and college. This can help students to make or conduct a survey easily. Expect to conduct a survey to analyze and make a solution to the challenges and problems faced of students in highschool and college. This can help students to make or conduct a survey easily. Expect to conduct a survey to analyze and make a solution to the challenges and problems faced of students in highschool and college. This can help students to make or conduct a survey easily. Expect to conduct a survey to analyze and make a solution to the challenges and problems faced of students in highschool and college. This can help students to make or conduct a survey easily. Expect to conduct a survey to analyze and make a solution to the challenges and problems faced of students in highschool and college. This can help students to make or conduct a survey easily. Expect to conduct a survey to analyze and make a solution to the challenges and problems faced of students in highschool and college. This can help students to make or conduct a survey easily. Expect to conduct a survey to analyze and make a solution to the challenges and problems faced of students in highschool and college. This can help students to make or conduct a survey easily. Expect to conduct a survey to analyze and make a solution to the challenges and problems faced of students in highschool and college. This can help students to make or conduct a survey easily. Expect to conduct a survey to analyze and make a solution to the challenges and problems faced of students in highschool and college. This can help students to make or conduct a survey easily. Expect to conduct a survey to analyze and make a solution to the challenges and problems faced of students in highschool and college. This can help students to make or conduct a survey easily. Expect to conduct a survey to analyze and make a solution to the challenges and problems faced of students in highschool and college. This can help students to make or conduct a survey easily. Expect to conduct a survey to analyze and make a solution to the challenges and problems faced of students in highschool and college. This can help students to make or conduct a survey easily. Expect to conduct a survey to analyze and make a solution to the challenges and problems faced of student
Tips for better surveys: better questions in your questionnaire, better overall survey process. From UPA2012 in Las Vegas.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The first part of a workshop on user experience surveys. Topics: (1) how to improve the questions in surveys and (2) how to assess UX using a survey.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Similar to Surveys that work: an introduction to using Total Survey Error for the UX Insight Festival 2020 (18)
In this workshop for the Virtual SDinGov 2024 , Caroline takes participants through two sets of guidelines in search of advice on how to make a single forms question accessible. She then introduces her own question protocol as a method of scrutinising and improving any question.
The Phylogenetic Tree in forms design - making forms work for complex academ...Caroline Jarrett
How can we guide busy academics in specialist fields through application processes that are complex, vary greatly depending on the funder, and always seem to be extra urgent? Especially when the stakes are high: awards can be in the millions, and research income is important to fund work that we can all benefit from.
For this year's HE Connect conference, Cambridge University Senior Product Manager Karen Fernandes and forms expert Caroline Jarrett reflected on how current work at Cambridge, and government forms patterns, can help (or hinder) this sort of multi-person, multi-challenge process.
What is a service designer SDinGOV 22 with all stickies.pptxCaroline Jarrett
In this case study for the 2022 Service Design in Government conference Caroline challenges people to think about their own definitions and shares her own - which is based on her three-layer model for creating good forms.
Helping teenage boys to become responsible adults.pptxCaroline Jarrett
Teenage boys use our services but many of us know little about them. In this session, Bukola (Kiki) Jolugbo and Caroline Jarrett shared some facts about teenage boys and some principles for helping them to become responsible adults.
Overview of how to make good forms that explains that a form builder can help, but it's essential to understand why you're asking the questions - and to write good questions.
Inwards and outwards research: choosing your research methods according to th...Caroline Jarrett
Is your user research looking inwards, at how your service works, or outwards, at the lives of those it affects?
The right research in the right direction at the right time can truly add value - but there’s usually no point in running a survey of 10,000 people in discovery or waiting until beta to look for high-level user needs.
This session, run with Clara Greo at the 2020 Service Design in Government conference, was a chance for colleagues to share their research questions, and think about how to map them to the right methods.
Write Clearly: take your web writing to the next level, May 2016Caroline Jarrett
These slides, setting out a series of rules for producing clear and effective web writing, come from a workshop delivered to staff of EBI/EMBL in May 2016
Understanding the costs of data capture: paper, automatic and with the intern...Caroline Jarrett
Organisations have sometimes been surprised and disappointed when they re-engineer a forms-based data capture process but fail to achieve their anticipated savings.
This paper, delivered to the CIMTECH conference, University of Hertfordshire, in 2000 explains:
how capture costs are built up from data entry plus dealing with problems
an example of costs for an automated process, and for dealing with the paper forms that are left after you bring in an internet process
four techniques for investigating the costs of your current process.
Storytelling For The Web: Integrate Storytelling in your Design ProcessChiara Aliotta
In this slides I explain how I have used storytelling techniques to elevate websites and brands and create memorable user experiences. You can discover practical tips as I showcase the elements of good storytelling and its applied to some examples of diverse brands/projects..
PDF SubmissionDigital Marketing Institute in NoidaPoojaSaini954651
https://www.safalta.com/online-digital-marketing/advance-digital-marketing-training-in-noidaTop Digital Marketing Institute in Noida: Boost Your Career Fast
[3:29 am, 30/05/2024] +91 83818 43552: Safalta Digital Marketing Institute in Noida also provides advanced classes for individuals seeking to develop their expertise and skills in this field. These classes, led by industry experts with vast experience, focus on specific aspects of digital marketing such as advanced SEO strategies, sophisticated content creation techniques, and data-driven analytics.
Can AI do good? at 'offtheCanvas' India HCI preludeAlan Dix
Invited talk at 'offtheCanvas' IndiaHCI prelude, 29th June 2024.
https://www.alandix.com/academic/talks/offtheCanvas-IndiaHCI2024/
The world is being changed fundamentally by AI and we are constantly faced with newspaper headlines about its harmful effects. However, there is also the potential to both ameliorate theses harms and use the new abilities of AI to transform society for the good. Can you make the difference?
White wonder, Work developed by Eva TschoppMansi Shah
White Wonder by Eva Tschopp
A tale about our culture around the use of fertilizers and pesticides visiting small farms around Ahmedabad in Matar and Shilaj.
Connect Conference 2022: Passive House - Economic and Environmental Solution...TE Studio
Passive House: The Economic and Environmental Solution for Sustainable Real Estate. Lecture by Tim Eian of TE Studio Passive House Design in November 2022 in Minneapolis.
- The Built Environment
- Let's imagine the perfect building
- The Passive House standard
- Why Passive House targets
- Clean Energy Plans?!
- How does Passive House compare and fit in?
- The business case for Passive House real estate
- Tools to quantify the value of Passive House
- What can I do?
- Resources
Connect Conference 2022: Passive House - Economic and Environmental Solution...
Surveys that work: an introduction to using Total Survey Error for the UX Insight Festival 2020
1. Surveys that work
An introduction to using
Total Survey Error in practice
Caroline Jarrett
@cjforms
#UXInsights2020
2. What would you do for a dollar?
Caroline Jarrett @cjforms (CC) BY SA-4.0 2
3. Caroline Jarrett @cjforms (CC) BY SA-4.0 3
$1 in the envelope beats $10 guaranteed later
Caroline Jarrett @cjforms (CC) BY SA-4.0 3
4. People will only respond if they trust
you. After that, it's a balance between
the perceived reward from filling in the
survey compared to the perceived
effort that's required. Strangely
enough, if a reward seems 'too good to
be true' that can also reduce the
response.
Response
Diagram inspired by Dillman, D.A. (2000)
“Internet, Mail and Mixed Mode Surveys: The Tailored Design Method” 4
6. Caroline Jarrett @cjforms (CC) BY SA-4.0 6
Why do people answer questions?
Caroline Jarrett @cjforms (CC) BY SA-4.0 6
7. Caroline Jarrett @cjforms (CC) BY SA-4.0 7
I’m the forms specialist
Image credit: Flickr, taxrebate.org.uk 7
8. People ask me about surveys
“Please have a look at this survey”
“Tell me whether this is a good question”
“How many people do I need in my sample?”
“I prefer 5 points in a rating scale, but my boss likes 7.
Who’s right?”
Caroline Jarrett @cjforms (CC) BY SA-4.0 8
9. The survey is a
systematic method
for gathering information from
(a sample of) entities
for the purpose of
constructing quantitative descriptors
of the attributes of the larger population
of which the entities are members.
Groves, Robert M.; Fowler, Floyd J.; Couper, Mick P.; Lepkowski, James M.; Singer, Eleanor &
Tourangeau, Roger (2004).Survey methodology. Hoboken, NJ: John Wiley & Sons.
Caroline Jarrett @cjforms (CC) BY SA-4.0
10. Caroline Jarrett @cjforms (CC) BY SA-4.0
I change the definition a bit
systematic method becomes process
gathering information becomes ask questions
entities become people
quantitative descriptors become numbers
attributes of the
larger population becomes make decisions
11. The survey is a process
for getting answers to questions
from (a sample of) people
for the purpose of
getting numbers
that you can use to
make decisions
Caroline Jarrett @cjforms (CC) BY SA-4.0
12. The survey is a
process for getting
answers to questions
To make decisions People
getting numbers
Caroline Jarrett @cjforms (CC) BY SA-4.0
13. Caroline Jarrett @cjforms (CC) BY SA-4.0 13
The aim of a survey is to get the number
that helps you to make a decision
The Survey
What you
want to ask
about
Who you
want to ask
The number
14. Caroline Jarrett @cjforms (CC) BY SA-4.0 14
The Survey Octopus has things to think about
What you
want to ask
about
Who you
want to ask
The number
15. Caroline Jarrett @cjforms (CC) BY SA-4.0 15
There are steps in the process for each area
Goals
Questions
Questionnaire
Response
Sample
Fieldwork
Response
Reports
15
16. Here are the 7 steps as a linear process
Establish
your goals for
the survey
Decide who
to ask and
how many
Build the
questionnaire
Run the
survey from
invitation to
follow-up
Clean and
analyse the
data
Present the
results
Questions
you need
answers to
People you
will invite to
answer
Goals Sample Questionnaire Fieldwork
People who
actually
answer
Responses Reports
Answers Decisions
Test the
questions
Questions
Questions
people can
answer
Questions
people can
interact with
Caroline Jarrett @cjforms (CC) BY SA-4.0
17. People ask me about surveys
“Please have a look at this survey”
“Tell me whether this is a good question”
“How many people do I need in my sample?”
“I prefer 5 points in a rating scale, but my boss likes 7.
Who is right?”
Caroline Jarrett @cjforms (CC) BY SA-4.0 17
18. Here are the 7 steps as a linear process
Establish
your goals for
the survey
Decide who
to ask and
how many
Build the
questionnaire
Run the
survey from
invitation to
follow-up
Clean and
analyse the
data
Present the
results
Questions
you need
answers to
People you
will invite to
answer
Goals Sample Questionnaire Fieldwork
People who
actually
answer
Responses Reports
Answers Decisions
Test the
questions
Questions
Questions
people can
answer
Questions
people can
interact with
Caroline Jarrett @cjforms (CC) BY SA-4.0
19. I have to know about goals first
Establish
your goals for
the survey
Questions
you need
answers to
Goals
Caroline Jarrett @cjforms (CC) BY SA-4.0
20. Caroline Jarrett @cjforms (CC) BY SA-4.0 20
The aim of a survey is to get the number
that helps you to make a decision
The Survey
What you
want to ask
about
Who you
want to ask
The number
21. Caroline Jarrett @cjforms (CC) BY SA-4.0 21
Establish your goals for the survey
What do you want to know?
Why do you want to know?
What decisions will you make
based on these answers?
22. People ask me about surveys
“Please have a look at this survey”
“Tell me whether this is a good question”
“How many people do I need in my sample?”
“I prefer 5 points in a rating scale, but my boss likes 7.
Who is right?”
Caroline Jarrett @cjforms (CC) BY SA-4.0 22
23. Caroline Jarrett @cjforms (CC) BY SA-4.0 23
Is this a good question?
Would you recommend us to
a friend or family member?
23
24. Caroline Jarrett @cjforms (CC) BY SA-4.0 24
“Recommend to friend/family” can be OK
A shop selling clothes A hospital ward
What do you want
to know?
Whether they will recommend ????
Why do you want to
know?
To compare numbers
over time
Because someone
said they had to ask
What number do
you need to make a
decision?
Some good reasons
(investigate and fix problems)
Some bad ones (punish staff)
???? What
decision?
24
25. People ask me about surveys
“Please have a look at this survey”
“Tell me whether this is a good question”
“How many people do I need in my sample?”
“I prefer 5 points in a rating scale, but my boss likes 7.
Who is right?”
Caroline Jarrett @cjforms (CC) BY SA-4.0 25
27. Caroline Jarrett @cjforms (CC) BY SA-4.0 27
Start with how many will answer
Fieldwork:
Who answers?
27
28. Caroline Jarrett @cjforms (CC) BY SA-4.0 28
Whether they’ll answer depends on effort
Questions:
What are you asking about?
How many questions?
28
29. Caroline Jarrett @cjforms (CC) BY SA-4.0 29
And on the reward you’re offering
Goals:
Why are you asking?
Is helping you a reward in itself?
Are you offering any other incentive?
29
30. Then there’s the ‘Justin Bieber North Korea’ problem
http://www.bbc.com/news/10506482 30
31. Caroline Jarrett @cjforms (CC) BY SA-4.0 31
So it matters where we get our sample
Sample:
the list you
sample from
31
32. Caroline Jarrett @cjforms (CC) BY SA-4.0 32
And now it’s easy to work out how many to ask
Sample:
the number of
people to ask
32
33. The response rate varies
It depends on
• Why you’re asking
• What you’re asking and
the type of questionnaire you make
• Who you’re asking
• How you’re asking
Goals
Questions
Questionnaire
Sample
Fieldwork
Caroline Jarrett @cjforms (CC) BY SA-4.0
34. People ask me about surveys
“Please have a look at this survey”
“Tell me whether this is a good question”
“How many people do I need in my sample?”
“I prefer 5 points in a rating scale, but my boss likes 7.
Who is right?”
Caroline Jarrett @cjforms (CC) BY SA-4.0 34
35. Caroline Jarrett @cjforms (CC) BY SA-4.0 35
A good question gets good answers
Response:
Is the question
easy to answer?
36. Caroline Jarrett @cjforms (CC) BY SA-4.0 36
Good answers help you to make decisions
Reports:
Is the answer
useful?
37. Caroline Jarrett @cjforms (CC) BY SA-4.0 37
Likert had several different types
of question in his response formats
Likert, Rensis. (1932). A Technique for the Measurement of Attitudes.
Archives of Psychology, 140, 1–55. 37
38. You can find an academic paper to support almost
any number of points
Krosnick and Presser refer to over 80 papers
Krosnick, J. A. and S. Presser (2009). Question and Questionnaire Design.
Handbook of Survey Research (2nd Edition) J. D. Wright and P. V. Marsden, Elsevier.
https://web.stanford.edu/dept/communication/faculty/krosnick/docs/2009/2009_handbook_krosnick.pdf
38
39. Caroline Jarrett @cjforms (CC) BY SA-4.0 39
Let’s try this question. Pick your answer.
Likert, Rensis. (1932). A Technique for the Measurement of Attitudes.
Archives of Psychology, 140, 1–55. 39
41. Tip
Don’t stress too much about
the number of points in your
rating scale
Picture credit: Flickr - Bill Soderman (BillsoPHOTO)Caroline Jarrett @cjforms (CC) BY SA-4.0 41
42. Caroline Jarrett @cjforms (CC) BY SA-4.0 42
Well, OK, stress a little bit.
This scale is
downright peculiar.
Avoid.
42
43. Let’s look at all the errors
Caroline Jarrett @cjforms (CC) BY SA-4.0 43
44. Caroline Jarrett @cjforms (CC) BY SA-4.0 44
There are errors all around the Survey Octopus
(Lack of)
validity
Measurement
error
Processing
error
Coverage
error
Sampling
error
Non-response
error
Adjustment
error
44
45. Caroline Jarrett @cjforms (CC) BY SA-4.0 45
The aim is to get the best number you can,
within the resources you have
What you want to ask about
The reason you’re doing it
The questions you ask
The answers you get
The answers you use
Who you want to ask
The list that you sample from
The sample you ask
The ones who answer
The ones whose answers
you can use
The number
(Lack of)
validity
Measurement
error
Processing
error
Coverage error
Sampling error
Non-response
error
Adjustment
error
47. Total Survey Error diagram as presented in
Groves, R. M., F. J. Fowler, M. P. Couper, J. M.
Lepkowski, E. Singer and R. Tourangeau (2009).
Survey methodology. Hoboken, N.J., Wiley.
47
People will only respond if they trust you. After that, it's a balance between the perceived reward from filling in the survey compared to the perceived effort that's required. Strangely enough, if a reward seems 'too good to be true' that can also reduce the response.
This is a genuine invitation from local government, but the layout and images in the invitation make it look as if it's an approach from some sort of spammer or scammer.
The survey sits between 'what you want to ask', 'who you want to ask' and 'the number'
The survey sits between 'what you want to ask', 'who you want to ask' and 'the number'
The octopus again; we've looked at 6 of the 8 tentacles.
The survey sits between 'what you want to ask', 'who you want to ask' and 'the number'
The question "What do you want to know" leads to "Why do you want to know" and then to "What decisions will you make based on these answers?"
If you're buying a baby carriage, then the 'recommend to a friend' question would probably be obvious, interesting and appropriate. If you're in hospital having a miscarriage, it would be cruelly inappropriate.
The survey octopus has 8 tentacles. We'll visit each one in the next few slides. We’ll get our survey to the people who will answer in what the survey methodologists call ‘fieldwork’ – that might be a pop-up a website, a mail survey, or face-to-face interviews.
The survey octopus has 8 tentacles. We'll visit each one in the next few slides. We’ll get our survey to the people who will answer in what the survey methodologists call ‘fieldwork’ – that might be a pop-up a website, a mail survey, or face-to-face interviews.
The octopus again. This time we're looking at 'the questions we ask'.
The resources you have will help you to decide on the reward you’re offering
Prank leaves Justin Bieber facing tour of North Korea
By Daniel Emery Technology reporter, BBC News
5 July 2010
Image caption It is highly unlikely Bieber would be given permission to enter North Korea Canadian singer Justin Bieber's has become the target of a viral campaign to send him to North Korea.
A website polled users as to which country he should tour next, with no restrictions on the nations that could be voted on.
There are now almost half a million votes to send the singer to the secretive communist nation.
The contest, which ends at 0600 on 7 July, saw North Korea move from 24th to 1st place in less than two days.
Many of the votes are thought to originate from imageboard website 4chan, which has built a reputation for triggering online viral campaigns.
The octopus, with focus on 'The list you sample from'
The octopus again; we've looked at 6 of the 8 tentacles.
This is a more conventional way of looking at the octopus tentacles
This slide translates the issues into the technical terms used by survey methodologists