Getting valid results from surveys: meet the Survey Octopus.
Surveys are a powerful research method, but not easy to get right. The Survey Octopus is a way of thinking through the issues that will ensure that you'll get solid results from your survey that you can use to make decisions. Presentation from the UX in the City conference, Oxford, March 2016
Surveys that work: an introduction to the Survey Octopus and Total Survey ErrorCaroline Jarrett
A presentation for Harvard University's User Research Community on some of the key issues in creating effective surveys, including: why run a survey, writing good questions, statistical significance and how to avoid errors.
A comprehensive (but not complete!) review of the Lean Analytics book (http://leananalyticsbook.com), which was presented at the Lean Startup Conference in 2012. Focuses on the
Presentation on how to got from MVP to product market fit. Its as given at StartMIT, a course that introduces members of the MIT community to the elements of entrepreneurship via professionals from the industry.
Why Can't We All Just Get Along? Four Generations Working Side by Side in Har...Andrew Krzmarzick
Presentation delivered at the Training Officer's Consortium (TOC) Institute in Williamsburg, VA, on April 27, 2009. Facilitated by a Gen X'er and a Traditional. Objectives included:
1. Understand and better navigate the intergenerational differences in your office
2. Participate in several training modalities that reach a multi-generational audience
3. Experience a truly blended approach to training that addresses the learning preferences of all four generations.
This presentation offers a comprehensive review of the 2017 Women in the Workplace report by McKinsey & Company and LeanIn.org with solutions companies can take to foster gender equality.
https://womenintheworkplace.com/
Lean Analytics: Using Data to Build a Better Business FasterLean Startup Co.
Alistair Croll, Solve for Interesting , @acroll
At the core of Lean Startup approaches is a continuous cycle of measurement and learning. But what should you measure? To find the right metric, you need to understand the stage you’re at and the business model you’re in, as well as where to draw the line so you know when to cut your losses—and when to step on the gas. In these two sessions, entrepreneur and best-selling author of Lean Analytics Alistair Croll will show you how to put data to work.
How to think about data and what makes a good metric
The importance of cohorts and proper analysis
The five stages every startup goes through
Six business model archetypes and how to find your own
What “good enough” looks like and how to run experiments
What works for larger organizations trying to change and innovate.
This session is relevant for both early-stage founders and intrapreneurs in large organizations. Based on interviews with over 130 analysts, entrepreneurs, and investors, this session is packed with practical information, hard numbers, and concrete steps you can put to work immediately. Attendees need not be technical but should come armed with a basic understanding of web analytics, business metrics, and their current business model, plus a willingness to share with one another.
This workshop is sponsored by Amplitude.
Surveys that work: an introduction to the Survey Octopus and Total Survey ErrorCaroline Jarrett
A presentation for Harvard University's User Research Community on some of the key issues in creating effective surveys, including: why run a survey, writing good questions, statistical significance and how to avoid errors.
A comprehensive (but not complete!) review of the Lean Analytics book (http://leananalyticsbook.com), which was presented at the Lean Startup Conference in 2012. Focuses on the
Presentation on how to got from MVP to product market fit. Its as given at StartMIT, a course that introduces members of the MIT community to the elements of entrepreneurship via professionals from the industry.
Why Can't We All Just Get Along? Four Generations Working Side by Side in Har...Andrew Krzmarzick
Presentation delivered at the Training Officer's Consortium (TOC) Institute in Williamsburg, VA, on April 27, 2009. Facilitated by a Gen X'er and a Traditional. Objectives included:
1. Understand and better navigate the intergenerational differences in your office
2. Participate in several training modalities that reach a multi-generational audience
3. Experience a truly blended approach to training that addresses the learning preferences of all four generations.
This presentation offers a comprehensive review of the 2017 Women in the Workplace report by McKinsey & Company and LeanIn.org with solutions companies can take to foster gender equality.
https://womenintheworkplace.com/
Lean Analytics: Using Data to Build a Better Business FasterLean Startup Co.
Alistair Croll, Solve for Interesting , @acroll
At the core of Lean Startup approaches is a continuous cycle of measurement and learning. But what should you measure? To find the right metric, you need to understand the stage you’re at and the business model you’re in, as well as where to draw the line so you know when to cut your losses—and when to step on the gas. In these two sessions, entrepreneur and best-selling author of Lean Analytics Alistair Croll will show you how to put data to work.
How to think about data and what makes a good metric
The importance of cohorts and proper analysis
The five stages every startup goes through
Six business model archetypes and how to find your own
What “good enough” looks like and how to run experiments
What works for larger organizations trying to change and innovate.
This session is relevant for both early-stage founders and intrapreneurs in large organizations. Based on interviews with over 130 analysts, entrepreneurs, and investors, this session is packed with practical information, hard numbers, and concrete steps you can put to work immediately. Attendees need not be technical but should come armed with a basic understanding of web analytics, business metrics, and their current business model, plus a willingness to share with one another.
This workshop is sponsored by Amplitude.
Unconscious biases affect our perceptions, decisions, and interactions every day. How do we address biases if we don't know about them? In this talk, you will learn how to recognize and counter the biases that play a part in interviewing, meeting a new team member, and day-to-day interactions. You’ll also see common scenarios and how to address bias as it happens or after the fact. Together, we can make Asynchrony a more diverse and inclusive place to work.
Covering workplace gender equality, innovative start-ups and how to get ahead, here are a collection of articles on women, leadership and the workplace
Understanding Lean Analytics (and how analytics helps businesses win)Lean Analytics
This is the workshop on Lean Analytics from the Web Analytics Congress (#wac13) in Amsterdam. It covers the basics of Lean Analytics + Lean Startup. It goes into details on specific business models such as media and e-commerce and includes many case studies from the Lean Analytics book.
Unlocking Growth: Building a sustainable growth engine with the new rules of...Sean Ellis
As Marc Andreessen says, startups fail for two primary reasons: either they try to grow when they're not ready to, or they're not aggressive enough when they're finally to grow. In this presentation Morgan Brown walks through the Startup Pyramid framework to show how startups can gain traction and drive sustainable growth.
Presentation on the 4 forces of customer progress, the JTBD framework to analyse customer acquisition and churn. Presented at the Sydney JTBD meetup. http://www.meetup.com/sydney_jtbd/
People are products of their history, their environment and all of their experiences. As a result, people from different generations often have very different life experiences that shape how they think, what they value, and what drives them. These influences affect people’s values and attitudes throughout their lives.
Protecting the Mental Wellbeing of Corporate InvestigatorsCase IQ
Vicarious trauma is a type of second-hand psychological response most often associated with first responders and front-line crisis workers. Although corporate incident investigators and HR and compliance professionals are usually not the first on the scene, they frequently find themselves listening to both victims and accused people who recount what could be the worst days of their professional lives.
There is no doubt that the case management process can be stressful and sometimes triggering for those who are managing it. The personal and organizational impacts can include compromised risk assessment and investigations, investigator burnout and absenteeism, and a less healthy workplace overall.
Join expert workplace investigator Ken McCarthy as he shares his experience with vicarious trauma and compassion fatigue resulting from his involvement in workplace investigations.
Your brain is capable of incredible healing and constant reshaping. Through a new relationship with your brain you can transform your life. With increased self-awareness and conscious intention, the brain can be taught to reach far beyond its present limitations.
This is a visual excerpt from my book "Super Brain," which I co-authored with Rudolph Tanzi, PhD. To read more, go to: http://www.chopra.com/
With practical and fun exercises, this visual workbook is essential reading for anyone who wants to get paid for who they are and what they believe in. This is a visual workbook for millennials who want meaningful work.
We're not the "me me me generation." We're a generation that refuses to settle for mediocrity because we know how great our impact can be when we find work we truly care about.
Stop climbing the career ladder. Move beyond FOMO. Embrace fear. Discover who you are and what you're good at. Start aligning your gifts with your impact. Find a community to support your breakthrough.
End your quarter-life crisis and find meaningful work today.
THE QUARTER-LIFE BREAKTHROUGH by Adam Smiley Poswolsky is an inspiring career guide for twentysomethings to get unstuck, pursue work that matters, and change the world.
Buy the book: http://www.thequarterlifebreakthrough.com/
Interfaces have always been areas of peril in software development. Ff you deliver software that has to be used by real people, distinguish between software that is “deployed”, actually “delivered”, and then 'used by real people'.
Unconscious biases affect our perceptions, decisions, and interactions every day. How do we address biases if we don't know about them? In this talk, you will learn how to recognize and counter the biases that play a part in interviewing, meeting a new team member, and day-to-day interactions. You’ll also see common scenarios and how to address bias as it happens or after the fact. Together, we can make Asynchrony a more diverse and inclusive place to work.
Covering workplace gender equality, innovative start-ups and how to get ahead, here are a collection of articles on women, leadership and the workplace
Understanding Lean Analytics (and how analytics helps businesses win)Lean Analytics
This is the workshop on Lean Analytics from the Web Analytics Congress (#wac13) in Amsterdam. It covers the basics of Lean Analytics + Lean Startup. It goes into details on specific business models such as media and e-commerce and includes many case studies from the Lean Analytics book.
Unlocking Growth: Building a sustainable growth engine with the new rules of...Sean Ellis
As Marc Andreessen says, startups fail for two primary reasons: either they try to grow when they're not ready to, or they're not aggressive enough when they're finally to grow. In this presentation Morgan Brown walks through the Startup Pyramid framework to show how startups can gain traction and drive sustainable growth.
Presentation on the 4 forces of customer progress, the JTBD framework to analyse customer acquisition and churn. Presented at the Sydney JTBD meetup. http://www.meetup.com/sydney_jtbd/
People are products of their history, their environment and all of their experiences. As a result, people from different generations often have very different life experiences that shape how they think, what they value, and what drives them. These influences affect people’s values and attitudes throughout their lives.
Protecting the Mental Wellbeing of Corporate InvestigatorsCase IQ
Vicarious trauma is a type of second-hand psychological response most often associated with first responders and front-line crisis workers. Although corporate incident investigators and HR and compliance professionals are usually not the first on the scene, they frequently find themselves listening to both victims and accused people who recount what could be the worst days of their professional lives.
There is no doubt that the case management process can be stressful and sometimes triggering for those who are managing it. The personal and organizational impacts can include compromised risk assessment and investigations, investigator burnout and absenteeism, and a less healthy workplace overall.
Join expert workplace investigator Ken McCarthy as he shares his experience with vicarious trauma and compassion fatigue resulting from his involvement in workplace investigations.
Your brain is capable of incredible healing and constant reshaping. Through a new relationship with your brain you can transform your life. With increased self-awareness and conscious intention, the brain can be taught to reach far beyond its present limitations.
This is a visual excerpt from my book "Super Brain," which I co-authored with Rudolph Tanzi, PhD. To read more, go to: http://www.chopra.com/
With practical and fun exercises, this visual workbook is essential reading for anyone who wants to get paid for who they are and what they believe in. This is a visual workbook for millennials who want meaningful work.
We're not the "me me me generation." We're a generation that refuses to settle for mediocrity because we know how great our impact can be when we find work we truly care about.
Stop climbing the career ladder. Move beyond FOMO. Embrace fear. Discover who you are and what you're good at. Start aligning your gifts with your impact. Find a community to support your breakthrough.
End your quarter-life crisis and find meaningful work today.
THE QUARTER-LIFE BREAKTHROUGH by Adam Smiley Poswolsky is an inspiring career guide for twentysomethings to get unstuck, pursue work that matters, and change the world.
Buy the book: http://www.thequarterlifebreakthrough.com/
Interfaces have always been areas of peril in software development. Ff you deliver software that has to be used by real people, distinguish between software that is “deployed”, actually “delivered”, and then 'used by real people'.
Speaker: Caroline Jarrett
To help us get the best out of this tricky research method, Caroline will describe the Survey Octopus, a friendly creature that helps her to tackle all the issues that may lie between 'What we want to ask, and who we want to ask', and a solid, reliable number that can be used to make decisions.
Along the way, we'll encounter the key concept in survey methodology, Total Survey Error, and the various types of error that can affect your survey.
Introduction to survey methods at LibDesign2016. A workshop led by Caroline Jarrett for people working in the library service and public sector in the Czech Republic. Caroline Jarrett led this workshop in Prague in September 2016 as part of the LibDesign 2016 conference.
Interfaces have always been areas of peril in software development. This presentation to the O'Reilly Software Architecture Conference 2016 in London challenges teams to include interfaces with people, and maybe even with paper, in their notions of where development begins and ends. For anyone delivering software that has to be used by real people, the presentation will help you distinguish between software that is “deployed” and actually “delivered.”
The UK Government Digital Service is the department that looks after GOV.UK, the UK government website. It's a single website that is home to over 300 government departments and agencies. This talk gives some examples of how the GDS Design Principles help our colleagues to create better public services, and how they help us to create better guidance for them too.
The Art of Asking Survey Questions: 7 Survey-Writing Don'tsHubSpot
What types of questions should you avoid the next time you have to write a survey to get feedback?
This presentation covers just a section of our guide: What not to do when writing survey questions. Get your free copy of the complete guide and workbook, The Art of Asking Survey Questions, right here: http://hub.am/1imzkQ6
Design Patterns for Government Services UXPA 2016Caroline Jarrett
The UK government now has several hundred designers working on services for citizens. How do we design at scale? By working with our designers as a community.
Presentation by Caroline Jarrett @cjforms at the UXPA 2016 conference in Seattle
How to get better results from a survey: Meet the Survey OctopusCaroline Jarrett
The Survey Octopus is a friendly creature who will help you to think about all the crucial issues in crafting a survey.
Presentation by Caroline Jarrett @cjforms for the 2014 Content Strategy Summit #CSSummit
Surveys that work: a webinar for FocusVision 2021Caroline Jarrett
Creating surveys that work for participants and deliver high quality insight is no mean feat. This is because the survey process is complex, with multiple considerations at every step in the journey.
In this webinar for FocusVision, I introduce the Survey Octopus, my friendly way of talking about the many issues that make surveys one of the most challenging research methods. I also explain how the Survey Octopus maps into the Total Survey Error concept that underpins the work of many survey methodologists.
The Survey Octopus will help you design better surveys by thoughtfully considering:
• What you want to ask about
• Who you want to ask
• The number of people you need to ask
Plain language to improve your survey houston 2022Caroline Jarrett
Plain language skills are vital for surveys - and especially to writing good questions and creating them for your survey audience. This presentation was prepared for the University of Houston's 8th Biannual Forum on Plain English, 24 February 2022.
Some thoughts on surveys: Boye and Company member conference callCaroline Jarrett
Slides from a short presentation on creating effective surveys. The event was a conference call for members of a community network organised by Janus Boye of Boye & Company.
Surveys that Work 2020: training course for HMRC user researchers 2020Caroline Jarrett
Slides from a training course on effective surveys, delivered to usability researchers at HMRC. The course took place at HMRC's Longbenton, Newcastle, offices, on January 30, 2020. Survey examples submitted by participants for review have been removed from this presentation.
Getting valid results from surveys: meet the Survey Octopus.
Surveys are a powerful research method, but not easy to get right. The Survey Octopus is a way of thinking through the issues that will ensure that you'll get solid results from your survey that you can use to make decisions. Presentation from the UX New Zealand conference 2015 #uxnz2015
Many of us receive multiple requests to complete surveys every day. Some of us find that colleagues or clients think of ‘doing a survey’ as the same as ‘doing some research’ – which may explain why organizations send out so many survey requests.
In this webinar, you’ll meet the Survey Octopus, Caroline Jarrett’s friendly way of talking about the many issues that make surveys one of the most challenging research methods.
The Survey Octopus will help you to:
Explain to colleagues that a survey may not be the first research method to try
Help to justify a choice to work with a “non significant” number of responses
Think about the steps that go into delivering a survey that works
As a bonus, Caroline will also explain how her Survey Octopus maps into the Total Survey Error concept that underpins the work of many survey methodologists.
Surveys that work: an introduction to using Total Survey Error for the UX Ins...Caroline Jarrett
Surveys are easy to do – but harder to do well. In this interactive workshop - delivered to the UX Insight Festival 2020 - I take you through using Total Survey Error as a way of balancing the issues and good practice in survey design to get the best results from your survey.
The session also covered my 7-step survey process, starting with Goals and thinking about Sampling, Questions, Questionnaires, Fieldwork, Responses and Reports. Plus we tackle some of the questions I'm most often asked about creating surveys that work.
Two ways to improve your surveys: the Most Crucial Question and the Burning I...Caroline Jarrett
In this webinar for product managers, Caroline introduces two key concepts from her book on surveys: identifying the most crucial question as part of getting clear on your goals, and allowing respondents to tell you the things that they want to - their burning issue. The webinar was organised by Productboard and held on March 30, 2023.
Slides from a workshop introduction to survey methods. The workshop was prepared for staff of the European Bioinformatics Institute in Cambridge, February 2017
Surveys are still really popular as a research method with colleagues (if not with service designers).
These slides are from a workshop at the 2021 Service Design in Government conference (@sdingov21) on 'how to improve the survey that is going to happen whether you like it or not'.
In the workshop we looked at a 7-step process for a survey and considered ways of encouraging colleagues to combine surveys with other research methods.
We also practiced techniques for looking at – and improving - a questionnaire.
In this half day workshop for ~WebExpo2023 Caroline Jarrett shares four ways to improve your survey so that you get plenty of useful responses.
Goals: Ruthlessly focus your survey on an immediate decision.
Sample: Write an invitation that makes people want to answer.
Questions: Ditch the rating scales.
Responses: Lose your fear of open answers.
Two ways to improve your survey, webinar for Delib 2023.pptxCaroline Jarrett
In this webinar for Delib, Caroline shows you how to get better results from shorter, more frequent surveys - with a special emphasis on local government and the requirement to run statutory consultations. Understanding and identifying the Most Crucial Question and making space for the Burning Issue are both helpful techniques for creating shorter more focused surveys.
Some thoughts on good survey design delivered to students at Olin College of Engineering. Caroline's talk covers her survey process, survey goals and focusing on a specific decision, sample and sampling error, ditching rating scales, and losing fear of open answers.
Surveys that work:training course for Rosenfeld Media, day 1Caroline Jarrett
Surveys seem easy: anyone can throw together a few questions, send them out, and hope that they are rewarded with a decent response. But we’ve all seen examples of poorly conceived surveys that couldn’t possibly deliver real insights for the organisation that sponsored them.
This highly participative three-session training - arranged by Rosenfeld Media as part of its Virtual Training with UX Industry Leaders programme - takes you through the whole process of creating an effective survey, from defining a goal through analysis of data and creating a presentation.
These slides come from day 1 of the course: goals and sample.
Did you love the form that you filled in most recently? Or did you hit some problems? Most of us find all sorts of small or major problems with lots of the forms we are forced to use.
In this talk for #WebExpo2023, Caroline turns that around. She points out the ways in which not fixing your forms is costing your organisation a lot of money. She then goes on to share plenty of practical tips for making improvements that will enable people to successfully complete your forms.
A presentation for the the Content Wrangler's coffee and content session on how to design and run surveys and gain actionable insights from the survey data.
Surveys that work: training course for Rosenfeld media, day 2Caroline Jarrett
Surveys seem easy: anyone can throw together a few questions, send them out, and hope that they are rewarded with a decent response. But we’ve all seen examples of poorly conceived surveys that couldn’t possibly deliver real insights for the organisation that sponsored them.
This highly participative three-session training - arranged by Rosenfeld Media as part of its Virtual Training with UX Industry Leaders programme - takes you through the whole process of creating an effective survey, from defining a goal through analysis of data and creating a presentation.
These slides come from day 2 of the course: questions, questionnaire and fieldwork
Total Survey Error for non-specialists: creating better conversations. A presentation of the Survey Octopus at the TSE2015 conference in Baltimore, September 2015.
Similar to The Survey Octopus - getting valid data from surveys, presentation for UX in the City 2016 (20)
In this workshop for the Virtual SDinGov 2024 , Caroline takes participants through two sets of guidelines in search of advice on how to make a single forms question accessible. She then introduces her own question protocol as a method of scrutinising and improving any question.
The Phylogenetic Tree in forms design - making forms work for complex academ...Caroline Jarrett
How can we guide busy academics in specialist fields through application processes that are complex, vary greatly depending on the funder, and always seem to be extra urgent? Especially when the stakes are high: awards can be in the millions, and research income is important to fund work that we can all benefit from.
For this year's HE Connect conference, Cambridge University Senior Product Manager Karen Fernandes and forms expert Caroline Jarrett reflected on how current work at Cambridge, and government forms patterns, can help (or hinder) this sort of multi-person, multi-challenge process.
In this member call for Boye & Co Caroline takes participants through her process for expert reviews of forms. She also shares some of her top tips for making them easier to use and more effective.
What is a service designer SDinGOV 22 with all stickies.pptxCaroline Jarrett
In this case study for the 2022 Service Design in Government conference Caroline challenges people to think about their own definitions and shares her own - which is based on her three-layer model for creating good forms.
Helping teenage boys to become responsible adults.pptxCaroline Jarrett
Teenage boys use our services but many of us know little about them. In this session, Bukola (Kiki) Jolugbo and Caroline Jarrett shared some facts about teenage boys and some principles for helping them to become responsible adults.
Surveys that work: training course for Rosenfeld Media, day 3 Caroline Jarrett
Surveys seem easy: anyone can throw together a few questions, send them out, and hope that they are rewarded with a decent response. But we’ve all seen examples of poorly conceived surveys that couldn’t possibly deliver real insights for the organisation that sponsored them.
This highly participative three-session training - arranged by Rosenfeld Media as part of its Virtual Training with UX Industry Leaders programme - takes you through the whole process of creating an effective survey, from defining a goal through analysis of data and creating a presentation.
These slides come from day 3 of the course: responses and reports.
Overview of how to make good forms that explains that a form builder can help, but it's essential to understand why you're asking the questions - and to write good questions.
Inwards and outwards research: choosing your research methods according to th...Caroline Jarrett
Is your user research looking inwards, at how your service works, or outwards, at the lives of those it affects?
The right research in the right direction at the right time can truly add value - but there’s usually no point in running a survey of 10,000 people in discovery or waiting until beta to look for high-level user needs.
This session, run with Clara Greo at the 2020 Service Design in Government conference, was a chance for colleagues to share their research questions, and think about how to map them to the right methods.
Write Clearly: take your web writing to the next level, May 2016Caroline Jarrett
These slides, setting out a series of rules for producing clear and effective web writing, come from a workshop delivered to staff of EBI/EMBL in May 2016
Understanding the costs of data capture: paper, automatic and with the intern...Caroline Jarrett
Organisations have sometimes been surprised and disappointed when they re-engineer a forms-based data capture process but fail to achieve their anticipated savings.
This paper, delivered to the CIMTECH conference, University of Hertfordshire, in 2000 explains:
how capture costs are built up from data entry plus dealing with problems
an example of costs for an automated process, and for dealing with the paper forms that are left after you bring in an internet process
four techniques for investigating the costs of your current process.
Book Formatting: Quality Control Checks for DesignersConfidence Ago
This presentation was made to help designers who work in publishing houses or format books for printing ensure quality.
Quality control is vital to every industry. This is why every department in a company need create a method they use in ensuring quality. This, perhaps, will not only improve the quality of products and bring errors to the barest minimum, but take it to a near perfect finish.
It is beyond a moot point that a good book will somewhat be judged by its cover, but the content of the book remains king. No matter how beautiful the cover, if the quality of writing or presentation is off, that will be a reason for readers not to come back to the book or recommend it.
So, this presentation points designers to some important things that may be missed by an editor that they could eventually discover and call the attention of the editor.
Dive into the innovative world of smart garages with our insightful presentation, "Exploring the Future of Smart Garages." This comprehensive guide covers the latest advancements in garage technology, including automated systems, smart security features, energy efficiency solutions, and seamless integration with smart home ecosystems. Learn how these technologies are transforming traditional garages into high-tech, efficient spaces that enhance convenience, safety, and sustainability.
Ideal for homeowners, tech enthusiasts, and industry professionals, this presentation provides valuable insights into the trends, benefits, and future developments in smart garage technology. Stay ahead of the curve with our expert analysis and practical tips on implementing smart garage solutions.
Unleash Your Inner Demon with the "Let's Summon Demons" T-Shirt. Calling all fans of dark humor and edgy fashion! The "Let's Summon Demons" t-shirt is a unique way to express yourself and turn heads.
https://dribbble.com/shots/24253051-Let-s-Summon-Demons-Shirt
7 Alternatives to Bullet Points in PowerPointAlvis Oh
So you tried all the ways to beautify your bullet points on your pitch deck but it just got way uglier. These points are supposed to be memorable and leave a lasting impression on your audience. With these tips, you'll no longer have to spend so much time thinking how you should present your pointers.
5. Caroline Jarrett @cjforms (CC) BY SA-4.0
Response relies on
effort, reward, and trust
People will only respond if they trust
you. After that, it's a balance between
the perceived reward from filling in the
survey compared to the perceived
effort that's required. Strangely
enough, if a reward seems 'too good to
be true' that can also reduce the
response.
Diagram from Jarrett, C, and Gaffney, G (2008)
“Forms that work: Designing web forms for usability”
inspired by Dillman, D.A. (2000)
“Internet, Mail and Mixed Mode Surveys: The Tailored Design Method”
5
6. Caroline Jarrett @cjforms (CC) BY SA-4.0
I’m a forms specialist
Image credit: Flickr, taxrebate.org.uk
6
7. Caroline Jarrett @cjforms (CC) BY SA-4.0
Why do people answer questions?
Caroline Jarrett @cjforms (CC) BY SA-4.0
7
9. Caroline Jarrett @cjforms
The survey is a
systematic method
for gathering information from
(a sample of) entities
for the purpose of
constructing quantitative descriptors
of the attributes of the larger population
of which the entities are members.
Groves, Robert M.; Fowler, Floyd J.; Couper, Mick P.; Lepkowski, James M.; Singer, Eleanor &
Tourangeau, Roger (2004).Survey methodology. Hoboken, NJ: John Wiley & Sons.
10. Caroline Jarrett @cjforms
The survey is a
process
for gathering information from
(a sample of) entities
for the purpose of
constructing quantitative descriptors
of the attributes of the larger population
of which the entities are members.
11. Caroline Jarrett @cjforms
The survey is a
process
for getting answers to questions from
(a sample of) entities
for the purpose of
constructing quantitative descriptors
of the attributes of the larger population
of which the entities are members.
12. Caroline Jarrett @cjforms
The survey is a
process
for getting answers to questions from
(a sample of) people
for the purpose of
constructing quantitative descriptors
of the attributes of the larger population
of which the entities are members.
13. Caroline Jarrett @cjforms
The survey is a
process
for getting answers to questions from
(a sample of) people
for the purpose of
getting numbers
of the attributes of the larger population
of which the entities are members.
14. Caroline Jarrett @cjforms
The survey is a
process
for getting answers to questions from
(a sample of) people
for the purpose of
getting numbers
that you can use to make decisions
15. Caroline Jarrett @cjforms
To make decisions People
The survey is a
process for getting
answers to questions
getting numbers
16. Caroline Jarrett @cjforms (CC) BY SA-4.0
The aim of a survey is to get a number
that helps you to make a decision
16
18. Caroline Jarrett @cjforms (CC) BY SA-4.0
Fieldwork used to be expensive
so a survey was a rare event.
Image credit: http://www.census.gov/history/www/genealogy/decennial_census_records/
18
19. Caroline Jarrett @cjforms (CC) BY SA-4.0
1950s mindset: “Ask Everything”
Survey =
Big Honkin’ Survey
19
20. Caroline Jarrett @cjforms (CC) BY SA-4.0
2016 mindset: the Light Touch survey
• Choose ONE question
• Find ONE person
• Ask the question, face-to-face
• See if you can make ONE decision
• Improve, iterate, increase
20
28. Caroline Jarrett @cjforms (CC) BY SA-4.0
Asking the right people is better
than asking lots of people Sample:
the list you
sample from
Caroline Jarrett @cjforms (CC) BY SA-4.0
28
31. Caroline Jarrett @cjforms (CC) BY SA-4.0
Feeling special is rewarding
People will only respond if they trust
you. After that, it's a balance between
the perceived reward from filling in the
survey compared to the perceived
effort that's required. Strangely
enough, if a reward seems 'too good to
be true' that can also reduce the
response.
Diagram from Jarrett, C, and Gaffney, G (2008)
“Forms that work: Designing web forms for usability”
inspired by Dillman, D.A. (2000)
“Internet, Mail and Mixed Mode Surveys: The Tailored Design Method”
31
32. Caroline Jarrett @cjforms (CC) BY SA-4.0
Ask fewer people to get better response
Sample:
the number of
people to ask
Caroline Jarrett @cjforms (CC) BY SA-4.0
32
33. Caroline Jarrett @cjforms (CC) BY SA-4.0
Try mail, phone or face-to-face for
better response rates
Fieldwork:
Who answers?
Caroline Jarrett @cjforms (CC) BY SA-4.0
33
34. Did we get answers from
the right people?
Is this sample representative?
Image credit: Caroline Jarrett / CorelDraw
34
35. Population of assorted birds
Is this sample representative?
Image credit: Caroline Jarrett / CorelDraw
35
36. Caroline Jarrett @cjforms (CC) BY SA-4.0
We don’t just want answers,
we want representative answers
Response
Caroline Jarrett @cjforms (CC) BY SA-4.0
36
40. Caroline Jarrett @cjforms
In your last five days at work, what
percentage of your work time do you
estimate that you spent using publicly-
available online services (not including
email, instant messaging, and search) to
do your work using a work computer or
other device?
%
Caroline Jarrett @cjforms (CC) BY SA-4.0
40
41. Caroline Jarrett @cjforms (CC) BY SA-4.0
Helps a lot if you ask good questions
Questions:
What are you
asking about?
How many
questions?
Caroline Jarrett @cjforms (CC) BY SA-4.0
41
42. Caroline Jarrett @cjforms (CC) BY SA-4.0
Response relies on
effort, reward and trust
People will only respond if they trust
you. After that, it's a balance between
the perceived reward from filling in the
survey compared to the perceived
effort that's required. Strangely
enough, if a reward seems 'too good to
be true' that can also reduce the
response.
Diagram from Jarrett, C, and Gaffney, G (2008)
“Forms that work: Designing web forms for usability”
inspired by Dillman, D.A. (2000)
“Internet, Mail and Mixed Mode Surveys: The Tailored Design Method”
42
43. Caroline Jarrett @cjforms (CC) BY SA-4.0
A good question works in three ways
Obvious Interesting
Appropriate
43
44. Caroline Jarrett @cjforms (CC) BY SA-4.0
Why did you visit our website today?
Obvious Interesting
Appropriate
44
45. Caroline Jarrett @cjforms (CC) BY SA-4.0
Would you recommend us to
a friend or family member?
In a shop,
buying a
baby buggy
In a hospital,
having a
miscarriage
Obvious Yes
Interesting Yes
Appropriate Yes Cruelly
inappropriate
45
46. Caroline Jarrett @cjforms (CC) BY SA-4.0
Would you recommend us to
a friend or family member?
In a shop,
buying a
baby buggy
In a hospital,
having a
miscarriage
Obvious Yes
Interesting Yes
Appropriate Yes Cruelly
inappropriate
46
47. Tip
Test your questions by
interviewing in context
Caroline Jarrett @cjforms (CC) BY SA-4.0
47
48. Caroline Jarrett @cjforms (CC) BY SA-4.0
We don’t want any old answers.
We want useful answers.
Response
Caroline Jarrett @cjforms (CC) BY SA-4.0
48
49. Caroline Jarrett @cjforms
In your last five days at work, what
percentage of your work time do you
estimate that you spent using publicly-
available online services (not including
email, instant messaging, and search) to
do your work using a work computer or
other device?
%
Caroline Jarrett @cjforms (CC) BY SA-4.0
49
50. Caroline Jarrett @cjforms
On your most recent working day,
what percentage of time did you spend
using the Internet?
%
Caroline Jarrett @cjforms (CC) BY SA-4.0
50
51. Caroline Jarrett @cjforms (CC) BY SA-4.0
The approximate curve of forgetting
Major life event
Noticeable, occasional
Unremarkable,
repetitive
Perfect
recall
Day
Gone
Week Month YearHour
51
52. Caroline Jarrett @cjforms (CC) BY SA-4.0
Can’t remember = can’t answer
Response
Caroline Jarrett @cjforms (CC) BY SA-4.0
52
53. Caroline Jarrett @cjforms (CC) BY SA-4.0
Insights are the answers that you use
to make the decisions
Insights Caroline Jarrett @cjforms (CC) BY SA-4.0
53
55. Caroline Jarrett @cjforms
"Phone photography" by Petar Milošević -
Own work. Licensed under CC BY-SA 3.0 via
Commons -
https://commons.wikimedia.org/wiki/File:Pho
ne_photography.jpg#/media/File:Phone_phot
ography.jpg
Modified by Caroline Jarrett
55
56. Tip
Always allow for ‘other’
Design by @RickyBuchanan; t-shirt from nopitycity.com or zazzle.co.uk
56
57. Caroline Jarrett @cjforms
Can we make the decision yet?
Can we make the decision yet?
Can we make the decision yet?
Caroline Jarrett @cjforms (CC) BY SA4.0
57
This is a genuine invitation from local government, but the layout and images in the invitation make it look as if it's an approach from some sort of spammer or scammer.
People will only respond if they trust you. After that, it's a balance between the perceived reward from filling in the survey compared to the perceived effort that's required. Strangely enough, if a reward seems 'too good to be true' that can also reduce the response.
The survey sits between 'what you want to ask', 'who you want to ask' and 'the number'
A process starting with one person face to face, continues through 10 people by phone, gets to 100 people by email or pop-up.
It’s best to check that your question works with one person before you hassle 10 people with it. Then check it works with 10 people before you send it to 100. Once you’ve tried it on 100 people, you might be more interested in a new question than getting more answers on this question
The survey sits between 'what you want to ask', 'who you want to ask' and 'the number'
Prank leaves Justin Bieber facing tour of North Korea
By Daniel Emery Technology reporter, BBC News
5 July 2010
Image caption It is highly unlikely Bieber would be given permission to enter North Korea Canadian singer Justin Bieber's has become the target of a viral campaign to send him to North Korea.
A website polled users as to which country he should tour next, with no restrictions on the nations that could be voted on.
There are now almost half a million votes to send the singer to the secretive communist nation.
The contest, which ends at 0600 on 7 July, saw North Korea move from 24th to 1st place in less than two days.
Many of the votes are thought to originate from imageboard website 4chan, which has built a reputation for triggering online viral campaigns.
The octopus, with focus on 'The list you sample from'
People will only respond if they trust you. After that, it's a balance between the perceived reward from filling in the survey compared to the perceived effort that's required. Strangely enough, if a reward seems 'too good to be true' that can also reduce the response.
We’ll get our survey to the people who will answer in what the survey methodologists call ‘fieldwork’ – that might be a pop-up a website, a mail survey, or face-to-face interviews.
The octopus, with focus on "the ones whose answers you use"
The octopus again. This time we're looking at 'the questions we ask'.
People will only respond if they trust you. After that, it's a balance between the perceived reward from filling in the survey compared to the perceived effort that's required. Strangely enough, if a reward seems 'too good to be true' that can also reduce the response.
Obvious questions require no effort to answer. Interesting questions are rewarding to answer. Appropriate questions are ones that inspire trust in the respondent.
People come to the web with their own questions, so they’re likely to know why they are on your website. It’s a relatively interesting question to answer, and it’s appropriate to ask visitors why they’re visiting.
In your last five days at work, what percentage of your work time do you estimate that you spent using publicly-available online services (not including email, instant messaging, and search) to do your work using a work computer or other device?
Two questions from a survey:
'24: Do you use a Windows or Mac computer'?
'25. What is your gender'?
Photo of a Samsung (android) mobile with the same questions as previous slide. If you only have an Android mobile, how do you answer ‘do you use a Windows or Mac computer’ when the answer options are ‘Windows’, ‘Mac’ and ‘Both’?
A model wears a t-shirt with Gender: 'Male' (crossed out), 'Female' (crossed out) and 'Other' (added and ticked'.
A process starting with one person face to face, continues through 10 people by phone, gets to 100 people by email or pop-up.
It’s best to check that your question works with one person before you hassle 10 people with it. Then check it works with 10 people before you send it to 100. Once you’ve tried it on 100 people, you might be more interested in a new question than getting more answers on this question
The 8 tentacles of the Survey Octopus are:
Left side:
Goals: the resources you have
Questions: the questions you ask
Response: the answers you get
Insights: the answers you use
Right side:
The list you sample from
The sample you ask
The ones who answer
The ones whose answers you use
The 8 tentacles of the Survey Octopus are:
Left side:
Goals: the resources you have
Questions: the questions you ask
Response: the answers you get
Insights: the answers you use
Right side:
The list you sample from
The sample you ask
The ones who answer
The ones whose answers you use