This document provides an overview of key considerations for designing and administering questionnaires. It discusses ethical issues to consider, approaches to planning a questionnaire, types of questionnaire items and response formats, question wording best practices, sequencing questions, layout, cover letters, piloting, and data processing. The document outlines purposes of piloting a questionnaire and practical design considerations like ensuring questions measure intended constructs comprehensively while keeping the questionnaire brief. It also reviews methods of administering questionnaires.
Critical Thinking in Emergency Services Education slide shareRommie Duckworth
Â
Itâs clear that critical thinking must be part of an education program in order to teach students to become intelligent, compassionate and skillful emergency responders. But how do we accomplish this within the constraints of current educational curricula? This presentation is for anyone who wants to better incorporate critical thinking skills into their course content and testing processes. Critical thinking is vital not only for effective delivery of emergency services, but as a lifelong learning skill necessary for our students to excel as fire, rescue and EMS responders.
Teaching Formats:
-Lecture
-Q & A
-Role-Play
Learning Objectives: Students will learn:
-How to improve test scores, retention of knowledge and the ability to apply that knowledge to street-level emergency services.
-The role of critical thinking in decision making for emergency responders.
-The selection and use of different motivators, presentation types and activities in the classroom.
-Barriers and inhibitors to critical thinking in education and how to overcome them.
-How the application of higher education theories such as Perryâs âjourney of growthâ from received knowledge to constructed knowledge is critical to create effective emergency responders.
More at www.romduckworth.com and www.rescuedigest.com
Critical Thinking in Emergency Services Education slide shareRommie Duckworth
Â
Itâs clear that critical thinking must be part of an education program in order to teach students to become intelligent, compassionate and skillful emergency responders. But how do we accomplish this within the constraints of current educational curricula? This presentation is for anyone who wants to better incorporate critical thinking skills into their course content and testing processes. Critical thinking is vital not only for effective delivery of emergency services, but as a lifelong learning skill necessary for our students to excel as fire, rescue and EMS responders.
Teaching Formats:
-Lecture
-Q & A
-Role-Play
Learning Objectives: Students will learn:
-How to improve test scores, retention of knowledge and the ability to apply that knowledge to street-level emergency services.
-The role of critical thinking in decision making for emergency responders.
-The selection and use of different motivators, presentation types and activities in the classroom.
-Barriers and inhibitors to critical thinking in education and how to overcome them.
-How the application of higher education theories such as Perryâs âjourney of growthâ from received knowledge to constructed knowledge is critical to create effective emergency responders.
More at www.romduckworth.com and www.rescuedigest.com
This presentation describes the steps in designing a questionnaire. Also includes video clips for the process in evaluating the questionnaires for its reliability analysis.
These slides are specific phd thesis help for a talk I gave at Dublin City University on 15 May 2014. They should be helpful for any in a European context about to turn in their final thesis pre viva
A 2 day seminar with 17 rural schools in Manitoba considering a systems look at reading growth. Day one the discussion focused on the what and why of our assessments while day two moved toward how to use the data we collect as part of our planning and instruction.
Critical Thinking course at Strathmore University is a core Unit and hence a requirement for graduation. This slides were used for the first class hence a lot of focus is on course content and and a little part of topic one.
Any question that we want answered and any assumption or assertion that we want to challenge or investigate can become a research problem or a research topic for our study
This presentation describes the steps in designing a questionnaire. Also includes video clips for the process in evaluating the questionnaires for its reliability analysis.
These slides are specific phd thesis help for a talk I gave at Dublin City University on 15 May 2014. They should be helpful for any in a European context about to turn in their final thesis pre viva
A 2 day seminar with 17 rural schools in Manitoba considering a systems look at reading growth. Day one the discussion focused on the what and why of our assessments while day two moved toward how to use the data we collect as part of our planning and instruction.
Critical Thinking course at Strathmore University is a core Unit and hence a requirement for graduation. This slides were used for the first class hence a lot of focus is on course content and and a little part of topic one.
Any question that we want answered and any assumption or assertion that we want to challenge or investigate can become a research problem or a research topic for our study
Questionnaires is one of the most popular tool of collecting data
They provide a convenient way to gathering information from a target population. A questionnaire is a planned self-reported form designed to elicit information though written or verbal responses of the subjects.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Â
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
DevOps and Testing slides at DASA ConnectKari Kakkonen
Â
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Dev Dives: Train smarter, not harder â active learning and UiPath LLMs for do...UiPathCommunity
Â
đĽ Speed, accuracy, and scaling â discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Miningâ˘:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing â with little to no training required
Get an exclusive demo of the new family of UiPath LLMs â GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
đ¨âđŤ Andras Palfi, Senior Product Manager, UiPath
đŠâđŤ Lenka Dulovicova, Product Program Manager, UiPath
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Â
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But thereâs more:
In a second workflow supporting the same use case, youâll see:
Your campaign sent to target colleagues for approval
If the âApproveâ button is clicked, a Jira/Zendesk ticket is created for the marketing design team
Butâif the âRejectâ button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
Â
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Â
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Â
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
2. STRUCTURE OF THE CHAPTER
⢠Ethical issues
⢠Approaching the planning of a questionnaire
⢠Types of questionnaire items
⢠Asking sensitive questions
⢠Avoiding pitfalls in question writing
⢠Sequencing questions
⢠Questionnaires containing few verbal items
⢠The layout of the questionnaire
⢠Covering letters/sheets and follow-up letters
⢠Piloting the questionnaire
⢠Practical considerations in questionnaire design
⢠Administering questionnaires
⢠Processing questionnaire data
3. ETHICAL ISSUES
⢠Intrusion
⢠Informed consent
⢠Rights to withdraw at any stage or not to
complete particular items
⢠Beneficence
⢠Non-maleficence
⢠Confidentiality, anonymity and non-traceability
⢠Threat or sensitivity
⢠Avoidance of bias
⢠Validity and reliability in the questionnaire
⢠Reactions of the respondent
4. APPROACHING THE PLANNING OF
A QUESTIONNAIRE
⢠Stage One: Decide the purposes/objectives/
research questions;
⢠Stage Two: Decide the population and sample
⢠Stage Three: Itemize the topics/constructs/
concepts;
⢠Stage Four: Decide the kinds of measures or
responses needed;
⢠Stage Five: Write the questionnaire items;
⢠Stage Six: Check that each research question
has been covered;
⢠Stage Seven: Pilot the questionnaire and refine;
⢠Stage Eight: Administer the questionnaire.
5. OPERATIONALIZING A
QUESTIONNAIRE
⢠Clarify the questionnaireâs general purposes and
then translate them into a specific, concrete aim
or set of aims.
⢠Identify and itemize subsidiary topics that relate
to its central purpose.
⢠Formulate specific information requirements
relating to each issue.
⢠Plan with the data analysis in mind.
6. STRUCTURED, SEMI-STRUCTURED AND
UNSTRUCTURED QUESTIONNAIRES
⢠The larger the size of the sample, the more
structured, closed and numerical the
questionnaire may have to be.
⢠The smaller the size of the sample, the less
structured, more open and word-based the
questionnaire may be.
⢠Structured questions take a lot of time to set up
but then a short time to process and analyze.
⢠Open questions take a shorter time to set up but
a longer time to process and analyze.
7. TYPES OF QUESTION
⢠Open to closed
⢠Choose the metric (scale of data):
â Nominal
â Ordinal
â Interval
â Ratio
⢠Do not assume that respondents have the
information/knowledge/views
9. TYPES OF QUESTION
RANKING
RATIO DATA
CONSTANT SUM
RATIO DATA
1st
/2nd
. ETC.
MANY
RESPONSES
DISTRIBUTING
MARKS
MARKS OUT
OF TEN
10. DICHOTOMOUS QUESTIONS
⢠Good for clear answers;
⢠Yes/no questions are often better rephrased as
âto what extentâ or âhow muchâ types of question.
11. MULTIPLE CHOICE
⢠Need for a pilot to gather exhaustive categories
of response;
⢠Do not allow for range of response;
⢠If more than one response permitted then each
choice is a separate variable.
12. LIKERT SCALES
⢠Useful for measuring degrees of intensity of feeling;
⢠No assumption of equal intervals;
⢠No assumptions of matched intensity of feeling;
⢠No way of knowing if respondents are telling the truth;
⢠No way of knowing if there should be other categories
or items;
⢠Halo effect;
⢠Allows for different scaling and mid-points, e.g.:
(a) strongly disagree â neither agree nor
disagree â strong agree;
(b) not at all â a very great deal;
⢠Central tendency;
⢠Ordinal data.
13. SEMANTIC DIFFERENTIAL SCALES
⢠A word and its semantic opposite, e.g.:
â Approachable . . . unapproachable
â Generous . . . Mean
â Friendly . . . hostile
⢠Same concerns as for Likert scales.
15. RANKING SCALES
⢠Enables comparisons to be made by
respondents across items;
⢠Enables sensitivity of response to be addressed;
⢠Can be âstrong on realityâ of decision making;
⢠Too many items to rank may result in unrealistic
ranking (people may not have strong enough
opinions to be able to rank)
⢠Too many decisions to be made;
⢠Ordinal data.
16. RATIO DATA: MANY RESPONSES
⢠Avoids forcing responses into categories;
⢠Allows for very great accuracy (e.g. âhow old are
you?â)
⢠Ratio data: mean, standard deviation, median
17. CONSTANT SUM
⢠Divide a fixed number of points between a range
of items;
⢠Yields priorities, comparative highs and lows and
equality of choice quickly and easily â in the
respondentsâ own terms;
⢠Requires participants to make comparative
judgements and choices across items;
⢠May be too difficult if there are too many items
across which to spread marks;
⢠People may make computational errors in
distributing marks;
⢠Ordinal data.
18. RATIO DATA: MARKS OUT OF TEN
⢠Enables proportions/ratios to be calculated;
⢠Enables high level statistics to be computed,
e.g. regression, factor analysis, structural
equation modelling.
19. ASKING SENSITIVE QUESTIONS
⢠Use open rather than closed questions about
socially undesirable behaviour.
⢠Use long rather than short questions about socially
undesirable behaviour.
⢠Use familiar words when asking about socially
undesirable behaviour.
⢠Use data from informants.
⢠Deliberately load questions so that overstatements
of socially desirable behaviour and understatements
of socially undesirable behaviour are reduced.
⢠With regard to socially undesirable behaviour, firstly
ask whether the respondent has engaged in that
behaviour previously, and then move to asking
about current behaviour.
20. ASKING SENSITIVE QUESTIONS
⢠Locate sensitive topics within a discussion of other
less sensitive matters.
⢠Use alternative ways of asking standard questions.
⢠Ask respondents to keep diaries.
⢠At the end of an interview ask respondents their
views on the sensitivity of the topics that have been
discussed.
⢠Find ways of validating the data.
⢠As questions become more threatening and
sensitive, expect greater bias and unreliability.
21. âRULESâ FOR QUESTIONNAIRE
DESIGN (1)
⢠Ensure that each issue is explored in more
than one question;
⢠Decide the most appropriate kind of question
and the kind of scale;
⢠Plan with the kind of analysis in mind;
⢠Avoid leading questions: âDo you prefer abstract,
academic-type courses, or down-to-earth, practical
courses that have some benefit in day-to-day work?â
⢠Ensure that the question stem does not
frame the answer: âThe tourism industry is
successful because . . . .â.
22. âRULESâ FOR QUESTIONNAIRE
DESIGN (2)
⢠Avoid highbrow questions: âWhat particular
aspects of the current positivistic/interpretive debate
would you like to see reflected in the course of
developmental psychology?â
⢠Avoid negatives and double negatives: âHow
far do you agree that without a Consumer Association
the public cannot discuss consumer matters?â
⢠Avoid complex questions: âWould you prefer a
short award-bearing course with part-day release and
one evening per week attendance, or a longer, non-
award-bearing course with full-day release, or the
whole day designed on part-day release without
evening attendance?â
⢠Avoid too many open-ended questions on self-
completion questionnaires
23. âRULESâ FOR QUESTIONNAIRE
DESIGN (3)
⢠Try to convert dichotomous questions into
rating scales: âDo you. . .â / âAre you. . .â become
âHow far . . .?â/ âHow much . . .?â
⢠Provide anchor statements for rating scales;
⢠Have a minimum five-point rating scale if you
opt for uneven numbers;
⢠Decide whether to have odd or even
numbered rating scales;
⢠Avoid extremes in rating scales ( e.g. âalwaysâ,
âneverâ)
⢠Avoid too many open-ended questions in
large surveys;
24. âRULESâ FOR QUESTIONNAIRE
DESIGN (4)
⢠Only ask one thing at a time: Avoid: âhow much
do you think Business courses should be available
to all students, or should they only be available to
higher ability students?â
⢠Avoid ambiguity: âAre your parents in
employment?â âHave you done your homework
this week?â Do you do your homework regularly?â
âHow many staff are there in your institution?â How
many computers does your institution have?â
⢠Avoid threatening/irritating questions: âHow
frequently do you drink alcohol each week?â
25. âRULESâ FOR QUESTIONNAIRE
DESIGN (5)
⢠Clarify the kind of response sought in an
open-ended question: âPlease indicate the most
important factors that reduce staff participation in
decision making.â
⢠Start with simple factual questions and
then move to more sensitive questions;
⢠Provide instructions for how to complete
and return the questionnaire;
⢠Have a cover sheet which explains the
purposes of the questionnaire and which
sets out ethical issues.
26. âRULESâ FOR QUESTIONNAIRE
DESIGN (6)
⢠Ensure categories are mutually exclusive and
comprehensive (covering all possible choices
of response);
⢠Avoid pressurising/biasing by association:
e.g. âdo you agree with your school principal
that boys are more troublesome than girls?â
⢠Have translations verified/use back
translation;
⢠Keep it as short, simple and clear as possible;
⢠Pilot the questionnaire.
27. SEQUENCING QUESTIONS
⢠Take care with order effects
â Earlier responses affect later responses
â Early tone/mood-setting affects later moods in
completing questionnaires
⢠Primacy effect
â Items high in a list tend to be chosen more than items
lower in a list
â Respondents choose the first reasonable answer
from a list, even though a later response statement
might be more fitting
⢠Avoid placing sensitive questions at the start
â Embed them later questions
⢠Move from objective facts to subjective views
28. QUESTIONNAIRES CONTAINING
FEW VERBAL ITEMS
⢠A questionnaire might:
â include visual information and ask participants
to respond to this (e.g. pictures, cartoons,
diagrams)
â might include some projective visual
techniques (e.g. draw a picture or diagram)
â join two related pictures with a line, write the
words or what someone is saying or thinking
in a âbubbleâ picture
29. THE LAYOUT OF THE QUESTIONNAIRE
⢠It must look easy, attractive and interesting.
⢠Keep it as uncomplicated as possible.
⢠Clarity of wording.
⢠Simplicity of design.
⢠Simple, short and clear instructions for completion.
⢠Avoid placing instructions at the bottom of a page.
⢠Break down long lists of numbered items into separate
sections, each item in the section starting with the number â1â.
⢠Make it clear if respondents are exempted from completing
certain questions or sections (filter), and where they go next if
they are exempted.
⢠Include a preliminary statement of anonymity/confidentiality.
⢠Place response categories to the immediate right of the text.
30. THE COVERING LETTER SHOULD . . .
⢠Provide a title to the research;
⢠Introduce the researcher and contact details;
⢠Indicate the purposes of the research;
⢠Indicate the importance and benefits of the
research;
⢠indicate why the respondent has been selected for
receipt of the questionnaire;
⢠Indicate any professional backing, endorsement,
or sponsorship of, or permission for, the research;
⢠Set out how, where and by what date to return the
questionnaire;
⢠Indicate what to do if questions or uncertainties
arise;
31. THE COVERING LETTER SHOULD . . .
⢠Indicate any incentives for completing the
questionnaire;
⢠Provide assurances of confidentiality, anonymity
and non-traceability;
⢠Indicate how the results will and will not be
disseminated, and to whom;
⢠Thank respondents in advance for their co-
operation.
32. ⢠To check clarity of items/layout/sections/presentation/
instructions;
⢠To gain feedback on appearance;
⢠To eliminate ambiguities/uncertainty/poor wording;
⢠To check readability;
⢠To gain feedback on question type (suitability/feasibility/
format (e.g. open/closed/multiple choice);
⢠To gain feedback on appropriateness of question stems;
⢠To generate categories for responses in multiple choices;
⢠To generate items for further exploration/discussion;
⢠To gain feedback on response categories;
⢠To gain feedback on length/timing (when to conduct the
data collection as well as how long each takes to complete
(e.g. each interview/questionnaire))/coverage/ease of
completion;
PURPOSES OF PILOTING
33. ⢠To identify redundant items/questions (those with little
discriminability);
⢠To identify irrelevant questions;
⢠To identify non responses;â
⢠To identify how motivating/non engaging/threatening/â
intrusive/offensive items may be;
⢠To identify sensitive topics and problems in conducting
interviews;
⢠To test for inter rater reliability;â
⢠To minimise counter transference;â
⢠To gain feedback on leading questions;
⢠To identify items which are too easy/difficult/complex/
remote from experience;
⢠To identify commonly misunderstood or non completedâ
items.
PURPOSES OF PILOTING
34. PRACTICAL CONSIDERATIONS IN
QUESTIONNAIRE DESIGN
⢠Operationalize the purposes of the questionnaire.
⢠Be prepared to have a pre-pilot to generate items for a
pilot questionnaire, and then be ready to modify the pilot
questionnaire for the final version.
⢠If the pilot includes many items, and the intention is to
reduce the number of items through statistical analysis
or feedback, then be prepared to have a second round of
piloting, after the first pilot has been modified.
⢠Decide on the most appropriate type of question â
dichotomous, multiple choice, rank orderings, rating
scales, constant sum, ratio, closed, open.
⢠Ensure that every issue has been explored exhaustively
and comprehensively; decide on the content and explore
it in depth and breadth.
⢠Use several items to measure a specific attribute,
concept or issue.
35. PRACTICAL CONSIDERATIONS IN
QUESTIONNAIRE DESIGN
⢠Ask more closed than open questions for ease of analysis
(particularly in a large sample).
⢠Balance comprehensiveness and exhaustive coverage of
issues with the demotivating factor of having respondents
complete several pages of a questionnaire.
⢠Ask only one thing at a time in a question. Use single
sentences per item wherever possible.
⢠Keep response categories simple.
⢠Avoid jargon.
⢠Keep statements in the present tense wherever possible.
⢠Strive to be unambiguous and clear in the wording.
⢠Be simple, clear and brief wherever possible.
⢠Clarify the kinds of responses required in open questions.
36. PRACTICAL CONSIDERATIONS IN
QUESTIONNAIRE DESIGN
⢠Balance brevity with politeness, e.g. replace a blunt phrase like
âmarital statusâ with a gentler âplease indicate whether you are
married, living with a partner, or single....â.
⢠Ensure a balance of questions which ask for facts and opinion.
⢠Avoid leading questions.
⢠Try to avoid threatening questions.
⢠Do not assume that respondents know the answers, or have
information to answer the questions, or will always tell the truth
(wittingly or not)..
⢠Avoid making the questions too hard.
⢠Balance the number of negative questions with the number of
positive questions.
⢠Consider the readability levels of the questionnaire and the
reading and writing abilities of the respondents.
⢠Put sensitive questions later in the questionnaire.
⢠Intersperse sensitive questions with non-sensitive questions.
37. PRACTICAL CONSIDERATIONS IN
QUESTIONNAIRE DESIGN
⢠Be very clear on the layout of the questionnaire so that it is
unambiguous and attractive.
⢠Avoid, where possible, splitting an item over more than one
page,.
⢠Ensure that the respondent knows how to enter a reply to
each question.
⢠Pilot the questionnaire.
⢠With the data analysis in mind.
⢠Decide how to avoid falsification of responses.
⢠Be satisfied if you receive a 50 per cent response to the
questionnaire
⢠Decide what you will do with missing data.
⢠Include a covering letter.
⢠If the questionnaire is going to be administered by someone
other than the researcher, ensure that instructions for
administration are provided and that they are clear.
38. METHODS OF ADMINISTERING
QUESTIONNAIRES
⢠Post
⢠Self-administration in presence of researcher
⢠Self-administration without researcher present
⢠In situ completion (e.g. workplace/ home)
⢠Face-to-face interview
⢠Telephone
⢠Internet