Prospective anomaly detection methods such as the Modified EARS C2 are commonly adapted and used in public health syndromic surveillance systems. These methods however can produce an excessive false alert rate. We present a combined use of retrospective (e.g., Change Point Analysis (or CPA)) and prospective (e.g., C2) anomaly detection methods. This combined approach will help detect sudden aberrations in addition to subtle changes in local trends, help rule out alarm investigations, and assist with retrospective follow-ups. Examples on the utility of this combined approach in working collaboratively with the scientific community are applied to BioSense emergency departments' visits due to ILI. Methods, limitations, future work, and invitation to the scientific community to collaborate with us will be discussed at this talk.
Prospective anomaly detection methods such as the Modified EARS C2 are commonly adapted and used in public health syndromic surveillance systems. These methods however can produce an excessive false alert rate. We present a combined use of retrospective (e.g., Change Point Analysis (or CPA)) and prospective (e.g., C2) anomaly detection methods. This combined approach will help detect sudden aberrations in addition to subtle changes in local trends, help rule out alarm investigations, and assist with retrospective follow-ups. Examples on the utility of this combined approach in working collaboratively with the scientific community are applied to BioSense emergency departments' visits due to ILI. Methods, limitations, future work, and invitation to the scientific community to collaborate with us will be discussed at this talk.
Please Subscribe to this Channel for more solutions and lectures
http://www.youtube.com/onlineteaching
Chapter 3: Describing, Exploring, and Comparing Data
3.2: Measures of Variation
A series of modules on project cycle, planning and the logical framework, aimed at team leaders of international NGOs in developing countries.
Part 7 of 11.
There are two handouts to go with this module, Population Indicators, and a Logframe with blanks. http://www.slideshare.net/Makewa/population-indicators-handout and http://www.slideshare.net/Makewa/exercise-watsan-logframe-with-blanks
Please Subscribe to this Channel for more solutions and lectures
http://www.youtube.com/onlineteaching
Chapter 3: Describing, Exploring, and Comparing Data
3.2: Measures of Variation
A series of modules on project cycle, planning and the logical framework, aimed at team leaders of international NGOs in developing countries.
Part 7 of 11.
There are two handouts to go with this module, Population Indicators, and a Logframe with blanks. http://www.slideshare.net/Makewa/population-indicators-handout and http://www.slideshare.net/Makewa/exercise-watsan-logframe-with-blanks
How to design surveys; describes differences between approaches to measuring awareness, opinions, perceptions, behaviors, needs and attitudes; describes roles of survey sponsor and researcher.
Improving and Demonstrating Impact for Youth Using Qualitative DataDetroitYDRC
This workshop provided an overview of how to use qualitative data for improving and demonstrating the impact of youth development programs. Tips for collecting, analyzing and using qualitative data are provided. Examples of creative ways to visualize qualitative data are also shared.
Methods of collecting data
Survey, methods and type, response rate, variable language
Hands on: Graphical techniques II, SPSS
Questionnaire design
Tips on writing a research paper
Individual project: article critique
This is a modified version of Master Class that Dr Siobhan O'Dwyer delivered at the Griffith University School of Nursing's Annual Research School for postgraduate students.
Michele Madden, managing director, nfpSynergy
Visit the CharityComms website to view slides from past events, see what events we have coming up and to check out what else we do: www.charitycomms.org.uk
This presentation provides an introduction to planning a small-scale project evaluation. It's aimed at small organisations, charities, purposeful businesses.
Honest Reviews of Tim Han LMA Course Program.pptxtimhan337
Personal development courses are widely available today, with each one promising life-changing outcomes. Tim Han’s Life Mastery Achievers (LMA) Course has drawn a lot of interest. In addition to offering my frank assessment of Success Insider’s LMA Course, this piece examines the course’s effects via a variety of Tim Han LMA course reviews and Success Insider comments.
Palestine last event orientationfvgnh .pptxRaedMohamed3
An EFL lesson about the current events in Palestine. It is intended to be for intermediate students who wish to increase their listening skills through a short lesson in power point.
Embracing GenAI - A Strategic ImperativePeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
2024.06.01 Introducing a competency framework for languag learning materials ...Sandy Millin
http://sandymillin.wordpress.com/iateflwebinar2024
Published classroom materials form the basis of syllabuses, drive teacher professional development, and have a potentially huge influence on learners, teachers and education systems. All teachers also create their own materials, whether a few sentences on a blackboard, a highly-structured fully-realised online course, or anything in between. Despite this, the knowledge and skills needed to create effective language learning materials are rarely part of teacher training, and are mostly learnt by trial and error.
Knowledge and skills frameworks, generally called competency frameworks, for ELT teachers, trainers and managers have existed for a few years now. However, until I created one for my MA dissertation, there wasn’t one drawing together what we need to know and do to be able to effectively produce language learning materials.
This webinar will introduce you to my framework, highlighting the key competencies I identified from my research. It will also show how anybody involved in language teaching (any language, not just English!), teacher training, managing schools or developing language learning materials can benefit from using the framework.
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
Model Attribute Check Company Auto PropertyCeline George
In Odoo, the multi-company feature allows you to manage multiple companies within a single Odoo database instance. Each company can have its own configurations while still sharing common resources such as products, customers, and suppliers.
Welcome to TechSoup New Member Orientation and Q&A (May 2024).pdfTechSoup
In this webinar you will learn how your organization can access TechSoup's wide variety of product discount and donation programs. From hardware to software, we'll give you a tour of the tools available to help your nonprofit with productivity, collaboration, financial management, donor tracking, security, and more.
Instructions for Submissions thorugh G- Classroom.pptxJheel Barad
This presentation provides a briefing on how to upload submissions and documents in Google Classroom. It was prepared as part of an orientation for new Sainik School in-service teacher trainees. As a training officer, my goal is to ensure that you are comfortable and proficient with this essential tool for managing assignments and fostering student engagement.
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
2. Lecture Overview
What should you measure?
What makes a good measure?
Measurement
Data Collection
Piloting
3. What do we measure and where does it fit into the whole project?
WHAT SHOULD YOU MEASURE?
4. What Should You Measure?
Follow the Theory of Change
• Characteristics: Who are the people the program works with, and
what is their environment?
Sub-groups, covariates, predictors of compliance
• Channels: How does the program work, or fail to work?
• Outcomes: What is the purpose of the program?
• Assumptions: What should have happened in order for the
program to succeed?
List all indicators you intend to measure
• Use participatory approach to develop indicators (existing
instruments, experts, beneficiaries, stakeholders)
• Assess based on feasibility, time, cost and importance
5. Methods of Data Collection
Administrative data
Surveys- household/individual
logs/diaries
Qualitative – eg. focus groups,
RRA
Games and choice problems
Observation
Health/Education tests and
measures
8. The Main Challenge in Measurement: Getting Accuracy
and Precision
More accurate
More precise
9. Terms “Biased” and “Unbiased” Used to Describe
Accuracy
More accurate
“Biased” “Unbiased
” On average, we get
the wrong answer
On average, we get
the right answer
10. Terms “Noisy” and “Precise” Used to Describe Precision
More precise
“Noisy”
Random error
causes answer to
bounce around
“Precise”
Measures of the
same thing cluster
together
11. Choices in Real Measurement Often Harder
More accurate
More precise
“Noisy” but
“Unbiased”
“Precise” but
“Biased”
Random error
causes answer to
bounce around
Measures of the
same thing cluster
together
12. The Main Challenge in Measurement: Getting Accuracy and
Precision
More accurate
More precise
13. Accuracy
In theory:
• How well does the indicator map to the outcome? (e.g.
intelligence IQ tests)
In practice: Are you getting unbiased answers?
• Social desirability bias (response bias)
• Anchoring bias (Strack and Mussweiler, 1997)
Did Mahatma Gandhi die before or after age 9?
Did Mahatma Gandhi die before or after age 140?
• Framing effect
Given that violence against women is a problem, should we impose
nighttime curfews?
14. Precision and Random Error
In theory: The measure is consistent, precise, but not necessarily valid
In practice:
• Length, fatigue
• “How much did you spend on broccoli yesterday?” (as a measure of
annual broccoli spending)
• Ambiguous wording (definitions, relationships, recall period, units of
question)
Eg. Definition of terms – ‘household’, ‘income’
• Recall period/units of question
• Type of answer -Open/Closed
• Choice of options for closed questions
Likert (i.e. Strongly disagree, disagree, neither agree nor disagree, . . .)
Rankings
• Surveyor training/quality
16. The Basics
Data that should be easy?
• E.g. Age, # of rooms in house, # in HH
What is the survey question identifying?
• E.g. Are HH members people who are related to the household
head? People who eat in the household? People who sleep in
the household?
Pre-test questions in local languages
17. The Basics: Units of Observation
Choosing Modules: Units of Observation
Often this is simple: For example, sex and age are clearly attributes of
individuals. Roofing material is attribute of the dwelling.
Not always obvious: To collect information on credit, one could ask
household’s
All current outstanding loans.
All loans taken and repaid in the last one year.
All “borrowing events” (all the times a household tried to borrow,
whether successfully or not).
Choice is determined by expected analytical use and reliability of
information
18. The Basics: Deciding Who to Ask
“Target respondent”: should be most informed person for each
module. Respondents for each module can vary.
For example: to measure use of Teaching Learning Materials,
should we survey the headmaster? Teachers? SMC? Parents?
Students?
Choice of modules decides target respondent, and target
respondent shapes the design of questions.
19. What is hard to measure in a survey?
(1) Things people do not know very well
(2) Things people do not want to talk about
(3) Abstract concepts
(4) Things that are not (always) directly observable
(5) Things that are best directly observed
20. How much tea did you consume last month?
A. <2 liters
B. 2-5 liters
C. 6-10 liters
D. >11 liters
21. 1. Things people do not know very well
What: Anything to estimate, particularly across time. Prone to
recall error and poor estimation
• Examples: distance to health center, profit, consumption,
income, plot size
Strategies:
• Consistency checks – How much did you spend in the last
week on x? How much did you spend in the last 4 weeks on x?
• Multiple measurements of same indicator – How many minutes
does it take to walk to the health center? How many kilometers
away is the health center?
22. How many cups of tea did you consume yesterday?
A. 0
B. 1-3
C. 4-6
D. >6
23. What is Hard to Measure?
(1) Things people do not know very well
(2) Things people do not want to talk about
(3) Abstract concepts
(4) Things that are not (always) directly observable
(5) Things that are best directly observed
24. How frequently do you yell at your partner?
A. Daily
B. Several times per week
C. Once per week
D. Once per month
E. Never
25. 2. Things people don’t want to talk about
What: Anything socially “risky” or something painful
• Examples: sexual activity, alcohol and drug use, domestic
violence, conduct during wartime, mental health
Strategies:
• Don’t start with the hard stuff!
• Consider asking questions in third person
• Always ensure comfort and privacy of respondent
• Think of innovative techniques – vignettes, list randomization
26. How frequently does your partner yell at you?
A. Daily
B. Several times per week
C. Once per week
D. Once per month
E. Never
27. What is Hard to Measure?
(1) Things people do not know very well
(2) Things people do not want to talk about
(3) Abstract concepts
(4) Things that are not (always) directly observable
(5) Things that are best directly observed
27
28. “I feel more empowered now than last year”
A. Strongly disagree
B. Disagree
C. Neither agree nor disagree
D. Agree
E. Strongly agree
29. 3. Abstract concepts
What: Potentially the most challenging and interesting type of
difficult-to-measure indicators
• Examples: empowerment, bargaining power, social cohesion, risk
aversion
Strategies:
• Three key steps when measuring “abstract concepts”
• Define what you mean by your abstract concept
• Choose the outcome that you want to serve as the measurement
of your concept
• Design a good question to measure that outcome
Often choice between choosing a self-reported measure and a
behavioral measure – both can add value!
30. What is Hard to Measure?
(1) Things people do not know very well
(2) Things people do not want to talk about
(3) Abstract concepts
(4) Things that are not (always) directly observable
(5) Things that are best directly observed
31. 4. Things that aren’t Directly Observable
What: You may want to measure outcomes that you can’t ask
directly about or directly observe
• Examples: corruption, fraud, discrimination
Strategies:
• Audit studies, e.g. CVs and racial discrimination
• Multiple sources of data, e.g. inputs of funds vs. outputs received by
recipients, pollution reports by different parties
• Don’t worry – there have already been lots of clever people before you
– so do literature reviews!
32. 5. Things that are Best Directly Observed
What: Behavioral preferences, anything that is more believable
when done than said
Strategies:
• Develop detailed protocols
• Ensure data collection of behavioral measures done under the same
circumstances for all individuals
34. Use of Data
Reporting
• On Inputs and Outputs (Achievement of physical and financial targets)
Monitoring
• Of Processes and Implementation (Doing things right)
Evaluation
• Of Outcomes and Impact (Doing the right thing)
Management and Decision Making
• Using relevant and timely information for decision making (reporting and
monitoring for mid term correction; evaluation for planning and scale up)
ALL OF THE ABOVE DEPEND ON THE AVAILABILITY OF RELIABLE,
ACCURATE AND TIMELY DATA
35. Problems in Data Collection
Data reliability (will we get the same data, when collected
again?)
Data validity (Are we measuring what we say we are
measuring?)
Data integrity (Is the data free of
manipulation?)
Data accuracy/precision (Is the data measuring the “indicator”
accurately?)
Data timeliness (Are we getting the data in
time?)
Data security/confidentiality (Loss of data / loss of
privacy)
36. Reliability of Data Collection
The process of collecting “good” data requires a lot of efforts and
thought
Need to make sure that the data collected is precise and accurate.
avoid false or misleading conclusions
The survey process:
• Design of questionnaire Survey printed on paper/electronic
filled in by enumerator interviewing the respondent data entry
electronic dataset
Where can this go wrong?
37. Reliability of Survey Data
Start with a pilot
Paper vs. electronic survey
Surveyors and supervision
Following up the respondents
Problems with respondents
Neutrality
39. Importance of Piloting
Finding the best way to procure required information
• choice of respondent
• type and wording of questions
• order of sections
Piloting and fine-tuning different response options and components
Understanding of time taken, respondent fatigue, and other
constraints
40. Steps in Piloting
ALWAYS allow time for piloting and back-and-forth between team on
the field and the researchers
Two phases of piloting
Phase 1: Early stages of questionnaire development
Understand the purpose of the questionnaire
test and develop new questions
adapt questions to context
build options and skips
Re-work, share and re-test
Build familiarity, adapt local terms, get a sense of time
41. Steps in Piloting
Phase 2: Field testing just before surveying
Final touches to translation
questions and instructions
Keep it as close to final survey as possible.
42. Things to Look for During the Pilot
Comprehension of questions
Ordering of questions - priming
Variation in responses
Missing answers
More questions for clarifications? Cut questions? consistency checks?
Is the choice of respondent appropriate?
Respondent fatigue or discomfort
Need to add or correct filters? Need to add clear surveyor instructions?
Is the format (phone or paper) user-friendly? Does it need to be improved?
43. Discuss Potentially Difficult Questions with the Respondent
Example 1: Simplify/clarify questions
Do you use Student Evaluation Sheets in your school?
• Yes
• No
• Don’t know/Not sure
• No response
They might not know it by this name (show them a sample)
You may need to break it up into several questions to get at what you want
• Do you have them?
• Have you been trained on how to use them?
• Do you use them?
44. Discuss Potentially Difficult Questions with the Respondent
Example 2 : Ordering questions and priming
Yesterday, how much time did you spend cooking, cleaning,
playing with your child, teaching/doing homework with your
child?
Do you think its important for mothers to play with children?
Do you think mothers or fathers should be more responsible
for a child’s education?
If Questions 2 and 3 had come before 1, there could’ve been a
possible
bias, order and wording of questions is important
45. Importance of Language and Translation
The local language is probably not English, which makes things
tricky as to the wording of certain questions
• But people may be familiar with “official” words in English
rather than the local language
Translate
• Ensures that every surveyor knows the exact wording of the
questions, instead of having to translate on the fly
Back-translate
• Helps clarify when local-language words are used that don’t
have the same meaning as the original English
46. Documentation and Feedback
Notes – time, difficulties, required or suggested changes
Meetings to share inputs
Draft document
Keep different versions of the questionnaire