Module 4RM
Data collection, Measuring, Sampling and Scaling
1 CLASSIFICATION OF DATA
The process of arranging data into homogenous group or classes according to some common
characteristics present in the data is called classification. For Example: The process of sorting
letters in a post office, the letters are classified according to the cities and further arranged
according to streets.
2 BASES OF CLASSIFICATION:
There are four important bases of classification:
(1) Qualitative Base
(2) Quantitative Base
(3) Geographical Base
(4) Chronological or Temporal Base
(1) Qualitative Base:
When the data are classified according to some quality or attributes such as sex, religion, literacy,
intelligence etc…
(2) Quantitative Base:
When the data are classified by quantitative characteristics like heights, weights, ages, income
etc…
(3) Geographical Base:
When the data are classified by geographical regions or location, like states, provinces, cities,
countries etc…
(4) Chronological or Temporal Base:
When the data are classified or arranged by their time of occurrence, such as years, months, weeks,
days etc… For Example: Time series data.
3 TYPES OF CLASSIFICATION:
(1) One -way Classification:
If we classify observed data keeping in view single characteristic, this type of classification is
known as one-way classification.
For Example: The population of world may be classified by religion as Muslim, Christians etc…
(2) Two -way Classification:
If we consider two characteristics at a time in order to classify the observed data then we are doing
two way classifications.
For Example: The population of world may be classified by Religion and Sex.
(3) Multi -way Classification:
We may consider more than two characteristics at a time to classify given data or observed data. In
this way we deal in multi-way classification.
For Example: The population of world may be classified by Religion, Sex and Literacy.
4 BENEFITS AND DRAWBACKS OF DATA
Primary data is data compiled from first-hand experience or observation. Advantages of
primary data include, the data is unbiased, the data is original and the data is basic.
Disadvantages of primary data include, gathering the data is time-consuming, the data is
raw and there can be too much data.
Primary Sources of Data- Primary sources are original sources from which the
researcher directly collects data that have not been previously collected e.g.., collection of
data directly by the researcher on brand awareness, brand preference, brand loyalty and
other aspects of consumer behaviour from a sample of consumers by interviewing them,.
Primary data are first hand information collected through various methods such as
observation, interviewing, mailing etc.
Advantage of Primary Data- It is original source of data
·It is possible to capture the changes occurring in the course of time. ·It flexible to the
advantage of researcher.
·Extensive research study is based of primary data
Disadvantage of Primary Data
1. Primary data is expensive to obtain
2. It is time consuming
3. It requires extensive research personnel who are skilled
4. It is difficult to administer.
1. Secondary data have several advantages
Data assembly is inexpensive. Company records, trade journals, and government
publications are all rather low-cost. No data collection forms, interviewers, and
tabulations are needed.
Data can be gathered quickly. Company records, library sources, and Web sites can be
accessed immediately. Many firms store reports in their retail information systems
connected with their pos equipment.
There may be several sources of secondary data--with many perspectives.
A secondary source may possess information that would otherwise be unavailable to the
retailer. Government publications often have statistics no private firm could acquire.
When data are assembled by a source such as Progressive Grocer, A.C. Nielsen, Stores,
or the government, results are usually quite credible.
The retailer may have only a rough idea of the topics to investigate. Secondary data can
then help to define issues more specifically. In addition, background information about a
given issue can be gathered from secondary sources before undertaking a primary study.
2. Secondary data also have several potential disadvantages
Available data may not suit the purposes of the current study because they have been
collected for other reasons. Neighborhood statistics may not be found in secondary
sources.
Secondary data may be incomplete.
Information may be dated.
The accuracy of secondary data must be carefully evaluated.
Some secondary data sources are known for poor data collection techniques; they should
be avoided.
In retailing, many secondary data projects are not retested and the user of secondary data
has to hope results from one narrow study are applicable to his or her firm.
3) Evaluation of data
How will the data be analyzed and presented in order to address the key evaluation
questions?
According to Owen's model, the second part of data management is analysis and
reporting, which has three components; data reduction, data display and conclusion
drawing and verification.
The general data analysis process in evaluation is one of reduction- that is, 'the process of
simplifying and transforming the raw information according to some logical set of
procedures or rules' (Owen 2006: 101). There is a wide range of processes for data
reduction for both quantitative and qualitative information. The processes used must be
explicitly described when reporting the data analysis results. There are two general
purposes for data analysis in evaluation; description and explanation. Both are important
because description enables the audience to understand the project, its intended processes
and outcomes, and the extent to which these were achieved, whereas explanation
provides evidence about the underlying logic of the project and the extent to which it is
sustainable, transferable and/or reproducible.
The display of data is a process of organising the information in ways that lead to the
drawing of explicit and defensible conclusions about the key evaluation questions. In
many evaluations, conclusions are the endpoint, however in others, the evaluators go
further to offer recommendations about the project. The former require placing values on
the conclusions such as stating the project is successful or not, whereas, the latter are
advice or suggestions for courses of action made to decision makers.
What ethical issues are involved in the evaluation and how will they addressed?
Ethical issues often arise in the data management process described above e.g. in the
selection of data sources, obtaining the information or reporting results. Data collection
activities in ALTC funded projects will usually require approval from the lead
institution’s research ethics committee, and may require complementary approval from
partner institutions. The main issues which are likely to arise include appropriate methods
of collecting, analysing, storing and reporting data from students to protect their
confidentiality and anonymity, ensuring students and staff are not impacted unfairly by
the evaluation activities (avoiding interruptions to the learning and teaching processes),
and unfairly disadvantaging students who are not receiving the project benefits. The
Australasian Evaluation Society has produced a Code of Conduct and a set of Guidelines
for the Ethical Conduct of Evaluations, which provide useful guidance for evaluation
activities
Qualitative methods of data collection, Methods of collecting qualitative data, Data
collection approaches for qualitative research usually involves:, Direct interaction with
individuals on a one to one basis , Or direct interaction with individuals in a group setting
Qualitative research data collection methods are time consuming, therefore data is usually
collected from a smaller sample than would be the case for quantitative approaches -
therefore this makes qualitative research more expensive.
The benefits of the qualitative approach is that the information is richer and has a deeper
insight into the phenomenon under study
The main methods for collecting qualitative data are: Individual interviews, Focus ,
groups Observations, Action Research
4. Interviews Interviews can be- Unstructured Can be referred to as 'depth' or 'in depth'
interviews. They have very little structure at all. The interviewer may just go with the aim
of discussing a limited number of topics, sometimes as few as just one or two. The
interviewer may frame the interview questions based on the interviewee and his/her
previous response
This allows the discussion to cover areas in great detail
They involve the researcher wanting to know or find out more about a specific topic
without there being a structure or a preconceived plan or expectation as to how they will
deal with the topic Semi structured-Semi structured interviews are sometimes also called
focused interviews
A series of open ended questions based on the topic areas the researcher wants to cover
A series of broad questions to ask and may have some prompts to help the interviewee
'The open ended nature of the question defines the topic under investigation but provides
opportunities for both interviewer and interviewee to discuss some topics in more detail'
Semi structured interviews allow the researcher to promt or encourage the interviewee if
they are looking for more information or find what they are saying interesting
This method gives the researcher the freedom to probe the interviewee to elaborate or to
follow a new line of inquiry introduced by what the interviewee is saying
Work best when the interviewed has a number of areas he/she wants to be sure to be
addressing Structured The interviewed asks the respondent the same questions in the
same way
A tightly structured schedule is used
The questions may be phrased in order that a limited range of responses may be given -
i.e. 'Do you rate our services as very good, good or poor'
A researcher needs to consider whether a questionnaire or structured interview is more
appropriate
'If the interview schedule is too tightly structured this may not enable the phenomena
under investigation to be explored in terms of either breadth or depth.'
Qualitative interviews should be fairly informal and participants feel they are taking part
in a conversation or discussion rather than in a formal question and answer situation.
There is skill required and involved in successful qualitative research approaches - which
requires careful consideration and planning Good quality qualitative research involves:
Thought Preparation The development of the interview schedule Conducting and
analysing the interview data with care and consideration
5. Focus groups -The use of focus groups is sometimes used when it is better to obtain
information from a group rather than individuals. Group interviews can be used when:
Limited resources (time, manpower, finances) The phenomena being researched requires
a collective discussion in order to understand the circumstances, behaviour or opinions
Greater insights may be developed of the group dynamic - or cause and consequence
Characteristics of a focus group: Recommended size of the sample group is 6 - 10 people
as smaller groups may limit the potential on the amount of information collected, and
more may make it difficult for all participants to participate and interact and for the
interviewer to be able to make sense of the information given Several focus groups
should be used in order to get a more objective and macro view of the investigation. i.e.
focussing on one group may give you idiosyncratic results. The use of several groups will
add to the breadth and depth of information. A minimum of three focus groups is
recommended for best practice approaches Members of the focus group should have
something in common which is important to the investigation Groups can either be put
together or existing groups - it is always useful to be mindful of the group dynamics of
both situations.
The aim of the focus group is to make use of participants' feelings, perceptions and
opinions
This method requires the researcher to use a range of skills: group skills, facilitating,
moderating, listening/observing, analysis
6. Observation
Observation involves may take place in natural settings and involve the researcher taking
lengthy and descriptive notes of what is happening.
It is argued that there are limits to the situations that can be observed in their 'natural'
settings and that the presence of the research may lead to problems with validity.
Limitations with observation include:
Change in people's behaviour when they know they are being observed
A 'snap shot' view of a whole situation
Think Big Brother...
The researcher may miss something while they are watching and taking notes
The researcher may make judgements of make value statements or misunderstand what
has been observed
Strengths of observation
Can offer a flavour for what is happening
Can give an insight into the bigger picture
Can demonstrate sub-groups
Can be used to assist in the design of the rest of the research
Sometimes, the researcher becomes or needs to become a participant observer, where
they are taking part in the situation in order to be accepted and further understand the
workings of the social phenomenon.
Observation can sometimes obtain more reliable information about certain things - for
example, how people actually behave (although it may not find out the reasons for why
they behave in a particular way).
Observation can also serve as a technique for verifying of nullifying information
provided in face to face encounters.'
People or environment can be observed.
When environment is researched, it can provide valuable background information that
may inform other aspects of the research.
Techniques for collecting data through observation
Written descriptions
The researcher makes written descriptions of the people, situations or environment
Limitations include
Researcher might miss out on an observation as they are taking notes
The researcher may be focussed on a particular event or situation
There is room for subjective interpretation of what is happening
Video recording
Allows the researcher to also record notes
Limitations may include people acting unnaturally towards the camera or others avoiding
the camera
The camera may not always see everything
Photographs and artefacts
Useful when there is a need to collect observable information or phenomena such as
buildings, neighbourhoods, dress and appearance
Artefacts include objects of significance - memorabilia, instruments, tools etc
Documentation
Any and all kinds of documentation may be used to provide information - a local paper,
information on a notice board, administrative policies and procedures...etc previous
research, even
7. Self Study
Consider an area within your work that you might want to observe in order to get an
answer, find out more or gain a better understanding.
Think about and plan:
What your aim/purpose is.
What permission, etc, you may need to gain.
What your role/presence will be.
How you will record your observation.
What you will record.
What you will do with your findings.
What are the pros and cons of this process?
5) METHODS OF QUALITATIVE RESEARCH
What is qualitative research?
Qualitative research is a type of scientific research. In general terms, scientific research
consists
Of an investigation that:
• seeks answers to a question
• Systematically uses a predefined set of procedures to answer the question
• collects evidence
• produces findings that were not determined in advance
• produces findings that are applicable beyond the immediate boundaries of the study
Qualitative research shares these characteristics. Additionally, it seeks to understand a
given
Research problem or topic from the perspectives of the local population it involves.
Qualitative
Research is especially effective in obtaining culturally specific information about the
values,
opinions, behaviors, and social contexts of particular populations.
What are some qualitative research methods?
The three most common qualitative methods, explained in detail in their respective
modules, are
participant observation, in-depth interviews, and focus groups. Each method is
particularly suited
for obtaining a specific type of data.
• Participant observation is appropriate for collecting data on naturally occurring
behaviors in
their usual contexts.
• In-depth interviews are optimal for collecting data on individuals’ personal histories,
perspectives,
and experiences, particularly when sensitive topics are being explored.
• Focus groups are effective in eliciting data on the cultural norms of a group and in
generating
broad overviews of issues of concern to the cultural groups or subgroups represented.
6) SAMPLING: SAMPLING STRATEGY, SAMPLE SIZE
Information is not readily found at a bargain price. Gathering it is costly in terms of
salaries, expenses and time. Taking samples of information can help ease these costs
because it is often impractical to collect all the data. Sound conclusions can often be
drawn from a relatively small amount of data; therefore, sampling is a more efficient way
to collect data. Using a sample to draw conclusions is known as statistical inference.
Making inferences is a fundamental aspect of statistical thinking.
Selecting the Most Appropriate Sampling Strategy
There are four primary sampling strategies:
Random sampling
Stratified random sampling
Systematic sampling
Rational sub-grouping
Before determining which strategy will work best, the analyst must determine what type
of study is being conducted. There are normally two types of studies: population and
process. With a population study, the analyst is interested in estimating or describing
some characteristic of the population (inferential statistics).
With a process study, the analyst is interested in predicting a process characteristic or
change over time. It is important to make the distinction for proper selection of a
sampling strategy. The “I Love Lucy” television show’s “Candy Factory” episode can be
used to illustrate the difference. For example, a population study, using samples, would
seek to determine the average weight of the entire daily run of candies. A process study
would seek to know whether the weight was changing over the day.
Random Sampling
Random samples are used in population sampling situations when reviewing historical or
batch data. The key to random sampling is that each unit in the population has an equal
probability of being selected in the sample. Using random sampling protects against bias
being introduced in the sampling process, and hence, it helps in obtaining a representative
sample.
In general, random samples are taken by assigning a number to each unit in the
population and using a random number table or Minitab to generate the sample list.
Absent knowledge about the factors for stratification for a population, a random sample is
a useful first step in obtaining samples.
For example, an improvement team in a human resources department wanted an accurate
estimate of what proportion of employees had completed a personal development plan
and reviewed it with their managers. The team used its database to obtain a list of all
associates. Each associate on the list was assigned a number. Statistical software was
used to generate a list of numbers to be sampled, and an estimate was made from the
sample.
Stratified Random Sampling
Like random samples, stratified random samples are used in population sampling
situations when reviewing historical or batch data. Stratified random sampling is used
when the population has different groups (strata) and the analyst needs to ensure that
those groups are fairly represented in the sample. In stratified random sampling,
independent samples are drawn from each group. The size of each sample is proportional
to the relative size of the group.
For example, the manager of a lending business wanted to estimate the average cycle
time for a loan application process. She knows there are three types (strata) of loans
(large, medium and small). Therefore, she wanted the sample to have the same proportion
of large, medium and small loans as the population. She first separated the loan
population data into three groups and then pulled a random sample from each group.
Systematic Sampling
Systematic sampling is typically used in process sampling situations when data is
collected in real time during process operation. Unlike population sampling, a frequency
for sampling must be selected. It also can be used for a population study if care is taken
that the frequency is not biased.
Systematic sampling involves taking samples according to some systematic rule – e.g.,
every fourth unit, the first five units every hour, etc. One danger of using systematic
sampling is that the systematic rule may match some underlying structure and bias the
sample.
For example, the manager of a billing center is using systematic sampling to monitor
processing rates. At random times around each hour, five consecutive bills are selected
and the processing time is measured.
Rational Subgrouping
Rational subgrouping is the process of putting measurements into meaningful groups to
better understand the important sources of variation. Rational subgrouping is typically
used in process sampling situations when data is collected in real time during process
operations. It involves grouping measurements produced under similar conditions,
sometimes called short-term variation. This type of grouping assists in understanding the
sources of variation between subgroups, sometimes called long-term variation.
The goal should be to minimize the chance of special causes in variation in the subgroup
and maximize the chance for special causes between subgroups. Subgrouping over time
is the most common approach; subgrouping can be done by other suspected sources of
variation (e.g., location, customer, supplier, etc.)
For example, an equipment leasing business was trying to improve equipment turnaround
time. They selected five samples per day from each of three processing centers. Each
processing center was formed into a subgroup.
When using subgrouping, form subgroups with items produced under similar conditions.
To ensure items in a subgroup were produced under similar conditions, select items
produced close together in time.
Determining Sample Size
This article focused on basic sampling strategies. An analyst must determine which
strategy applies to a particular situation before determining how much data is required for
the sample. Depending on the question the analyst wants to answer, the amount of sample
data needed changes. The analyst should collect enough baseline data to capture an entire
iteration (or cycle) of the process.
An iteration should account for the different types of variation seen within the process,
such as cycles, shifts, seasons, trends, product types, volume ranges, cycle time ranges,
demographic mixes, etc. If historical data is not available, a data collection plan should
be instituted to collect the appropriate data.
Factors affecting sample size include:
How confident (accurate) the analyst wants to be that the sample will provide a good
estimate of the true population mean. The more confidence required, the greater the
sample size needed.
How close (precision) to the “truth” the analyst wants to be. For more precision, a greater
sample size is needed.
How much variation exists or is estimated to exist in the population. If there is more
variation, a greater sample size is needed.
Sample size calculators are available to make the determination of sample size much
easier; it is best, however, that an analyst consults with a Master Black Belt and/or Black
Belt coach until he or she is comfortable with determining sample size.
7) ATTITUDE MEASUREMENTS AND SCALING
8) TYPES OF MEASUREMENTS
Measurement
Types of Measurements
Measurement is the assigning of numbers or codes according to prior-set rules.* It is
how we get the numbers upon
which we perform statistical operations. There are many ways to classify measurements.
SPSS classifies
measurements as either nominal, ordinal, or scale.
Nominal variables are named categories or attributes. For example, SEX (male or
female) is a nominal variable, and so
is NAME and EYECOLOR. Nominal variables are often referred to as qualitative
variables or categorical variables.
Ordinal variables are rank-ordered characteristics and responses. For example, an
OPINION graded 5 = strongly
agree, 4 = agree, 3 = undecided, 2 = disagree, 1 = strongly disagree is ordinal. Notice that
ordinal categories can be
put in ascending or descending order, but difference (“distances”) between possible
responses are uneven. For
example, the difference between 5 (strongly agree) and 4 (agree) is not the same as the
difference between a 4 (agree)
and 3 (undecided).
Scale variables represent quantitative measurements in which differences between
possible responses are uniform.
For example, LENGTH (measured in inches) is a scale measurement since the difference
between 2 inches and 1 inch is
the same as the difference between 3 inches and 2 inches. Scale variables are also called
quantitative variables and
continuous variables.
Notice that each step up the measurement scale hierarchy takes on the assumptions of the
step below it and then
adds another restriction. That is, nominal variables are named categories, ordinal
variables are named categories that
can be put in logical order, and scale variables are ordinal variables that have equal
distances between ordered
responses.
Comment: Although distinctions between measurement scales appears to be
straightforward, the lines between
them occasionally get blurred. For example, an IQ score is usually considered to be a
scale variable. However,
strictly speaking, we have no assurance that the difference between an IQ of 70 and 80,
for instance, means the
same as the difference between an IQ of 80 and 90. Therefore, IQ scores can be thought
of as either ordinal or
scale. In fact, some statisticians believe that adhering to any type of measurement
taxonomy can be misleading,
and discourage the use of any single scheme.†
9) CRITERIA OF GOOD MEASUREMENTS
Now that we have seen how to operationally define variables, it is important to make
sure that the instrument that we develop to measure a particular concept is indeed
accurately measuring the variable, and in fact, we are actually measuring the concept that
we set out to measure. This ensures that in operationally defining perceptual and
attitudinal variables, we have not over looked some important dimensions and elements
or included some irrelevant ones. The scales developed could often be
imperfect and errors are prone to occur in the measurement of attitudinal variables. The
use of better instruments will ensure more accuracy in results, which in turn, will enhance
the scientific quality of the research. Hence, in some way, we need to assess the
"goodness" of the measure developed. What should be the characteristics of a good
measurement? An intuitive answer to this question is that the tool should be an accurate
indicator of what we are interested in measuring. In addition, it should be
easy and efficient to use. There are three major criteria for evaluating a measurement tool
validity, reliability, and sensitivity.
Validity
Validity is the ability of an instrument (for example measuring an attitude) to measure
what it is supposed to measure. That is, when we ask a set of questions (i.e. develop a
measuring instrument)with the hope that we are tapping the concept, how can we be
reasonably certain that we are indeed measuring the concept we set out to do and not
something else? There is no quick answer.
Researchers have attempted to assess validity in different ways, including asking
questions such as "Is there consensus among my colleagues that my attitude scale
measures what it is supposed to measure?" and "Does my measure correlate with others'
measures of the `same' concept?" and "Does the behavior expected from my measure
predict the actual observed behavior?" Researchers expect the answers to provide some
evidence of a measure's validity. What is relevant depends on the nature of the research
problem and the researcher's judgment. One way to approach this question is to organize
the answer according to measure-relevant types of validity. One widely accepted
classification consists of three major types of validity: (1) content validity, (2) criterion-
related validity, and (3) construct validity.
(1) Content Validity
The content validity of a measuring instrument(the composite of measurement scales) is
the extent to which it provides adequate coverage of the investigative questions guiding
the study. If the instrument contains a representative sample of the universe of subject
matter of interest, then the content validity is good. To evaluate the content validity of an
instrument, one must first agree on what dimensions and elements constitute adequate
coverage. To put it differently, content validity is a function of how well
the dimensions and elements of a concept have been delineated. Look at the concept of
feminism which implies a person's commitment to a set of beliefs creating full equality
between men and women in areas of the arts, intellectual pursuits, family, work, politics,
and authority relations. Does this definition provide adequate coverage of the different
dimensions of the concept? Then we have the following two questions to measure
feminism:
1. Should men and women get equal pay for equal work?
2. Should men and women share house hold tasks?
These two questions do not provide coverage to all the dimensions delineated earlier. It
definitely falls short of adequate content validity for measuring feminism. A panel of
persons to judge how well the instrument mets the standard can attest to the content
validity of the instrument. A panel independently assesses the test items for a
performance test. It judges each item to be essential, useful but not essential, or not
necessary in assessing performance of a relevant behavior.
Face validity is considered as a basic and very minimum index of content validity. Face
validity indicates that the items that are intended to measure a concept, do on the face of
it look like they measure the concept. For example a few people would accept a measure
of college student math ability using a question that asked students: 2 + 2 = ? This is not a
valid measure of college-level math ability on the face of it. Nevertheless, it is a
subjective agreement among professionals that a scale logically appears to reflect
accurately what it is supposed to measure. When it appears evident to experts that the
measure provides adequate coverage of the concept, a measure has face validity.
(2)Criterion-Related Validity
Criterion validity uses some standard or criterion to indicate a construct accurately. The
validity of an indicator is verified by comparing it with another measure of the same
construct in which research has confidence. There are two subtypes of this kind of
validity.
Concurrent validity: To have concurrent validity, an indicator must be associated with a
preexisting indicator that is judged to be valid. For example we create a new test to
measure intelligence. For it to be concurrently valid, it should be highly associated with
existing IQ tests (assuming the same definition of intelligence is used). It means that most
people who score high on the old measure should also score high on the new one, and
vice versa. The two measures may not be perfectly associated, but if they
measure the same or a similar construct, it is logical for them to yield similar results.
Predictive validity:
Criterion validity whereby an indicator predicts future events that are logically related to
a construct is called a predictive validity. It cannot be used for all measures. The measure
and the action predicted must be distinct from but indicate the same construct. Predictive
measurement validity should not be confused with prediction in hypothesis testing, where
one variable predicts a different variable in future.
Look at the scholastic assessment tests being given to candidates seeking admission in
different subjects. These are supposed to measure the scholastic aptitude of the
candidates the ability to perform in institution as well as in the subject. If this test has
high predictive validity, then candidates who get high test score will subsequently do well
in their subjects. If students with high scores perform the same as students with average
or low score, then the test has low predictive validity.
(3) Construct Validity
Construct validity is for measures with multiple indicators. It addresses the question: If
the measure is valid, do the various indicators operate in consistent manner? It requires a
definition with clearly specified conceptual boundaries. In order to evaluate construct
validity, we consider both theory and the measuring instrument being used. This is
assessed through convergent validity and discriminant validity.
Convergent Validity: This kind of validity applies when multiple indicators converge or
are associated with one another. Convergent validity means that multiple measures of the
same construct hang together or operate in similar ways. For example, we construct
"education" by asking people how much education they have completed, looking at their
institutional records, and asking people to complete a test of school level knowledge. If
the measures do not converge (i.e. people who claim to have college
Degree but have no record of attending college, or those with college degree perform no
better than high school dropouts on the test), then our test has weak convergent validity
and we should not combine all three indicators into one measure.
Discriminant Validity: Also called divergent validity, discriminant validity is the
opposite of convergent validity. It means that the indicators of one construct hang
together or converge, but also diverge or are negatively associated with opposing
constructs. It says that if two constructs A and B are very different, then measures of A
and B should not be associated .For example, we have 10 items that measure political
conservatism. People answer all 10 in similar ways. But we have also put 5 questions in
the same questionnaire that measure political liberalism. Our measure of conservatism
has discriminant validity if the 10 conservatism items hang together and are negatively
associated with 5 liberalism ones.
Reliability
There liability of a measure indicates the extent to which it is with outbid as (error free)
and hence ensures consistent measurement across time and across the various items in the
instrument. In other words, the reliability of a measure is an indication of the stability and
consistency with which the instrument measures the concept and helps to assess the
`goodness" of measure.
Stability of Measures
The ability of the measure to remain the same over time despite uncontrollable testing
conditions or the state of the respondents themselves is indicative of its stability and low
vulnerability to changes in the situation. This attests to its "goodness" because the
concept is stably measured, no matter when it is done. Two tests of stability are test-retest
reliability and parallel-form reliability.
(1)Test-retest Reliability: Test-retest method of determining reliability involves
administering the same scale to the same respondents at two separate times to test for
stability. If the measure is stable overtime, the test, administered under the same
conditions each time, should obtain similar results. For example, suppose a researcher
measures job satisfaction and finds that 64 percent of the population is satisfied with their
jobs. If the study is repeated a few weeks later under similar conditions, and the
Researcher again finds that 64 percent of the population is satisfied with their jobs, it
appears that the measure has repeatability. The high stability correlation or consistency
between the two measures at time 1 and at time 2 indicates high degree of reliability. This
was at the aggregate level; the same exercise can be applied at the individual level. When
the measuring instrument produces unpredictable results from one testing to the next, the
results are said to be unreliable because of error in measurement. There are two problems
with measures of test-retest reliability that are common to all longitudinal studies. Firstly,
the first measure may sensitize the respondents to their participation in a research project
and subsequently influence the results of the second measure. Further if the time between
the measures is long, there may be attitude change or other maturation of the subjects.
Thus it is possible for a reliable measure to indicate low or moderate correlation between
the first and the second administration, but this low correlation may be due an attitude
change over time rather than to lack of reliability.
(2)Parallel-Form Reliability: When responses on two comparable sets of measures
tapping the same construct are highly correlated, we have parallel-form reliability. It is
also called equivalent-form reliability. Both forms have similar items and same response
format, the only changes being the wording and the order or sequence of the questions.
What we try to establish here is the error variability resulting from wording and ordering
of the questions. If two such comparable forms are highly correlated, we may be fairly
certain that the measures are reasonably reliable, with minimal error variancecaused by
wording, ordering, or other factors.
Internal Consistency of Measures
Internal consistency of measures is indicative of the homogeneity of the items in the
measure that tap the construct. In other words, the items should `hang together as a set,'
and be capable of independently measuring the same concept so that the respondents
attach the same overall meaning to each of the items. This can be seen by examining if
the items and the subsets of items in the measuring instrument are highly correlated.
Consistency can be examined through the inter-item consistency reliability and split-half
reliability.
(1)Inter-item Consistency reliability: This is a test of consistency of respondents'
answers to all the items in a measure. To the degree that items are independent measures
of the same concept, they will be correlated with one another.
(2)Split-Half reliability: Split half reliability reflects the correlations between two halves
of an instrument. The estimates could vary depending on how the items in the measure
are split into two halves. The technique of splitting halves is the most basic method for
checking internal consistency when measures contain a large number of items. In the
split-half method the researcher may take the results obtained from one half of the scale
items (e.g. odd-numbered items) and check them against the
Results from the other half of the items (e.g. even numbered items). The high correlation
tells us there is similarity (or homogeneity) among its items. It is important to note that
reliability is a necessary but not sufficient condition of the test of goodness of
a measure. For example, one could reliably measure a concept establishing high stability
and consistency, but it may not be the concept that one had set out to measure. Validity
ensures the ability of a scale to measure the intended concept.
Sensitivity
The sensitivity of a scale is an important measurement concept, particularly when
changes in attitudes or other hypothetical constructs are under investigation. Sensitivity
refers to an instrument's ability to accurately measure variability in stimuli or responses.
A dichotomous response category, such as "agree or disagree," does not allow the
recording of subtle attitude changes. A more sensitive measure, with numerous items on
the scale, may be needed. For example adding "strongly agree,""mildly agree,""neither
agree nor disagree," "mildly disagree," and "strongly disagree" as categories increases a
scale's sensitivity. The sensitivity of a scale based on a single question or single item can
also be increased by adding additional questions or items. In other words, because index
measures allow for greater range of possible scores, they are more sensitive than single
item.
Practicality: The scientific requirements of a project call for the measurement process to
be reliable and valid, while the operational requirements call for it to be practical.
Practicality has been defined as economy, convenience, and interpretability
10) CLASSIFICATION OF SCALES
1.1 Measurement Scales: Traditional Classification
Statisticians call an attribute on which observations differ a variable. The type of unit on
which a variable is measured is called a scale. Traditionally, statisticians talk of four
types of measurement scales: (1) nominal, (2) ordinal, (3) interval, and (4) ratio.
1.1.1 Nominal Scales
The word nominal is derived from no men, the Latin word for name. Nominal scales
merely name differences and are used most often for qualitative variables in which
observations are classified into discrete groups. The key attribute for a nominal scale is
that there is no inherent quantitative difference among the categories. Sex, religion, and
race are three classic nominal scales used in the behavioral sciences. Taxonomic
categories (rodent, primate, canine) are nominal scales in biology. Variables on a
nominal scale are often called categorical variables.
1.1.2 Ordinal Scales
Ordinal scales rank-order observations. Class rank and horse race results are examples.
There are two salient attributes of an ordinal scale. First, there is an underlying
quantitative measure on which the observations differ. For class rank, this underlying
quantitative attribute might be composite grade point average, and for horse race results it
would be time to the finish line. The second attribute is that individual differences
individual on the underlying quantitative measure are either unavailable or
ignored. As a result, ranking the horses in a race as 1st, 2nd, 3rd, etc. hides the
information about whether the first-place horse won by several lengths or by a nose.
There are a few occasions in which ordinal scales may be preferred to using a
quantitative index of the underlying scale. College admission officers, for example, favor
class rank to overcome the problem of the different criteria used by school districts in
calculating GPA. In general, however, measurement of the underlying quantitative
dimension is preferred to rank-ordering observations because the resulting scale has
greater statistical power than the ordinal scale.
1.1.3 Interval Scales
In ordinal scales, the interval between adjacent values is not constant. For example, the
difference in finishing time between the 1st place horse and the 2nd horse need not the
same as that between the 2nd and 3rd place horses. An interval scale has a constant
interval but lacks a true 0 point. As a result, one can add and subtract values on an
interval scale, but one cannot multiply or divide units. Temperature used in day-to-day
weather reports is the classic example of an interval scale. The assignment of the number
0 to a particular height in a column of mercury is an arbitrary convenience apparent to
everyone anyone familiar with the difference between the Celsius and Fahrenheit scales.
As a result, one cannot say that 30oC is twice as warm as 15o C because that statement
involved implied multiplication. To convince yourself, translate these two into Fahrenheit
and ask whether 86o F is twice as warm as 50o F. Nevertheless, temperature has constant
intervals between numbers, permitting one to add and subtract. The difference between
28o C and 21o C is 7 Celsius units as is the difference between 53o C and 46o C. Again,
convert these to Fahrenheit and ask whether the difference between 82.4o F and 69.8o F
is the same in Fahrenheit units as the difference between 127.4o F and 114.8o F?
1.1.4 Ratio Scales
A ratio scale has the property of equal intervals but also has a true 0 point. As a result,
one can multiply and divide as well as add and subtract using ratio scales. Units of time
(msec, hours), distance and length (cm, kilometers), weight (mg, kilos), and volume (cc)
are all ratio scales. Scales involving division of two ratio scales are also themselves ratio
scales. Hence, rates (miler per hour) and adjusted volumetric measures (mg/dL) are ratio
scales. Note that even though a ratio scale has a true 0 point, it is possible that the nature
of the variable is such that a value of 0 will never be observed. Human height is measured
on a ratio scale but every human has a height greater than 0. Because of the multiplicative
property of ratio scales, it is possible to make statements that 60 mg of fluoexetine is
three times as great as 20 mg.
Unit 1: Introduction to Research
1. WHAT IS RESEARCH?
This module considers the role, purpose, structure and process of research. It aims to
answer the following questions:
What is research?
Why do research?
What types of research are there?
What ethical considerations are there when conducting research?
How might research findings be used?
2. RESEARCH IS A SIGN OF INTELLIGENCE
Intelligence can be defined as the adaptation of an environment to suit needs, which is
why humans can be acknowledged as the most 'intelligent' of species.
Humans observe, identify, plan and then effect change. Humans have social gain through
information as well as resource sharing.
As apart from any other species, humans have complex language structures and the
written word to share information from one person to another. Literate societies with well
structured, permanent means of communicating information have immense evolutionary
advantage.
3. WE RESEARCH EVERYDAY
Humans are 'intuitive' scientists ....always asking questions and testing theories about
themselves, others, events, the environment and the world around them.
Research is asking a question and finding out the answer.....
It is looking into something.
It is looking for something.
It is comparing and contrasting things.
It is finding out more information...it is counting things ...making enquiries...being
curious...finding out what people think...finding out what people do....finding out what
works.... finding out what doesn't work...finding out what people want...
What research have you conducted recently?
What decisions have you made about your day?
What decisions have you made today?
What influenced your decision to take this course?
How do you prepare and write assignments?
How do you decide how to provide the best quality of service for your service users?
We all engage in or do social research as we act on the basis and results of our own
research and theorising, therefore, what we think affects the way we behave....
4. WHAT DO WE RESEARCH?
We research people and their behaviour, opinions, attitudes, trends and patterns, also
politics, animals, health and illness. Research can be conducted either informally for our
own benefit, through asking questions, watching, counting or reading and formally, for
medical or academic purposes, as a marketing strategy, to inform and influence politics
and policy.
Research may be carried out in our own lives, through the media, in our place of work,
with our friends and family or through reading past research.
Our views - personal, social, community and worldwide and our own identities are
socially constructed through our own the orising.
5. WHAT DOES RESEARCH TELL US?
Research gives us information about: Thoughts and opinions, Attitudes, Habits, Culture,
Norms Scientific facts Medical information
What do we do with research?
Have it as interesting fact , Use it to make decisions , Use it to persuade influence others ,
Use it to affect change , Use it to change behaviour , Use it to better use...medical
...improve customer care...write better funding applications....monitor and evaluate our
provision...., We research in order to understand society and social processes, as well as
to test and or create theories in order that we are better able to inform about social action
and potentially 'improve' social conditions.
6. KNOWLEDGE,
Interpretation and dissemination, Research involves gaining knowledge, interpreting data
and disseminating the findings.
Gathering data from direct and indirect sources:, observations, questionnaires,
interviews, experiments, other research
Processing data for interpretation numerically and or verbally:, statistics, themes or
perspectives
Dissemination of findings, written reports , presentations, seminars, supply to media
7. WHEN WE CONDUCT RESEARCH, IT SHOULD BE...,
Systematic, Non-discriminatory, Open to criticism, Independent and free from and direct
and or indirect censors
8. RESEARCH THEORY-
Research is approached in a variety of ways...in its methods, analysis and
presentation...which may be influenced by the theoretical approach the researcher takes.
The appendix of "Research theory" offers a brief introduction to some of the theoretical
positions as well as some links which you can use to research further

Research Methodology Module-04

  • 1.
    Module 4RM Data collection,Measuring, Sampling and Scaling 1 CLASSIFICATION OF DATA The process of arranging data into homogenous group or classes according to some common characteristics present in the data is called classification. For Example: The process of sorting letters in a post office, the letters are classified according to the cities and further arranged according to streets. 2 BASES OF CLASSIFICATION: There are four important bases of classification: (1) Qualitative Base (2) Quantitative Base (3) Geographical Base (4) Chronological or Temporal Base (1) Qualitative Base: When the data are classified according to some quality or attributes such as sex, religion, literacy, intelligence etc… (2) Quantitative Base: When the data are classified by quantitative characteristics like heights, weights, ages, income etc… (3) Geographical Base: When the data are classified by geographical regions or location, like states, provinces, cities, countries etc… (4) Chronological or Temporal Base: When the data are classified or arranged by their time of occurrence, such as years, months, weeks, days etc… For Example: Time series data. 3 TYPES OF CLASSIFICATION: (1) One -way Classification: If we classify observed data keeping in view single characteristic, this type of classification is known as one-way classification.
  • 2.
    For Example: Thepopulation of world may be classified by religion as Muslim, Christians etc… (2) Two -way Classification: If we consider two characteristics at a time in order to classify the observed data then we are doing two way classifications. For Example: The population of world may be classified by Religion and Sex. (3) Multi -way Classification: We may consider more than two characteristics at a time to classify given data or observed data. In this way we deal in multi-way classification. For Example: The population of world may be classified by Religion, Sex and Literacy. 4 BENEFITS AND DRAWBACKS OF DATA Primary data is data compiled from first-hand experience or observation. Advantages of primary data include, the data is unbiased, the data is original and the data is basic. Disadvantages of primary data include, gathering the data is time-consuming, the data is raw and there can be too much data. Primary Sources of Data- Primary sources are original sources from which the researcher directly collects data that have not been previously collected e.g.., collection of data directly by the researcher on brand awareness, brand preference, brand loyalty and other aspects of consumer behaviour from a sample of consumers by interviewing them,. Primary data are first hand information collected through various methods such as observation, interviewing, mailing etc. Advantage of Primary Data- It is original source of data ·It is possible to capture the changes occurring in the course of time. ·It flexible to the advantage of researcher. ·Extensive research study is based of primary data Disadvantage of Primary Data 1. Primary data is expensive to obtain 2. It is time consuming 3. It requires extensive research personnel who are skilled 4. It is difficult to administer.
  • 3.
    1. Secondary datahave several advantages Data assembly is inexpensive. Company records, trade journals, and government publications are all rather low-cost. No data collection forms, interviewers, and tabulations are needed. Data can be gathered quickly. Company records, library sources, and Web sites can be accessed immediately. Many firms store reports in their retail information systems connected with their pos equipment. There may be several sources of secondary data--with many perspectives. A secondary source may possess information that would otherwise be unavailable to the retailer. Government publications often have statistics no private firm could acquire. When data are assembled by a source such as Progressive Grocer, A.C. Nielsen, Stores, or the government, results are usually quite credible. The retailer may have only a rough idea of the topics to investigate. Secondary data can then help to define issues more specifically. In addition, background information about a given issue can be gathered from secondary sources before undertaking a primary study. 2. Secondary data also have several potential disadvantages Available data may not suit the purposes of the current study because they have been collected for other reasons. Neighborhood statistics may not be found in secondary sources. Secondary data may be incomplete. Information may be dated. The accuracy of secondary data must be carefully evaluated. Some secondary data sources are known for poor data collection techniques; they should be avoided. In retailing, many secondary data projects are not retested and the user of secondary data has to hope results from one narrow study are applicable to his or her firm. 3) Evaluation of data How will the data be analyzed and presented in order to address the key evaluation questions?
  • 4.
    According to Owen'smodel, the second part of data management is analysis and reporting, which has three components; data reduction, data display and conclusion drawing and verification. The general data analysis process in evaluation is one of reduction- that is, 'the process of simplifying and transforming the raw information according to some logical set of procedures or rules' (Owen 2006: 101). There is a wide range of processes for data reduction for both quantitative and qualitative information. The processes used must be explicitly described when reporting the data analysis results. There are two general purposes for data analysis in evaluation; description and explanation. Both are important because description enables the audience to understand the project, its intended processes and outcomes, and the extent to which these were achieved, whereas explanation provides evidence about the underlying logic of the project and the extent to which it is sustainable, transferable and/or reproducible. The display of data is a process of organising the information in ways that lead to the drawing of explicit and defensible conclusions about the key evaluation questions. In many evaluations, conclusions are the endpoint, however in others, the evaluators go further to offer recommendations about the project. The former require placing values on the conclusions such as stating the project is successful or not, whereas, the latter are advice or suggestions for courses of action made to decision makers. What ethical issues are involved in the evaluation and how will they addressed? Ethical issues often arise in the data management process described above e.g. in the selection of data sources, obtaining the information or reporting results. Data collection activities in ALTC funded projects will usually require approval from the lead institution’s research ethics committee, and may require complementary approval from partner institutions. The main issues which are likely to arise include appropriate methods of collecting, analysing, storing and reporting data from students to protect their confidentiality and anonymity, ensuring students and staff are not impacted unfairly by the evaluation activities (avoiding interruptions to the learning and teaching processes), and unfairly disadvantaging students who are not receiving the project benefits. The Australasian Evaluation Society has produced a Code of Conduct and a set of Guidelines for the Ethical Conduct of Evaluations, which provide useful guidance for evaluation activities Qualitative methods of data collection, Methods of collecting qualitative data, Data collection approaches for qualitative research usually involves:, Direct interaction with individuals on a one to one basis , Or direct interaction with individuals in a group setting Qualitative research data collection methods are time consuming, therefore data is usually collected from a smaller sample than would be the case for quantitative approaches - therefore this makes qualitative research more expensive.
  • 5.
    The benefits ofthe qualitative approach is that the information is richer and has a deeper insight into the phenomenon under study The main methods for collecting qualitative data are: Individual interviews, Focus , groups Observations, Action Research 4. Interviews Interviews can be- Unstructured Can be referred to as 'depth' or 'in depth' interviews. They have very little structure at all. The interviewer may just go with the aim of discussing a limited number of topics, sometimes as few as just one or two. The interviewer may frame the interview questions based on the interviewee and his/her previous response This allows the discussion to cover areas in great detail They involve the researcher wanting to know or find out more about a specific topic without there being a structure or a preconceived plan or expectation as to how they will deal with the topic Semi structured-Semi structured interviews are sometimes also called focused interviews A series of open ended questions based on the topic areas the researcher wants to cover A series of broad questions to ask and may have some prompts to help the interviewee 'The open ended nature of the question defines the topic under investigation but provides opportunities for both interviewer and interviewee to discuss some topics in more detail' Semi structured interviews allow the researcher to promt or encourage the interviewee if they are looking for more information or find what they are saying interesting This method gives the researcher the freedom to probe the interviewee to elaborate or to follow a new line of inquiry introduced by what the interviewee is saying Work best when the interviewed has a number of areas he/she wants to be sure to be addressing Structured The interviewed asks the respondent the same questions in the same way A tightly structured schedule is used The questions may be phrased in order that a limited range of responses may be given - i.e. 'Do you rate our services as very good, good or poor' A researcher needs to consider whether a questionnaire or structured interview is more appropriate 'If the interview schedule is too tightly structured this may not enable the phenomena under investigation to be explored in terms of either breadth or depth.'
  • 6.
    Qualitative interviews shouldbe fairly informal and participants feel they are taking part in a conversation or discussion rather than in a formal question and answer situation. There is skill required and involved in successful qualitative research approaches - which requires careful consideration and planning Good quality qualitative research involves: Thought Preparation The development of the interview schedule Conducting and analysing the interview data with care and consideration 5. Focus groups -The use of focus groups is sometimes used when it is better to obtain information from a group rather than individuals. Group interviews can be used when: Limited resources (time, manpower, finances) The phenomena being researched requires a collective discussion in order to understand the circumstances, behaviour or opinions Greater insights may be developed of the group dynamic - or cause and consequence Characteristics of a focus group: Recommended size of the sample group is 6 - 10 people as smaller groups may limit the potential on the amount of information collected, and more may make it difficult for all participants to participate and interact and for the interviewer to be able to make sense of the information given Several focus groups should be used in order to get a more objective and macro view of the investigation. i.e. focussing on one group may give you idiosyncratic results. The use of several groups will add to the breadth and depth of information. A minimum of three focus groups is recommended for best practice approaches Members of the focus group should have something in common which is important to the investigation Groups can either be put together or existing groups - it is always useful to be mindful of the group dynamics of both situations. The aim of the focus group is to make use of participants' feelings, perceptions and opinions This method requires the researcher to use a range of skills: group skills, facilitating, moderating, listening/observing, analysis 6. Observation Observation involves may take place in natural settings and involve the researcher taking lengthy and descriptive notes of what is happening. It is argued that there are limits to the situations that can be observed in their 'natural' settings and that the presence of the research may lead to problems with validity. Limitations with observation include: Change in people's behaviour when they know they are being observed A 'snap shot' view of a whole situation Think Big Brother...
  • 7.
    The researcher maymiss something while they are watching and taking notes The researcher may make judgements of make value statements or misunderstand what has been observed Strengths of observation Can offer a flavour for what is happening Can give an insight into the bigger picture Can demonstrate sub-groups Can be used to assist in the design of the rest of the research Sometimes, the researcher becomes or needs to become a participant observer, where they are taking part in the situation in order to be accepted and further understand the workings of the social phenomenon. Observation can sometimes obtain more reliable information about certain things - for example, how people actually behave (although it may not find out the reasons for why they behave in a particular way). Observation can also serve as a technique for verifying of nullifying information provided in face to face encounters.' People or environment can be observed. When environment is researched, it can provide valuable background information that may inform other aspects of the research. Techniques for collecting data through observation Written descriptions The researcher makes written descriptions of the people, situations or environment Limitations include Researcher might miss out on an observation as they are taking notes The researcher may be focussed on a particular event or situation There is room for subjective interpretation of what is happening Video recording Allows the researcher to also record notes
  • 8.
    Limitations may includepeople acting unnaturally towards the camera or others avoiding the camera The camera may not always see everything Photographs and artefacts Useful when there is a need to collect observable information or phenomena such as buildings, neighbourhoods, dress and appearance Artefacts include objects of significance - memorabilia, instruments, tools etc Documentation Any and all kinds of documentation may be used to provide information - a local paper, information on a notice board, administrative policies and procedures...etc previous research, even 7. Self Study Consider an area within your work that you might want to observe in order to get an answer, find out more or gain a better understanding. Think about and plan: What your aim/purpose is. What permission, etc, you may need to gain. What your role/presence will be. How you will record your observation. What you will record. What you will do with your findings. What are the pros and cons of this process? 5) METHODS OF QUALITATIVE RESEARCH What is qualitative research? Qualitative research is a type of scientific research. In general terms, scientific research consists Of an investigation that:
  • 9.
    • seeks answersto a question • Systematically uses a predefined set of procedures to answer the question • collects evidence • produces findings that were not determined in advance • produces findings that are applicable beyond the immediate boundaries of the study Qualitative research shares these characteristics. Additionally, it seeks to understand a given Research problem or topic from the perspectives of the local population it involves. Qualitative Research is especially effective in obtaining culturally specific information about the values, opinions, behaviors, and social contexts of particular populations. What are some qualitative research methods? The three most common qualitative methods, explained in detail in their respective modules, are participant observation, in-depth interviews, and focus groups. Each method is particularly suited for obtaining a specific type of data. • Participant observation is appropriate for collecting data on naturally occurring behaviors in their usual contexts. • In-depth interviews are optimal for collecting data on individuals’ personal histories, perspectives, and experiences, particularly when sensitive topics are being explored. • Focus groups are effective in eliciting data on the cultural norms of a group and in generating broad overviews of issues of concern to the cultural groups or subgroups represented.
  • 10.
    6) SAMPLING: SAMPLINGSTRATEGY, SAMPLE SIZE Information is not readily found at a bargain price. Gathering it is costly in terms of salaries, expenses and time. Taking samples of information can help ease these costs because it is often impractical to collect all the data. Sound conclusions can often be drawn from a relatively small amount of data; therefore, sampling is a more efficient way to collect data. Using a sample to draw conclusions is known as statistical inference. Making inferences is a fundamental aspect of statistical thinking. Selecting the Most Appropriate Sampling Strategy There are four primary sampling strategies: Random sampling Stratified random sampling Systematic sampling Rational sub-grouping Before determining which strategy will work best, the analyst must determine what type of study is being conducted. There are normally two types of studies: population and process. With a population study, the analyst is interested in estimating or describing some characteristic of the population (inferential statistics). With a process study, the analyst is interested in predicting a process characteristic or change over time. It is important to make the distinction for proper selection of a sampling strategy. The “I Love Lucy” television show’s “Candy Factory” episode can be used to illustrate the difference. For example, a population study, using samples, would seek to determine the average weight of the entire daily run of candies. A process study would seek to know whether the weight was changing over the day. Random Sampling Random samples are used in population sampling situations when reviewing historical or batch data. The key to random sampling is that each unit in the population has an equal probability of being selected in the sample. Using random sampling protects against bias being introduced in the sampling process, and hence, it helps in obtaining a representative sample. In general, random samples are taken by assigning a number to each unit in the population and using a random number table or Minitab to generate the sample list. Absent knowledge about the factors for stratification for a population, a random sample is a useful first step in obtaining samples.
  • 11.
    For example, animprovement team in a human resources department wanted an accurate estimate of what proportion of employees had completed a personal development plan and reviewed it with their managers. The team used its database to obtain a list of all associates. Each associate on the list was assigned a number. Statistical software was used to generate a list of numbers to be sampled, and an estimate was made from the sample. Stratified Random Sampling Like random samples, stratified random samples are used in population sampling situations when reviewing historical or batch data. Stratified random sampling is used when the population has different groups (strata) and the analyst needs to ensure that those groups are fairly represented in the sample. In stratified random sampling, independent samples are drawn from each group. The size of each sample is proportional to the relative size of the group. For example, the manager of a lending business wanted to estimate the average cycle time for a loan application process. She knows there are three types (strata) of loans (large, medium and small). Therefore, she wanted the sample to have the same proportion of large, medium and small loans as the population. She first separated the loan population data into three groups and then pulled a random sample from each group. Systematic Sampling Systematic sampling is typically used in process sampling situations when data is collected in real time during process operation. Unlike population sampling, a frequency for sampling must be selected. It also can be used for a population study if care is taken that the frequency is not biased. Systematic sampling involves taking samples according to some systematic rule – e.g., every fourth unit, the first five units every hour, etc. One danger of using systematic sampling is that the systematic rule may match some underlying structure and bias the sample. For example, the manager of a billing center is using systematic sampling to monitor processing rates. At random times around each hour, five consecutive bills are selected and the processing time is measured. Rational Subgrouping Rational subgrouping is the process of putting measurements into meaningful groups to better understand the important sources of variation. Rational subgrouping is typically used in process sampling situations when data is collected in real time during process operations. It involves grouping measurements produced under similar conditions, sometimes called short-term variation. This type of grouping assists in understanding the sources of variation between subgroups, sometimes called long-term variation.
  • 12.
    The goal shouldbe to minimize the chance of special causes in variation in the subgroup and maximize the chance for special causes between subgroups. Subgrouping over time is the most common approach; subgrouping can be done by other suspected sources of variation (e.g., location, customer, supplier, etc.) For example, an equipment leasing business was trying to improve equipment turnaround time. They selected five samples per day from each of three processing centers. Each processing center was formed into a subgroup. When using subgrouping, form subgroups with items produced under similar conditions. To ensure items in a subgroup were produced under similar conditions, select items produced close together in time. Determining Sample Size This article focused on basic sampling strategies. An analyst must determine which strategy applies to a particular situation before determining how much data is required for the sample. Depending on the question the analyst wants to answer, the amount of sample data needed changes. The analyst should collect enough baseline data to capture an entire iteration (or cycle) of the process. An iteration should account for the different types of variation seen within the process, such as cycles, shifts, seasons, trends, product types, volume ranges, cycle time ranges, demographic mixes, etc. If historical data is not available, a data collection plan should be instituted to collect the appropriate data. Factors affecting sample size include: How confident (accurate) the analyst wants to be that the sample will provide a good estimate of the true population mean. The more confidence required, the greater the sample size needed. How close (precision) to the “truth” the analyst wants to be. For more precision, a greater sample size is needed. How much variation exists or is estimated to exist in the population. If there is more variation, a greater sample size is needed. Sample size calculators are available to make the determination of sample size much easier; it is best, however, that an analyst consults with a Master Black Belt and/or Black Belt coach until he or she is comfortable with determining sample size.
  • 13.
  • 23.
    8) TYPES OFMEASUREMENTS Measurement Types of Measurements Measurement is the assigning of numbers or codes according to prior-set rules.* It is how we get the numbers upon which we perform statistical operations. There are many ways to classify measurements. SPSS classifies measurements as either nominal, ordinal, or scale. Nominal variables are named categories or attributes. For example, SEX (male or female) is a nominal variable, and so is NAME and EYECOLOR. Nominal variables are often referred to as qualitative variables or categorical variables.
  • 24.
    Ordinal variables arerank-ordered characteristics and responses. For example, an OPINION graded 5 = strongly agree, 4 = agree, 3 = undecided, 2 = disagree, 1 = strongly disagree is ordinal. Notice that ordinal categories can be put in ascending or descending order, but difference (“distances”) between possible responses are uneven. For example, the difference between 5 (strongly agree) and 4 (agree) is not the same as the difference between a 4 (agree) and 3 (undecided). Scale variables represent quantitative measurements in which differences between possible responses are uniform. For example, LENGTH (measured in inches) is a scale measurement since the difference between 2 inches and 1 inch is the same as the difference between 3 inches and 2 inches. Scale variables are also called quantitative variables and continuous variables. Notice that each step up the measurement scale hierarchy takes on the assumptions of the step below it and then adds another restriction. That is, nominal variables are named categories, ordinal variables are named categories that can be put in logical order, and scale variables are ordinal variables that have equal distances between ordered responses. Comment: Although distinctions between measurement scales appears to be straightforward, the lines between them occasionally get blurred. For example, an IQ score is usually considered to be a scale variable. However, strictly speaking, we have no assurance that the difference between an IQ of 70 and 80, for instance, means the same as the difference between an IQ of 80 and 90. Therefore, IQ scores can be thought of as either ordinal or
  • 25.
    scale. In fact,some statisticians believe that adhering to any type of measurement taxonomy can be misleading, and discourage the use of any single scheme.† 9) CRITERIA OF GOOD MEASUREMENTS Now that we have seen how to operationally define variables, it is important to make sure that the instrument that we develop to measure a particular concept is indeed accurately measuring the variable, and in fact, we are actually measuring the concept that we set out to measure. This ensures that in operationally defining perceptual and attitudinal variables, we have not over looked some important dimensions and elements or included some irrelevant ones. The scales developed could often be imperfect and errors are prone to occur in the measurement of attitudinal variables. The use of better instruments will ensure more accuracy in results, which in turn, will enhance the scientific quality of the research. Hence, in some way, we need to assess the "goodness" of the measure developed. What should be the characteristics of a good measurement? An intuitive answer to this question is that the tool should be an accurate indicator of what we are interested in measuring. In addition, it should be easy and efficient to use. There are three major criteria for evaluating a measurement tool validity, reliability, and sensitivity. Validity Validity is the ability of an instrument (for example measuring an attitude) to measure what it is supposed to measure. That is, when we ask a set of questions (i.e. develop a measuring instrument)with the hope that we are tapping the concept, how can we be reasonably certain that we are indeed measuring the concept we set out to do and not something else? There is no quick answer. Researchers have attempted to assess validity in different ways, including asking questions such as "Is there consensus among my colleagues that my attitude scale measures what it is supposed to measure?" and "Does my measure correlate with others' measures of the `same' concept?" and "Does the behavior expected from my measure predict the actual observed behavior?" Researchers expect the answers to provide some evidence of a measure's validity. What is relevant depends on the nature of the research problem and the researcher's judgment. One way to approach this question is to organize the answer according to measure-relevant types of validity. One widely accepted classification consists of three major types of validity: (1) content validity, (2) criterion- related validity, and (3) construct validity.
  • 26.
    (1) Content Validity Thecontent validity of a measuring instrument(the composite of measurement scales) is the extent to which it provides adequate coverage of the investigative questions guiding the study. If the instrument contains a representative sample of the universe of subject matter of interest, then the content validity is good. To evaluate the content validity of an instrument, one must first agree on what dimensions and elements constitute adequate coverage. To put it differently, content validity is a function of how well the dimensions and elements of a concept have been delineated. Look at the concept of feminism which implies a person's commitment to a set of beliefs creating full equality between men and women in areas of the arts, intellectual pursuits, family, work, politics, and authority relations. Does this definition provide adequate coverage of the different dimensions of the concept? Then we have the following two questions to measure feminism: 1. Should men and women get equal pay for equal work? 2. Should men and women share house hold tasks? These two questions do not provide coverage to all the dimensions delineated earlier. It definitely falls short of adequate content validity for measuring feminism. A panel of persons to judge how well the instrument mets the standard can attest to the content validity of the instrument. A panel independently assesses the test items for a performance test. It judges each item to be essential, useful but not essential, or not necessary in assessing performance of a relevant behavior. Face validity is considered as a basic and very minimum index of content validity. Face validity indicates that the items that are intended to measure a concept, do on the face of it look like they measure the concept. For example a few people would accept a measure of college student math ability using a question that asked students: 2 + 2 = ? This is not a valid measure of college-level math ability on the face of it. Nevertheless, it is a subjective agreement among professionals that a scale logically appears to reflect accurately what it is supposed to measure. When it appears evident to experts that the measure provides adequate coverage of the concept, a measure has face validity. (2)Criterion-Related Validity Criterion validity uses some standard or criterion to indicate a construct accurately. The validity of an indicator is verified by comparing it with another measure of the same construct in which research has confidence. There are two subtypes of this kind of validity. Concurrent validity: To have concurrent validity, an indicator must be associated with a preexisting indicator that is judged to be valid. For example we create a new test to measure intelligence. For it to be concurrently valid, it should be highly associated with
  • 27.
    existing IQ tests(assuming the same definition of intelligence is used). It means that most people who score high on the old measure should also score high on the new one, and vice versa. The two measures may not be perfectly associated, but if they measure the same or a similar construct, it is logical for them to yield similar results. Predictive validity: Criterion validity whereby an indicator predicts future events that are logically related to a construct is called a predictive validity. It cannot be used for all measures. The measure and the action predicted must be distinct from but indicate the same construct. Predictive measurement validity should not be confused with prediction in hypothesis testing, where one variable predicts a different variable in future. Look at the scholastic assessment tests being given to candidates seeking admission in different subjects. These are supposed to measure the scholastic aptitude of the candidates the ability to perform in institution as well as in the subject. If this test has high predictive validity, then candidates who get high test score will subsequently do well in their subjects. If students with high scores perform the same as students with average or low score, then the test has low predictive validity. (3) Construct Validity Construct validity is for measures with multiple indicators. It addresses the question: If the measure is valid, do the various indicators operate in consistent manner? It requires a definition with clearly specified conceptual boundaries. In order to evaluate construct validity, we consider both theory and the measuring instrument being used. This is assessed through convergent validity and discriminant validity. Convergent Validity: This kind of validity applies when multiple indicators converge or are associated with one another. Convergent validity means that multiple measures of the same construct hang together or operate in similar ways. For example, we construct "education" by asking people how much education they have completed, looking at their institutional records, and asking people to complete a test of school level knowledge. If the measures do not converge (i.e. people who claim to have college Degree but have no record of attending college, or those with college degree perform no better than high school dropouts on the test), then our test has weak convergent validity and we should not combine all three indicators into one measure. Discriminant Validity: Also called divergent validity, discriminant validity is the opposite of convergent validity. It means that the indicators of one construct hang together or converge, but also diverge or are negatively associated with opposing constructs. It says that if two constructs A and B are very different, then measures of A and B should not be associated .For example, we have 10 items that measure political conservatism. People answer all 10 in similar ways. But we have also put 5 questions in
  • 28.
    the same questionnairethat measure political liberalism. Our measure of conservatism has discriminant validity if the 10 conservatism items hang together and are negatively associated with 5 liberalism ones. Reliability There liability of a measure indicates the extent to which it is with outbid as (error free) and hence ensures consistent measurement across time and across the various items in the instrument. In other words, the reliability of a measure is an indication of the stability and consistency with which the instrument measures the concept and helps to assess the `goodness" of measure. Stability of Measures The ability of the measure to remain the same over time despite uncontrollable testing conditions or the state of the respondents themselves is indicative of its stability and low vulnerability to changes in the situation. This attests to its "goodness" because the concept is stably measured, no matter when it is done. Two tests of stability are test-retest reliability and parallel-form reliability. (1)Test-retest Reliability: Test-retest method of determining reliability involves administering the same scale to the same respondents at two separate times to test for stability. If the measure is stable overtime, the test, administered under the same conditions each time, should obtain similar results. For example, suppose a researcher measures job satisfaction and finds that 64 percent of the population is satisfied with their jobs. If the study is repeated a few weeks later under similar conditions, and the Researcher again finds that 64 percent of the population is satisfied with their jobs, it appears that the measure has repeatability. The high stability correlation or consistency between the two measures at time 1 and at time 2 indicates high degree of reliability. This was at the aggregate level; the same exercise can be applied at the individual level. When the measuring instrument produces unpredictable results from one testing to the next, the results are said to be unreliable because of error in measurement. There are two problems with measures of test-retest reliability that are common to all longitudinal studies. Firstly, the first measure may sensitize the respondents to their participation in a research project and subsequently influence the results of the second measure. Further if the time between the measures is long, there may be attitude change or other maturation of the subjects. Thus it is possible for a reliable measure to indicate low or moderate correlation between the first and the second administration, but this low correlation may be due an attitude change over time rather than to lack of reliability. (2)Parallel-Form Reliability: When responses on two comparable sets of measures tapping the same construct are highly correlated, we have parallel-form reliability. It is also called equivalent-form reliability. Both forms have similar items and same response format, the only changes being the wording and the order or sequence of the questions.
  • 29.
    What we tryto establish here is the error variability resulting from wording and ordering of the questions. If two such comparable forms are highly correlated, we may be fairly certain that the measures are reasonably reliable, with minimal error variancecaused by wording, ordering, or other factors. Internal Consistency of Measures Internal consistency of measures is indicative of the homogeneity of the items in the measure that tap the construct. In other words, the items should `hang together as a set,' and be capable of independently measuring the same concept so that the respondents attach the same overall meaning to each of the items. This can be seen by examining if the items and the subsets of items in the measuring instrument are highly correlated. Consistency can be examined through the inter-item consistency reliability and split-half reliability. (1)Inter-item Consistency reliability: This is a test of consistency of respondents' answers to all the items in a measure. To the degree that items are independent measures of the same concept, they will be correlated with one another. (2)Split-Half reliability: Split half reliability reflects the correlations between two halves of an instrument. The estimates could vary depending on how the items in the measure are split into two halves. The technique of splitting halves is the most basic method for checking internal consistency when measures contain a large number of items. In the split-half method the researcher may take the results obtained from one half of the scale items (e.g. odd-numbered items) and check them against the Results from the other half of the items (e.g. even numbered items). The high correlation tells us there is similarity (or homogeneity) among its items. It is important to note that reliability is a necessary but not sufficient condition of the test of goodness of a measure. For example, one could reliably measure a concept establishing high stability and consistency, but it may not be the concept that one had set out to measure. Validity ensures the ability of a scale to measure the intended concept. Sensitivity The sensitivity of a scale is an important measurement concept, particularly when changes in attitudes or other hypothetical constructs are under investigation. Sensitivity refers to an instrument's ability to accurately measure variability in stimuli or responses. A dichotomous response category, such as "agree or disagree," does not allow the recording of subtle attitude changes. A more sensitive measure, with numerous items on the scale, may be needed. For example adding "strongly agree,""mildly agree,""neither agree nor disagree," "mildly disagree," and "strongly disagree" as categories increases a scale's sensitivity. The sensitivity of a scale based on a single question or single item can also be increased by adding additional questions or items. In other words, because index
  • 30.
    measures allow forgreater range of possible scores, they are more sensitive than single item. Practicality: The scientific requirements of a project call for the measurement process to be reliable and valid, while the operational requirements call for it to be practical. Practicality has been defined as economy, convenience, and interpretability 10) CLASSIFICATION OF SCALES 1.1 Measurement Scales: Traditional Classification Statisticians call an attribute on which observations differ a variable. The type of unit on which a variable is measured is called a scale. Traditionally, statisticians talk of four types of measurement scales: (1) nominal, (2) ordinal, (3) interval, and (4) ratio. 1.1.1 Nominal Scales The word nominal is derived from no men, the Latin word for name. Nominal scales merely name differences and are used most often for qualitative variables in which observations are classified into discrete groups. The key attribute for a nominal scale is that there is no inherent quantitative difference among the categories. Sex, religion, and race are three classic nominal scales used in the behavioral sciences. Taxonomic categories (rodent, primate, canine) are nominal scales in biology. Variables on a nominal scale are often called categorical variables. 1.1.2 Ordinal Scales Ordinal scales rank-order observations. Class rank and horse race results are examples. There are two salient attributes of an ordinal scale. First, there is an underlying quantitative measure on which the observations differ. For class rank, this underlying quantitative attribute might be composite grade point average, and for horse race results it would be time to the finish line. The second attribute is that individual differences individual on the underlying quantitative measure are either unavailable or ignored. As a result, ranking the horses in a race as 1st, 2nd, 3rd, etc. hides the information about whether the first-place horse won by several lengths or by a nose. There are a few occasions in which ordinal scales may be preferred to using a quantitative index of the underlying scale. College admission officers, for example, favor class rank to overcome the problem of the different criteria used by school districts in calculating GPA. In general, however, measurement of the underlying quantitative dimension is preferred to rank-ordering observations because the resulting scale has greater statistical power than the ordinal scale.
  • 31.
    1.1.3 Interval Scales Inordinal scales, the interval between adjacent values is not constant. For example, the difference in finishing time between the 1st place horse and the 2nd horse need not the same as that between the 2nd and 3rd place horses. An interval scale has a constant interval but lacks a true 0 point. As a result, one can add and subtract values on an interval scale, but one cannot multiply or divide units. Temperature used in day-to-day weather reports is the classic example of an interval scale. The assignment of the number 0 to a particular height in a column of mercury is an arbitrary convenience apparent to everyone anyone familiar with the difference between the Celsius and Fahrenheit scales. As a result, one cannot say that 30oC is twice as warm as 15o C because that statement involved implied multiplication. To convince yourself, translate these two into Fahrenheit and ask whether 86o F is twice as warm as 50o F. Nevertheless, temperature has constant intervals between numbers, permitting one to add and subtract. The difference between 28o C and 21o C is 7 Celsius units as is the difference between 53o C and 46o C. Again, convert these to Fahrenheit and ask whether the difference between 82.4o F and 69.8o F is the same in Fahrenheit units as the difference between 127.4o F and 114.8o F? 1.1.4 Ratio Scales A ratio scale has the property of equal intervals but also has a true 0 point. As a result, one can multiply and divide as well as add and subtract using ratio scales. Units of time (msec, hours), distance and length (cm, kilometers), weight (mg, kilos), and volume (cc) are all ratio scales. Scales involving division of two ratio scales are also themselves ratio scales. Hence, rates (miler per hour) and adjusted volumetric measures (mg/dL) are ratio scales. Note that even though a ratio scale has a true 0 point, it is possible that the nature of the variable is such that a value of 0 will never be observed. Human height is measured on a ratio scale but every human has a height greater than 0. Because of the multiplicative property of ratio scales, it is possible to make statements that 60 mg of fluoexetine is three times as great as 20 mg.
  • 32.
    Unit 1: Introductionto Research 1. WHAT IS RESEARCH? This module considers the role, purpose, structure and process of research. It aims to answer the following questions: What is research? Why do research? What types of research are there? What ethical considerations are there when conducting research? How might research findings be used? 2. RESEARCH IS A SIGN OF INTELLIGENCE Intelligence can be defined as the adaptation of an environment to suit needs, which is why humans can be acknowledged as the most 'intelligent' of species. Humans observe, identify, plan and then effect change. Humans have social gain through information as well as resource sharing. As apart from any other species, humans have complex language structures and the written word to share information from one person to another. Literate societies with well structured, permanent means of communicating information have immense evolutionary advantage. 3. WE RESEARCH EVERYDAY Humans are 'intuitive' scientists ....always asking questions and testing theories about themselves, others, events, the environment and the world around them. Research is asking a question and finding out the answer..... It is looking into something. It is looking for something. It is comparing and contrasting things. It is finding out more information...it is counting things ...making enquiries...being curious...finding out what people think...finding out what people do....finding out what works.... finding out what doesn't work...finding out what people want... What research have you conducted recently?
  • 33.
    What decisions haveyou made about your day? What decisions have you made today? What influenced your decision to take this course? How do you prepare and write assignments? How do you decide how to provide the best quality of service for your service users? We all engage in or do social research as we act on the basis and results of our own research and theorising, therefore, what we think affects the way we behave.... 4. WHAT DO WE RESEARCH? We research people and their behaviour, opinions, attitudes, trends and patterns, also politics, animals, health and illness. Research can be conducted either informally for our own benefit, through asking questions, watching, counting or reading and formally, for medical or academic purposes, as a marketing strategy, to inform and influence politics and policy. Research may be carried out in our own lives, through the media, in our place of work, with our friends and family or through reading past research. Our views - personal, social, community and worldwide and our own identities are socially constructed through our own the orising. 5. WHAT DOES RESEARCH TELL US? Research gives us information about: Thoughts and opinions, Attitudes, Habits, Culture, Norms Scientific facts Medical information What do we do with research? Have it as interesting fact , Use it to make decisions , Use it to persuade influence others , Use it to affect change , Use it to change behaviour , Use it to better use...medical ...improve customer care...write better funding applications....monitor and evaluate our provision...., We research in order to understand society and social processes, as well as to test and or create theories in order that we are better able to inform about social action and potentially 'improve' social conditions. 6. KNOWLEDGE, Interpretation and dissemination, Research involves gaining knowledge, interpreting data and disseminating the findings. Gathering data from direct and indirect sources:, observations, questionnaires, interviews, experiments, other research
  • 34.
    Processing data forinterpretation numerically and or verbally:, statistics, themes or perspectives Dissemination of findings, written reports , presentations, seminars, supply to media 7. WHEN WE CONDUCT RESEARCH, IT SHOULD BE..., Systematic, Non-discriminatory, Open to criticism, Independent and free from and direct and or indirect censors 8. RESEARCH THEORY- Research is approached in a variety of ways...in its methods, analysis and presentation...which may be influenced by the theoretical approach the researcher takes. The appendix of "Research theory" offers a brief introduction to some of the theoretical positions as well as some links which you can use to research further