Remya.G
OPERATIONAL
RESEARCH
Dr Remya.G
Guide- Dr Althaf
Contents- Operational Research(OR)
1. Overview
2. History of OR
3. Definition OR
4. Scope of OR
5. Phases of OR
6. OR in public health
7. Societies and journals of OR
8. Other Relevant forms of research
9. Gaps& Challenges
10.Conclusion
Overview-Operational Research
 Operations- The activities carried out in an organization
related to attain its goals and objectives
Kargil Operation
 Research- Any form of systematic and organized
investigation to establish facts
Overview- Operational Research
I need a washing machine
Should I buy it now?
Is it affordable now?
Which company should I go for?
 Decision making is a key part of our daily
life.
 Final decision should be to maximise
benefits and minimise effort and time
Overview - OR
 Operational research is the scientific
study of operations to make better
decisions
 In simple terms OR is described as
“The science of better”
History
 During 2 nd World War how to use the limited
military resources effectively to win the battle
by UK?
 They studied strategic and tactical problems
associated with air and land defense of the
country, and won the war
 This technique was named OR (British Air Ministry
official named A. P. Rowe )
History
 Immediately following World War II, O.R.
grew rapidly
 Scientists realized these principles can
be equally applicable to many problems
in the civilian sector.
History
That ranged from
 Short-term problems such as scheduling
(PERT/CPM)
 Long-term problems such as strategic planning
and resource allocation in various fields
History
 The technique of OR have been applied and
proven in many industries under various
names
Lean ,six sigma in manufacturing
Supply chain in logistics
Yield management in airlines
Definitions
(WHO 2003)
The use of systematic research
techniques for programme decision
making to achieve a specific outcome
Definition OR
OR is also defined as the search for knowledge
on interventions, strategies, or tools that can
improve the quality, effectiveness, or coverage
of programs in which the research is being done
“(Zachariah and colleagues 2009)”
Goals of OR
 Identifying areas that could be
improved
 Solve problems in timely manner
 Strengthen the programme
 Scale up programme
Scope of OR
 Health services & national health programmes
 Public sector- Govt development
 Private sector- Industrial establishment
 National defense service
 Engineering
 Business management
 Agriculture
 Education & training
 Transportation and communication
Phases of OR
Different OR techniques
 Multi- criteria decision analysis
 Simulation technique
 Linear and non linear programming
 Optimization method
 Queuing and stochastic process modeling
 Data - Analysis method
Useful in Public Health
Model Solution
 How believable is the Solution
Design field tests
 Experimental
 Nonexperimental
 Quasi experimental
Implementation of results
 Beauty OR process lies in obtaining, results
which are implementable
o Results of OR technique are information which
helps in making a decision
The OR process
OR in Public Health
“Any research producing practically usable
knowledge (evidence, findings, information,
etc.) which can improve program
implementation (e.g. effectiveness, efficiency,
quality,access, scale up, sustainability)
regardless of the type of research (design,
methodology, approach) falls within the
boundaries of operations research”
Global Meeting Geneva April 2008
How OR differes from Basic Research?
OR
Examines
a system
Systematic
data
collection
Treating
the system
in its real
environme
ntCollaboration
b/w
researchers
and
programme
managers
Deals with
specific
health
issues
Successful
when results
implemented
Basic
Reserach
Not
implementd
in to practice
Focusing
on an
individual Routinely
collected
data can be
used
Deals general
health issues
Research
under
controlled
conditions
May be only
researcher/
s involved
Operation
Research
Models
Techniques
Analysis
Too
complicated?
OR in Public Health
 In management science OR
Systems modeling
Analytical techniques.
 Operational research in global health
does not use these complicated tools
 The simple epidemiological tools and
designs aptly can be applied in health
Operational Research Team
Varies with the type of research.
 public health administrator/Programme
Manager
 Epidemiologist
 Statistician
 Social scientist.
 Field workers
 Clerks, Peons
In health OR - Steps
 Defining the problem
 Collect relevant data- formulate a
hypothesis(Model)
 Derive solution from hypothesis/Model
 Choosing the optimal solution.
 Testing of solution eg : Pilot projects.
 Implementing the solution in the whole system.
Example of developing an OR
 Reports at a health clinic showed opportunistic
infection in HIV positive clients on ART are
risen.
WHY?
Because HIV positive clients are not adhering
to their treatment.
Problem
 Why they are not adhering to their
treatment regimen?
 .
 .Poor communication Between the client and staff
Poor patients can not afford transportation to clinic
Clients cannot afford missing their work at day
Reluctant to visit clinic- Social stigma
Frequent drug shortages
Solutions
 In-service training for staff to improve drug
forecasting.
 Develop clinic based performance standers for
problem solving among staff.
 Here an OR can be designed to find the
effectivenss of both these approaches
Types of OR in Public health
 4 types of operation research (Defined
by Population Council)
1. Exploratory/diagnostic
2. Field intervention
3. Evaluative
4. Cost effectiveness studies.
1. Exploratory/diagnostic
 First step in research.
 Carried out to find out the extent of problem.
 Both Qualitative & Quantitative methods
Interviews cross-sectional
group discussions case control
observations cohort studies
2. Field intervention
Truly randomized,
 Intervention studies
Non randomized
(Quasi-experimental)
3.Monitoring and evaluation
 Monitoring - Looking after inputs, processes and
outputs in programmes.
 Evaluation - Outcome/Impact evaluation
 Important in National Health Programmes
 Data collection on continuous basis
 Implementation of program at different sites is
measured.
4.Economic analysis
 Costing and calculating cost
effectiveness ratios of the interventions
 Less expensive pathways for achieving
optimal outcomes
Operational Researches in India
 Implementation of DOTS by RNTCP.
 Diagnosis and Treatment of Malaria
 Immunisation,Antenatal,Postnatalcare,neo
natal Care – In RCH
 Integration of health system and ICDS
 Joint home visits by AWW and ASHA
 Institutional Deliveries – ‘HUTS’ in
subcentre and PHCs
 PPP
 Decentralisation
Operational Researches in India cont
 Delivery of ORS packets
 Delivery of contraceptives at village level
 Peoples participation in Primary health care
 Alternate model of adolescent health (Teachers
to adolescent, adolescent to teachers)
 Optimal size of area and population to be covered
by a midwife, or basic health unit.
 Ideal vehicle for local health worker.
 Architectural design of hospitals and health
centers.
 Queuing problem in OPD and hospital waiting lists.
 Study of bed load and nursing services in hospitals.
 Length of stay in hospitals, length of sickness
absence.
Operational Researches in India cont
ORs in RNTCP by ICMR
 Increased yield of smear positive
pulmonary TB patients by screening patients
with > 2 weeks cough, compared to > 3
weeks and adequacy of 2 sputum for
diagnosis of sputum positive patients
Aims:
 To assess the yield of sputum positive cases
among CSs with cough of >2 weeks
compared to cough of > 3 weeks
 To compare the efficacy of 2 smear
examination instead of 3 smears, among
CSs
Methods:
 cross sectional multicentric study carried out in
five states.
 Andhra Pradesh, Maharashtra,Orissa, West Bengal
and Rajasthan.
 From each of these states, three districts were
selected.
 A convenient sample of 90 primary and secondary
level health facilities.
 10,000 samples from each state.
Other OR studies in RNTCP
 Implementation of DOTS
 Reliability of involving community volunteers as DOTS
providers
 Involving private practitioners in the RNTCP
 HIV seroprevalence in TB patients.
 MDR TB management
 Assessing relapse among cat 1 patients who have
successfully completed DOTS.
ORs by NVBDCP in Malaria (2012-2017)
 To ensure by 2017 that, at least 80% of
malaria cases in targeted districts receive
prompt and effective treatment as per
national drug policy within 24 hours of first
contact with the health care provider.
 To strengthen the capacity for managing
severe malaria cases and reducing deaths.
ORs by NVBDCP in Malaria
 To ensure that at least 80% of people in high-
risk areas (target areas) sleep under effective
ITNs/ LLINs by 2017.
 To place 80% of the sanctioned staff in target
areas and ensure they are trained in malaria
control.
 To ensure that 100% of districts in target areas
will collect, analyse, and effectively
use routine data and estimate their impact.
Ethical
Informed consent should be received
The use of medical records without informed consent justified When
 There is minimal risk of harm to the individuals.
 Access to the records is essential to achieve the objectives of the
research.
 There is a public benefit to undertaking the research.
 Informed consent is logically or economically impracticable
 There is consent to use the data from the custodian of the records
 The data are protected against those not involved in the research
 The research is approved by an ethics committee
Other Relevant Forms of
Researches
 Action Oriented Research
○ Participatory Action Research
○ Empowerment Research
 Formative research
 Implementation Research
 Health Services Research
Action Oriented Researches
 Research trying to change the system at the
same time generating knowledge.
 Action research may not have any prescribed
methodology.
 Collaboration with non researcher participants.
 Sensitive to needs and perspectives of non
researcher collaborators
Participatory Action Research
= Community Based Participatory Research
CBPR begins with a research topic of
importance to the community with the aim of
combining knowledge and action for social
change to improve community (health).
CBPR
Egs- promotion of sanitary latrines in rural
area(1986)- Demand Driven Approach
Steps followed …..
 Behavioural research promote behavior to use
SL
 Promote low cost and affordable SL- requring
less amount of H2O
 Generate demand for SL
 Involve community/Leaders for popularization
of SL
Empowerment Research
Diarrhoea prevention and managemnt
Continue breast feeding/feeding
Promotion of ORS
Avoid unncessary antibiotis
When women groups and mother groups are
empowered/trained by health workers most
mothers succeeded in preventing diarrhoea.
Formative Research
 When a new intervention is designed
for a relatively new population
 Formative research finds out what will
work for them(needs and acceptability
of clients)
 Formative research occurs before a
program is designed and implemented
Implementation Research
“Research to significantly improve access to
efficacious interventions by developing
practical solutions to common implementation
problems” (TDR)
IR - Intervention specific
 Strategies of Implementation
Factors affecting implementation,
Processes of implementation
Outcomes
End-products of implementation .
Health Services Research
Health services research (HSR) examines how
people get access to
health care practitioners
health care services,
how much care costs,
what happens to patients as a result of this
care
Examples for 3 research Domains
Researches to improve Health Systems
Socities and Journals of OR
 IFORS- International Federation of Operational
Research Society
 ORSI- Operational Research Society of India-
Journal Opsearch.
 CORT- Centre for Operational Research and
Training in India.
CORT
 Is an NGO – Vadodhara Gujarat
 It covers health,Demography,Social and
Development issues
 Special interest operations research in
reproductive health, HIV/AIDS and violence.
 CORT runs state Training and Resource
Centre.
Institutions in Kerala
undertaking OR
 Medical Colleges
 State TB Cell
(Eg-Evaluation of the use of fluorescent
smear microscopy in high‐workload
settings)
Funding and capacity Building
in OR
 UNAID Global Fund to fight AIDS,Malaria and
TB allows up to 10% each grant for OR
 WHO
 CTD
 ICMR
 NACO
 RNTCP
 Training cources in OR MCH and Public Health
Institutions in India (NIHFW,Public health
Foundation India)
Funding and capacity Building
in OR - RNTCP
 State OR committees and Zonal OR
committees
 PGs in MCH are allotted funds( Rs 20,000) by
RNTCP for doing their thesis on OR topics
related to the programme.
Gaps & Challenges of OR
 Shortage of funding.
(But under use of OR in global health - average of 3% or less
for OR and actually spend).
 Trained work force.
 Researchers
 Managers
 Time constaints
Conclusion
 Operational research and management science
approaches can lead to a better health system
in terms of four A’s— accessibility,
affordability, availability and acceptability.
 Countries should become producers of
research, as well as consumers (WHO report
2013)
References
 WHO Bulletin
 ICMR Reports
 India Medcial Times
 Indian Journal of Public Health
 Lancet Global Health
 UNICEF/UNDP/ World Bank/WHO
 The Global Fund to Fight AIDS,Tuberculosis and Malaria
tropical Diseases Research.
 Operational Research in Management
 AFMC community Medicine Text Book
‘Family Farming’
“Feeding the world Caring for the Earth”
World Food Day – OCT 16

Operational Research

  • 1.
  • 2.
    Contents- Operational Research(OR) 1.Overview 2. History of OR 3. Definition OR 4. Scope of OR 5. Phases of OR 6. OR in public health 7. Societies and journals of OR 8. Other Relevant forms of research 9. Gaps& Challenges 10.Conclusion
  • 3.
    Overview-Operational Research  Operations-The activities carried out in an organization related to attain its goals and objectives Kargil Operation  Research- Any form of systematic and organized investigation to establish facts
  • 4.
    Overview- Operational Research Ineed a washing machine Should I buy it now? Is it affordable now? Which company should I go for?
  • 5.
     Decision makingis a key part of our daily life.  Final decision should be to maximise benefits and minimise effort and time
  • 6.
    Overview - OR Operational research is the scientific study of operations to make better decisions  In simple terms OR is described as “The science of better”
  • 7.
    History  During 2nd World War how to use the limited military resources effectively to win the battle by UK?  They studied strategic and tactical problems associated with air and land defense of the country, and won the war  This technique was named OR (British Air Ministry official named A. P. Rowe )
  • 8.
    History  Immediately followingWorld War II, O.R. grew rapidly  Scientists realized these principles can be equally applicable to many problems in the civilian sector.
  • 9.
    History That ranged from Short-term problems such as scheduling (PERT/CPM)  Long-term problems such as strategic planning and resource allocation in various fields
  • 10.
    History  The techniqueof OR have been applied and proven in many industries under various names Lean ,six sigma in manufacturing Supply chain in logistics Yield management in airlines
  • 11.
    Definitions (WHO 2003) The useof systematic research techniques for programme decision making to achieve a specific outcome
  • 12.
    Definition OR OR isalso defined as the search for knowledge on interventions, strategies, or tools that can improve the quality, effectiveness, or coverage of programs in which the research is being done “(Zachariah and colleagues 2009)”
  • 13.
    Goals of OR Identifying areas that could be improved  Solve problems in timely manner  Strengthen the programme  Scale up programme
  • 14.
    Scope of OR Health services & national health programmes  Public sector- Govt development  Private sector- Industrial establishment  National defense service  Engineering  Business management  Agriculture  Education & training  Transportation and communication
  • 15.
  • 16.
    Different OR techniques Multi- criteria decision analysis  Simulation technique  Linear and non linear programming  Optimization method  Queuing and stochastic process modeling  Data - Analysis method Useful in Public Health
  • 17.
    Model Solution  Howbelievable is the Solution Design field tests  Experimental  Nonexperimental  Quasi experimental
  • 18.
    Implementation of results Beauty OR process lies in obtaining, results which are implementable o Results of OR technique are information which helps in making a decision
  • 19.
  • 20.
    OR in PublicHealth “Any research producing practically usable knowledge (evidence, findings, information, etc.) which can improve program implementation (e.g. effectiveness, efficiency, quality,access, scale up, sustainability) regardless of the type of research (design, methodology, approach) falls within the boundaries of operations research” Global Meeting Geneva April 2008
  • 21.
    How OR differesfrom Basic Research?
  • 22.
    OR Examines a system Systematic data collection Treating the system inits real environme ntCollaboration b/w researchers and programme managers Deals with specific health issues Successful when results implemented
  • 23.
    Basic Reserach Not implementd in to practice Focusing onan individual Routinely collected data can be used Deals general health issues Research under controlled conditions May be only researcher/ s involved
  • 24.
  • 25.
    OR in PublicHealth  In management science OR Systems modeling Analytical techniques.  Operational research in global health does not use these complicated tools  The simple epidemiological tools and designs aptly can be applied in health
  • 26.
    Operational Research Team Varieswith the type of research.  public health administrator/Programme Manager  Epidemiologist  Statistician  Social scientist.  Field workers  Clerks, Peons
  • 27.
    In health OR- Steps  Defining the problem  Collect relevant data- formulate a hypothesis(Model)  Derive solution from hypothesis/Model  Choosing the optimal solution.  Testing of solution eg : Pilot projects.  Implementing the solution in the whole system.
  • 28.
    Example of developingan OR  Reports at a health clinic showed opportunistic infection in HIV positive clients on ART are risen. WHY? Because HIV positive clients are not adhering to their treatment. Problem
  • 29.
     Why theyare not adhering to their treatment regimen?  .  .Poor communication Between the client and staff Poor patients can not afford transportation to clinic Clients cannot afford missing their work at day Reluctant to visit clinic- Social stigma Frequent drug shortages
  • 30.
    Solutions  In-service trainingfor staff to improve drug forecasting.  Develop clinic based performance standers for problem solving among staff.  Here an OR can be designed to find the effectivenss of both these approaches
  • 31.
    Types of ORin Public health  4 types of operation research (Defined by Population Council) 1. Exploratory/diagnostic 2. Field intervention 3. Evaluative 4. Cost effectiveness studies.
  • 32.
    1. Exploratory/diagnostic  Firststep in research.  Carried out to find out the extent of problem.  Both Qualitative & Quantitative methods Interviews cross-sectional group discussions case control observations cohort studies
  • 33.
    2. Field intervention Trulyrandomized,  Intervention studies Non randomized (Quasi-experimental)
  • 34.
    3.Monitoring and evaluation Monitoring - Looking after inputs, processes and outputs in programmes.  Evaluation - Outcome/Impact evaluation  Important in National Health Programmes  Data collection on continuous basis  Implementation of program at different sites is measured.
  • 35.
    4.Economic analysis  Costingand calculating cost effectiveness ratios of the interventions  Less expensive pathways for achieving optimal outcomes
  • 36.
    Operational Researches inIndia  Implementation of DOTS by RNTCP.  Diagnosis and Treatment of Malaria  Immunisation,Antenatal,Postnatalcare,neo natal Care – In RCH  Integration of health system and ICDS  Joint home visits by AWW and ASHA  Institutional Deliveries – ‘HUTS’ in subcentre and PHCs  PPP  Decentralisation
  • 37.
    Operational Researches inIndia cont  Delivery of ORS packets  Delivery of contraceptives at village level  Peoples participation in Primary health care  Alternate model of adolescent health (Teachers to adolescent, adolescent to teachers)
  • 38.
     Optimal sizeof area and population to be covered by a midwife, or basic health unit.  Ideal vehicle for local health worker.  Architectural design of hospitals and health centers.  Queuing problem in OPD and hospital waiting lists.  Study of bed load and nursing services in hospitals.  Length of stay in hospitals, length of sickness absence. Operational Researches in India cont
  • 39.
    ORs in RNTCPby ICMR  Increased yield of smear positive pulmonary TB patients by screening patients with > 2 weeks cough, compared to > 3 weeks and adequacy of 2 sputum for diagnosis of sputum positive patients
  • 40.
    Aims:  To assessthe yield of sputum positive cases among CSs with cough of >2 weeks compared to cough of > 3 weeks  To compare the efficacy of 2 smear examination instead of 3 smears, among CSs
  • 41.
    Methods:  cross sectionalmulticentric study carried out in five states.  Andhra Pradesh, Maharashtra,Orissa, West Bengal and Rajasthan.  From each of these states, three districts were selected.  A convenient sample of 90 primary and secondary level health facilities.  10,000 samples from each state.
  • 42.
    Other OR studiesin RNTCP  Implementation of DOTS  Reliability of involving community volunteers as DOTS providers  Involving private practitioners in the RNTCP  HIV seroprevalence in TB patients.  MDR TB management  Assessing relapse among cat 1 patients who have successfully completed DOTS.
  • 43.
    ORs by NVBDCPin Malaria (2012-2017)  To ensure by 2017 that, at least 80% of malaria cases in targeted districts receive prompt and effective treatment as per national drug policy within 24 hours of first contact with the health care provider.  To strengthen the capacity for managing severe malaria cases and reducing deaths.
  • 44.
    ORs by NVBDCPin Malaria  To ensure that at least 80% of people in high- risk areas (target areas) sleep under effective ITNs/ LLINs by 2017.  To place 80% of the sanctioned staff in target areas and ensure they are trained in malaria control.  To ensure that 100% of districts in target areas will collect, analyse, and effectively use routine data and estimate their impact.
  • 45.
    Ethical Informed consent shouldbe received The use of medical records without informed consent justified When  There is minimal risk of harm to the individuals.  Access to the records is essential to achieve the objectives of the research.  There is a public benefit to undertaking the research.  Informed consent is logically or economically impracticable  There is consent to use the data from the custodian of the records  The data are protected against those not involved in the research  The research is approved by an ethics committee
  • 46.
    Other Relevant Formsof Researches  Action Oriented Research ○ Participatory Action Research ○ Empowerment Research  Formative research  Implementation Research  Health Services Research
  • 47.
    Action Oriented Researches Research trying to change the system at the same time generating knowledge.  Action research may not have any prescribed methodology.  Collaboration with non researcher participants.  Sensitive to needs and perspectives of non researcher collaborators
  • 48.
    Participatory Action Research =Community Based Participatory Research CBPR begins with a research topic of importance to the community with the aim of combining knowledge and action for social change to improve community (health).
  • 49.
    CBPR Egs- promotion ofsanitary latrines in rural area(1986)- Demand Driven Approach Steps followed …..  Behavioural research promote behavior to use SL  Promote low cost and affordable SL- requring less amount of H2O  Generate demand for SL  Involve community/Leaders for popularization of SL
  • 50.
    Empowerment Research Diarrhoea preventionand managemnt Continue breast feeding/feeding Promotion of ORS Avoid unncessary antibiotis When women groups and mother groups are empowered/trained by health workers most mothers succeeded in preventing diarrhoea.
  • 51.
    Formative Research  Whena new intervention is designed for a relatively new population  Formative research finds out what will work for them(needs and acceptability of clients)  Formative research occurs before a program is designed and implemented
  • 52.
    Implementation Research “Research tosignificantly improve access to efficacious interventions by developing practical solutions to common implementation problems” (TDR) IR - Intervention specific  Strategies of Implementation Factors affecting implementation, Processes of implementation Outcomes End-products of implementation .
  • 53.
    Health Services Research Healthservices research (HSR) examines how people get access to health care practitioners health care services, how much care costs, what happens to patients as a result of this care
  • 54.
    Examples for 3research Domains
  • 55.
    Researches to improveHealth Systems
  • 56.
    Socities and Journalsof OR  IFORS- International Federation of Operational Research Society  ORSI- Operational Research Society of India- Journal Opsearch.  CORT- Centre for Operational Research and Training in India.
  • 57.
    CORT  Is anNGO – Vadodhara Gujarat  It covers health,Demography,Social and Development issues  Special interest operations research in reproductive health, HIV/AIDS and violence.  CORT runs state Training and Resource Centre.
  • 58.
    Institutions in Kerala undertakingOR  Medical Colleges  State TB Cell (Eg-Evaluation of the use of fluorescent smear microscopy in high‐workload settings)
  • 59.
    Funding and capacityBuilding in OR  UNAID Global Fund to fight AIDS,Malaria and TB allows up to 10% each grant for OR  WHO  CTD  ICMR  NACO  RNTCP  Training cources in OR MCH and Public Health Institutions in India (NIHFW,Public health Foundation India)
  • 60.
    Funding and capacityBuilding in OR - RNTCP  State OR committees and Zonal OR committees  PGs in MCH are allotted funds( Rs 20,000) by RNTCP for doing their thesis on OR topics related to the programme.
  • 61.
    Gaps & Challengesof OR  Shortage of funding. (But under use of OR in global health - average of 3% or less for OR and actually spend).  Trained work force.  Researchers  Managers  Time constaints
  • 62.
    Conclusion  Operational researchand management science approaches can lead to a better health system in terms of four A’s— accessibility, affordability, availability and acceptability.  Countries should become producers of research, as well as consumers (WHO report 2013)
  • 63.
    References  WHO Bulletin ICMR Reports  India Medcial Times  Indian Journal of Public Health  Lancet Global Health  UNICEF/UNDP/ World Bank/WHO  The Global Fund to Fight AIDS,Tuberculosis and Malaria tropical Diseases Research.  Operational Research in Management  AFMC community Medicine Text Book
  • 64.
    ‘Family Farming’ “Feeding theworld Caring for the Earth” World Food Day – OCT 16

Editor's Notes

  • #11 Logistics is the management of the flow of goods between the point of origin and the point of consumption  Yield management-Variable pricing strategy, based on understanding, anticipating and influencing consumer behavior (airlines, hotel rum reservations).
  • #23 System- health system,Health programmes
  • #31 Note- Sometimes jz the problem becomes the topic for research
  • #43 The currently ongoing OR- Daily DOTS
  • #46 Strict adherence to the principle of autonomy may not be appropriate or practical in an OR study.
  • #50 This programme is now known as Total Sanitation Campain
  • #53 Special Programme for Research and Training in Tropical Diseases (TDR)
  • #54 Research on health system challenges addresses more complex, systems problems and is geared towards improving the understanding of what works for whom and under what circumstances.
  • #55 No clearcut bounderis between all 3.