SlideShare a Scribd company logo
1 of 18
Download to read offline
Replacing peer review by metrics in
the UK REF?
CWTS, Leiden University
8 September 2017
V.A. Traag & L. Waltman
UK REF
• Research Excellence Framework (REF) is UK’s system “to assess
the quality of research” of institutions.
• Each institution assessed separately per Unit of Assessment
(UoA) e.g. Physics, Chemistry, etc….
• Three profiles are assessed:
1 Research output;
2 Impact beyond academia; and
3 Research environment.
• Each output is awarded 1–4 stars.
4∗
is “world-leading”, 1∗
is “recognised nationally”.
• Publications are awarded 1–4 stars based on peer review.
• Results are divulged at institutional/UoA level (as % of 1–4∗).
2/18
Metrics
• Metric Tide (2015) correlate peer review results with various
citation metrics on article level. Find correlations roughly 0.3 –
0.6.
• Myrglod et al. (2015) use departmental h-index, correlations
range from 0.39 (Physics) – 0.71 (Chemistry).
• Elsevier (2015) use proportion highly cited publications. Find
correlations roughly 0.20 (Physics) – 0.75 (Chemistry).
3/18
Problems
• Correlation analysis of Metric Tide at wrong level. Should be
institutional/UoA level, not article level.
Possible to have low correlations at lower level but high
correlations at higher level.
• Difference in set of publications. Elsevier & Myrglod based on
coverage in Scopus subject categories of UoA.
• Central problem: when is a correlation high enough? Outliers.
• Peer review also has uncertainties.
4/18
Analysis
• Re-do citation analysis of all submitted publications.
• Calculate correlations at institutional level.
• Compare to uncertainty in peer-review.
Unfortunately, peer review results per article are unavailable.
Create model of peer review uncertainty.
Central idea: what if we repeat the REF?
5/18
Citation analysis
• Match all publications to WoS using DOI.
• Use citation window until 2014 (publications from 2008–2014).
• Normalise based on CWTS publication classification.
• Calculate PP(top 10%) indicator per institution/UoA.
• Correlations between PP(top 10%) and PP(4∗) per UoA.
6/18
Correlation results
10 20 30 40 50 60 70
PP(Top 10%)
0
5
10
15
20
25
30
35PP(4*)
Correlation = 0.86
Unit of Assessment: Physics
7/18
Correlation results
0 10 20 30 40 50 60
PP(Top 10%)
10
0
10
20
30
40
50PP(4*)
Correlation = 0.83
Unit of Assessment: Chemistry
8/18
Correlation results
10 20 30 40 50 60 70
PP(Top 10%)
10
0
10
20
30
40
50
60PP(4*)
Correlation = 0.69
Unit of Assessment: Biological Sciences
9/18
Review
Model
1 Each paper has some value vi.
2 Review yields some perceived value pi = vi .
3 Award 4∗ if perceived value exceeds some threshold pi > p4∗
.
Resampling
1 Sample a value vi given a paper’s 4∗ status.
2 Sample perceived value pi = vi .
3 Award 4∗ if perceived value exceeds some threshold pi > p4∗
.
10/18
Probability 4∗
0.0 0.5 1.0 1.5 2.0
Value 
0.0
0.2
0.4
0.6
0.8
1.0(*=)
=
= .
= .
11/18
Probability of value (σ2
)
0.0 0.5 1.0 1.5 2.0 2.5 3.0
0.00
0.05
0.10
0.15
0.20
0.25
0.30(=*)
=
= .
= .
12/18
UK REF Correlations
EconomicsandEconometrics
ClinicalMedicine
Physics
Chemistry
PublicHealth,HealthServicesandPrimaryCare
EarthSystemsandEnvironmentalSciences
Psychology,PsychiatryandNeuroscience
ElectricalandElectronicEngineering,MetallurgyandMaterials
BiologicalSciences
Agriculture,VeterinaryandFoodScience
Geography,EnvironmentalStudiesandArchaeology
BusinessandManagementStudies
PoliticsandInternationalStudies
GeneralEngineering
AlliedHealthProfessions,Dentistry,NursingandPharmacy
ComputerScienceandInformatics
Communication,CulturalandMediaStudies,LibraryandInformationManagement
Philosophy
Education
ArtandDesign:History,PracticeandTheory
Law
History
MathematicalSciences
Aeronautical,Mechanical,ChemicalandManufacturingEngineering
SocialWorkandSocialPolicy
SportandExerciseSciences,LeisureandTourism
TheologyandReligiousStudies
EnglishLanguageandLiterature
Music,Drama,DanceandPerformingArts
Architecture,BuiltEnvironmentandPlanning
CivilandConstructionEngineering
Sociology
ModernLanguagesandLinguistics
Classics
AreaStudies
AnthropologyandDevelopmentStudies
0.0
0.2
0.4
0.6
0.8
1.0Correlation
PP(top 10%) Bootstrap (0.1) Bootstrap (0.5) Bootstrap (1)
trics
icine
ysics
istry
Care
nces
ence
rials
0.0
0.2
0.4
0.6
0.8
1.0
Correlationtrics
icine
ysics
istry
Care
nces
ence
rials
nces
ence
0.0
0.2
0.4
0.6
0.8
1.0
PP(to
13/18
Conclusions
Conclusions
• Correlations at article level = correlations at institutional level.
• Correlations on metrics should consider peer review uncertainty.
• For UK REF, physics metrics correlate almost equally well as
(model of) peer review.
Future
• Longer citation window?
• Compare to Scopus correlations.
• Corroborate model of peer review uncertainty.
14/18
Thank you!
Questions?
v.a.traag@cwts.leidenuniv.nl @vtraag
15/18
Model technicalities
Distributions
• Value distributed as vi ∼ LogN(µv , 1).
• Error distributed as ∼ LogN(−σ2
2 , σ2), so that = 1.
• Perceived value distributed as pi ∼ LogN(µv − σ2
2 , 1 + σ2).
Estimation
• Assume overall value µv = 1.
• Calculate p4∗
such that Pr(pi > p4∗
) = PP(4∗) overall.
• Assume values for institution/uoa is LogN(µv , 1).
• Calculate µv such that Pr(pi > p4∗
) = PP(4∗) per
institution/uoa.
16/18
Probability of value (µv)
0 1 2 3 4 5
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45(=*)
= .
=
=
17/18
Accuracy
0.0 0.2 0.4 0.6 0.8 1.0
PP(4*)
0.0
0.2
0.4
0.6
0.8
1.0Accuracy
Accuracy (4*) Accuracy (< 4*) Accuracy (overall)
18/18

More Related Content

What's hot

Scientific Method Notes
Scientific Method NotesScientific Method Notes
Scientific Method Notesmgitterm
 
BioSHaRE: Maelstrom Research tools for data harmonization and co-analysis - I...
BioSHaRE: Maelstrom Research tools for data harmonization and co-analysis - I...BioSHaRE: Maelstrom Research tools for data harmonization and co-analysis - I...
BioSHaRE: Maelstrom Research tools for data harmonization and co-analysis - I...Lisette Giepmans
 
Beyond Journal Impact and Usage Statistics: Using Citation Analysis for Colle...
Beyond Journal Impact and Usage Statistics: Using Citation Analysis for Colle...Beyond Journal Impact and Usage Statistics: Using Citation Analysis for Colle...
Beyond Journal Impact and Usage Statistics: Using Citation Analysis for Colle...NASIG
 
INSTRUMENTAL ANALYSIS INTRODUCTION
INSTRUMENTAL ANALYSIS INTRODUCTIONINSTRUMENTAL ANALYSIS INTRODUCTION
INSTRUMENTAL ANALYSIS INTRODUCTIONHamunyare Ndwabe
 
A scientometric perspective on university ranking
A scientometric perspective on university rankingA scientometric perspective on university ranking
A scientometric perspective on university rankingNees Jan van Eck
 
Comparing scientific performance across disciplines: Methodological and conce...
Comparing scientific performance across disciplines: Methodological and conce...Comparing scientific performance across disciplines: Methodological and conce...
Comparing scientific performance across disciplines: Methodological and conce...Ludo Waltman
 
Antigen antibody reactions
Antigen antibody reactionsAntigen antibody reactions
Antigen antibody reactionsDr NEETHU ASOKAN
 
Veterinary pathologist
Veterinary pathologistVeterinary pathologist
Veterinary pathologist15CTLe
 

What's hot (14)

Statistical test discuss 5
Statistical test discuss 5Statistical test discuss 5
Statistical test discuss 5
 
Scientific Method Notes
Scientific Method NotesScientific Method Notes
Scientific Method Notes
 
83341 ch24 jacobsen
83341 ch24 jacobsen83341 ch24 jacobsen
83341 ch24 jacobsen
 
BioSHaRE: Maelstrom Research tools for data harmonization and co-analysis - I...
BioSHaRE: Maelstrom Research tools for data harmonization and co-analysis - I...BioSHaRE: Maelstrom Research tools for data harmonization and co-analysis - I...
BioSHaRE: Maelstrom Research tools for data harmonization and co-analysis - I...
 
Beyond Journal Impact and Usage Statistics: Using Citation Analysis for Colle...
Beyond Journal Impact and Usage Statistics: Using Citation Analysis for Colle...Beyond Journal Impact and Usage Statistics: Using Citation Analysis for Colle...
Beyond Journal Impact and Usage Statistics: Using Citation Analysis for Colle...
 
INSTRUMENTAL ANALYSIS INTRODUCTION
INSTRUMENTAL ANALYSIS INTRODUCTIONINSTRUMENTAL ANALYSIS INTRODUCTION
INSTRUMENTAL ANALYSIS INTRODUCTION
 
Brief overview on meta analysis
Brief overview  on meta analysisBrief overview  on meta analysis
Brief overview on meta analysis
 
A scientometric perspective on university ranking
A scientometric perspective on university rankingA scientometric perspective on university ranking
A scientometric perspective on university ranking
 
83341 ch25 jacobsen
83341 ch25 jacobsen83341 ch25 jacobsen
83341 ch25 jacobsen
 
07 data analysis
07 data analysis07 data analysis
07 data analysis
 
Comparing scientific performance across disciplines: Methodological and conce...
Comparing scientific performance across disciplines: Methodological and conce...Comparing scientific performance across disciplines: Methodological and conce...
Comparing scientific performance across disciplines: Methodological and conce...
 
Antigen antibody reactions
Antigen antibody reactionsAntigen antibody reactions
Antigen antibody reactions
 
Veterinary pathologist
Veterinary pathologistVeterinary pathologist
Veterinary pathologist
 
Scientific Inquiry
Scientific InquiryScientific Inquiry
Scientific Inquiry
 

Similar to UK REF Metrics Correlate Nearly as Well as Peer Review

Paper 6: World University's Evaluation (Qiu & Zhao)
Paper 6: World University's Evaluation (Qiu & Zhao)Paper 6: World University's Evaluation (Qiu & Zhao)
Paper 6: World University's Evaluation (Qiu & Zhao)Kent Business School
 
Cnr elsevier workshop nov 14 2012 bonaccorsi
Cnr elsevier workshop nov 14 2012 bonaccorsiCnr elsevier workshop nov 14 2012 bonaccorsi
Cnr elsevier workshop nov 14 2012 bonaccorsiElsevier Events
 
From RAE to REF
From RAE to REFFrom RAE to REF
From RAE to REFDavid Clay
 
Urap 2014 2015 world ranking and urap tr presentation nov 12 2014 sa (1)
Urap 2014 2015 world ranking and urap tr presentation nov 12 2014 sa (1)Urap 2014 2015 world ranking and urap tr presentation nov 12 2014 sa (1)
Urap 2014 2015 world ranking and urap tr presentation nov 12 2014 sa (1)Murat KOÇAK
 
Research evaluation in the Netherlands : a library perspective
Research evaluation in the Netherlands : a library perspectiveResearch evaluation in the Netherlands : a library perspective
Research evaluation in the Netherlands : a library perspectiveWouter Gerritsma
 
A scientometric perspective on university ranking
A scientometric perspective on university rankingA scientometric perspective on university ranking
A scientometric perspective on university rankingLudo Waltman
 
Applying to competitive universities
Applying to competitive universitiesApplying to competitive universities
Applying to competitive universitiesMeltonVale
 
Statistical analysis of world university rankings (report for PDMU-2010)
Statistical analysis of world university rankings (report for PDMU-2010)Statistical analysis of world university rankings (report for PDMU-2010)
Statistical analysis of world university rankings (report for PDMU-2010)Vladimir Bakhrushin
 
International ranking system
International ranking system International ranking system
International ranking system Kirsten Siksne
 
Ct lecture 17. introduction to logistic regression
Ct lecture 17. introduction to logistic regressionCt lecture 17. introduction to logistic regression
Ct lecture 17. introduction to logistic regressionHau Pham
 
AMEE 2013 poster Does ePortfolio engagement predict performance in undergradu...
AMEE 2013 poster Does ePortfolio engagement predict performance in undergradu...AMEE 2013 poster Does ePortfolio engagement predict performance in undergradu...
AMEE 2013 poster Does ePortfolio engagement predict performance in undergradu...Laura-Jane Smith
 
Epidemiology and Public Policy
Epidemiology and Public PolicyEpidemiology and Public Policy
Epidemiology and Public Policyemphemory
 
Open access policies - Policy effectiveness, Alma Swan
Open access policies - Policy effectiveness, Alma SwanOpen access policies - Policy effectiveness, Alma Swan
Open access policies - Policy effectiveness, Alma SwanSPARC Europe
 

Similar to UK REF Metrics Correlate Nearly as Well as Peer Review (20)

LEIDEN UNIVERSITY 2010
LEIDEN UNIVERSITY 2010LEIDEN UNIVERSITY 2010
LEIDEN UNIVERSITY 2010
 
Paper 6: World University's Evaluation (Qiu & Zhao)
Paper 6: World University's Evaluation (Qiu & Zhao)Paper 6: World University's Evaluation (Qiu & Zhao)
Paper 6: World University's Evaluation (Qiu & Zhao)
 
Cnr elsevier workshop nov 14 2012 bonaccorsi
Cnr elsevier workshop nov 14 2012 bonaccorsiCnr elsevier workshop nov 14 2012 bonaccorsi
Cnr elsevier workshop nov 14 2012 bonaccorsi
 
From RAE to REF
From RAE to REFFrom RAE to REF
From RAE to REF
 
Research Excellence Framework 2014 [Michael Rayner & Shirley Leckie]
Research Excellence Framework 2014 [Michael Rayner & Shirley Leckie]Research Excellence Framework 2014 [Michael Rayner & Shirley Leckie]
Research Excellence Framework 2014 [Michael Rayner & Shirley Leckie]
 
Urap 2014 2015 world ranking and urap tr presentation nov 12 2014 sa (1)
Urap 2014 2015 world ranking and urap tr presentation nov 12 2014 sa (1)Urap 2014 2015 world ranking and urap tr presentation nov 12 2014 sa (1)
Urap 2014 2015 world ranking and urap tr presentation nov 12 2014 sa (1)
 
Research evaluation in the Netherlands : a library perspective
Research evaluation in the Netherlands : a library perspectiveResearch evaluation in the Netherlands : a library perspective
Research evaluation in the Netherlands : a library perspective
 
Research Excellence Framework 2014 [Michael Rayner & Shirley Leckie]
Research Excellence Framework 2014 [Michael Rayner & Shirley Leckie]Research Excellence Framework 2014 [Michael Rayner & Shirley Leckie]
Research Excellence Framework 2014 [Michael Rayner & Shirley Leckie]
 
A scientometric perspective on university ranking
A scientometric perspective on university rankingA scientometric perspective on university ranking
A scientometric perspective on university ranking
 
ETE presentation
ETE presentationETE presentation
ETE presentation
 
Applying to competitive universities
Applying to competitive universitiesApplying to competitive universities
Applying to competitive universities
 
Statistical analysis of world university rankings (report for PDMU-2010)
Statistical analysis of world university rankings (report for PDMU-2010)Statistical analysis of world university rankings (report for PDMU-2010)
Statistical analysis of world university rankings (report for PDMU-2010)
 
International ranking system
International ranking system International ranking system
International ranking system
 
Presentation for GIRS
Presentation for GIRSPresentation for GIRS
Presentation for GIRS
 
Ct lecture 17. introduction to logistic regression
Ct lecture 17. introduction to logistic regressionCt lecture 17. introduction to logistic regression
Ct lecture 17. introduction to logistic regression
 
AMEE 2013 poster Does ePortfolio engagement predict performance in undergradu...
AMEE 2013 poster Does ePortfolio engagement predict performance in undergradu...AMEE 2013 poster Does ePortfolio engagement predict performance in undergradu...
AMEE 2013 poster Does ePortfolio engagement predict performance in undergradu...
 
Epidemiology and Public Policy
Epidemiology and Public PolicyEpidemiology and Public Policy
Epidemiology and Public Policy
 
ARWU 2010
ARWU 2010ARWU 2010
ARWU 2010
 
Open access policies - Policy effectiveness, Alma Swan
Open access policies - Policy effectiveness, Alma SwanOpen access policies - Policy effectiveness, Alma Swan
Open access policies - Policy effectiveness, Alma Swan
 
Author Seminar NUI Galway July 2019
Author Seminar NUI Galway July 2019Author Seminar NUI Galway July 2019
Author Seminar NUI Galway July 2019
 

More from Vincent Traag

Peer review uncertainty at the institutional level
Peer review uncertainty at the institutional levelPeer review uncertainty at the institutional level
Peer review uncertainty at the institutional levelVincent Traag
 
Use of the journal impact factor for assessing individual articles need not b...
Use of the journal impact factor for assessing individual articles need not b...Use of the journal impact factor for assessing individual articles need not b...
Use of the journal impact factor for assessing individual articles need not b...Vincent Traag
 
Uncovering important intermediate publications
Uncovering important intermediate publicationsUncovering important intermediate publications
Uncovering important intermediate publicationsVincent Traag
 
Complex contagion of campaign donations
Complex contagion of campaign donationsComplex contagion of campaign donations
Complex contagion of campaign donationsVincent Traag
 
Polarization and consensus in citation networks
Polarization and consensus in citation networksPolarization and consensus in citation networks
Polarization and consensus in citation networksVincent Traag
 
Community structure in complex networks
Community structure in complex networksCommunity structure in complex networks
Community structure in complex networksVincent Traag
 
Introduction to complex networks
Introduction to complex networksIntroduction to complex networks
Introduction to complex networksVincent Traag
 
Public thesis defence: groups and reputation in social networks
Public thesis defence: groups and reputation in social networksPublic thesis defence: groups and reputation in social networks
Public thesis defence: groups and reputation in social networksVincent Traag
 
Structure of media attention
Structure of media attentionStructure of media attention
Structure of media attentionVincent Traag
 
Dynamics of Media Attention
Dynamics of Media AttentionDynamics of Media Attention
Dynamics of Media AttentionVincent Traag
 
Dynamical Models Explaining Social Balance
Dynamical Models Explaining Social BalanceDynamical Models Explaining Social Balance
Dynamical Models Explaining Social BalanceVincent Traag
 
Significant scales in community structure
Significant scales in community structureSignificant scales in community structure
Significant scales in community structureVincent Traag
 
Reconstructing Third World Elite Rotation Events from Newspapers
Reconstructing Third World Elite Rotation Events from NewspapersReconstructing Third World Elite Rotation Events from Newspapers
Reconstructing Third World Elite Rotation Events from NewspapersVincent Traag
 
Reputation Dynamics Through Gossiping
Reputation Dynamics Through GossipingReputation Dynamics Through Gossiping
Reputation Dynamics Through GossipingVincent Traag
 
Limits of community detection
Limits of community detectionLimits of community detection
Limits of community detectionVincent Traag
 
Cooperation, Reputation & Gossiping
Cooperation, Reputation & GossipingCooperation, Reputation & Gossiping
Cooperation, Reputation & GossipingVincent Traag
 
Resolution-free community detection
Resolution-free community detectionResolution-free community detection
Resolution-free community detectionVincent Traag
 
Cooperation, Reputation & Gossiping
Cooperation, Reputation & GossipingCooperation, Reputation & Gossiping
Cooperation, Reputation & GossipingVincent Traag
 
Exponential Ranking: Taking into account negative links.
Exponential Ranking: Taking into account negative links.Exponential Ranking: Taking into account negative links.
Exponential Ranking: Taking into account negative links.Vincent Traag
 
Social Event Detection
Social Event DetectionSocial Event Detection
Social Event DetectionVincent Traag
 

More from Vincent Traag (20)

Peer review uncertainty at the institutional level
Peer review uncertainty at the institutional levelPeer review uncertainty at the institutional level
Peer review uncertainty at the institutional level
 
Use of the journal impact factor for assessing individual articles need not b...
Use of the journal impact factor for assessing individual articles need not b...Use of the journal impact factor for assessing individual articles need not b...
Use of the journal impact factor for assessing individual articles need not b...
 
Uncovering important intermediate publications
Uncovering important intermediate publicationsUncovering important intermediate publications
Uncovering important intermediate publications
 
Complex contagion of campaign donations
Complex contagion of campaign donationsComplex contagion of campaign donations
Complex contagion of campaign donations
 
Polarization and consensus in citation networks
Polarization and consensus in citation networksPolarization and consensus in citation networks
Polarization and consensus in citation networks
 
Community structure in complex networks
Community structure in complex networksCommunity structure in complex networks
Community structure in complex networks
 
Introduction to complex networks
Introduction to complex networksIntroduction to complex networks
Introduction to complex networks
 
Public thesis defence: groups and reputation in social networks
Public thesis defence: groups and reputation in social networksPublic thesis defence: groups and reputation in social networks
Public thesis defence: groups and reputation in social networks
 
Structure of media attention
Structure of media attentionStructure of media attention
Structure of media attention
 
Dynamics of Media Attention
Dynamics of Media AttentionDynamics of Media Attention
Dynamics of Media Attention
 
Dynamical Models Explaining Social Balance
Dynamical Models Explaining Social BalanceDynamical Models Explaining Social Balance
Dynamical Models Explaining Social Balance
 
Significant scales in community structure
Significant scales in community structureSignificant scales in community structure
Significant scales in community structure
 
Reconstructing Third World Elite Rotation Events from Newspapers
Reconstructing Third World Elite Rotation Events from NewspapersReconstructing Third World Elite Rotation Events from Newspapers
Reconstructing Third World Elite Rotation Events from Newspapers
 
Reputation Dynamics Through Gossiping
Reputation Dynamics Through GossipingReputation Dynamics Through Gossiping
Reputation Dynamics Through Gossiping
 
Limits of community detection
Limits of community detectionLimits of community detection
Limits of community detection
 
Cooperation, Reputation & Gossiping
Cooperation, Reputation & GossipingCooperation, Reputation & Gossiping
Cooperation, Reputation & Gossiping
 
Resolution-free community detection
Resolution-free community detectionResolution-free community detection
Resolution-free community detection
 
Cooperation, Reputation & Gossiping
Cooperation, Reputation & GossipingCooperation, Reputation & Gossiping
Cooperation, Reputation & Gossiping
 
Exponential Ranking: Taking into account negative links.
Exponential Ranking: Taking into account negative links.Exponential Ranking: Taking into account negative links.
Exponential Ranking: Taking into account negative links.
 
Social Event Detection
Social Event DetectionSocial Event Detection
Social Event Detection
 

Recently uploaded

GLYCOSIDES Classification Of GLYCOSIDES Chemical Tests Glycosides
GLYCOSIDES Classification Of GLYCOSIDES  Chemical Tests GlycosidesGLYCOSIDES Classification Of GLYCOSIDES  Chemical Tests Glycosides
GLYCOSIDES Classification Of GLYCOSIDES Chemical Tests GlycosidesNandakishor Bhaurao Deshmukh
 
Pests of Bengal gram_Identification_Dr.UPR.pdf
Pests of Bengal gram_Identification_Dr.UPR.pdfPests of Bengal gram_Identification_Dr.UPR.pdf
Pests of Bengal gram_Identification_Dr.UPR.pdfPirithiRaju
 
trihybrid cross , test cross chi squares
trihybrid cross , test cross chi squarestrihybrid cross , test cross chi squares
trihybrid cross , test cross chi squaresusmanzain586
 
Loudspeaker- direct radiating type and horn type.pptx
Loudspeaker- direct radiating type and horn type.pptxLoudspeaker- direct radiating type and horn type.pptx
Loudspeaker- direct radiating type and horn type.pptxpriyankatabhane
 
FREE NURSING BUNDLE FOR NURSES.PDF by na
FREE NURSING BUNDLE FOR NURSES.PDF by naFREE NURSING BUNDLE FOR NURSES.PDF by na
FREE NURSING BUNDLE FOR NURSES.PDF by naJASISJULIANOELYNV
 
User Guide: Pulsar™ Weather Station (Columbia Weather Systems)
User Guide: Pulsar™ Weather Station (Columbia Weather Systems)User Guide: Pulsar™ Weather Station (Columbia Weather Systems)
User Guide: Pulsar™ Weather Station (Columbia Weather Systems)Columbia Weather Systems
 
Call Girls in Majnu Ka Tilla Delhi 🔝9711014705🔝 Genuine
Call Girls in Majnu Ka Tilla Delhi 🔝9711014705🔝 GenuineCall Girls in Majnu Ka Tilla Delhi 🔝9711014705🔝 Genuine
Call Girls in Majnu Ka Tilla Delhi 🔝9711014705🔝 Genuinethapagita
 
Dubai Calls Girl Lisa O525547819 Lexi Call Girls In Dubai
Dubai Calls Girl Lisa O525547819 Lexi Call Girls In DubaiDubai Calls Girl Lisa O525547819 Lexi Call Girls In Dubai
Dubai Calls Girl Lisa O525547819 Lexi Call Girls In Dubaikojalkojal131
 
final waves properties grade 7 - third quarter
final waves properties grade 7 - third quarterfinal waves properties grade 7 - third quarter
final waves properties grade 7 - third quarterHanHyoKim
 
GenAI talk for Young at Wageningen University & Research (WUR) March 2024
GenAI talk for Young at Wageningen University & Research (WUR) March 2024GenAI talk for Young at Wageningen University & Research (WUR) March 2024
GenAI talk for Young at Wageningen University & Research (WUR) March 2024Jene van der Heide
 
Pests of Blackgram, greengram, cowpea_Dr.UPR.pdf
Pests of Blackgram, greengram, cowpea_Dr.UPR.pdfPests of Blackgram, greengram, cowpea_Dr.UPR.pdf
Pests of Blackgram, greengram, cowpea_Dr.UPR.pdfPirithiRaju
 
User Guide: Magellan MX™ Weather Station
User Guide: Magellan MX™ Weather StationUser Guide: Magellan MX™ Weather Station
User Guide: Magellan MX™ Weather StationColumbia Weather Systems
 
projectile motion, impulse and moment
projectile  motion, impulse  and  momentprojectile  motion, impulse  and  moment
projectile motion, impulse and momentdonamiaquintan2
 
Thermodynamics ,types of system,formulae ,gibbs free energy .pptx
Thermodynamics ,types of system,formulae ,gibbs free energy .pptxThermodynamics ,types of system,formulae ,gibbs free energy .pptx
Thermodynamics ,types of system,formulae ,gibbs free energy .pptxuniversity
 
Fertilization: Sperm and the egg—collectively called the gametes—fuse togethe...
Fertilization: Sperm and the egg—collectively called the gametes—fuse togethe...Fertilization: Sperm and the egg—collectively called the gametes—fuse togethe...
Fertilization: Sperm and the egg—collectively called the gametes—fuse togethe...D. B. S. College Kanpur
 
Base editing, prime editing, Cas13 & RNA editing and organelle base editing
Base editing, prime editing, Cas13 & RNA editing and organelle base editingBase editing, prime editing, Cas13 & RNA editing and organelle base editing
Base editing, prime editing, Cas13 & RNA editing and organelle base editingNetHelix
 
User Guide: Orion™ Weather Station (Columbia Weather Systems)
User Guide: Orion™ Weather Station (Columbia Weather Systems)User Guide: Orion™ Weather Station (Columbia Weather Systems)
User Guide: Orion™ Weather Station (Columbia Weather Systems)Columbia Weather Systems
 
User Guide: Capricorn FLX™ Weather Station
User Guide: Capricorn FLX™ Weather StationUser Guide: Capricorn FLX™ Weather Station
User Guide: Capricorn FLX™ Weather StationColumbia Weather Systems
 
LESSON PLAN IN SCIENCE GRADE 4 WEEK 1 DAY 2
LESSON PLAN IN SCIENCE GRADE 4 WEEK 1 DAY 2LESSON PLAN IN SCIENCE GRADE 4 WEEK 1 DAY 2
LESSON PLAN IN SCIENCE GRADE 4 WEEK 1 DAY 2AuEnriquezLontok
 

Recently uploaded (20)

GLYCOSIDES Classification Of GLYCOSIDES Chemical Tests Glycosides
GLYCOSIDES Classification Of GLYCOSIDES  Chemical Tests GlycosidesGLYCOSIDES Classification Of GLYCOSIDES  Chemical Tests Glycosides
GLYCOSIDES Classification Of GLYCOSIDES Chemical Tests Glycosides
 
Pests of Bengal gram_Identification_Dr.UPR.pdf
Pests of Bengal gram_Identification_Dr.UPR.pdfPests of Bengal gram_Identification_Dr.UPR.pdf
Pests of Bengal gram_Identification_Dr.UPR.pdf
 
trihybrid cross , test cross chi squares
trihybrid cross , test cross chi squarestrihybrid cross , test cross chi squares
trihybrid cross , test cross chi squares
 
Loudspeaker- direct radiating type and horn type.pptx
Loudspeaker- direct radiating type and horn type.pptxLoudspeaker- direct radiating type and horn type.pptx
Loudspeaker- direct radiating type and horn type.pptx
 
FREE NURSING BUNDLE FOR NURSES.PDF by na
FREE NURSING BUNDLE FOR NURSES.PDF by naFREE NURSING BUNDLE FOR NURSES.PDF by na
FREE NURSING BUNDLE FOR NURSES.PDF by na
 
User Guide: Pulsar™ Weather Station (Columbia Weather Systems)
User Guide: Pulsar™ Weather Station (Columbia Weather Systems)User Guide: Pulsar™ Weather Station (Columbia Weather Systems)
User Guide: Pulsar™ Weather Station (Columbia Weather Systems)
 
Call Girls in Majnu Ka Tilla Delhi 🔝9711014705🔝 Genuine
Call Girls in Majnu Ka Tilla Delhi 🔝9711014705🔝 GenuineCall Girls in Majnu Ka Tilla Delhi 🔝9711014705🔝 Genuine
Call Girls in Majnu Ka Tilla Delhi 🔝9711014705🔝 Genuine
 
Let’s Say Someone Did Drop the Bomb. Then What?
Let’s Say Someone Did Drop the Bomb. Then What?Let’s Say Someone Did Drop the Bomb. Then What?
Let’s Say Someone Did Drop the Bomb. Then What?
 
Dubai Calls Girl Lisa O525547819 Lexi Call Girls In Dubai
Dubai Calls Girl Lisa O525547819 Lexi Call Girls In DubaiDubai Calls Girl Lisa O525547819 Lexi Call Girls In Dubai
Dubai Calls Girl Lisa O525547819 Lexi Call Girls In Dubai
 
final waves properties grade 7 - third quarter
final waves properties grade 7 - third quarterfinal waves properties grade 7 - third quarter
final waves properties grade 7 - third quarter
 
GenAI talk for Young at Wageningen University & Research (WUR) March 2024
GenAI talk for Young at Wageningen University & Research (WUR) March 2024GenAI talk for Young at Wageningen University & Research (WUR) March 2024
GenAI talk for Young at Wageningen University & Research (WUR) March 2024
 
Pests of Blackgram, greengram, cowpea_Dr.UPR.pdf
Pests of Blackgram, greengram, cowpea_Dr.UPR.pdfPests of Blackgram, greengram, cowpea_Dr.UPR.pdf
Pests of Blackgram, greengram, cowpea_Dr.UPR.pdf
 
User Guide: Magellan MX™ Weather Station
User Guide: Magellan MX™ Weather StationUser Guide: Magellan MX™ Weather Station
User Guide: Magellan MX™ Weather Station
 
projectile motion, impulse and moment
projectile  motion, impulse  and  momentprojectile  motion, impulse  and  moment
projectile motion, impulse and moment
 
Thermodynamics ,types of system,formulae ,gibbs free energy .pptx
Thermodynamics ,types of system,formulae ,gibbs free energy .pptxThermodynamics ,types of system,formulae ,gibbs free energy .pptx
Thermodynamics ,types of system,formulae ,gibbs free energy .pptx
 
Fertilization: Sperm and the egg—collectively called the gametes—fuse togethe...
Fertilization: Sperm and the egg—collectively called the gametes—fuse togethe...Fertilization: Sperm and the egg—collectively called the gametes—fuse togethe...
Fertilization: Sperm and the egg—collectively called the gametes—fuse togethe...
 
Base editing, prime editing, Cas13 & RNA editing and organelle base editing
Base editing, prime editing, Cas13 & RNA editing and organelle base editingBase editing, prime editing, Cas13 & RNA editing and organelle base editing
Base editing, prime editing, Cas13 & RNA editing and organelle base editing
 
User Guide: Orion™ Weather Station (Columbia Weather Systems)
User Guide: Orion™ Weather Station (Columbia Weather Systems)User Guide: Orion™ Weather Station (Columbia Weather Systems)
User Guide: Orion™ Weather Station (Columbia Weather Systems)
 
User Guide: Capricorn FLX™ Weather Station
User Guide: Capricorn FLX™ Weather StationUser Guide: Capricorn FLX™ Weather Station
User Guide: Capricorn FLX™ Weather Station
 
LESSON PLAN IN SCIENCE GRADE 4 WEEK 1 DAY 2
LESSON PLAN IN SCIENCE GRADE 4 WEEK 1 DAY 2LESSON PLAN IN SCIENCE GRADE 4 WEEK 1 DAY 2
LESSON PLAN IN SCIENCE GRADE 4 WEEK 1 DAY 2
 

UK REF Metrics Correlate Nearly as Well as Peer Review