This document proposes a model for assessing the scientific performance of social science researchers in Mexico that considers factors beyond bibliometric indicators. It will integrate variables of scientific production, teaching activities, professional involvement, and scholarly recognition. A stratified random sample of 227 researchers from the Mexican National Researchers System will be analyzed using their CVs. Preliminary findings show differences in publishing patterns between researcher levels and fields. The study aims to determine the relative impacts of the four variable groups on scientific performance.

1. Methodological Proposal
for Evaluating Performance
of Social Science
Researchers in Mexico
Cristina Restrepo-Arango and Jane M. Russell
Universidad Nacional Autónoma de México

2. Background
 Bibliometric techniques inadequate to assess
research outputs of Social Scientists, especially
those from developing world.
 Web of Science, primary source for data
collection, the majority of journals are published
in English.
 Metric evaluation does not take into account
indicators of “cultural capital” .
 Particularly in Social Sciences, bibliometrics is
recommended in combination with qualitative
techniques.

3. Overall aim
Propose a model for assessing scientific
performance that takes into consideration
other factors in addition to bibliometric
indicators, and apply it to a selected
sample of Social Science researchers in
Mexico.

4. Our approach
• Integrates variables of scientific production with
other variables: teaching activities; professional
involvement; and scholarly recognition.
• Builds a holistic view of scientific achievement.
• Recognizes that research is a social process
inserted in a specific cultural context and
influenced by the behaviour of the individual
researchers.

5. Main research question
What are the relative impacts of these four
main groups of variables on the scientific
performance of Mexican researchers in
Social Sciences, members of the Mexican
National Researchers System (SNI)?

6. Mexican National Researchers
System (SNI)
 Created in 1984 by Mexican government to recognise
the work of individuals in the creation of knowledge.
 Recognition and financial incentives given at four
levels: I, II, III y Emeritus.
 Peer review by expert committees in 7 knowledge
areas.
 Area V. Social Sciences (Administration, Political
Science, Communication, Accountancy, Demography,
Law, Economics, Human Geography, Sociology).

7. Methodology : Stratified random
sample
Fields
Levels
I II III Emeritus Total
Sociology
449
(Sample 50)
181
(Sample 20)
75
(Sample 9)
6
(Sample 1)
711
(Sample 81)
Economics
439
(Sample 49)
101
(Sample 11)
58
(Sample 6)
1
(Sample 1)
599
(Sample 62)
Political
Science
212
(Sample 24)
98
(Sample 11)
37
(Sample 4)
5
(Sample 1)
352
(Sample 39)
Law
206
(Sample 23)
69
(Sample 8)
31
(Sample 3)
4
(Sample 1)
310
(Sample 38)
Demography
31
(Sample 3)
21
(Sample 2)
14
(Sample 2) --
66
(Sample 7)
Total 1337
(Sample 149)
470
(Sample 52)
217
(Sample 24)
16
(Sample 3)
2038
(Sample 227)

8. Data Collection and Analysis
 Data were collected from the CVs requested via email.
 Each CV was carefully examined for any mention of activities relating
to our four main groups, from the onset of their scientific careers.
 Individual criteria were constructed on the basis of previous studies,
as well as preliminary analysis of a representative sample of CVs.
 Data matrix was created for number of occurrences of each criterion,
extracted from the CVs.
 SPSS software version 20 for Windows is used to process the data.
 Analysis of results of multivariate techniques by multiple
correspondence analysis and principal components analysis, is under
evaluation.

9. Methodological Considerations: CV as
a data source
 Lack of uniformity in the presentation.
 Not all include information on all four groups of
variables and not all were entirely up to date.
 Public access to unified formats of researcher’s
CVs is not an option in Mexico.

10. Scientific Production
 Author or co-author scholarly book.
 Author or co-author scholarly book chapter.
 Compiler, editor, scholarly publication.
 Article in foreign or national refereed scholarly journal.
 Article in foreign or national non- scholarly journal.
 Article in foreign or national newspaper.
 Other publications (brochures, reports, etc.).
 Book reviews.
 Preface, foreword, introduction of scholarly book.
 Translations of scholarly books or articles.
 Paper presented at foreign or national event.

11. Teaching Activities
 Teaching at secondary, preparatory, specialization,
undergraduate, master´s and Ph. D. levels.
 Thesis director at specialization, undergraduate, master’s
and Ph. D levels.
 Thesis committee at undergraduate, master’s and Ph. D
levels.
 Reader of thesis at undergraduate, master’s and Ph. D
levels.
 Visiting professor or researcher.

12. Professional Involvement
 Consultant in others institutions.
 Administrative post or member of institutional committees.
 Membership of professional bodies (scientific societies, professional
associations, etc.)
 Reviewer of scholarly journal article, book chapter, book and research project
or research grant.
 Member of editorial board.
 Member of research group or research project.
 Commentator, chair, book presenter, etc. in event.
 Interviewed in radio or television programme.
 Invited speaker in foreign or national event.
 Coordinator and organizer of events, congress, seminars, etc.

13. Scholarly Recognition
 Awards presented for academic papers.
 Awards for distinguished service (emeritus,
distinctions, etc.)
 Tributes for academic career.
 Research grants obtained by public competition.
 Scholarships (graduate, postdoctoral and research).
 Honourable mentions and academic incentives.
 Doctorate Honoris Causa.

14. Preliminary Findings
Our preliminary data reveal important differences between
researchers in the various category levels of the SNI:
o Emeritus & Level III publish more scholarly books and
newspaper articles.
o Level II more articles in peer-reviewed foreign journals.
We also found differences with respect to the fields of the
researchers:
o Law: predominance of scholarly books
o Economics: articles in peer-reviewed foreign journals
o Political science: newspaper articles

15. Thank you very much for your
attention

16. Contacts
Cristina Restrepo-Arango
crestrepoarango@gmail.com
Programa de Posgrado en Bibliotecología y Estudios de la
Información.
Universidad Nacional Autónoma de México, Unidad de Posgrado,
Edificio H, 1er. Nivel, H-104. Circuito de Posgrado, Ciudad
Universitaria.
Coyoacán, México D.F. C.P. 04510 (Mexico)
Jane M. Russell
jrussell@unam.mx
Instituto de Investigaciones Bibliotecológicas y de la Información,
Universidad Nacional Autónoma de México, Torre II de
Humanidades pisos 11-13, Ciudad Universitaria.
Coyoacán, México D.F. C.P. 04510 (Mexico)