The document discusses the issues with using quantitative metrics and rankings to assess university performance and research. It provides examples of how rankings have been influential in policy decisions around the world despite their questionable scientific validity. Concerns raised include the lack of transparency in methodology, arbitrary weighting of indicators, and incentives for gaming the system. While metrics provide objective measures, they fail to capture many important qualitative aspects of universities and research. Overall, the document questions the practice of "governing by numbers" and emphasizes the need for more holistic evaluation that considers national priorities and systemic impacts.
Presentazione di Giuseppe De Nicolao al II Convegno Roars: “Higher Education and Research Policies in Europe: Challenges for Italy”, 21 febbraio 2014
CNR, Piazzale A. Moro 7, Roma
This presentation covered the UK university application process through UCAS. It began with an overview of the application timeline and deadlines. Key parts of the application like choosing courses, writing a personal statement, and obtaining a reference letter were discussed in detail. The presentation also addressed UCAS registration, navigating the application, and important application components like IB predicted grades. Overall, the presentation aimed to guide students through completing the UCAS application and understanding the requirements for UK university admission.
11th International Conference on Social Science and Humanities (ICSSH)Global R & D Services
Conference Name: 11th International Conference on Social Science and Humanities (ICSSH), 19-20 Sept, 2016, London
Conference Dates: 19-20 Sept, 2016
Conference Venue: Imperial College London, South Kensington Campus | London SW7 2AZ
Deadline for Abstract/Paper Submissions: Sept 16, 2016
Contact E-Mail ID: info@gahssr.org
Conference Convener: Dr. Dorothy C
Languages: English, Chinese, Arabic, Thai, Persian
http://gahssr.org/11th-international-conference-on-social-science-and-humanities-icssh-19-20-sept-2016-london-about-18
The document provides information about studying in the Netherlands for international students. It discusses impressions of the country, things to do before departure and upon arrival, and what to expect regarding life and study in the Netherlands. Key points covered include an overview of the Dutch education system, credits and grading scales, expectations around independent and interactive learning styles, and the relationship between teachers and students being quite informal.
Tra assenze e refusi, A Napoli la primavera CRUI parte maleGiuseppe De Nicolao
Per il 21 Marzo 2016 la CRUI ha indetto la Primavera dell’Università, promossa con un video patinato, ma non senza qualche refuso. A Napoli, politici assenti e docenti contestati. Anche il Presidente della CRUI dà forfait e va a Roma in TV per “Porta a Porta”.
La ricerca umanistica: i risultati della VQR 2004-2010 (di Paola Galimberti)Giuseppe De Nicolao
La ricerca umanistica: i risultati della VQR 2004-2010
Milano 23 ottobre 2013
La valutazione della ricerca nelle aree 10 e 11: dati, contesti, risultati
I risultati della valutazione del nucleo / i risultati della VQR
Valutazione della ricerca 2012-2014: tre anni vissuti pericolosamenteGiuseppe De Nicolao
Slide della relazione di Giuseppe De Nicolao tenuta nel corso dell'Assemblea dell’area CUN 09 (Ingegneria industriale e dell’informazione), Roma, 3 dicembre 2014, Università La Sapienza
Presentazione di Giuseppe De Nicolao al II Convegno Roars: “Higher Education and Research Policies in Europe: Challenges for Italy”, 21 febbraio 2014
CNR, Piazzale A. Moro 7, Roma
This presentation covered the UK university application process through UCAS. It began with an overview of the application timeline and deadlines. Key parts of the application like choosing courses, writing a personal statement, and obtaining a reference letter were discussed in detail. The presentation also addressed UCAS registration, navigating the application, and important application components like IB predicted grades. Overall, the presentation aimed to guide students through completing the UCAS application and understanding the requirements for UK university admission.
11th International Conference on Social Science and Humanities (ICSSH)Global R & D Services
Conference Name: 11th International Conference on Social Science and Humanities (ICSSH), 19-20 Sept, 2016, London
Conference Dates: 19-20 Sept, 2016
Conference Venue: Imperial College London, South Kensington Campus | London SW7 2AZ
Deadline for Abstract/Paper Submissions: Sept 16, 2016
Contact E-Mail ID: info@gahssr.org
Conference Convener: Dr. Dorothy C
Languages: English, Chinese, Arabic, Thai, Persian
http://gahssr.org/11th-international-conference-on-social-science-and-humanities-icssh-19-20-sept-2016-london-about-18
The document provides information about studying in the Netherlands for international students. It discusses impressions of the country, things to do before departure and upon arrival, and what to expect regarding life and study in the Netherlands. Key points covered include an overview of the Dutch education system, credits and grading scales, expectations around independent and interactive learning styles, and the relationship between teachers and students being quite informal.
Tra assenze e refusi, A Napoli la primavera CRUI parte maleGiuseppe De Nicolao
Per il 21 Marzo 2016 la CRUI ha indetto la Primavera dell’Università, promossa con un video patinato, ma non senza qualche refuso. A Napoli, politici assenti e docenti contestati. Anche il Presidente della CRUI dà forfait e va a Roma in TV per “Porta a Porta”.
La ricerca umanistica: i risultati della VQR 2004-2010 (di Paola Galimberti)Giuseppe De Nicolao
La ricerca umanistica: i risultati della VQR 2004-2010
Milano 23 ottobre 2013
La valutazione della ricerca nelle aree 10 e 11: dati, contesti, risultati
I risultati della valutazione del nucleo / i risultati della VQR
Valutazione della ricerca 2012-2014: tre anni vissuti pericolosamenteGiuseppe De Nicolao
Slide della relazione di Giuseppe De Nicolao tenuta nel corso dell'Assemblea dell’area CUN 09 (Ingegneria industriale e dell’informazione), Roma, 3 dicembre 2014, Università La Sapienza
Il finanziamento del sistema universitario: alla ricerca di un equilibrio.Giuseppe De Nicolao
Intervento di Gianfranco Viesti al
Terzo Convegno di ROARS, 19 Giugno 2015, Camera dei Deputati, Sala del Refettorio, Palazzo di Via del Seminario 76, Roma
Paola Galimberti: Via il velo - la trasparenza dei dati nella produzione scie...Giuseppe De Nicolao
Slides presentate nel convegno ROARS "Il sistema dell’Università e della Ricerca - Fatti leggende futuro" (Istituto della Enciclopedia Italiana, 15 novembre 2012)
Un quadro della situazione a quattro anni dalla legge 240: le principali crit...Giuseppe De Nicolao
Intervento di Andrea Stella al
Terzo Convegno di ROARS, 19 Giugno 2015, Camera dei Deputati, Sala del Refettorio, Palazzo di Via del Seminario 76, Roma
la prevista grande manifestazione proposta dalla CRUI per riaffermare il ruolo dell'università (o forse per tenere i docenti un po' più tranquilli nel momento clou del dibattito sulla VQR) si limiterà a Milano a una mezza giornata di dibattiti lontano dal Poli. La locandina dell'evento milanese nemmeno ne cita le motivazioni. Certo il ministro resterà impressionato da una simile "prova di forza", ma purtroppo molti di noi e tutti i nostri studenti non vi potranno partecipare.
Vi propongo quindi una nostra iniziativa fai-da-te, per la quale ho preparato solo sette slide powerpoint che riassumono la situazione di finanziamenti, ruolo e situazione dei docenti dell'università italiana. Le potete scaricare da:
http://home.deib.polimi.it/guariso/inverno universita.pptx
L'idea è, dopo averle corrette e modificate a vostro piacimento, di dedicare 10 minuti di un quarto d'ora accademico di lunedì 21 marzo e farle vedere agli studenti, per illustrare i motivi del nostro disagio e cercare di coinvolgerli nella battaglia per un'università (e quindi un paese) migliore.
Stante la situazione evidenziata nelle slide, mi sembra che il titolo più opportuno sia "L'inverno dell'università", ma ovviamente potete modificare anche questo.
Buon lavoro!
La compressione selettiva e cumulativa dell'università italianaGiuseppe De Nicolao
Intervento di Gianfranco Viesti all'incontro "Per il diritto allo studio e alla ricerca", Università degli Studi di Napoli Federico II Aula Pessina, Corso Umberto 1 11 febbraio 2016
WebShool Tours du 19/02/2015 - Les CMS : Pourquoi ? comment ?
Sommaire : Les bases et les solutions de la création de site web. Les solutions de gestion de contenu (CMS). Exemple de mise en pratique avec le logiciel libre Drupal.
This document summarizes Dan York's presentation on deploying DNSSEC. It discusses how DNSSEC helps ensure the integrity of DNS data by using digital signatures to authenticate DNS responses, preventing cache poisoning and man-in-the-middle attacks. It provides examples of normal DNS interactions versus attacks, and how DNSSEC establishes a chain of trust. The presentation also covers the current state of DNSSEC deployment, how it works in combination with TLS/SSL through DANE, and business reasons for organizations to deploy DNSSEC.
ION Bucharest, 12 October 2016 - Today, we can say that IPv6 is already happening all around the world. It’s interesting to see the main reasons that made it happen, how it’s happening, and to make the audience think about their deployment status and strategy. Statistics from different sources are showed, including data from RIPE NCC measurements.
Quell'oscuro oggetto del desiderio: le classifiche internazionali degli ateneiGiuseppe De Nicolao
This document discusses university rankings and issues with various ranking systems. It begins by looking at the World Reputation Rankings 2014 where the University of Pavia obtained a good position, ranking fifth among the 15 Italian universities present and gaining 76 positions from 2013. It then discusses problems that have occurred with rankings, such as one scholar artificially inflating the ranking of their university by frequently self-publishing. Overall, the document examines the methodologies and shortcomings of different international university rankings.
Paper 7: Ranking Methodology of Times Higher Education (Baty)Kent Business School
The document discusses plans to improve the Times Higher Education World University Rankings by developing a new methodology in consultation with academics and university managers. It outlines criticisms of the previous methodology developed with QS, including the small and non-representative peer review survey sample and biases in the citation data. The new rankings will be developed in partnership with Thomson Reuters and will incorporate additional institutional data variables and make the methodology more transparent.
What is ‘research impact’ in an interconnected world?Danny Kingsley
This talk looks at what researchers need to do to ensure their research is widely disseminated and reaches the largest audience possible. In summary: Publishing a paper is the beginning not the end; Making work open access does not mean it is accessible; Writing in plain language is translating, not dumbing it down; Sharing work involves peer networks and publishing platforms and If you don't take control of your online presence someone/something else will. The presentation was originally given as part of the Cambridge University Alumni Festival on 27 September 2015.
“Meritocracy, evaluation, excellence: The case of universities and research”Francesco Sylos Labini
This document discusses the rise of meritocracy and inequality. It argues that the idea of meritocracy is used to justify growing inequalities, as resources are increasingly concentrated among a few based on measures of "merit" and "excellence". This is seen in higher education and research systems which are restructured according to rankings and quantitative evaluations. While metrics claim to measure quality, in reality they often only capture certain quantities. The document advocates for more diversity and risk-taking in research to enable disruptive discoveries, rather than focusing only on rewarding current "excellence".
The document discusses the impact and implications of university rankings. It notes that while rankings aim to measure quality and compare institutions, they often reduce quality to a few quantifiable indicators and ignore important factors like teaching quality, student experience, and community engagement. As a result, rankings can distort institutions' priorities and behaviors. The document reviews research showing that rankings significantly influence students, employers, universities, governments, and academic work. Many countries are using rankings to restructure their higher education systems and concentrate resources in a small number of elite institutions.
This document discusses the methodology and results of the Academic Ranking of World Universities (ARWU) conducted by Shanghai Jiao Tong University. It outlines the purposes of developing the ARWU, which were to evaluate Chinese universities' positions globally and measure the gap to becoming world-class. The methodology uses 6 objective indicators and internationally comparable data to rank over 1000 universities. It acknowledges issues with the methodology and ways to improve the rankings, such as addressing biases against certain fields and languages.
Il finanziamento del sistema universitario: alla ricerca di un equilibrio.Giuseppe De Nicolao
Intervento di Gianfranco Viesti al
Terzo Convegno di ROARS, 19 Giugno 2015, Camera dei Deputati, Sala del Refettorio, Palazzo di Via del Seminario 76, Roma
Paola Galimberti: Via il velo - la trasparenza dei dati nella produzione scie...Giuseppe De Nicolao
Slides presentate nel convegno ROARS "Il sistema dell’Università e della Ricerca - Fatti leggende futuro" (Istituto della Enciclopedia Italiana, 15 novembre 2012)
Un quadro della situazione a quattro anni dalla legge 240: le principali crit...Giuseppe De Nicolao
Intervento di Andrea Stella al
Terzo Convegno di ROARS, 19 Giugno 2015, Camera dei Deputati, Sala del Refettorio, Palazzo di Via del Seminario 76, Roma
la prevista grande manifestazione proposta dalla CRUI per riaffermare il ruolo dell'università (o forse per tenere i docenti un po' più tranquilli nel momento clou del dibattito sulla VQR) si limiterà a Milano a una mezza giornata di dibattiti lontano dal Poli. La locandina dell'evento milanese nemmeno ne cita le motivazioni. Certo il ministro resterà impressionato da una simile "prova di forza", ma purtroppo molti di noi e tutti i nostri studenti non vi potranno partecipare.
Vi propongo quindi una nostra iniziativa fai-da-te, per la quale ho preparato solo sette slide powerpoint che riassumono la situazione di finanziamenti, ruolo e situazione dei docenti dell'università italiana. Le potete scaricare da:
http://home.deib.polimi.it/guariso/inverno universita.pptx
L'idea è, dopo averle corrette e modificate a vostro piacimento, di dedicare 10 minuti di un quarto d'ora accademico di lunedì 21 marzo e farle vedere agli studenti, per illustrare i motivi del nostro disagio e cercare di coinvolgerli nella battaglia per un'università (e quindi un paese) migliore.
Stante la situazione evidenziata nelle slide, mi sembra che il titolo più opportuno sia "L'inverno dell'università", ma ovviamente potete modificare anche questo.
Buon lavoro!
La compressione selettiva e cumulativa dell'università italianaGiuseppe De Nicolao
Intervento di Gianfranco Viesti all'incontro "Per il diritto allo studio e alla ricerca", Università degli Studi di Napoli Federico II Aula Pessina, Corso Umberto 1 11 febbraio 2016
WebShool Tours du 19/02/2015 - Les CMS : Pourquoi ? comment ?
Sommaire : Les bases et les solutions de la création de site web. Les solutions de gestion de contenu (CMS). Exemple de mise en pratique avec le logiciel libre Drupal.
This document summarizes Dan York's presentation on deploying DNSSEC. It discusses how DNSSEC helps ensure the integrity of DNS data by using digital signatures to authenticate DNS responses, preventing cache poisoning and man-in-the-middle attacks. It provides examples of normal DNS interactions versus attacks, and how DNSSEC establishes a chain of trust. The presentation also covers the current state of DNSSEC deployment, how it works in combination with TLS/SSL through DANE, and business reasons for organizations to deploy DNSSEC.
ION Bucharest, 12 October 2016 - Today, we can say that IPv6 is already happening all around the world. It’s interesting to see the main reasons that made it happen, how it’s happening, and to make the audience think about their deployment status and strategy. Statistics from different sources are showed, including data from RIPE NCC measurements.
Quell'oscuro oggetto del desiderio: le classifiche internazionali degli ateneiGiuseppe De Nicolao
This document discusses university rankings and issues with various ranking systems. It begins by looking at the World Reputation Rankings 2014 where the University of Pavia obtained a good position, ranking fifth among the 15 Italian universities present and gaining 76 positions from 2013. It then discusses problems that have occurred with rankings, such as one scholar artificially inflating the ranking of their university by frequently self-publishing. Overall, the document examines the methodologies and shortcomings of different international university rankings.
Paper 7: Ranking Methodology of Times Higher Education (Baty)Kent Business School
The document discusses plans to improve the Times Higher Education World University Rankings by developing a new methodology in consultation with academics and university managers. It outlines criticisms of the previous methodology developed with QS, including the small and non-representative peer review survey sample and biases in the citation data. The new rankings will be developed in partnership with Thomson Reuters and will incorporate additional institutional data variables and make the methodology more transparent.
What is ‘research impact’ in an interconnected world?Danny Kingsley
This talk looks at what researchers need to do to ensure their research is widely disseminated and reaches the largest audience possible. In summary: Publishing a paper is the beginning not the end; Making work open access does not mean it is accessible; Writing in plain language is translating, not dumbing it down; Sharing work involves peer networks and publishing platforms and If you don't take control of your online presence someone/something else will. The presentation was originally given as part of the Cambridge University Alumni Festival on 27 September 2015.
“Meritocracy, evaluation, excellence: The case of universities and research”Francesco Sylos Labini
This document discusses the rise of meritocracy and inequality. It argues that the idea of meritocracy is used to justify growing inequalities, as resources are increasingly concentrated among a few based on measures of "merit" and "excellence". This is seen in higher education and research systems which are restructured according to rankings and quantitative evaluations. While metrics claim to measure quality, in reality they often only capture certain quantities. The document advocates for more diversity and risk-taking in research to enable disruptive discoveries, rather than focusing only on rewarding current "excellence".
The document discusses the impact and implications of university rankings. It notes that while rankings aim to measure quality and compare institutions, they often reduce quality to a few quantifiable indicators and ignore important factors like teaching quality, student experience, and community engagement. As a result, rankings can distort institutions' priorities and behaviors. The document reviews research showing that rankings significantly influence students, employers, universities, governments, and academic work. Many countries are using rankings to restructure their higher education systems and concentrate resources in a small number of elite institutions.
This document discusses the methodology and results of the Academic Ranking of World Universities (ARWU) conducted by Shanghai Jiao Tong University. It outlines the purposes of developing the ARWU, which were to evaluate Chinese universities' positions globally and measure the gap to becoming world-class. The methodology uses 6 objective indicators and internationally comparable data to rank over 1000 universities. It acknowledges issues with the methodology and ways to improve the rankings, such as addressing biases against certain fields and languages.
The value of Scopus: an eye on global research "Search | Discover | Analyze"Genevieve Musasa
Scopus is the largest abstract and citation database
of peer-reviewed research literature from around the world
and features bibliometrics tools that allow you to track, analyze and visualize scholarly research
This document summarizes Kristy Rawson's presentation on "Alt-Ac" and "Post-Ac" careers for humanities PhDs. It provides an overview of Rawson's background and credentials, defines key terms like "Alt-Ac" and "Post-Ac", lists various career paths outside of academia that utilize research and writing skills, and outlines practical steps and resources for exploring non-faculty career options like networking, informational interviews, projects, and internships. The presentation emphasizes using skills like coding, blogging, data analysis and visualization to expand career opportunities.
From research student to academic: thinking about and preparing for academic ...Joss Winn
Preparing for academic life (or not). See also: http://josswinn.org/2015/07/from-research-student-to-academic-thinking-about-and-preparing-for-academic-work/
The Impact of World University Rankings on Research and Curriculum Developmen...Leslie Chan
The talk identifies the impact of “world university rankings” on research and curriculum development, particularly for universities in the global South. It is argued that relevance of local research and community engagement are better indicators of ‘excellence’ in the university and we need to rethink assumptions behind "objective" indicators that are underlying most of the major world rankings. These rankings have the effect of rendering research from the developing world invisible and dictating curriculum development oriented towards market needs of the global North. Such rankings thus represent a form of social and cognitive exclusion and institution of higher education should strongly resist this agenda, and formulate other forms of "excellence" based on social inclusion and community engagement.
Presentation on the usefulness of benchmarking for Research Deans - part of a course on Research Leadership by the European Foundation for Management Development
1. The diversity and equality agenda cannot ignore questions about quality, as pursuing diversity raises equality issues and pursuing equality raises fears about standards.
2. Quality is a complex concept that goes beyond rankings, and a university's performance should be viewed relative to its context and mission to contribute to society.
3. Supporters of diversity must confront claims that equality erodes quality by articulating a broader view of quality as excellence relative to context, upholding standards through support programs, and contributing to social justice and innovation.
- International university rankings provide composite measures of academic quality, research output, teaching quality, and international reputation based on metrics like peer review surveys, citation counts, faculty-student ratios, and percentages of international students and faculty.
- While rankings are not intended as strict league tables, they have significant real-world impacts by influencing government and university policies and priorities around the world as well as student choice.
- Rankings also create competition between institutions to improve their standing through strategies like increasing research productivity, attracting top faculty and students internationally, and enhancing teaching quality and graduate employability.
Valuing professional staff in higher education august atem conference septemb...Melissa Bradley
This document provides an overview of a presentation by Melissa Bradley and Carroll Graham on valuing the roles of professional staff in higher education from UK and Australian perspectives. The presentation covers topics such as setting the context of professional staff roles through historical examples, examining the debate on professional identity and value, exploring career pathways and interfaces between administration and academics. It also reports on a straw poll of attendees and poses questions on promoting higher education administration as a career choice and enhancing professional identity through recognition and qualifications. Suggestions for further reading on the topic from UK and Australian sources are also provided.
Universties global ranking systems for pub.Ahmed Metwaly
The document discusses several global university rankings systems, including the Academic Ranking of World Universities (ARWU), Times Higher Education World University Rankings, and QS World University Rankings. It provides details on the methodology, metrics, and historical development of each ranking system. In particular, it explains that ARWU was first published in 2003 based on benchmarking of Chinese and US universities, Times Higher Education rankings began in 2004 in response to ARWU, and QS rankings were previously a collaboration with Times Higher Education before becoming independent.
The document discusses the Times Higher Education World University Rankings for 2010-2011. It provides background on why university rankings are important given increasing globalization of higher education. It outlines improvements made to the 2010-2011 methodology, including greater emphasis on research excellence and citations over reputation surveys. The results are announced, with Harvard, Caltech, and MIT taking the top three spots.
2nd Year Orientations - Natural Sciences - Imperial College London (2013)ICGS
This document provides information for doctoral students in their second year of study. It discusses common feelings mid-stage doctoral students may experience like uncertainty about next steps. It encourages students to focus on consolidating their skills and preparing for their career. The document outlines resources and support available from the Graduate School, including professional development courses, library resources, and alumni speakers. Students hear perspectives from representatives of different support services on developing career plans, using library resources, balancing coursework and research, and getting involved with the Graduate Student Union. The presentation aims to help second year doctoral students feel supported and informed about next steps.
“The University is a community of scholars engaged in the task of seeking truth”. Karl Jaspers, 1923
“I find the three major administrative problems on campus are sex for the students, athletics for the alumni and car parking for the faculty”.
Clark Kerr President,
University of California, 1958
Faculty of Humanities, University of Manchester postgraduate research open da...Anusarin Lowe
Learn about doing a PhD in humanities and social science at the University of Manchester - what it entails, support available and development opportunities.
Faculty of Humanities, University of Manchester postgraduate research open day 2017
Similar to Harassing with Numbers: the Uses and Abuses of Bureaucracy and Bibliometry (20)
I misteri dei dipartimenti di eccellenza: genesi, anatomia, ideologiaGiuseppe De Nicolao
Per primeggiare nella classifica 2017 dei dipartimenti di eccellenza, un premio Nobel come Giorgio Parisi contava assai poco, dato che un bravo storico della fisica pesava tre volte tanto. Ciò nonostante, la classifica "si fonda su un modello matematico solido", se si vuole prestar fede alla rassicurazione inviata dall'Anvur al Consiglio Universitario Nazionale. Talmente solido che l'Anvur e il Ministero hanno preferito tenere nascosto che ben 119 dipartimenti su 767 avevano meritato zero in pagella e che nel 2022 la situazione è verosimilmente peggiorata. Ma come nasce e come funziona questa classifica così strana, che oltre a penalizzare i Nobel, funge da "ammazza-Sud"? Nel seminario, vengono spiegati a uno a uno gli ingranaggi dell'algoritmo che genera i punteggi dell'ISPD, l'indicatore standardizzato di performance dipartimentale. La sua nascita risale al cosiddetto voto standardizzato, ideato nel 2014 da una commissione della CRUI. Nascita segnata da un peccato originale che si cercò di rendere invisibile attraverso modifiche che condussero all’indicatore ISPD, infine adottato nel 2017 come generatore della classifica dei dipartimenti di eccellenza. La ricostruzione storica e la dissezione anatomica aiuteranno a capire perché si diventa eccellenti o paria; per esempio perché nel 2017 il miglior dipartimento di fisica era quello che aveva pochi fisici e tanti psicologi. Verrà anche spiegato perché la presenza o l’assenza di ben precisi settori scientifici rende più o meno probabile la scalata verso l’eccellenza. Da ultimo, si cerca di capire quali motivazioni stanno alla base dell’adozione di una classifica così irrazionale.
_________
I misteri dei dipartimenti di eccellenza: genesi, anatomia, ideologia
Seminario di Giuseppe De Nicolao.
Organizzato da Roars, Rete 29 Aprile e Circolo Errera.
24 giugno 2022
Contenuti del video:
- Presentazione di Massimiliano Tabusi (Rete 29 Aprile)
- Introduzione
- La pietra filosofale della competizione
- Un grecista eccellente? Vale 4 Nobel per la Fisica!
- La legge dell’imbuto
- Perché ISPD è distribuito «a vasca da bagno»?
- Perché la classifica deve rimanere segreta?
- Una svista che vale un ricorso al TAR
- Ammazzare il Sud giustifica i mezzi
Video: https://www.youtube.com/watch?v=03E_5srlLPo
Slides dell'intervento di Giuseppe De Nicolao nella giornata di studi
Le ragioni dell'emergenza, le ragiomi della didattica
Università di Padova, 26 novembre 2020
La didattica a distanza, aspetti critici, limiti e potenzialitàGiuseppe De Nicolao
In vista dell'anno accademico 2020/21 si parla molto di didattica universitaria cosiddetta "blended", nel senso di erogata simultaneamente in presenza per una parte degli studendi e a distanza, per esempio in streaming, per quelli che non sono in aula. Nel video, si prova a rispondere a tre domande.
1. Come variano le probabilità di contagio in ambienti chiusi frequentati da più persone in funzione della cubatura, del ricambio d'aria e del tempo di permanenza?
2. Quali sono e come sono gestibili i principali problemi audio e video della didattica a distanza?
3. Quali relazioni tra didattica blended, datafication e università delle piattaforme?
Intervento di Giuseppe De Nicolao presentato nel corso del convegno “L' Università tra crisi e rinascita: quale direzione per il futuro?”, organizzato dal Comitato Nazionale Universitario (CNU), 10 luglio 2020
Più citazioni se volete far carriera! La valutazione della ricerca universitariaGiuseppe De Nicolao
La valutazione della ricerca Universitaria: webinar con Giuseppe De Nicolao di ROARS: 29 maggio ore 18:00
Quarto appuntamento di “Immaginiamo il futuro, dopo la crisi e oltre”, incontri in webinar con personalità della cultura e della politica per ragionare e discutere del futuro e degli scenari del dopo crisi. Gli incontri sono organizzati dai prof. Plinio Innocenzi e Quirico Migheli.
Slides dell'intervento di Rossella Latempa al Convegno "SCUOLA DI COMPETENZE: VERSO UN NUOVO MODELLO DIDATTICO. QUALE?", Gilda degli Insegnanti, Vicenza 18 marzo 2019.
State-fostered immaturity? Kant, Galileo and the Grand EvaluatorGiuseppe De Nicolao
Slide dell'intervento di Giuseppe De Nicolao al convegno “Academic Freedom Today - Insights from Law, Philosophy, and Institutional Practice”, 6-7 dicembre 2018, Libera Università di Bolzano, https://academicfreedom.events.unibz.it/
_________
State-fostered immaturity? Kant, Galileo and the Grand Evaluator
In 2017, the president of Anvur, the Italian agency for research evaluation, was asked if bibliometrics-based research evaluation could discourage innovative research. The answer was disconcerting, yet revealing: a scientific genius “will be rewarded twenty years from now, when he will become the most famous scientist in the world. In the meantime, he should be grateful that he maintained his academic position without being burnt alive. Frankly speaking, we are not all Galilei and Newton”. No less worrying is the creeping effect of research evaluation on the public use of reason by academics. Linking administrative actions such as hiring, career, funding and wages to the outcomes of centralized research evaluation can subtly impair academic freedom in fields such as health, environment, economics, education and research policy. These issues will be illustrated through the analysis of methods, outcomes and unintended effects of research evaluations run by Anvur.
Una rivista ad accesso aperto, senza costi per gli autori e di alta qualità: ...Giuseppe De Nicolao
«Times have changed. Articles now circulate easily via the Internet, but unfortunately MLJ publications are under restricted access. […] In summary, our resignation from the editorial board of MLJ reflects our belief that journals should principally serve the needs of the intellectual community, in particular by providing the immediate and universal access to journal articles that modern technology supports, and doing so at a cost that excludes no one.» Questa lettera di dimissioni, firmata dalla maggioranza degli editor della rivista Machine Learning (Kluwer ), sancì la nascita del Journal of Machine Learning Research, una rivista open access, subito accreditatasi tra le più qualificate sedi di pubblicazione per le ricerche nel settore del machine learning. Una vicenda che risale al 2000, ma che, ad anni di distanza, non cessa di far discutere. Come quando nel 2011 Kent Anderson (Scholarly Kitchen) mise in dubbio la sostenibilità di riviste open access e senza costi per gli autori. «In my field (computer science) one of the most prominent journals is entirely free and open access (Journal of Machine Learning Research)» fu la secca replica di Yann LeCun. Ne nacque una discussione, proseguita sul blog di Stuart Shieber, che appare emblematica sotto diversi aspetti e di cui proveremo a riassumere i punti salienti.
Conferenza AISA (Associazione Italiana per la promozione della Scienza Aperta), ospitata dal Dipartimento di Scienze Politiche dell'Università di Pisa, dal titolo Publishing without perishing? La scienza aperta all’epoca della valutazione della ricerca (link is external), con gli interventi di Alberto Baccini (Università di Siena), Roberto Caso (Università di Trento), Giuseppe De Nicolao (Università di Pavia) e Paola Galimberti (Università di Milano) e con la partecipazione di Paolo Rossi (Università di Pisa) in veste di discussant. Giovedì 19 gennaio 2017 dalle ore 15 in aula 2, Polo Piagge
Credo quia absurdum, l'apologetica delle "Weapons of math destruction” Giuseppe De Nicolao
Intervento presentato al Convegno "Teocrazia e Tecnocrazia", organizzato da Fondazione Roma Sapienza e da Nuova Accademia.
giovedì 3 maggio alle ore 10.30 presso l’aula Multimediale Palazzo del Rettorato
Sapienza Università di Roma
Abstract. “There is no alternative” è la legittimazione principe della tecnocrazia. Non sorprende, pertanto, che solo di rado i tecnocrati cerchino di legittimare, sul piano razionale ed etico, il loro ruolo e i loro strumenti. Tra i pochi che si sono cimentati, vi sono due membri del Consiglio direttivo dell’Anvur, l’Agenzia nazionale per la valutazlone del sistema universitario e della ricerca. Sul versante del dibattito razionale, quella che potremmo chiamare “apologetica della valutazione” si scontra con l’obiezione “Caesar non est supra grammaticos”, attuale ogni qual volta chi detiene il potere pretende di prendere parte al dibattito scientifico, che per sua natura deve svolgersi tra pari. Sul versante etico, il vulnus che la valutazione di stato infligge alla libertà della scienza è giustificato dalla necessità di guidare una comunità scientifica (quella italiana) debole e bisognosa di indirizzi, un argomento curiosamente simile a quello invocato dal Grande Inquisitore per legittimare il suo regime teocratico. Una coincidenza già notata da Aldous Huxley che, commentando nel 1959 la distopia tecnocratica descritta nel suo romanzo “Brave New World”, ne riconduceva il dilemma etico fondamentale proprio al famoso personaggio di Dostoevskij.
Intervento di Rossella Latempa al CONVEGNO DI AGGIORNAMENTO CESP: PER RIPARTIRE DALLA SCUOLA. DISCUSSIONE INTORNO A SETTE TEMI:
9 febbraio 2018 - IIS USUELLI-RUZZA - Padova
http://www.cesp-pd.it/spip/spip.php?article1464
Intervento di Rossella Latempa al CONVEGNO DI AGGIORNAMENTO CESP: PER RIPARTIRE DALLA SCUOLA. DISCUSSIONE INTORNO A SETTE TEMI:
9 febbraio 2018 - IIS USUELLI-RUZZA - Padova
http://www.cesp-pd.it/spip/spip.php?article1464
The EUA Public Funding Observatory was launched in 2008 with the aim to monitor the impact of the financial crisis on higher education in different countries across Europe. Since then, EUA has been collecting quantitative and qualitative data on public funding received by European higher education institutions, and analysing both long- term trends and recent changes.
The funding data and other relevant figures are made available to EUA by its collective members, the national rectors’ conferences, whose support has been invaluable. Processed and analysed in view of evolving student numbers, as well as the overall economic context adjusted to inflation and GDP growth, this data provides some empirical evidence on public funding trajectories in the field of higher education in Europe.
Dai sistemi di valutazione agli effetti sull'etica dei comportamentiGiuseppe De Nicolao
Slide dell'interventod di Giuseppe De Nicolao al convegno "Frode Scientifica, come nasce e come si previene", Venerdì 17 e Sabato 18 Novembre a Pavia. Partecipazione gratuita, previa registrazione obbligatoria http://www-3.unipv.it/max3/fs/programma-1.html
Terza missione: è lei la prima vittima della "cultura della valutazione"?Giuseppe De Nicolao
Intervento di Giuseppe De Nicolao al Convegno "Culture della valutazione. Università e Terza Missione. Conoscenza, formazione, territorio°. IULM, Martedì 10 OTTOBRE 2017
P. Galimberti: Valutazione/classificazione delle riviste scientificheGiuseppe De Nicolao
Intervento di Paola Galimberti (Università degli Studi di Milano) nel WORKSHOP INTERNO AIPDA (ASSOCIAZIONE ITALIANA DEI PROFESSORI DI DIRITTO AMMINISTRATIVO)
LA VALUTAZIONE DELLE RIVISTE
dalla Scientificità alla Classificazione
(Ragioni, Metodi, Modelli, Impatto)
13 gennaio 2017, Milano, ore 11.15-17.00 Università Milano IULM
Sala delle Conferenze-VI° piano-IULM 1-Via Carlo Bo, 1
ANVUR ASN
Bibliometria Bufale Benito
CRUI Cattaneo Checchi Costituzione Cingolani Cantone Consiglio di Stato Cammellate Coda colpo di
Diritto allo studio Declino Duce
Elena Cattaneo ERC
Ferraro/ Figlio di MIUR/Fascia A Formazione Feticismo
Graziosi/ Giannini
Human Technopole
IIT Impact Factor Inguscio immortale innovativi
Junk arithmetic
K
Londra radio Licenza di copiare Ludi dipartimentali
Miccoli Manfredi
Natta Cattedre nannicini Nature
OCSE Oro alla Patria Omonimie
Perquisizioni Primavera PISA Pavia PRRR! Professionalizzanti Premiale
Quinlan
Renzi - Referendum Rignano Science Foundation Rottamazione del Sud
Stop-VQR Sole 24 Ore Scatti
Turchia Tasse Times Higher Education Tesoretto IIT Truppe Terremoto
Ultimi OCSE Uomo quasi immortale
VQR fantasma Viesti
W Roars
Xylella
Y
Blob Roars 2015, di tutto, di più. Un anno di università e ricerca attraverso una selezione delle copertine di Roars
Level 3 NCEA - NZ: A Nation In the Making 1872 - 1900 SML.pptHenry Hollis
The History of NZ 1870-1900.
Making of a Nation.
From the NZ Wars to Liberals,
Richard Seddon, George Grey,
Social Laboratory, New Zealand,
Confiscations, Kotahitanga, Kingitanga, Parliament, Suffrage, Repudiation, Economic Change, Agriculture, Gold Mining, Timber, Flax, Sheep, Dairying,
How to Setup Default Value for a Field in Odoo 17Celine George
In Odoo, we can set a default value for a field during the creation of a record for a model. We have many methods in odoo for setting a default value to the field.
Creative Restart 2024: Mike Martin - Finding a way around “no”Taste
Ideas that are good for business and good for the world that we live in, are what I’m passionate about.
Some ideas take a year to make, some take 8 years. I want to share two projects that best illustrate this and why it is never good to stop at “no”.
How to Download & Install Module From the Odoo App Store in Odoo 17Celine George
Custom modules offer the flexibility to extend Odoo's capabilities, address unique requirements, and optimize workflows to align seamlessly with your organization's processes. By leveraging custom modules, businesses can unlock greater efficiency, productivity, and innovation, empowering them to stay competitive in today's dynamic market landscape. In this tutorial, we'll guide you step by step on how to easily download and install modules from the Odoo App Store.
Information and Communication Technology in EducationMJDuyan
(𝐓𝐋𝐄 𝟏𝟎𝟎) (𝐋𝐞𝐬𝐬𝐨𝐧 2)-𝐏𝐫𝐞𝐥𝐢𝐦𝐬
𝐄𝐱𝐩𝐥𝐚𝐢𝐧 𝐭𝐡𝐞 𝐈𝐂𝐓 𝐢𝐧 𝐞𝐝𝐮𝐜𝐚𝐭𝐢𝐨𝐧:
Students will be able to explain the role and impact of Information and Communication Technology (ICT) in education. They will understand how ICT tools, such as computers, the internet, and educational software, enhance learning and teaching processes. By exploring various ICT applications, students will recognize how these technologies facilitate access to information, improve communication, support collaboration, and enable personalized learning experiences.
𝐃𝐢𝐬𝐜𝐮𝐬𝐬 𝐭𝐡𝐞 𝐫𝐞𝐥𝐢𝐚𝐛𝐥𝐞 𝐬𝐨𝐮𝐫𝐜𝐞𝐬 𝐨𝐧 𝐭𝐡𝐞 𝐢𝐧𝐭𝐞𝐫𝐧𝐞𝐭:
-Students will be able to discuss what constitutes reliable sources on the internet. They will learn to identify key characteristics of trustworthy information, such as credibility, accuracy, and authority. By examining different types of online sources, students will develop skills to evaluate the reliability of websites and content, ensuring they can distinguish between reputable information and misinformation.
Information and Communication Technology in Education
Harassing with Numbers: the Uses and Abuses of Bureaucracy and Bibliometry
1. Harassing with Numbers:
the Uses and Abuses of
Bureaucracy and Bibliometry
Giuseppe De Nicolao
Università di Pavia
2. • Quality assurance, accountability, rankings,
efficient allocation of resources, objective
measures of performance, ...
• Boring stuff, lots of bureaucracy .. but
changes our lives and impacts also research
and teaching
• Let’s begin with a story
3. Outline
• The most accurate ranking ever produced
• Should you believe in university rankings?
• Governing by numbers
• The end is not near. It is here
• The Good, the Bad, and the Ugly
• The quest for the Holy Grail
• You’ll gonna hear me roar!
10. New York Times, November 14, 2010
Alexandria’s surprising prominence was actually
due to “the high output from one scholar in one
journal” — soon identified on various blogs as
Mohamed El Naschie, an Egyptian academic
who published over 320 of his own articles in a
scientific journal of which he was also the
editor.
15. • ... of the 400 papers by El Naschie indexed in
Web of Science, 307 were published in Chaos,
Solitons and Fractals alone while he was editor-
in-chief.
• El Naschie’s papers in CSF make 4992 citations,
about 2000 of which are to papers published in
CSF, largely his own.
16. All ingredients in one story
• reputation race at work
• “the most accurate picture ... ever produced”
• gaming affecting:
– university rankings
– journal Impact Factor
– individual bibliometrics
Objection: this is just an outlier
Be serious: no one can be so stupid to let important
decisions depend on questionable rankings
18. Who is a “highly skilled migrant”
in the Netherlands?
Decided by the rankings
19. Highly skilled migrants
Can I become a highly skilled migrant in the Netherlands - even if I
haven't got a job yet?
To be eligible, you must be in possession of one of the following
diplomas or certificates:
• a master's degree or doctorate from a recognised Dutch institution
of higher education or
• a master's degree or doctorate from a non-Dutch institution of
higher education which is ranked in the top 150 establishments in
either the Times Higher Education 2007 list or the Academic
Ranking of World Universities 2007 issued by Jiao Ton Shanghai
University in 2007
20. Sardegna (an Italian region): am I
eligible for a scholarship
to attend a PhD?
Decided by the rankings
21. APPLICATION WILL BE SCORED
BASED ON PRESTIGE OF PHD
SCHOOL ACCORDING TO QS WORLD
UNIVERSITY RANKINGS
22. Outline
• The most accurate ranking ever produced
• Should you believe in university rankings?
• Governing by numbers
• The end is not near. It is here
• The Good, the Bad, and the Ugly
• The quest for the Holy Grail
• You’ll gonna hear me roar!
29. Should you believe in the Shanghai ranking? An MCDM view
J.-C. Billaut D. Bouyssou P. Vincke
• all criteria used are only loosely connected with what they
intended to capture.
• several arbitrary parameters and many micro-decisions that
are not documented.
• flawed and nonsensical aggregation method
• «the Shanghai ranking is a poorly conceived quick and dirty
exercise»
«any of our MCDM student that would have proposed such a
methodology in her Master’s Thesis would have surely failed
according to our own standards»
34. Twenty Ways to Rise in the Rankings (1/3)
by Richard Holmes http://rankingwatch.blogspot.it/2013/12/twenty-ways-to-rise-in-rankings-quickly.html
1. Get rid of students. The university will therefore do better in
the faculty student ratio indicators.
2. Kick out the old and bring in the young. Get rid of ageing
professors, especially if unproductive and expensive, and hire
lots of temporary teachers and researchers.
5. Get a medical school. Medical research produces a
disproportionate number of papers and citations which is good
for the QS citations per faculty indicator and the ARWU
publications indicator. Remember this strategy may not help
with THE who use field normalisation.
35. 7. Amalgamate. What about a new mega university formed by
merging LSE, University College London and Imperial College? Or
a tres grande ecole from all those little grandes ecoles around
Paris?
9. The wisdom of crowds. Focus on research projects in those
fields that have huge multi - “author” publications, particle
physics, astronomy and medicine for example. Such
publications often have very large numbers of citations.
10. Do not produce too much. If your researchers are
producing five thousand papers a year, then those five hundred
citations from a five hundred “author” report on the latest
discovery in particle physics will not have much impact.
Twenty Ways to Rise in the Rankings (2/3)
by Richard Holmes http://rankingwatch.blogspot.it/2013/12/twenty-ways-to-rise-in-rankings-quickly.html
36. 13. The importance of names. Make sure that your researchers
know which university they are affiliated to and that they know
its correct name. Keep an eye on Scopus and ISI and make sure
they know what you are called.
18. Support your local independence movement. Increasing the
number of international students and faculty is good for both
the THE and QS rankings. If it is difficult to move students
across borders why not create new borders?
20. Get Thee to an Island. Leiden Ranking has a little known
ranking that measures the distance between collaborators. At
the moment the first place goes to the Australian National
University.
Twenty Ways to Rise in the Rankings (3/3)
by Richard Holmes http://rankingwatch.blogspot.it/2013/12/twenty-ways-to-rise-in-rankings-quickly.html
37. Rankings
• Fragile scientific grounds
• Cost of providing data
• Incentive to gaming
• Raw data are obscured
Why, then?
38. Outline
• The most accurate ranking ever produced
• Should you believe in university rankings?
• Governing by numbers
• The end is not near. It is here.
• The Good, the Bad, and the Ugly
• The quest for the Holy Grail
• You’ll gonna hear me roar!
42. Conflicting opinions
• Non-aggregators:
key objection to aggregation: the arbitrary
nature of the weighting process by which the
variables are combined
• Aggregators:
value in combining indicators: extremely
useful in garnering media interest and hence
the attention of policy makers
43.
44. Can you really govern by numbers?
Let’s see a survey of reactions
to university rankings
45. Germany
• “We look back decades and people came to
German universities; today they go to US
universities.”
• The Exzellenzinitiative (2005): from
traditional emphasis on egalitarianism
towards competition and hierarchical
stratification
46. France
• The Shanghai ranking
“generated considerable embarrassment
among the French intelligentsia, academia
and government: the first French higher
education institution in the ranking came only
in 65th position, mostly behind American
universities and a few British ones”
47. Australia
• The Shanghai and QS: at least two Australian
universities among the top 100.
• Opposing strategic options:
– fund a small number of top-tier competitive
universities
– “creation of a diverse set of high performing,
globally-focused institutions, each with its own
clear, distinctive mission”.
48. Japan
• “The government wants a first class university
for international prestige ”
• “in order for Japanese HEIs to compete
globally, the government will close down some
regional and private universities and direct
money to the major universities”
• some institutions will become teaching only.
49. The Education and University
Minister: «We are lagging behind in
the world rankings. For this reason
we are going to present the reform of
the University [...] I wish I will never
see again that the first Italian
university is ranked 174-th »
Italy
54. E. Hazelkorn on rankings
• 90% or 95% of our students do not attend elite
institutions. Why are we spending so much on what
people aren’t attending as opposed to what they are
attending?
• May detract resources from pensions, health,
housing, ....
• Are “elite” institutions really driving national or
regional economic and social development?
55. Does trickle-down work?
E. Hazelkorn: “Governments and universities must stop obsessing
about global rankings and the top 1% of the world's 15,000
institutions. Instead of simply rewarding the achievements of
elites and flagship institutions, policy needs to focus on the
quality of the system-as-a-whole.”
There is little evidence
that trickle-down works.
56. Where are we?
• (Even) Phil Baty (Times Higher Education)
admits that there are aspects of academic life
where rankings are of little value
• Can we/you afford the ‘reputation race’?
• We will have to live in a world in which
extremely poor rankings are regularly
published and used.
What can be done then?
57. some advices from the authors of
“Should you believe
in the Shanghai Ranking?”
58. “Stop being naive”
• There is no such thing as a ‘‘best university’’ in
abstracto.
• Stop talking about these ‘‘all purpose
rankings’’. They are meaningless.
• Lobby in our own institution so that these
rankings are never mentioned in institutional
communication
63. from “Is There Life After Rankings?”
• Not cooperating with the rankings affects my
life and the life of the college in several ways.
Some are relatively trivial; for instance, we are
saved the trouble of filling out U.S. News's
forms, which include a statistical survey that has
gradually grown to 656 questions
• The most important consequence of sitting out
the rankings game, however, is the freedom to
pursue our own educational philosophy, not
that of some newsmagazine.
64. Outline
• The most accurate ranking ever produced
• Should you believe in university rankings?
• Governing by numbers
• The end is not near. It is here
• The Good, the Bad, and the Ugly
• The quest for the Holy Grail
• You’ll gonna hear me roar!
70. Let us ask SCOPUS:
no evidence of collapse
Italy’s scientific documents 1996-2010
71. No trace of collapse fo all other countries
(Europe and non-Europe)
Then, what’s the explanation of the graph
in the scientific paper?
72. EXAMPLE: DUE TO WELL KNOWN RECORDING DELAYS IN
BIBLIOMETRIC DATABASES, IN 2010 THE NATIONAL SCIENCE
FOUNDATION REGARDED 2008 E 2009 DATA AS UNRELIABLE
IT’S JUST A MATTER
OF DELAYS
73. The moral of
the story
Bibliometric data of last two years are not in
steady-state: do not use for scientific
(or assessment) purposes
74. 1. For the (bibliometric) bureaucrat a number is
something objective and trustable.
2. Awareness about errors, uncertainty, relevance,
manipulability is usually very low.
3. Administrative and normative use of bibliometry
is extremely fragile.
4. In the last two years, hundreds if not thousands
of Italian researchers spent a lot of time asking
Web-of-Science and Scopus to update/correct
their bibliometric profile.
Comment
75. Outline
• The most accurate ranking ever produced
• Should you believe in university rankings?
• Governing by numbers
• The end is not near. It is here
• The Good, the Bad, and the Ugly
• The quest for the Holy Grail
• You’ll gonna hear me roar!
89. Outline
• The most accurate ranking ever produced
• Should you believe in university rankings?
• Governing by numbers
• The end is not near. It is here
• The Good, the Bad, and the Ugly
• The quest for the Holy Grail
• You’ll gonna hear me roar!
92. The Holy Grail of research assessment
• Peer review is subjective, lengthy, and expensive
• We have a lot of bibliometric data relative to
journals (e.g. IF) and scientists:
– # papers
– # cites
– h-index
– ...
• Solution: work out indicators to obtain objective,
quick and unexpensive bibliometric assessment
also at individual level
95. Assessing poetry
... determining a poem's
greatness becomes a
relatively simple matter. If
the poem's score for
perfection is plotted
along the horizontal of a
graph, and its importance
is plotted on the vertical,
then calculating the total
area of the poem yields
the measure of its
greatness.
97. For the matrix entries labeled IR we rely
on the informed peer review
ANVUR proposal: Use bibliometry, # of citations
(and informed peer review)
97
A
B
C
D
Citations
A
B
C
D
A B C D
A
B
C
D
Citations
A
B
C
D
A B C D
A A A?
D D
D
A
A
A?
D
IR
IR
IR
IRIR
IRIR
IR IR
IR
IR
IR IRIR
Bibliometry (IF,…) Bibliometry (IF,…)
Recent
articles
Old
articles
98. Research as target shooting
A paper is
an arrow aiming
at high IF & cites
99. E = 1 B = 0,8 A = 0,5 L = 0
ITALIAN RESEARCH ASSESSMENT: COLORS AND SCORESITALIAN RESEARCH ASSESSMENT: COLORS AND SCORES
102. Due to a flawed design, the actual targets did not match the
specification and did differ between scientific areas
Target specification
103. Medical Sc. vs Industr. & Inform. Eng.
40%
25%
14%
21%
22%
21%
13%
44% Ingegneria Industriale
e dell’InformazioneScienze
Mediche
Industr. &
Information Eng.
Medical
Sciences
104. The moral of the story: since the
targets in different areas (and
disciplines!) are not matched, scores
are not comparable and any
subsequent aggregation (e.g. for
funding) becomes nonsensical.
We failed the quest, but the Grail
may still exist ...
106. Report on the pilot exercise to develop bibliometric
indicators for the REF [the research assessment]
Bibliometrics are not sufficiently robust at this stage to
be used formulaically or to replace expert review in the
REF
http://www.hefce.ac.uk/pubs/year/2009/200939/
bibliometry
And the Holy Grail? Let’s ask HEFCE
107. Code of Practice – European Mathematical Society
http://www.euro-math-soc.eu/system/files/COP-approved.pdf
1.... the Committee sees grave danger in the routine use of bibliometric
and other related measures to assess the alleged quality of
mathematical research and the performance of individuals or small
groups of people.
2.It is irresponsible for institutions or committees assessing individuals
for possible promotion or the award of a grant or distinction to base
their decisions on automatic responses to bibliometric data.
And the Holy Grail? Let’s ask EMS
108. On the use of bibliometric indices during assessment –
European Physical Society
http://www.eps.org/news/94765/
... the European Physical Society considers it essential
that the use of bibliometric indices is always
complemented by a broader assessment of scientific
content taking into account the research environment, to
be carried out by peers in the framework of a clear code
of conduct.
And the Holy Grail? Let’s ask EPS
109. And the Holy Grail? Let’s ask
Académie des Sciences
Du Bon Usage de la Bibliometrie pour l’Évaluation
Individuelle des Chercheurs”- Institut de France, Académie
des Sciences
http://www.academie-sciences.fr/activite/rapport/avis170111gb.pd
Any bibliometric evaluation should be tightly associated to a
close examination of a researcher’s work, in particular to
evaluate its originality, an element that cannot be assessed
through a bibliometric study.
110. And the Holy Grail? Let’s ask IEEE
IEEE Board of Directors: Position Statement on “Appropriate
Use of Bibliometric Indicators for the Assessment of Journals,
Research Proposals, and Individuals”.
http://www.ieee.org/publications_standards/publications/rights/iee
Any journal-based metric is not designed to capture qualities
of individual papers and must therefore not be used alone
as a proxy for single-article quality or to evaluate individual
scientists.
112. 1. Avoid using journal metrics to judge individual papers or
individuals for hiring, promotion and funding decisions.
2. Judge the content of individual papers and take into
account other research outputs, such as data sets,
software and patents, as well as a researcher’s influence
on policy and practice.
113. Signed by 484 organizations including:
- American Association for the Advancement of Science (AAAS)
- American Society for Cell Biology
- British Society for Cell Biology
- European Association of Science Editors
- European Mathematical Society
- European Optical Society
- European Society for Soil Conservation
- Federation of European Biochemical Societies
- Fondazione Telethon
- Higher Education Funding Council for England (HEFCE)
- Proceedings of The National Academy Of Sciences (PNAS)
- Public Library of Science (PLOS)
- The American Physiological Society
- The Journal of Cell Biology
- Institute Pasteur
- CNRS – University Paris Diderot
- INGM, National Institute of Molecular Genetics; Milano, Italy
- Université de Paris VIII, France
- University of Florida
- The European Association for Cancer Research (EACR)
- Ben-Gurion University of the Negev
- Université de Louvain
114. And the Holy Grail?
Let’s ask the literature
Interpretation and Impact ”... analysts should also
be aware of the potential effect of the results in terms
of future behavioural changes by institutions and
individuals seeking to improve their subsequent
'ranking'."
115.
116.
117.
118.
119.
120.
121. Outline
• The most accurate ranking ever produced
• Should you believe in university rankings?
• Governing by numbers
• The end is not near. It is here
• The Good, the Bad, and the Ugly
• The quest for the Holy Grail
• You’ll gonna hear me roar!
123. • Media provide distorted information about
expenditure, performance and efficiency of
higher education.
• This justified heavy budget cuts
(- 18,7% from 2009 to 2013 in real terms)
• “Bureaucratic delirium”
• Flawed bibliometric assessments
incentivating gaming
What to do?
Italy
124.
125. A blog devoted to research evaluation
and higher-education policy
•Birth: October 2011
•Members of the Editorial Board: 14
•Collaborators: > 200
•Contacts from November 2011 to June 2014: 8 million
•More than 13,000 daily contacts in 2014
•Articles published: 1.627
•25,000 comments by readers
•Funding: donations from the readers
•Often cited by national newspapers and magazines
•Good visibility among cultural blogs
(see e.g. 8-th position in http://labs.ebuzzing.it/top-blogs/cultura)