Simple, Complex, and Compound Sentences Exercises.pdf
The challenges of implementing responsible research metrics at TU Dublin
1. The challenges of implementing
responsible research metrics at
TU Dublin
Stewart Killeen, MSc
https://stewartkilleen.wordpress.com/
https://www.linkedin.com/in/stewart-killeen-760875104/
https://orcid.org/0000-0001-9402-6874
2. TU Dublin: “infinite possibilities”
30,000
Students
3,500
Staff
members
140
Nationalities
400+
Sustainable
Businesses
7. Open Science and research
evaluation
“Open science represents an innovation in
the way research is performed: in how
scientists collaborate and share knowledge
with the wider world, and how scientific
institutions are organized for greater
societal impact” (Wilsdon, 2017, p.5).
8. Towards responsible metrics
a “responsible use of research metrics” to
ensure fairer and more transparent
research evaluation processes
9. Irish Research Landscape
Qualitative expert assessment
Explicit
Transparent and verifiable
Intrinsic merits
Diversity of outputs
Equality, diversity and inclusion
10. Challenges
Cultural shift in the values and
reward structures that
incentivize research
Buy-in from all stakeholders
Academic integrity
11. Research Funding
A clear commitment to reward
and recognise more open, more
diverse and more robust research
practices
12. Buy-in
Buy-in from publishers can help
transform the current mindset
from “publishing as fast as
possible” to “sharing knowledge
as early as possible” (Wilsdon,
2017, p.5)
13. Academic Integrity
Academic integrity can be
maintained by developing policies
and statements that explicitly
support and endorse responsible
research metrics
14. References
Dalton, M., Collins, S., Schwamm, H., & Smith, S. (2021); Responsible Research
Metrics Policy Development and Alignment. National Open Research Forum, Ireland.
Available at: https://doi.org/10.7486/DRI.jw82n294d (Accessed: 9 June 2023)
F1000. (2023). What can open research values bring to research assessment
reform? Available at: https://www.youtube.com/watch?v=ppTWQg-3Fu4 (Accessed:
24 May 2023)
15. References
Higher Education Policy Institute (2022) ‘What does Open Research mean for the
future of the REF? The next REF could better recognise the diversity of roles
contributing to research’. Available at: https://www.hepi.ac.uk/2022/05/30/what-
does-open-research-mean-for-the-future-of-the-ref-the-next-ref-could-better-
recognize-the-diversity-of-roles-contributing-to-research/ (Accessed: 10 June 2023).
Irish Research Council (2023) ‘Irish Research Council Strategic Plan 2020-2024’.
Available at: https://research.ie/about-us/our-strategy/ (Accessed: 8 June 2023).
16. References
San Francisco Declaration on Research Assessment (2023) ‘San Francisco
Declaration on Research Assessment’. Available at: https://sfdora.org/read/
(Accessed: 7 June 2023).
TU Dublin (2023) ‘Facts and Figures’. Available at: https://www.tudublin.ie/
(Accessed: 7 June 2023).
17. References
TU Dublin (2023) ‘Open Research Support Unit (ORSU)’. Available at:
https://www.tudublin.ie/research/support-for-researchers/orsu/
(Accessed: 7 June 2023).
TU Dublin Library Services (2022); Service Design Team Report: Organisation Design
Strategy & Implementation Project. (Accessed: 19 March 2023).
18. References
Wilsdon, J.R., Bar-Ilan, J., Frodeman, R., Lex, E., Peters, I. and Wouters, P., (2017);
Next-generation metrics: Responsible metrics and evaluation for open science,
European Commission. Available at: https://eprints.whiterose.ac.uk/113919/
(Accessed: 8 June 2023).
TU Dublin:
The university supports the largest cohort of students of business, media, culinary arts, and the creative and performing arts. Committed to life-long learning, it is the largest provider of part-time education and it also provides a comprehensive range of apprenticeships.
Recognised as a leader in the STEM disciplines, the university boasts an impressive resume of “distinguished researchers and innovators” who play active roles in informing policy and standards and contributing to the creative life of Ireland.
With its international outlook and its strong international partnerships, the university welcomes students and staff from around the world and provides many opportunities for student and staff exchange programmes, major cross-collaboration research projects and employment opportunities.
The university’s award-winning technology transfer and business incubation activity have delivered over 400 sustainable news businesses with an economic value of almost €700 million.
Aspiring to “create a better world together” and guided by the UN Sustainability Goals, TU Dublin’s strategic intent 2030 is built on 3 interrelated pillars:
People, Planet and Partnership.
By creating “transformational educational opportunities”, TU Dublin will strive to inspire, support and develop the individual in reaching their full potential within an ever changing world.
By being a “voice for” sustainability, TU Dublin will champion ways of living and working that safeguard the planet for future generations.
By sharing ideas and connecting people, TU Dublin will leverage the benefits of partnerships and synergies to tackle the complex problems of today (TU, 2023).
TU Dublin is one of the first Irish universities to endorse and develop the practices of open research (TU Dublin, 2023). The Open Research Support Unit (ORSU), a joint initiative between TU Dublin Library Services and the Research Support & Development Unit (RSDU), provides the resources and expertise to support the development of open research in the university (TU Dublin, 2023). The ORSU can provide information and support in the areas of Open Access to publications, management of data, managing open scholarship profiles, and research evaluation (TU Dublin, 2023).
Research evaluation is the attempt to measure the quality and impact of scientific outputs. It is situated within a system that values and rewards academic prestige and reputation and, as such, is “intrinsically linked” to those processes which support the aspiring researcher, including:
Promotion
Recruitment
Allocation of funding
Development of research strategy. (Dalton, 2021, p.2).
Assessing the quality and impact of research has relied largely on the use of bibliometrics, which measure scholarly output in terms of publications, especially journal publications (Wilsdon, 2017, p.11). A recent study comparing the different categories of research outputs listed in the UK’s Research Excellence Framework (REF), found that publications dominated the research system there (F1000, 2023; Higher Education Policy Institute, 2022). As the authors noted, the data reflects the distorting effect of publication metrics in the UK’s diverse research environment (Higher Education Policy Institute, 2022). In other words, current evaluation is skewed towards publication metrics (DORA, 2O23).
The growing popularity of Open Science and Research practices in recent years has led many to rethink research assessment. “Open science represents an innovation in the way research is performed: in how scientists collaborate and share knowledge with the wider world, and how scientific institutions are organized for greater societal impact” (Wilsdon, 2017, p.5). It recognises and rewards research based on “its intrinsic quality and the many broader forms of activity and impact beyond publications” (Wilsdon, 2017, p.5).
The inherently democratic and highly collaborative nature of Open Research requires a “responsible use of research metrics” to ensure fairer and more transparent research evaluation processes (Dalton, 2021, p.2; F1000, 2023). Initiatives such as DORA and COARA are calling on researchers to develop more “sophisticated and meaningful” approaches to assessing the quality and impact of scientific research (DORA, 2023). By taking a more holistic view of research, responsible research metrics (RRM) will complement traditional bibliometrics with both article-level metrics and qualitative indicators that seek to measure the influence of research on policy and practise (DORA, 2023).
According to The National Open Research Forum (NORF), RRM should address the numerous deficiencies within current research assessment practices (Dalton, 2021, p.4; DORA, 2023):
Metrics should be used in parallel with qualitative expert assessment
Criteria used in research evaluation should be explicit
Calculation and use of metrics should be transparent and easily verified
Research should be evaluated based on its intrinsic merits, rather than the venue it is published in
Broader outputs and activities beyond publications should be incorporated as part of research evaluation
Research evaluation processes should support equality, diversity and inclusion initiatives
The adoption of RRMs will, of course, face significant challenges. The most significant of these, perhaps, will entail a cultural shift in the values and reward structures that incentivize research. It will also be critical to get all the relevant stakeholders on board, including publishers. And last, but not least, RRMs must respect academic integrity (Dalton, 2021, p.6; F1000, 2023).
Research funders can play a crucial role in driving change in how metrics are used as part of the grant application and award process. By widening the scope of research evaluation, beyond the narrow confines of publication metrics, funding bodies can play an important part in rewarding and recognising more open and robust research practices (Dalton, 2021, p.2). As NORF note, “a clear commitment” by funding organisations and other stakeholders will be an “important first step in achieving the necessary cultural change” needed to ensure the success of RRMs (Dalton, 2021, p.2).
In spite of the changes brought about by digital technologies, publishers continues to play a vital role in the dissemination of research. As such, they can help drive the move to more RRMs. Indeed, by endorsing the outputs and practices of open research and incorporating next generation metrics, such as Altmetrics, publishers will transform the current mindset from “publishing as fast as possible” to “sharing knowledge as early as possible” (Dalton, 2021, p.3; DORA, 2023; F1000, 2023; Wilsdon, 2017, p.5).
Ensuring academic integrity throughout is essential. This can be achieved by developing policies and statements that explicitly support and endorse RRMs, according to local need (Dalton, 2021, p.5). Moreover, the adoption of RRMs, while it may add “additional workload” to the process of evaluation, will undoubtedly supplement that process with greater transparency, thus leading to more empowered and responsible researchers (F1000, 2023; Wilsdon, 2017, p.15)
