Review on reform of international research evaluation in past decade

Authors

Fang XU, Institutes of Science and Development, Chinese Academy of Sciences, Beijing 100190, China School of Public Policy and Management, University of Chinese Academy of Sciences, Beijing 100049, China S&T Management and Evaluation Committee of CASSSP, Beijing 100080, ChinaFollow
Xiaoxuan LI, Institutes of Science and Development, Chinese Academy of Sciences, Beijing 100190, China School of Public Policy and Management, University of Chinese Academy of Sciences, Beijing 100049, China S&T Management and Evaluation Committee of CASSSP, Beijing 100080, ChinaFollow

Keywords

international, research evaluation, reform, San Francisco Declaration on Research Assessment (DORA)

Document Type

Strategy & Policy Decision Research

Abstract

In May 2013, with the release of the San Francisco Declaration on Research Assessment (DORA) as a symbol, the reform of international research evaluation began. Over the past decade, the reform of international research evaluation has undergone extensive theoretical preparation and initiatives, and is now moving towards practice. More than 350 institutions from over 40 countries have signed agreements to reform research evaluation and carried out pilot reforms, with initial results. This study systematically summarizes and analyzes the ten-year reform of international research evaluation in three aspects: reform goals, measures, and cases. From the perspective of comparing with China’s research evaluation reform, research conclusions and inspirations are drawn.

First page

121

Last Page

130

Language

Chinese

Publisher

Bulletin of Chinese Academy of Sciences

References

1 徐芳, 晋新新, 吕雯洁, 等. 国际科研评价改革实践与启示——以DORA为代表的探索. 华南理工大学学报（社会科学版）, 2022, 24(3): 94-100. Xu F, Jin X X, Lyu W J, et al. International science and technology evaluation reform’s practice and enlightenment—A exploration represented by DORA. Journal of South China University of Technology (Social Science Edition), 2022, 24(3): 94-100. (in Chinese)

2 徐芳, 李晓轩. 科技评价改革十年评述. 中国科学院院刊, 2022, 37(5): 603-612. Xu F, Li X X. Review on reform of research evaluation in past decade. Bulletin of Chinese Academy of Sciences, 2022, 37(5): 603-612. (in Chinese)

3 Garfield E. Citation indexes for science: A new dimension in documentation through association of ideas. Science, 1955, 122: 108-111.

4 盛小平, 毕畅畅, 唐筠杰. 国内外开放科学主题研究综述. 图书情报知识, 2022, 39(4): 101-113. Sheng X P, Bi C C, Tang J J. A review of open science research at home and abroad. Documentation, Information & Knowledge, 2022, 39(4): 101-113. (in Chinese)

5 United Nations Educational, Scientific and Cultural Organization. UNESCO Recommendation on Open Science. (2021-11-09)[2023-12-26]. https://unesdoc.unesco.org/ark:/48223/pf000037994.

6 王飞跃, 缪青海. 人工智能驱动的科学研究新范式：从AI4S到智能科学. 中国科学院院刊, 2023, 38(4): 536-540. Wang F Y, Miao Q H. Novel paradigm for AI-driven scientific research: From AI4S to intelligent science. Bulletin of Chinese Academy of Sciences, 2023, 38(4): 536-540. (in Chinese)

7 Hicks D, Wouters P, Waltman L, et al. Bibliometrics: The Leiden Manifesto for research metrics. Nature, 2015, 520: 429-431.

8 Wilsdon J, Allen L, Belfiore E, et al. The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management. (2015-07-06)[2023-12-26]. https://www.ukri.org/wp-content/uploads/2021/12/RE-151221-TheMetricTideFullReport2015.pdf.

9 The HuMetricsHSS Team. Humane Metrics Initiative. [2023-12-26]. https://humetricshss.org/humetricshss-principles/.

10 Wilsdon J, Bar-Ilan J, Frodeman R, et al. Next-generation metrics: responsible metrics and evaluation for open science. (2017-03-20)[2023-12-26]. https://www.researchgate.net/publication/315385216_Next-generation_metrics_responsible_metrics_and_evaluation_for_open_science.

11 Moher D, Naudet F, Cristea I A, et al. Assessing scientists for hiring, promotion, and tenure. PLoS Biology, 2018, 16(3): e2004089.

12 INORMS-RWG. SCOPE Framework for Research Evaluation. [2024-01-04]. https://inorms.net/scope-framework-for-research-evaluation/.

13 European University Association. EUA Roadmap on Research Assessment in the Transition to Open Science. (2018-06-18)[2024-01-04]. https://eua.eu/downloads/publications/eua-roadmap-on-research-assessment-in-the-transition-to-open-science.pdf.

14 Dominik M, Badre A, Almas T A. et al. Publishing models, assessment, and open science. Report and outcomes from a workshop held by the Global Young Academy. (2018-10-29)[2023-12-26]. https://globalyoungacademy.net/wp-content/uploads/2018/10/APOS-Report-29.10.2018.pdf.

15 Global Research Council. Statement of Principles on Peer/Merit Review. [2023-12-26]. https://globalresearchcouncil.org/fileadmin//documents/GRC_Publications/French/GRC_2018_SoP_Declaration_de_Principes_sur_l_Analyse_Pair_Merite.pdf.

16 Moher D, Bouter L, Kleinert S, et al. The Hong Kong Principles for assessing researchers: Fostering research integrity. PLoS Biology, 2020, 18(7): e3000737.

17 Technopolis. Science Europe Study on Research Assessment Practices. (2020-02-04)[2023-12-26]. https://www.scienceeurope.org/our-resources/science-europe-study-on-research-assessment-practices/.

18 Global Research Council. Responsible Research Assessment—Global Research Council (GRC) Conference Report 2021. (2020-11-23)[2023-12-26]. https://globalresearchcouncil.org/fileadmin/documents/GRC_Publications/GRC_RRA_Conference_Summary_Report.pdf.

19 CommissionEuropean. Reforming research assessment: The Agreement is now final. (2022-07-20)[2023-12-26]. https://research-and-innovation.ec.europa.eu/news/all-research-and-innovation-news/reforming-research-assessment-agreement-now-final-2022-07-20_en.

20 Curry S, Rijcke S D, Hatch A,et al. The changing role of funders in responsible research assessment: Pprogress, obstacles and the way ahead. (2020-11-18)[2023-12-26]. https://rori.figshare.com/ndownloader/files/25518674.

21 Ghent University. Vision Statement for Evaluating Research at Ghent University. (2016-11-08)[2023-12-26]. https://www.ugent.be/en/research/research-strategy/evaluation/research-evaluation-principles.pdf.

22 Ghent University. Responsible use of quantitative indicators. (2023-10-9)[2024-01-04]. https://onderzoektips.ugent.be/en/tips/00001832/

23 Kozlov M. NIH plans grant-review overhaul to reduce bias. Nature, 2022, 612: 602-603.

24 National Institutes of Health. Biographical Sketch. [2023-12-26]. https://grants.nih.gov/grants/forms/biosketch.html.

25 Kozlov M. NIH issues a seismic mandate: Share data publicly. Nature, 2022, 602: 558-559.

26 徐芳, 刘文斌, 李晓轩. 英国REF科研影响力评价的方法及启示. 科学学与科学技术管理, 2014, 35(7): 9-15. Xu F, Liu W B, Li X X. Research impact evaluation within UK REF and its implications. Science of Science and Management of S & T, 2014, 35(7): 9-15.(in Chinese)

27 Research and Innovation UK. Future Research Assessment Programme. (2022-12-09)[2023-12-26]. https://www.ukri.org/who-we-are/research-england/research-excellence/future-research-assessment-programme-frap/.

28 关于完善科技成果评价机制的指导意见. (2021-07-16)[2023-12-26]. https://www.gov.cn/zhengce/zhengceku/2021-08/02/content_5628987.htm?eqid=efd6cb8a0001575900000002646f098b.

29 郝若扬. 3种引文规范化指标RCR、CNCI和JNCI的相关性研究. 现代情报, 2023, 43(12): 133-142. Hao R Y. Study on correlations among three standardized evaluation indices: RCR, CNCI and JNCI. Journal of Modern Information, 2023, 43(12): 133-142.(in Chinese)

30 徐芳, 李晓轩. 在“破”与“立”之间：科技评价的BRIDGE理论. 中国科学院院刊, 2022, 37(8): 1099-1107. Xu F, Li X X. Between “Breaking” and “Building”: The BRIDGE Theory of research evaluation. Bulletin of Chinese Academy of Sciences, 2022, 37(8): 1099-1107. (in Chinese)

Recommended Citation

XU, Fang and LI, Xiaoxuan (2024) "Review on reform of international research evaluation in past decade," Bulletin of Chinese Academy of Sciences (Chinese Version): Vol. 39 : Iss. 1 , Article 14.
DOI: https://doi.org/10.16418/j.issn.1000-3045.20231217002
Available at: https://bulletinofcas.researchcommons.org/journal/vol39/iss1/14