Unravelling the Panel Contribution upon Peer Review Evaluation of Numerous, Unstructured and Highly Interdisciplinary Research Proposals

Autor/inn/en	Baimpos, Theodoros; Dittel, Nils; Borissov, Roumen
Titel	Unravelling the Panel Contribution upon Peer Review Evaluation of Numerous, Unstructured and Highly Interdisciplinary Research Proposals
Quelle	In: Research Evaluation, 29 (2020) 3, S.316-326 (11 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0958-2029
DOI	10.1093/reseval/rvz013
Schlagwörter	Peer Evaluation; Research Proposals; Interdisciplinary Approach; Financial Support; Grantsmanship; Interrater Reliability; Scores + Suchen Sie Ihr Suchwort? Forschungsgegenstand; Fächerübergreifender Unterricht; Fächerverbindender Unterricht; Interdisziplinarität; Finanzielle Förderung; Interrater-Reliabilität
Abstract	In this study, we analyze the two-phase bottom-up procedure applied by the Future and Emerging Technologies Program (FET-Open) at the Research Executive Agency (REA) of the European Commission (EC), for the evaluation of highly interdisciplinary, multi-beneficiary research proposals which request funding. In the first phase, remote experts assess the proposals and draft comments addressing the pre-defined (by FET-Open) evaluation criteria. In the second phase, a new set of additional experts (of more general expertise and different from the remote ones), after cross reading the proposals and their remote evaluation reports, they convene in an on-site panel where they discuss the proposals. They complete the evaluation by reinforcing per proposal and per criterion one or another assessment, as assigned remotely during the first phase. We analyze the level of the inter-rater agreement among the remote experts and we identify its relative correlation with the funded proposals resulted after the end of the evaluation. Our study also provides comparative figures of the evolution of the proposals' scores during the two phases of the evaluation process. Finally, by carrying out an appropriate quantitative and qualitative analysis of all scores from the seven past cut-offs, we elaborate on the significant contribution of the panel (the second phase of the evaluation) in identifying and promoting the best proposals for funding. (As Provided).
Anmerkungen	Oxford University Press. Great Clarendon Street, Oxford, OX2 6DP, UK. Tel: +44-1865-353907; Fax: +44-1865-353485; e-mail: jnls.cust.serv@oxfordjournals.org; Web site: http://applij.oxfordjournals.org/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01