Governments Harnessing the Power of Data to Get 'Value for Money': A Simulation Study of England's Office for Students B3 Proceed Metric

Autor/inn/en	Bradley, Alex; Quigley, Martyn
Titel	Governments Harnessing the Power of Data to Get 'Value for Money': A Simulation Study of England's Office for Students B3 Proceed Metric
Quelle	In: Studies in Higher Education, 48 (2023) 8, S.1289-1302 (14 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Bradley, Alex) ORCID (Quigley, Martyn)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0307-5079
DOI	10.1080/03075079.2023.2196292
Schlagwörter	Foreign Countries; Higher Education; Education Work Relationship; Outcomes of Education; College Graduates; Universities; Institutional Evaluation; Accuracy; Probability; Statistical Bias; Sample Size; Federal Regulation; Employment Potential; Measurement; Simulation; Outcome Measures; Classification; United Kingdom (England) + Suchen Sie Ihr Suchwort? Ausland; Hochschulbildung; Hochschulsystem; Hochschulwesen; Lernleistung; Schulerfolg; Hochschulabsolvent; Hochschulabsolventin; University; Universität; Wahrscheinlichkeitsrechnung; Wahrscheinlichkeitstheorie; Bundeskompetenz; Arbeitsmarktbezogene Qualifikation; Beschäftigungsfähigkeit; Messverfahren; Simulation program; Simulationsprogramm; Classification system; Klassifikation; Klassifikationssystem
Abstract	The mass participation in higher education has led to greater spending by governments and students which has increased the focus on graduate outcomes. In England, the Office for Students (OfS) is planning to take regulatory action, using the Proceed metric, against universities and their courses which do not have 60% of students with positive outcomes within 15 months of graduation. This study uses simulations to explore how effectively the Proceed metric can (a) identify the true population level of positive outcomes, (b) explore the precision of those estimates, and (c) accurately classify courses below the 60% threshold. The simulation varied: level of positive outcomes within the population (20-90%), sample size (40-1000), and the percentage of the population sampled (30-90%). The bias (difference between sample and population estimate), coverage probability (proportion of true population estimates within the intervals), and precision of confidence intervals (average range of confidence intervals) were calculated. Three main findings were (a) the Proceed metric is accurate in terms of bias and coverage probability, (b) the estimates lack precision, especially with small sample sizes, and (c) the imprecision will impact the Proceed metric ability to correctly classify courses below the 60% threshold. Governments seeking to collect graduate survey data and use it to regulate universities should make every effort to maximise sample size and should resist the temptation to regulate at a micro-level (i.e. courses) since classification will likely be inaccurate, especially for those just below the threshold. (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01