Credibility, Relevance, and Policy Impact in the Evaluation of Adult Basic Skills Programs: The Case of the New Opportunities Initiative in Portugal

Autor/inn/en	Carpentieri, J.D.; Mallows, David; Amorim, José Pedro
Titel	Credibility, Relevance, and Policy Impact in the Evaluation of Adult Basic Skills Programs: The Case of the New Opportunities Initiative in Portugal
Quelle	In: Adult Literacy Education, 2 (2020) 1, S.6-21 (16 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2642-3669
Schlagwörter	Adult Basic Education; Information Technology; Literacy; Numeracy; Educational Policy; Program Descriptions; Literacy Education; Program Evaluation; Educational Finance; Case Studies; Foreign Countries; Longitudinal Studies; Problems; Credibility; Portugal + Suchen Sie Ihr Suchwort? Adult; Adults; Education; Adult education; Erwachsenenbildung; Informationstechnologie; Alphabetisierung; Schreib- und Lesefähigkeit; Rechenkompetenz; Politics of education; Bildungspolitik; Programme evaluation; Programmevaluation; Bildungsfonds; Case study; Fallstudie; Case Study; Ausland; Longitudinal study; Longitudinal method; Longitudinal methods; Längsschnittuntersuchung; Problemsituation; Glaubwürdigkeit
Abstract	Adult basic education (ABE) policies aim to help adults improve their literacy, numeracy and information and communications technology skills, as well as their qualifications, often in pursuit of economic gains such as better employment and earnings. The large-scale improvement of skills and qualifications has been referred to as a wicked policy problem, suggesting that it is extremely difficult and perhaps even impossible to achieve success in this policy domain. Evaluations have highlighted these challenges, with many programs showing little or no impact. Between 2006 and 2012, the Portuguese government ran a large-scale adult education program, the New Opportunities Initiative (NOI), which focused on the recognition and validation of adults' existing skills and the development of literacy and numeracy. The NOI was evaluated twice, in 2009 and in 2012. These two evaluations produced very different findings and outcomes: the first evaluation found the NOI to be a success, and led to continued investment, but the second evaluation reached more negative conclusions and was used as a rationale for de-funding the program. In this article we analyze these two sets of evaluations, investigating the reasons for their starkly different conclusions. We find that, while both evaluations had strengths, they also suffered from serious methodological and/or theoretical weaknesses. These weaknesses are "part of a broader pattern of evaluation errors" that characterize the field of ABE more generally and which make it more likely that ABE policies will continue to fail. Using the conflicting NOI evaluations as case studies, we offer potential solutions to ABE's evaluation problem, emphasizing the need to collect long-term longitudinal evidence on the causal mechanisms through which policy goals may be achieved. (As Provided).
Anmerkungen	ProLiteracy. 101 Wyoming Street, Syracuse, NY 13204. Tel: 888-528-2224; Tel: 315-214-2400; Fax: 315-422-6369; e-mail: info@proliteracy.org; Web site: https://proliteracy.org/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2022/1/01