Setting the Record Straight: Strong Positive Impacts Found from the National Evaluation of Upward Bound. Re-Analysis Documents Significant Positive Impacts Masked by Errors in Flawed Contractor Reports

Literaturnachweis - Detailanzeige

Trefferliste

Autor/inn/en	Cahalan, Margaret; Goodwin, David
Institution	Pell Institute for the Study of Opportunity in Higher Education
Titel	Setting the Record Straight: Strong Positive Impacts Found from the National Evaluation of Upward Bound. Re-Analysis Documents Significant Positive Impacts Masked by Errors in Flawed Contractor Reports
Quelle	(2014), (28 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Quantitative Daten; Federal Programs; College Readiness; Access to Education; Data Analysis; Error Correction; Quality Assurance; Program Evaluation; Statistical Analysis; Surveys; Student Records; Databases; Educational Research; Standards; Research Methodology; Program Effectiveness; Postsecondary Education; College Bound Students; High School Graduates; College Attendance; Reports; Academic Advising; Counseling Services; Academic Support Services; Low Income Groups; At Risk Students; First Generation College Students; Longitudinal Studies; Middle School Students; High School Students; Followup Studies; Outcomes of Education; Weighted Scores; Experimental Groups; Control Groups; Comparative Analysis; Student Financial Aid; Bachelors Degrees; Academic Degrees + Suchen Sie Ihr Suchwort? Education; Access; Bildung; Zugang; Bildungszugang; Auswertung; Korrektur; Qualitätssicherung; Programme evaluation; Programmevaluation; Statistische Analyse; Survey; Umfrage; Befragung; Schülerakte; Datenbank; Bildungsforschung; Pädagogische Forschung; Standard; Research method; Forschungsmethode; Post-secondary education; Tertiäre Bildung; High school; High schools; Graduate; Graduates; Oberschule; Absolvent; Absolventin; College; Colleges; Attendance; Hochschule; Fachhochschule; Anwesenheit; Abschlussbericht; Berichten; Akademischer Rat; Longitudinal study; Longitudinal method; Longitudinal methods; Längsschnittuntersuchung; Middle school; Middle schools; Student; Students; Mittelschule; Mittelstufenschule; Schüler; Schülerin; Studentin; Follow-up studies; Kontaktstudium; Lernleistung; Schulerfolg; Finanzielle Beihilfe; Studienfinanzierung; Studienförderung; 'Bachelor''s degrees'; Bachelor-Studiengang; Degree; Degrees; Academic level graduation; Akademischer Grad; Hochschulabschluss
Abstract	In January 2009, in the last week of the Bush Administration, the U.S. Department of Education (ED), upon orders from the departing political appointee staff, published the final report in a long running National Evaluation of Upward Bound (UB). The study was conducted by the contractor, Mathematica Policy Research. After more than a year in review, and over a year after the third and final contract had ended, the report was published over objections from the Policy and Program Studies Services (PPSS) ED career technical staff who were assigned to monitor the final Mathematica contract. The report was also published after a "disapproval to publish" rating in the formal review process from the Office of Postsecondary Education (OPE), out of whose program allocation the evaluation was funded. The Mathematica reports from the Upward Bound (UB) study (Myers et. al. 2004; and Seftor et. al. 2009) have had a large impact on policy development for more than a decade. They have resulted in an OMB "ineffective rating" and were used to justify the zero funding requests for all of the federal pre-college programs, UB, Upward Bound Math/Science (UBMS), Talent Search and GEAR UP in President Bush's budgets in FY 2005 and FY 2006. As the original and final Contracting Officers Technical Representatives (COTRs) for the study within the US Department of Education, it was the official job of Drs. Goodwin and Callahan to provide Technical Monitoring of the Upward Bound evaluation contracts. In the final of three sequential contracts, after concerns about the study were raised, Dr Goodwin and Dr. Callahan conducted a Quality Assurance Review (ED-PPSS QA review), and found that the impact estimations from the study being reported by the contractor were seriously flawed--so much so that the basic conclusions Mathematica made concerning the efficacy of the Upward Bound program were impacted. While they have spoken out before on this topic, Dr. Goodwin and Dr. Callahan are speaking out again in 2014, because of the on-going and recent citations of the erroneous findings from the study in Congressional testimony, policy briefs, and public speeches (Whitehurst, 2011, Haskins and Rouse 2013; Decker 2013). These erroneous findings continue to do unwarranted and non-transparent serious reputational harm to the Upward Bound program. The ED-PPSS QA review involved an internal review and analysis of all data files from the study, as well as consultation and replication of results by external statistical experts. The data files reviewed included: the initial sampling frame, the baseline survey, five follow-up surveys, student transcripts, 10 years of federal aid files and 10 years of National Student Clearinghouse (NSC) data. The ED-PPSS QA found that the Mathematica reports were seriously flawed, made unwarranted conclusions about the Upward Bound program, and were not transparent in reporting. Moreover statistically significant and educationally meaningful positive impacts on the key legislative goals of the Upward Bound program were clearly found when the study errors were addressed using standards-based statistical methods. These positive impacts are unacknowledged in the Mathematica reports. This report provides: (1) Highlights from the ED-PPSS Review and Re-Analysis: (2) Major Errors Identified in the Technical Monitors' Quality Assurance Review; and (3) Major Impact Findings from the Re-Analyses. (ERIC).
Anmerkungen	Pell Institute for the Study of Opportunity in Higher Education. 1025 Vermont Avenue NW Suite 1020, Washington, DC 20005. Tel: 202-638-2887; Fax: 202-638-3808; e-mail: info@pellinstitute.org; Web site: http://www.pellinstitute.org
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01