Investigating the Process of Consequential Validity with the Ambassador Questionnaire

Autor/in	Kuhn, Melissa Gayle
Titel	Investigating the Process of Consequential Validity with the Ambassador Questionnaire
Quelle	(2022), (95 Seiten) PDF als Volltext Verfügbarkeit Ph.D. Dissertation, Old Dominion University
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
ISBN	979-8-3719-7673-4
Schlagwörter	Hochschulschrift; Dissertation; Test Validity; Psychometrics; Test Interpretation; Scores; Questionnaires; Engineering Education; Factor Analysis; Factor Structure + Suchen Sie Ihr Suchwort? Thesis; Dissertations; Academic thesis; Testvalidität; Psychometry; Psychometrie; Test analysis; Testauswertung; Fragebogen; Ingenieurausbildung; Faktorenanalyse; Faktorenstruktur
Abstract	Validity in psychometrics refers to the degree to which evidence and theory supports the interpretations drawn from a test, and Messick's Contemporary Validity Theory (1994) includes several facets with well-established evidence collection methods. However, there is a lack of consensus on appropriate methods of evaluating the facet of consequential validity, which is the degree to which interpretation of scores could have consequences for test-takers. The primary objective of this study was to illustrate a method of identifying potential consequences of survey in the stage of manual development. This method was placed in the context of the Ambassador Questionnaire (AQ) used in Engineering Ambassador (EA) programs that aim to broaden participation in the discipline with outreach. Because this program focuses on equity and inclusion, a second objective was to determine if the AQ could include a subscale sensitive to equity perceptions. A mixed method was employed to address these objectives. Qualitative interviews informed both the capacity for interviews to solicit potential consequences of use and the utility of an equity perception subscale to EA chapter advisors. Quantitative analyses included factor analytic techniques. Existing AQ data was proportionately stratified for separate exploratory and confirmatory factor analyses to determine if controlling for demographic differences would reinforce the AQ's existing factor structure that omits pilot items sensitive to perceptions of equity differences. Ten advisors were interviewed and a total of eleven hours of interview data was collected. Several themes related to consequences of use emerged from these interviews, including use of the AQ to track student growth, potential influences on students as a result of taking the AQ, programmatic changes that could be made with AQ data, and sharing of AQ data with interested parties. These findings suggest that interviewing stakeholders during the development of an instrument's manual can provide developers with foreseeable consequences. Integration of advisor feedback on items sensitive to perceptions of equity differences with quantitative findings that reinforce the AQ's existing factor structure also suggests that a scale sensitive to said perceptions would be best served by an additional subscale with added items or an independent equity perceptions scale. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.] (As Provided).
Anmerkungen	ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://www.proquest.com/en-US/products/dissertations/individuals.shtml
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01