Latent Program Modeling: Inferring Latent Problem-Solving Strategies from a PISA Problem-Solving Task

Autor/in	Lundgren, Erik
Titel	Latent Program Modeling: Inferring Latent Problem-Solving Strategies from a PISA Problem-Solving Task
Quelle	In: Journal of Educational Data Mining, 14 (2022) 1, S.46-80 (35 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Zusatzinformation	ORCID (Lundgren, Erik)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
Schlagwörter	Problem Solving; Classification; Accuracy; Foreign Countries; Achievement Tests; International Assessment; Secondary School Students; Cognitive Processes; Test Items; Item Analysis; Simulation; Traffic Safety; Transportation; Inferences; Program for International Student Assessment + Suchen Sie Ihr Suchwort? Problemlösen; Classification system; Klassifikation; Klassifikationssystem; Ausland; Achievement test; Achievement; Testing; Test; Tests; Leistungsbeurteilung; Leistungsüberprüfung; Leistung; Testdurchführung; Testen; Sekundarschüler; Cognitive process; Kognitiver Prozess; Test content; Testaufgabe; Itemanalyse; Simulation program; Simulationsprogramm; Verkehrswesen; Inference; Inferenz
Abstract	Response process data have the potential to provide a rich description of test-takers' thinking processes. However, retrieving insights from these data presents a challenge for educational assessments and educational data mining as they are complex and not well annotated. The present study addresses this challenge by developing a computational model that simulates how different problem-solving strategies would behave while searching for a solution to a Program for International Student Assessment (PISA) 2012 problem-solving item, and uses n-gram processing of data together with a naïve Bayesian classifier to infer latent problem-solving strategies from the test-takers' response process data. The retrieval of simulated strategies improved with increased n-gram length, reaching an accuracy of 0.72 on the original PISA task. Applying the model to generalized versions of the task showed that classification accuracy increased with problem size and the mean number of actions, reaching a classification accuracy of 0.90 for certain task versions. The strategy that was most efficient and effective in the PISA Traffic task evaluated paths based on the labeled travel time. However, in generalized versions of the task, a straight line strategy was more effective. When applying the classifier to empirical data, most test-takers were classified as using a random path strategy (46%). Test-takers classified as using the travel time strategy had the highest probability of solving the task. The test-takers classified as using the random actions strategy had the lowest probability of solving the task. The effect of (classified) strategy on general PISA problem-solving performance was overall weak, except for a negative effect for the random actions strategy ([beta] [approximately equal to] -65, CI[subscript 95%] [approximately equal to] [-96, -36]). The study contributes with a novel approach to inferring latent problem-solving strategies from action sequences. The study also illustrates how simulations can provide valuable information about item design by exploring how changing item properties could affect the accuracy of inferences about unobserved problem-solving strategies. (As Provided).
Anmerkungen	International Educational Data Mining. e-mail: jedm.editor@gmail.com; Web site: https://jedm.educationaldatamining.org/index.php/JEDM
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01