Using Learning Analytics to Evaluate the Quality of Multiple-Choice Questions: A Perspective with Classical Test Theory and Item Response Theory

Autor/inn/en	Azevedo, Jose Manuel; Oliveira, Ema P.; Beites, Patrícia Damas
Titel	Using Learning Analytics to Evaluate the Quality of Multiple-Choice Questions: A Perspective with Classical Test Theory and Item Response Theory
Quelle	In: International Journal of Information and Learning Technology, 36 (2019) 4, S.322-341 (20 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Azevedo, Jose Manuel)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2056-4880
DOI	10.1108/IJILT-02-2019-0023
Schlagwörter	Learning Analytics; Multiple Choice Tests; Test Theory; Item Response Theory; Test Items; Difficulty Level; College Students + Suchen Sie Ihr Suchwort? Multiple choice examinations; Multiple-choice tests, Multiple-choice examinations; Multiple-Choice-Verfahren; Testtheorie; Item-Response-Theorie; Test content; Testaufgabe; Schwierigkeitsgrad; Collegestudent
Abstract	Purpose: The purpose of this paper is to find appropriate forms of analysis of multiple-choice questions (MCQ) to obtain an assessment method, as fair as possible, for the students. The authors intend to ascertain if it is possible to control the quality of the MCQ contained in a bank of questions, implemented in Moodle, presenting some evidence with Item Response Theory (IRT) and Classical Test Theory (CTT). The used techniques can be considered a type of Descriptive Learning Analytics since they allow the measurement, collection, analysis and reporting of data generated from students' assessment. Design/methodology/approach: A representative data set of students' grades from tests, randomly generated with a bank of questions implemented in Moodle, was used for analysis. The data were extracted from a Moodle database using MySQL with an ODBC connector, and collected in MS Excel[TM] worksheets, and appropriate macros programmed with VBA were used. The analysis with the CTT was done through appropriate MS Excel[TM] formulas, and the analysis with the IRT was approached with an MS Excel[TM] add-in. Findings: The Difficulty and Discrimination Indexes were calculated for all the questions having enough answers. It was found that the majority of the questions presented values for these indexes, which leads to a conclusion that they have quality. The analysis also showed that the bank of questions presents some internal consistency and, consequently, some reliability. Groups of questions with similar features were obtained, which is very important for the teacher to develop tests as fair as possible. Originality/value: The main contribution and originality that can be found in this research is the definition of groups of questions with similar features, regarding their difficulty and discrimination properties. These groups allow the identification of difficulty levels in the questions on the bank of questions, thus allowing teachers to build tests, randomly generated with Moodle, that include questions with several difficulty levels in the tests, as it should be done. As far as the authors' knowledge, there are no similar results in the literature. (As Provided).
Anmerkungen	Emerald Group Publishing Limited. Howard House, Wagon Lane, Bingley, West Yorkshire, BD16 1WA, UK. Tel: +44-1274-777700; Fax: +44-1274-785201; e-mail: emerald@emeraldinsight.com; Web site: http://www.emeraldinsight.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01