A Closer Look at the Differences between Graders in Introductory Computer Science Exams

Autor/in	Albluwi, Ibrahim
Titel	A Closer Look at the Differences between Graders in Introductory Computer Science Exams
Quelle	In: IEEE Transactions on Education, 61 (2018) 3, S.253-260 (8 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Albluwi, Ibrahim)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0018-9359
DOI	10.1109/TE.2018.2805706
Schlagwörter	Introductory Courses; Computer Science Education; Teacher Attitudes; Grading; Tests; Objective Tests; Evaluators; Summative Evaluation; Formative Evaluation; Teaching Assistants; College Faculty; College Students; Evaluation Methods + Suchen Sie Ihr Suchwort? Einführungskurs; Computer science lessons; Informatikunterricht; Lehrerverhalten; Notengebung; Schulnote; Examination; Prüfung; Examen; Objektiver Test; Fakultät; Collegestudent
Abstract	Contribution: This paper shows how significantly computer science instructors can disagree when grading code writing questions in introductory computer science (CS1) exams. Certain solutions are considered "very good" by some instructors and "very bad" by others. The study identifies four factors as possible contributors to such differences between graders: 1) disagreement on the role of syntax in CS1; 2) absence of objective measures for the seriousness of logic errors; 3) variance in how rubrics are constructed; and 4) the use of subjective judgement while grading. Background: Code writing questions are widely used in CS1 exams. Understanding differences in how graders approach such questions can help the CS education community become more informed about how to design valid assessments for summative and formative purposes. Research Questions: Do graders differ significantly in how they score student programs? What are the main factors to which differences between graders could be attributed? Methodology: A number of CS1 instructors and TAs were asked to score pieces of code resembling student solutions to CS1 exam questions, and were asked to answer open ended questions on how they approach grading in CS1 exams. Responses were then analyzed qualitatively and quantitatively. Findings: There is no consensus on how code writing questions in CS1 exams should be graded. Differences in opinion on how many points to award can be very significant, even for very simple programs and between graders in the same institution. Some graders assign grades that do not necessarily agree with how many points they believe the student solution is worth. (As Provided).
Anmerkungen	Institute of Electrical and Electronics Engineers, Inc. 445 Hoes Lane, Piscataway, NJ 08854. Tel: 732-981-0060; Web site: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=13
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01