Does Test Method Matter? An Investigation of the Effect of a Two-Facet Reading Comprehension Test on Test-Takers’ Scores (Journal of Academic Research, 2025)
The degree to which test-takers perform on a given assessment may vary depending on the method of presentation. This research investigated the impact of test format—constructed-response (CR) and multiple-choice (MC)—on reading comprehension performance among EFL university students. Twenty-four students from a Reading III course participated in both assessments, with the CR test administered first, followed by the MC test a few days later. A paired-sample t-test revealed significant differences in performance, with students typically achieving higher scores on the MC format. To explore response discrepancies, follow-up interviews were conducted with students who provided different answers on the two test formats. The results highlighted several key factors contributing to these differences, including time pressure during the CR test, the presence of answer options in the MC format, and the ability to make educated guesses. While these findings suggest that MC items may be more accessible under timed conditions, the study acknowledges the potential influence of order effects and individual student attributes. Future research should further investigate these factors, employing counterbalanced designs, and exploring the impact of test formats on higher-order cognitive skills.
Keywords: constructed-response; effect, Multiple-choice; performance; reading