A study on reducing nonfunctional distractors of multiple-choice questions in medical examination

Li Guojian; Lu Yan; He Ju; Liu Maowei; Feng Pan

doi:10.3760/cma.j.cn116021-20230509-01565

您当前的位置：

首页 >

文章列表页 >

A study on reducing nonfunctional distractors of multiple-choice questions in medical examination

Educational Evaluation | 更新时间：2024-02-27

- A study on reducing nonfunctional distractors of multiple-choice questions in medical examination
- Chinese Journal of Medical Education Research Vol. 23, Issue 1, Pages: 12-16(2024)
- 作者机构：
  
  1.国家医学考试中心试题开发一部，北京　100097
  2.国家医学考试中心发展研究部，北京　100097
  3.国家医学考试中心，北京　100097
  4.国家医学考试中心综合管理部，北京　100097
- 作者简介：
  
  Lu Yan, Email: mingtiyichu@163.com
- 基金信息：
- DOI：10.3760/cma.j.cn116021-20230509-01565
  CLC： R-05
- Published：20 January 2024，
  
  Received：09 May 2023，
扫描看全文
李国建, 卢燕, 何惧, 等. 医学考试中减少选择题无效干扰选项方法研究[J]. 中华医学教育探索杂志, 2024, 23(1): 12-16.

Li Guojian, Lu Yan, He Ju, et al. A study on reducing nonfunctional distractors of multiple-choice questions in medical examination. [J]. Chinese Journal of Medical Education Research 23(1):12-16(2024)
李国建, 卢燕, 何惧, 等. 医学考试中减少选择题无效干扰选项方法研究[J]. 中华医学教育探索杂志, 2024, 23(1): 12-16. DOI： 10.3760/cma.j.cn116021-20230509-01565.

Li Guojian, Lu Yan, He Ju, et al. A study on reducing nonfunctional distractors of multiple-choice questions in medical examination. [J]. Chinese Journal of Medical Education Research 23(1):12-16(2024) DOI： 10.3760/cma.j.cn116021-20230509-01565.

摘要

目的

观察无效干扰选项修改后试题难度、区分度及被修改选项选择率变化，探讨通过修改无效干扰选项提高试题质量的可行性。

方法

以某医学基本知识考试中包含无效干扰选项的试题32题为研究对象。请专家根据命审题工作要求，在实测数据提示下，对无效干扰选项进行分析修改，观察修改前后试题的难度、区分度变化及其相关性和被修改选项选择率变化。采用SPSS 24.0进行数据处理。试题修改前后的难度、区分度及修改选项的选择率变化采用配对Wilcoxon检验；修改前和修改后试题难度、区分度相关性采用斯皮尔曼等级相关法。

结果

少数试题（6题）修改原因为技术性因素，多数试题（26题）修改原因为知识内容方面干扰性不强。对一个无效干扰选项进行修改后，修改前后试题难度分别为0.676（0.558，0.893）和0.637（0.531，0.839），区分度（点二列相关）分别为0.261（0.150，0.316）和0.262（0.138，0.358），区分度（鉴别指数）分别为0.215（0.113，0.352）和0.259（0.138，0.346）。修改前后难度和鉴别指数差异有统计学意义。修改前后难度和区分度均相关。被修改选项的选择率修改前为0.009（0.003，0.015），修改后为0.044（0.021，0.092），差异有统计学意义。

结论

本研究涉及的无效干扰选项出现的原因多数为试题内容方面，技术性缺陷相对较少。通过对无效干扰选项的分析和修改，可以有效提高无效干扰选项的选择率，提高试题难度和区分度。

Abstract

Objective

To discuss the possibility of modifying nonfunctional distractors to improve the quality of multiple-choice questions in medical examination through analyzing the changes in the difficulty and discrimination of items and the selection rate of modified distractors.

Methods

Thirty-two multiple-choice questions involving nonfunctional distractors from a medical examination were studied. According to the item-writing guidelines

experts modified nonfunctional distractors based on actual measurements. We analyzed the changes in the difficulty and discrimination of items and the selection rate of modified distractors before and after item modification using the paired Wilcoxon test

and investigated the correlation of item difficulty and discrimination before and after item modification by Spearman's rank correlation analysis.

Results

Six questions were modified due to technical factors

and 26 questions were modified due to weak interference in knowledge content. Before and after nonfunctional distractor modification

the difficulty values of the questions were 0.676 (0.558

0.893) and 0.637 (0.531

0.839)

respectively; the discrimination values (point-biserial correlation) were 0.261 (0.150

0.316) and 0.262 (0.138

0.358)

respectively; and the discrimination index values were 0.215 (0.113

0.352) and 0.259 (0.138

0.346)

respectively. There were significant differences in the difficulty and discrimination index of items before and after modification. The difficulty and discrimination of items before and after modification were both significantly correlated. The selection rates of modified distractors were 0.009 (0.003

0.015) and 0.044 (0.021

0.092) before and after modification

respectively

which were significantly different.

Conclusions

Nonfunctional distractors in this study were mainly caused by reasons in the contents of the test questions

less in technical defects. Through the analysis and modification of nonfunctional distractors

the selection rate of nonfunctional distractors can be effectively increased

and item difficulty and discrimination can be improved.

关键词

医学考试选择题干扰选项难度区分度

Keywords

Medical examinationMultiple-choice questionDistractorDifficultyDiscrimination

references

Gierl MJ, Bulut O, Guo Q, et al. Developing, analyzing, and using distractors for multiple-choice tests in education: a comprehensive review [J]. Rev Educ Res, 2017, 87(6): 1082-1116. DOI: 10.3102/0034654317726529http://dx.doi.org/10.3102/0034654317726529.

李萍，苏小兵. 地理单项选择题干扰项功能分析：基于测试结果的评价[J]. 地理教学， 2021（14）： 16-20. DOI: 10.3969/j.issn.1000-078X.2021.14.006http://dx.doi.org/10.3969/j.issn.1000-078X.2021.14.006.

王来民. 追寻英语阅读理解干扰项踪迹，破解选项迷津[J]. 教学考试， 2022（39）： 26-29.

刘泽军，赵明德. 识别“雷区” 巧用排除法：基于高考选择题的思考[J]. 教学考试， 2022（44）： 13-15.

刘拓，张佳慧，辛涛. 多项选择题中干扰项信息的利用[J]. 心理学探新， 2015， 35（3）： 261-265.

Rogausch A, Hofer R, Krebs R. Rarely selected distractors in high stakes medical multiple-choice examinations and their recognition by item authors: a simulation and survey [J]. BMC Med Educ, 2010, 10: 85. DOI: 10.1186/1472-6920-10-85http://dx.doi.org/10.1186/1472-6920-10-85.

Haladyna TM, Downing SM. A taxonomy of multiple-choice item-writing rules [J]. Appl Meas Educ, 1989, 2(1): 37-50. DOI: 10.1207/s15324818ame0201_3http://dx.doi.org/10.1207/s15324818ame0201_3.

冯攀，何佳. 医师资格考试单项选择题最优干扰选项数量分析[J]. 中国考试， 2019（3）： 48-53. DOI: 10.19360/j.cnki.11-3303/g4.2019.03.008http://dx.doi.org/10.19360/j.cnki.11-3303/g4.2019.03.008.

Tarrant M, Ware J, Mohammed AM. An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis [J]. BMC Med Educ, 2009(9): 40. DOI: 10.1186/1472-6920-9-40http://dx.doi.org/10.1186/1472-6920-9-40.

Raymond MR, Stevens C, Bucak SD. The optimal number of options for multiple-choice questions on high-stakes tests: application of a revised index for detecting nonfunctional distractors [J]. Adv Health Sci Educ Theory Pract, 2019, 24(1): 141-150. DOI: 10.1007/s10459-018-9855-9http://dx.doi.org/10.1007/s10459-018-9855-9.

Kilgour JM, Tayyaba S. An investigation into the optimal number of distractors in single-best answer exams [J]. Adv Health Sci Educ Theory Pract, 2016, 21(3): 571-585. DOI: 10.1007/s10459-015-9652-7http://dx.doi.org/10.1007/s10459-015-9652-7.

Haladyna TM, Downing SM. How many options is enough for a multiple-choice test item? [J]. Educ Psychol Meas, 1993, 53(4): 999-1010. DOI: 10.1177/0013164493053004013http://dx.doi.org/10.1177/0013164493053004013.

Abdulghani H, Ahmad F, Aldrees A, et al. The relationship between non-functioning distractors and item difficulty of multiple choice questions: a descriptive analysis [J]. J Health Spec, 2014, 2(4): 148. DOI: 10.4103/1658-600x.142784http://dx.doi.org/10.4103/1658-600x.142784.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰