Progress in the application of large language models to medical multiple-choice question generation

Li Ke; Yan Hui; Jin Zhe; She Kang; Qi Jianguang

doi:10.3760/cma.j.cn116021-20260221-02386

您当前的位置：

首页 >

文章列表页 >

Progress in the application of large language models to medical multiple-choice question generation

更新时间：2026-10-28

- Progress in the application of large language models to medical multiple-choice question generation
- Chinese Journal of Medical Education Research Pages: 1-6(2026)
- 作者机构：
  
  1.北京大学第一医院教育处，北京 100034
  2.北京大学第一医院儿童医学中心，北京 100034
  3.北京大学第一医院呼吸与危重症医学科，北京 100034
  4.北京大学第一医院介入血管外科，北京 100034
- 作者简介：
  
  Qi Jianguang, Email: qjg2006@126.com
- 基金信息：
  
  2025-2026 National Medical Examination Center Innovative Research Project: Research on High-Quality Item Generation and Evaluation Systems for Medical Licensing Examinations Based on Generative AI
- DOI：10.3760/cma.j.cn116021-20260221-02386
  CLC： R-0;R-4;R78
- Received：21 February 2026，
  
  Online First：28 October 2026，
- 稿件说明：
移动端阅览
Li Ke, Yan Hui, Jin Zhe, et al. Progress in the application of large language models to medical multiple-choice question generation[J/OL]. Chinese Journal of Medical Education Research, 2026, 1-6.
DOI：

Li Ke, Yan Hui, Jin Zhe, et al. Progress in the application of large language models to medical multiple-choice question generation[J/OL]. Chinese Journal of Medical Education Research, 2026, 1-6. DOI： 10.3760/cma.j.cn116021-20260221-02386.

摘要

使用大语言模型（large language model，LLM）生成医学多项选择题（multiple-choice question，MCQ）已成为国外医学教育评价领域的研究热点之一。本文对LLM用于MCQ命题的应用现状、影响因素及评价方法进行综述。现有研究显示，LLM在缩短命题周期、生成结构规范的题目初稿方面优势明显，部分题目在内容准确性与拟真度上已接近专家命题水平；但在高阶临床推理考查、事实准确性、干扰项质量及心理测量学属性稳定性等方面仍存在局限。提示词设计、模型选择以及外部权威知识源的引入，均为影响生成题目质量的重要因素。在此基础上，本文提出了AI命题提示词框架和初步的AI命题质量评估框架，并提出专家与AI协同命题的模式，以期为LLM辅助医学MCQ命题提供参考。

Abstract

The application of large language models (LLMs) to generate medical multiple-choice questions (MCQs) has become one of the research hotspots in medical education assessment abroad. This article reviews the current application status

influencing factors

and evaluation methods of using LLMs for MCQ generation. Existing studies show that LLMs have clear advantages in shortening the item-writing cycle and generating structurally standardized first drafts of questions

and some generated questions have approached the level of expert-written ones in terms of content accuracy and authenticity. However

limitations still remain in the assessment of higher-order clinical reasoning

factual accuracy

distractor quality

and the stability of psychometric properties. Prompt design

model selection

and the introduction of external authoritative knowledge sources are all important factors affecting the quality of generated questions. On this basis

this paper proposes an artificial intelligence prompt framework for question generation and a preliminary quality evaluation framework for artificial intelligence-generated questions

along with a model of expert-artificial intelligence collaborative item writing

in order to provide a reference for LLM-assisted medical MCQ generation.

关键词

Keywords

references

曾令铝，方天乐，李婧，等 . 发达国家及地区国际医学毕业生参加临床执业医师资格考试的比较研究 [J ] . 循证医学， 2025 ， 25 （ 3 ）： 181 - 190 . DOI: 10.12019/j.issn.1671-5144.202412058 http://dx.doi.org/10.12019/j.issn.1671-5144.202412058 .

Stadler M , Horrer A , Fischer MR . Crafting medical MCQs with generative AI: a how-to guide on leveraging ChatGPT [J ] . GMS J Med Educ , 2024 , 41 ( 2 ): Doc20 . DOI: 10.3205/zma001675 http://dx.doi.org/10.3205/zma001675 .

江哲涵，奉世聪，王维民 . 生成式大语言模型在医学考试题库建设中的实践探索 [J ] . 中华医学教育杂志， 2024 ， 44 （ 8 ）： 561- 569. DOI: 10.3760/cma.j.cn115259-20240524-00520 http://dx.doi.org/10.3760/cma.j.cn115259-20240524-00520 .

李丽，朱朝阳，李冬梅，等 . 智能题库模型quizLlama的构建与教学应用 [J ] . 基础医学教育， 2025 ， 27 （ 7 ）： 674 - 679 . DOI: 10.13754/j.issn2095-1450.2025.07.16 http://dx.doi.org/10.13754/j.issn2095-1450.2025.07.16 .

Magzoub ME , Zafar I , Munshi F , et al . Ten tips to harnessing generative AI for high-quality MCQS in medical education assessment [J ] . Med Educ Online , 2025 , 30 ( 1 ): 2532682 . DOI: 10.1080/10872981.2025.2532682 http://dx.doi.org/10.1080/10872981.2025.2532682 .

Nielsen JPS , Mikkelsen AK , Kuenzel J , et al . Evaluation of multiple-choice tests in head and neck ultrasound created by physicians and large language models [J ] . Diagnostics (Basel) , 2025 , 15 ( 15 ): 1848 . DOI: 10.3390/diagnostics15151848 http://dx.doi.org/10.3390/diagnostics15151848 .

Emekli E , Karahan BN . Artificial intelligence in radiology examinations: a psychometric comparison of question generation methods [J ] . Diagn Interv Radiol , 2026 , 32 ( 5 ): 548 - 554 . DOI: 10.4274/dir.2025.253407 http://dx.doi.org/10.4274/dir.2025.253407 .

Elzayyat M , Mohammad JN , Zaqout S . Assessing LLM-generated vs . expert-created clinical anatomy MCQs: a student perception-based comparative study in medical education [J ] . Med Educ Online , 2025 , 30 : 2554678 . DOI: 10.1080/10872981.2025.2554678 http://dx.doi.org/10.1080/10872981.2025.2554678 .

Ma X , Pan W , Yu XN . Evaluating AI-generated examination papers in periodontology: a comparative study with human-designed counterparts [J ] . BMC Med Educ , 2025 , 25 ( 1 ): 1099 . DOI: 10.1186/s12909-025-07706-6 http://dx.doi.org/10.1186/s12909-025-07706-6 .

Laohawetwanit T , Apornvirat S , Kantasiripitak C . ChatGPT as a teaching tool: Preparing pathology residents for board examination with AI-generated digestive system pathology tests [J ] . Am J Clin Pathol , 2024 , 162 ( 5 ): 471 - 479 . DOI: 10.1093/ajcp/aqae062 http://dx.doi.org/10.1093/ajcp/aqae062 .

Gholami S , Mummert DB , Wilson B , et al . Leveraging large language models to generate multiple-choice questions for ophthalmology education [J ] . JAMA Ophthalmol , 2025 , 143 ( 11 ): 955 - 961 . DOI: 10.1001/jamaophthalmol.2025.3622 http://dx.doi.org/10.1001/jamaophthalmol.2025.3622 .

Mistry NP , Saeed H , Rafique S , et al . Large language models as tools to generate radiology board-style multiple-choice questions [J ] . Acad Radiol , 2024 , 31 ( 9 ): 3872 - 3878 . DOI: 10.1016/j.acra.2024.06.046 http://dx.doi.org/10.1016/j.acra.2024.06.046 .

Jackson FI , Keller NA , Kouba I , et al . Large language model clinical vignettes and multiple-choice questions for postgraduate medical education [J ] . Acad Med , 2025 , 100 ( 10 ): 1163 - 1166 . DOI: 10.1097/acm.0000000000006137 http://dx.doi.org/10.1097/acm.0000000000006137 .

Safadi S , Amirahmadi R , Tlimat A , et al . Quality of human expert vs large language model-generated multiple-choice questions in the field of mechanical ventilation [J ] . Chest , 2025 , 168 ( 6 ): 1425 - 1432 . DOI: 10.1016/j.chest.2025.07.005 http://dx.doi.org/10.1016/j.chest.2025.07.005 .

Law AK , So J , Lui CT , et al . AI versus human-generated multiple-choice questions for medical education: a cohort study in a high-stakes examination [J ] . BMC Med Educ , 2025 , 25 ( 1 ): 208 . DOI: 10.1186/s12909-025-06796-6 http://dx.doi.org/10.1186/s12909-025-06796-6 .

Kıyak YS , Emekli E . ChatGPT prompts for generating multiple-choice questions in medical education and evidence on their validity: a literature review [J ] . Postgrad Med J , 2024 , 100 ( 1189 ): 858 - 865 . DOI: 10.1093/postmj/qgae065 http://dx.doi.org/10.1093/postmj/qgae065 .

Cheung BHH , Lau GKK , Wong GTC , et al . ChatGPT versus human in generating medical graduate exam multiple choice questions-a multinational prospective study [J ] . PLoS One , 2023 , 18 ( 8 ): e0290691 . DOI: 10.1371/journal.pone.0290691 http://dx.doi.org/10.1371/journal.pone.0290691 .

Zuckerman M , Flood R , Tan RJB , et al . ChatGPT for assessment writing [J ] . Med Teach , 2023 , 45 ( 11 ): 1224 - 1227 . DOI: 10.1080/0142159x.2023.2249239 http://dx.doi.org/10.1080/0142159x.2023.2249239 .

Karahan BN , Emekli E . Comparison of applicability, difficulty, and discrimination indices of multiple-choice questions on medical imaging generated by different AI-based chatbots [J ] . Radiography , 2025 , 31 ( 5 ): 103087 . DOI: 10.1016/j.radi.2025.103087 http://dx.doi.org/10.1016/j.radi.2025.103087 .

Ahmed A , Kerr E , O'Malley A . Quality assurance and validity of AI-generated single best answer questions [J ] . BMC Med Educ , 2025 , 25 ( 1 ): 300 . DOI: 10.1186/s12909-025-06881-w http://dx.doi.org/10.1186/s12909-025-06881-w .

Wu H , Zerner T , Lee D , et al . GPT-4 versus human authors in clinically complex MCQ creation: a blinded analysis of item quality [J ] . Med Teach , 2025 , 47 ( 12 ): 1961 - 1974 . DOI: 10.1080/0142159x.2025.2505122 http://dx.doi.org/10.1080/0142159x.2025.2505122 .

Abouzeid E , Wassef R , Jawwad A , et al . Chatbots' role in generating single best answer questions for undergraduate medical student assessment: comparative analysis [J ] . JMIR Med Educ , 2025 , 11 : e69521 . DOI: 10.2196/69521 http://dx.doi.org/10.2196/69521 .

Ch'en PY , Day W , Pekson RC , et al . GPT-4 generated answer rationales to multiple choice assessment questions in undergraduate medical education [J ] . BMC Med Educ , 2025 , 25 ( 1 ): 333 . DOI: 10.1186/s12909-025-06862-z http://dx.doi.org/10.1186/s12909-025-06862-z .

Sawamura S , Kohiyama K , Takenaka T , et al . Potential of large language models in generating multiple-choice questions for the Japanese national licensure examination for physical therapists [J ] . Cureus , 2025 , 17 ( 2 ): e79183 . DOI: 10.7759/cureus.79183 http://dx.doi.org/10.7759/cureus.79183 .

Singhal K , Azizi S , Tu T , et al . Large language models encode clinical knowledge [J ] . Nature , 2023 , 620 ( 7972 ): 172 - 180 . DOI: 10.1038/s41586-023-06291-2 http://dx.doi.org/10.1038/s41586-023-06291-2 .

Singhal K , Tu T , Gottweis J , et al . Toward expert-level medical question answering with large language models [J ] . Nat Med , 2025 , 31 ( 3 ): 943 - 950 . DOI: 10.1038/s41591-024-03423-7 http://dx.doi.org/10.1038/s41591-024-03423-7 .

Wu C , Lin W , Zhang X , et al . PMC-LLaMA: toward building open-source language models for medicine [J ] . J Am Med Inform Assoc , 2024 , 31 ( 9 ): 1833 - 1843 . DOI: 10.1093/jamia/ocae045 http://dx.doi.org/10.1093/jamia/ocae045 .

Xie Q , Chen Q , Chen A , et al . Medical foundation large language models for comprehensive text analysis and beyond [J ] . NPJ Digit Med , 2025 , 8 ( 1 ): 141 . DOI: 10.1038/s41746-025-01533-1 http://dx.doi.org/10.1038/s41746-025-01533-1 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Construction of an intelligent knowledge base for venous thromboembolism and evaluation of its application in medical education

Application and prospect of large language model in the education of public health and preventive medicine in colleges and universities

Feasibility of DeepSeek-assisted test item generation for prostate cancer educational assessment

Artificial intelligence-enhanced teaching of neurological physical examination: an exploratory study on its effectiveness

Development and application of an artificial intelligence-assisted learning platform for the entire process of medical education

Related Author

She Kang

Hao Yudi

Wang Yaqing

Li Junmei

Ma Yinglin

Mu Guangyan

Li Ke

Zhang Xiaolin

Related Institution

Department of Interventional Vascular Surgery, Peking University First Hospital

Department of Pharmacy, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College

Department of Pharmacy, Peking University First Hospital

Department of Education, Peking University First Hospital

Department of Burns and Plastic Surgery, Peking University First Hospital

⁰