浏览全部资源
扫码关注微信
大语言模型在医学检验技术教育中的能力评估
Proficiency evaluation of large language models in medical laboratory technology education
- 中华医学教育探索杂志 2025年24卷第11期 页码：1447-1453
- 作者机构：
中山大学附属第一医院医学检验科，广州 510080
- 作者简介：
李俊勋，Email：lijunx5@mail.sysu.edu.cn
- 基金信息：2023年广东省基础与应用基础研究基金项目(2023A1515220150);2023年吴阶平医学基金会临床科研专项资助基金(H2023053);中山大学2025年校级教学质量与教学改革工程项目
DOI：10.3760/cma.j.cn116021-20250516-02108
中图分类号： R-05收稿：2025-05-16，
纸质出版：2025-11-20
- 稿件说明：
移动端阅览
汪洋, 吴嘉豪, 张帆, 等. 大语言模型在医学检验技术教育中的能力评估[J]. 中华医学教育探索杂志, 2025, 24(11): 1447-1453.
Wang Yang, Wu Jiahao, Zhang Fan, et al. Proficiency evaluation of large language models in medical laboratory technology education[J]. Chinese Journal of Medical Education Research, 2025, 24(11): 1447-1453.
汪洋, 吴嘉豪, 张帆, 等. 大语言模型在医学检验技术教育中的能力评估[J]. 中华医学教育探索杂志, 2025, 24(11): 1447-1453. DOI： 10.3760/cma.j.cn116021-20250516-02108.
Wang Yang, Wu Jiahao, Zhang Fan, et al. Proficiency evaluation of large language models in medical laboratory technology education[J]. Chinese Journal of Medical Education Research, 2025, 24(11): 1447-1453. DOI： 10.3760/cma.j.cn116021-20250516-02108.
摘要
目的
2
评估主流大语言模型（large language model，LLM）在医学检验教育中的专业知识水平，探究其是否具备辅导医学检验技术专业学习的可能性。
方法
2
选取2023年我国临床医学检验技术（师）资格考试真题共400题，分2轮测试5款LLM（Copilot、Grok、元宝、豆包及Kimi）：首轮采用零提示词策略，次轮采用交互优化策略，评估其答题正确率及生成内容质量。通过Cochran’s
Q
检验比较LLM性能差异，并基于CLEAR框架（完整性、准确性、循证性、恰当性、相关性）对生成内容进行评分。
结果
2
首轮测试中，豆包总正确率最高（375/400），且豆包与元宝的总正确率显著优于Copilot与Kimi（
P
<
0.001）。次轮优化交互后，Kimi的正确率显著提升（
P
<
0.05）；其他LLM的正确率均略有提升，但差异无统计学意义（
P
>
0.05）。豆包的总正确率仍最高（380/400），豆包和元宝的总正确率显著高于Copilot（
P
<
0.005）。基于CLEAR框架的评估显示，元宝、豆包、Kimi在循证性（
P
<
0.003）和完整性（
P
<
0.05）维度显著优于国外LLM，能规范引用权威证据，生成内容质量更优。
结论
2
主流LLM具备丰富的医学检验知识，通过单题输入、明确证据要求及启用高级推理功能可提升答题正确率与生成内容质量。国产LLM在正确率上与国外LLM相当，且在循证性和完整性维度具有显著优势。研究表明，主流LLM可作为辅助工具用于医学检验技术专业知识学习。
Abstract
Objective
2
To assess the professional knowledge proficiency of mainstream large language models (LLMs) in medical laboratory education and to explore their potential as educational aids for medical laboratory technology students.
Methods
2
A comprehensive evaluation was conducted using 400 authentic questions from the 2023 Chinese National Clinical Medical Laboratory Technician Qualification Examination. Five LLMs (Copilot
Grok
Yuanbao
Doubao
and Kimi) were tested through two-round interactions using zero-shot prompting and interaction-optimized prompting strategies. The accuracy of answers and the quality of generated content were evaluated. Performance disparities were analyzed using Cochran's Q test. Content quality was scored through the CLEAR framework (completeness
lack of false information
evidence-based reasoning
appropriateness
relevance).
Results
2
In the first-round test
Doubao achieved the highest overall accuracy rate (375/400). The overall accuracy rates of Doubao and Yuanbao significantly outperformed Copilot and Kimi (
P
<
0.001). After the second-round interactive optimization
the accuracy rate of Kimi significantly improved (
P
<
0.05)
whereas other LLMs showed slight improvements (
P
>
0.05). Doubao still had the highest overall accuracy rate (380/400). The overall accuracy rates of Doubao and Yuanbao significantly outperformed Copilot (
P
<
0.005). Evaluation based on the CLEAR framework revealed that Yuanbao
Doubao
and Kimi significantly outperformed foreign models in the dimensions of evidence-based reasoning (
P
<
0.003) and completeness (
P
<
0.05)
demonstrating standardized citation of authoritative evidence and superior content quality.
Conclusions
2
The tested LLMs possess extensive medical laboratory knowledge. The accuracy of their answers and the quality of the generated content can be improved through single-question input
specifying evidence requirements
and enabling advanced reasoning functions. Domestic LLMs are comparable to foreign LLMs in terms of accuracy
and have significant advantages in the dimensions of evidence-based reasoning and completeness. LLMs can serve as auxiliary tools for learning professional knowledge in medical laboratory technology.
关键词
Keywords
references
Sallam M , Al-Salahat K , Eid H , et al . Human versus artificial intelligence: ChatGPT-4 outperforming Bing, bard, ChatGPT-3.5 and humans in clinical chemistry multiple-choice questions [J ] . Adv Med Educ Pract , 2024 , 15 : 857 - 871 . DOI: 10.2147/AMEP.S479801 http://dx.doi.org/10.2147/AMEP.S479801 .
Triola MM , Rodman A . Integrating generative artificial intelligence into medical education: curriculum, policy, and governance strategies [J ] . Acad Med , 2025 , 100 ( 4 ): 413 - 418 . DOI: 10.1097/acm.0000000000005963 http://dx.doi.org/10.1097/acm.0000000000005963 .
Gan W , Ouyang J , Li H , et al . Integrating ChatGPT in orthopedic education for medical undergraduates: randomized controlled trial [J ] . J Med Internet Res , 2024 , 26 : e57037 . DOI: 10.2196/57037 http://dx.doi.org/10.2196/57037 .
Kung TH , Cheatham M , Medenilla A , et al . Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models [J ] . PLoS Digit Health , 2023 , 2 ( 2 ): e0000198 . DOI: 10.1371/journal.pdig.0000198 http://dx.doi.org/10.1371/journal.pdig.0000198 .
Long C , Lowe K , Zhang J , et al . A novel evaluation model for assessing ChatGPT on otolaryngology-head and neck surgery certification examinations: performance study [J ] . JMIR Med Educ , 2024 , 10 : e49970 . DOI: 10.2196/49970 http://dx.doi.org/10.2196/49970 .
Patel EA , Fleischer L , Filip P , et al . Comparative performance of ChatGPT 3.5 and GPT4 on rhinology standardized board examination questions [J ] . OTO Open , 2024 , 8 ( 2 ): e164 . DOI: 10.1002/oto2.164 http://dx.doi.org/10.1002/oto2.164 .
Kung JE , Marshall C , Gauthier C , et al . Evaluating ChatGPT performance on the orthopaedic in-training examination [J ] . JB JS Open Access , 2023 , 8 ( 3 ): e23.00056 . DOI: 10.2106/JBJS.OA.23.00056 http://dx.doi.org/10.2106/JBJS.OA.23.00056 .
van Merriënboer JJG , Sweller J . Cognitive load theory in health professional education: design principles and strategies [J ] . Med Educ , 2010 , 44 ( 1 ): 85 - 93 . DOI: 10.1111/j.1365-2923.2009.03498.x http://dx.doi.org/10.1111/j.1365-2923.2009.03498.x .
Vygotsky LS . Mind in society: development of higher psychological processes [M/OL ] //Cole M, Jolm-Steiner V, Scribner S, et al. Cambridge , MA, USA: Harvard University Press , 1980 [2023-03-01] . https://www.jstor.org/stable/10.2307/j.ctvjf9vz4 https://www.jstor.org/stable/10.2307/j.ctvjf9vz4 .
Li J , Ouyang J , Liu J , et al . Artificial Intelligence-based online platform assists blood cell morphology learning: a mixed-methods sequential explanatory designed research [J ] . Med Teach , 2023 , 45 ( 6 ): 596 - 603 . DOI: 10.1080/0142159X.2023.2190483 http://dx.doi.org/10.1080/0142159X.2023.2190483 .
Groot F , Jonker G , Rinia M , et al . Simulation at the frontier of the zone of proximal development: a test in acute care for inexperienced learners [J ] . Acad Med , 2020 , 95 ( 7 ): 1098 - 1105 . DOI: 10.1097/ACM.0000000000003265 http://dx.doi.org/10.1097/ACM.0000000000003265 .
Lindgren H , Lu MH , Hong Y , et al . Applying the zone of proximal development when evaluating clinical decision support systems: a case study [J ] . Stud Health Technol Inform , 2018 , 247 : 131 - 135 .
李俊勋 ， 张帆 ， 李润钊 ， 等 . 基于人工智能的线上学习平台在医学检验技术专业血细胞形态学实习教学中的应用 [J ] . 中华医学教育杂志 ， 2025 ， 45 （ 3 ）： 204 - 209 .
0
浏览量
3
下载量
0
CSCD
- 网络PDF下载
- WORD下载
- Meta-XML下载
- BibTeX下载
- Endnote下载
- RefMan 下载
- NoteExpress下载
- NoteFirst下载
关联资源
相关文章
相关作者
相关机构
- 技术支持由北京北大方正电子有限公司提供 京ICP备09064830号-19
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies
