Proficiency evaluation of large language models in medical laboratory technology education

Wang Yang; Wu Jiahao; Zhang Fan; Cheng Jing; Tan Hongxia; Ouyang Juan; Li Junxun

doi:10.3760/cma.j.cn116021-20250516-02108

您当前的位置：

首页 >

文章列表页 >

Proficiency evaluation of large language models in medical laboratory technology education

2025 Sun Yat-sen Medical Education Development Conference Call for Papers | 更新时间：2025-11-21

- Proficiency evaluation of large language models in medical laboratory technology education
- Chinese Journal of Medical Education Research Vol. 24, Issue 11, Pages: 1447-1453(2025)
- 作者机构：
  
  中山大学附属第一医院医学检验科，广州 510080
- 作者简介：
  
  Li Junxun, Email: lijunx5@mail.sysu.edu.cn
- 基金信息：
  
  2023 Guangdong Province Basic and Applied Basic Research Fund(2023A1515220150);2023 Wu Jieping Medical Foundation Clinical Research Special Fund(H2023053);2025 Sun Yat-sen University Teaching Quality and Reform Project
- DOI：10.3760/cma.j.cn116021-20250516-02108
  CLC： R-05
- Received：16 May 2025，
  
  Published：20 November 2025
- 稿件说明：
移动端阅览
汪洋, 吴嘉豪, 张帆, 等. 大语言模型在医学检验技术教育中的能力评估[J]. 中华医学教育探索杂志, 2025, 24(11): 1447-1453.

Wang Yang, Wu Jiahao, Zhang Fan, et al. Proficiency evaluation of large language models in medical laboratory technology education[J]. Chinese Journal of Medical Education Research, 2025, 24(11): 1447-1453.
汪洋, 吴嘉豪, 张帆, 等. 大语言模型在医学检验技术教育中的能力评估[J]. 中华医学教育探索杂志, 2025, 24(11): 1447-1453. DOI： 10.3760/cma.j.cn116021-20250516-02108.

Wang Yang, Wu Jiahao, Zhang Fan, et al. Proficiency evaluation of large language models in medical laboratory technology education[J]. Chinese Journal of Medical Education Research, 2025, 24(11): 1447-1453. DOI： 10.3760/cma.j.cn116021-20250516-02108.

摘要

目的

评估主流大语言模型（large language model，LLM）在医学检验教育中的专业知识水平，探究其是否具备辅导医学检验技术专业学习的可能性。

方法

选取2023年我国临床医学检验技术（师）资格考试真题共400题，分2轮测试5款LLM（Copilot、Grok、元宝、豆包及Kimi）：首轮采用零提示词策略，次轮采用交互优化策略，评估其答题正确率及生成内容质量。通过Cochran’s

检验比较LLM性能差异，并基于CLEAR框架（完整性、准确性、循证性、恰当性、相关性）对生成内容进行评分。

结果

首轮测试中，豆包总正确率最高（375/400），且豆包与元宝的总正确率显著优于Copilot与Kimi（

0.001）。次轮优化交互后，Kimi的正确率显著提升（

0.05）；其他LLM的正确率均略有提升，但差异无统计学意义（

0.05）。豆包的总正确率仍最高（380/400），豆包和元宝的总正确率显著高于Copilot（

0.005）。基于CLEAR框架的评估显示，元宝、豆包、Kimi在循证性（

0.003）和完整性（

0.05）维度显著优于国外LLM，能规范引用权威证据，生成内容质量更优。

结论

主流LLM具备丰富的医学检验知识，通过单题输入、明确证据要求及启用高级推理功能可提升答题正确率与生成内容质量。国产LLM在正确率上与国外LLM相当，且在循证性和完整性维度具有显著优势。研究表明，主流LLM可作为辅助工具用于医学检验技术专业知识学习。

Abstract

Objective

To assess the professional knowledge proficiency of mainstream large language models (LLMs) in medical laboratory education and to explore their potential as educational aids for medical laboratory technology students.

Methods

A comprehensive evaluation was conducted using 400 authentic questions from the 2023 Chinese National Clinical Medical Laboratory Technician Qualification Examination. Five LLMs (Copilot

Grok

Yuanbao

Doubao

and Kimi) were tested through two-round interactions using zero-shot prompting and interaction-optimized prompting strategies. The accuracy of answers and the quality of generated content were evaluated. Performance disparities were analyzed using Cochran's Q test. Content quality was scored through the CLEAR framework (completeness

lack of false information

evidence-based reasoning

appropriateness

relevance).

Results

In the first-round test

Doubao achieved the highest overall accuracy rate (375/400). The overall accuracy rates of Doubao and Yuanbao significantly outperformed Copilot and Kimi (

0.001). After the second-round interactive optimization

the accuracy rate of Kimi significantly improved (

0.05)

whereas other LLMs showed slight improvements (

0.05). Doubao still had the highest overall accuracy rate (380/400). The overall accuracy rates of Doubao and Yuanbao significantly outperformed Copilot (

0.005). Evaluation based on the CLEAR framework revealed that Yuanbao

Doubao

and Kimi significantly outperformed foreign models in the dimensions of evidence-based reasoning (

0.003) and completeness (

0.05)

demonstrating standardized citation of authoritative evidence and superior content quality.

Conclusions

The tested LLMs possess extensive medical laboratory knowledge. The accuracy of their answers and the quality of the generated content can be improved through single-question input

specifying evidence requirements

and enabling advanced reasoning functions. Domestic LLMs are comparable to foreign LLMs in terms of accuracy

and have significant advantages in the dimensions of evidence-based reasoning and completeness. LLMs can serve as auxiliary tools for learning professional knowledge in medical laboratory technology.

关键词

Keywords

references

Sallam M , Al-Salahat K , Eid H , et al . Human versus artificial intelligence: ChatGPT-4 outperforming Bing, bard, ChatGPT-3.5 and humans in clinical chemistry multiple-choice questions [J ] . Adv Med Educ Pract , 2024 , 15 : 857 - 871 . DOI: 10.2147/AMEP.S479801 http://dx.doi.org/10.2147/AMEP.S479801 .

Triola MM , Rodman A . Integrating generative artificial intelligence into medical education: curriculum, policy, and governance strategies [J ] . Acad Med , 2025 , 100 ( 4 ): 413 - 418 . DOI: 10.1097/acm.0000000000005963 http://dx.doi.org/10.1097/acm.0000000000005963 .

Gan W , Ouyang J , Li H , et al . Integrating ChatGPT in orthopedic education for medical undergraduates: randomized controlled trial [J ] . J Med Internet Res , 2024 , 26 : e57037 . DOI: 10.2196/57037 http://dx.doi.org/10.2196/57037 .

Kung TH , Cheatham M , Medenilla A , et al . Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models [J ] . PLoS Digit Health , 2023 , 2 ( 2 ): e0000198 . DOI: 10.1371/journal.pdig.0000198 http://dx.doi.org/10.1371/journal.pdig.0000198 .

Long C , Lowe K , Zhang J , et al . A novel evaluation model for assessing ChatGPT on otolaryngology-head and neck surgery certification examinations: performance study [J ] . JMIR Med Educ , 2024 , 10 : e49970 . DOI: 10.2196/49970 http://dx.doi.org/10.2196/49970 .

Patel EA , Fleischer L , Filip P , et al . Comparative performance of ChatGPT 3.5 and GPT4 on rhinology standardized board examination questions [J ] . OTO Open , 2024 , 8 ( 2 ): e164 . DOI: 10.1002/oto2.164 http://dx.doi.org/10.1002/oto2.164 .

Kung JE , Marshall C , Gauthier C , et al . Evaluating ChatGPT performance on the orthopaedic in-training examination [J ] . JB JS Open Access , 2023 , 8 ( 3 ): e23.00056 . DOI: 10.2106/JBJS.OA.23.00056 http://dx.doi.org/10.2106/JBJS.OA.23.00056 .

van Merriënboer JJG , Sweller J . Cognitive load theory in health professional education: design principles and strategies [J ] . Med Educ , 2010 , 44 ( 1 ): 85 - 93 . DOI: 10.1111/j.1365-2923.2009.03498.x http://dx.doi.org/10.1111/j.1365-2923.2009.03498.x .

Vygotsky LS . Mind in society: development of higher psychological processes [M/OL ] //Cole M, Jolm-Steiner V, Scribner S, et al. Cambridge , MA, USA: Harvard University Press , 1980 [2023-03-01] . https://www.jstor.org/stable/10.2307/j.ctvjf9vz4 https://www.jstor.org/stable/10.2307/j.ctvjf9vz4 .

Li J , Ouyang J , Liu J , et al . Artificial Intelligence-based online platform assists blood cell morphology learning: a mixed-methods sequential explanatory designed research [J ] . Med Teach , 2023 , 45 ( 6 ): 596 - 603 . DOI: 10.1080/0142159X.2023.2190483 http://dx.doi.org/10.1080/0142159X.2023.2190483 .

Groot F , Jonker G , Rinia M , et al . Simulation at the frontier of the zone of proximal development: a test in acute care for inexperienced learners [J ] . Acad Med , 2020 , 95 ( 7 ): 1098 - 1105 . DOI: 10.1097/ACM.0000000000003265 http://dx.doi.org/10.1097/ACM.0000000000003265 .

Lindgren H , Lu MH , Hong Y , et al . Applying the zone of proximal development when evaluating clinical decision support systems: a case study [J ] . Stud Health Technol Inform , 2018 , 247 : 131 - 135 .

李俊勋，张帆，李润钊，等 . 基于人工智能的线上学习平台在医学检验技术专业血细胞形态学实习教学中的应用 [J ] . 中华医学教育杂志， 2025 ， 45 （ 3 ）： 204 - 209 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A chest CT report conclusion generation system based on mT5 large language model for residency training

Accuracy and quality of answer reasoning of Chinese large language model in Chinese middle level professional qualification examination of radiology

Correlation analysis and countermeasures between learning burnout and social support of medical laboratory technology interns

Reform thought and practice on training application-oriented talents of medical laboratory technology specialty by the trinity of "teaching, clinics and research" method

Related Author

Li Junxun

Zhao Keke

Zhang Yaping

Wang Ai

Hu Yanfei

Pan Zhijie

Li Qingyao

Xu Min

Related Institution

Department of Radiology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine

Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine

Editorial Office of Journal of Diagnostics Concepts & Practice, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine

Dean's Office, Tongren Hospital, Shanghai Jiao Tong University School of Medicine

Department of Education, Tongren Hospital, Shanghai Jiao Tong University School of Medicine

⁰