Comparative audit capabilities of five Chinese large language models in detecting drug-drug interaction in outpatient prescriptions

WANG Zihao; LI Jinbao; ZHANG Yan; YAN Sen

doi:10.6039/j.issn.1001-0408.2026.19.16

您当前的位置：

首页 >

文章列表页 >

Comparative audit capabilities of five Chinese large language models in detecting drug-drug interaction in outpatient prescriptions

更新时间：2026-10-12

- Comparative audit capabilities of five Chinese large language models in detecting drug-drug interaction in outpatient prescriptions
- ZHONGGUO YAOFANG Vol. 37, Issue 19, Pages: 2577-2581(2026)
- 作者机构：
  
  1.黄河三门峡医院药学部，河南三门峡　472000
  2.黄河三门峡医院眼科，河南三门峡　472000
- 作者简介：
- 基金信息：
- DOI：10.6039/j.issn.1001-0408.2026.19.16
  CLC： R95
- Received：25 May 2026，
  
  Revised：2026-09-26，
  
  Accepted：28 September 2026，
  
  Published：15 October 2026
- 稿件说明：
移动端阅览
王紫灏,李晋宝,张研,等.5种中文大语言模型对门诊处方中药物相互作用的审核能力比较 [J].中国药房,2026,37(19):2577-2581.

WANG Zihao,LI Jinbao,ZHANG Yan,et al.Comparative audit capabilities of five Chinese large language models in detecting drug-drug interaction in outpatient prescriptions[J].ZHONGGUO YAOFANG,2026,37(19):2577-2581.
王紫灏,李晋宝,张研,等.5种中文大语言模型对门诊处方中药物相互作用的审核能力比较 [J].中国药房,2026,37(19):2577-2581. DOI： 10.6039/j.issn.1001-0408.2026.19.16.

WANG Zihao,LI Jinbao,ZHANG Yan,et al.Comparative audit capabilities of five Chinese large language models in detecting drug-drug interaction in outpatient prescriptions[J].ZHONGGUO YAOFANG,2026,37(19):2577-2581. DOI： 10.6039/j.issn.1001-0408.2026.19.16.

摘要

目的

为审方软件的优化及医疗机构合理用药辅助审方工具的选择提供参考。

方法

收集某三甲医院门诊2025年1月至2026年3月美康合理用药监测系统中符合纳入标准的处方559张。根据建立的复合参照标准（CRS）进行判定，其中阳性[存在药物相互作用（DDI）]处方206张、阴性（不存在DDI）处方353张。分别采用DeepSeek、豆包、文心一言、Kimi和通义千问对559张处方的DDI进行判定，并对其输出的DDI文本进行二值化判定。以CRS判定的结果为基准，构建混淆矩阵并计算模型的性能评价指标；比较任意2种大语言模型间敏感性的差异，计算二者的相对敏感性（RS）及其95%置信区间（CI），并以森林图展示。

结果

文心一言的敏感性最高（74.8%），漏报风险最低；但其特异性仅为40.8%，且准确性最低（53.3%），存在较高的误报风险。通义千问的准确性最高（65.3%），F1分数最优（0.587）。DeepSeek的敏感性与特异性均居中。豆包与Kimi的敏感性均较低（分别为38.8%、45.1%），漏报风险均较高。两两比较结果显示，文心一言的敏感性显著高于DeepSeek[

＝4.84，

＝0.028；RS＝1.124，95%CI（1.018，1.230）

]

，通义千问的敏感性显著高于豆包[

＝4.77，

＝0.029；RS＝1.239，95%CI（1.035，1.443）

]

，DeepSeek的敏感性显著高于豆包[

＝37.65，

＜0.001；RS＝1.715，95%CI（1.491，1.939）

]

；Kimi与豆包、通义千问与Kimi的敏感性差

异均无统计学意义（

＞0.05）。

结论

文心一言适用于对漏报容忍度较低的初步筛查场景，且需联合人工复核；通义千问适合用于门诊辅助审核；DeepSeek可作为无审方软件医疗机构的过渡方案；豆包与Kimi不宜独立用于处方中DDI审核。

Abstract

OBJECTIVE

To provide a reference for the optimization of prescription-review software and the selection of auxiliary prescription-review tools for medical institutions.

METHODS

A total of 559 eligible prescriptions were collected from the Meikang Prescription Automatic Screening System of a tertiary grade-A hospital from January 2025 to March 2026. Judged according to the composite reference standard （CRS） established in this study， there were 206 positive prescriptions [with drug-drug interaction （DDI）] and 353 negative prescriptions （without DDI）. DeepSeek， Doubao， Wenxin yiyan， Kimi and Tongyi qianwen were separately adopted to identify DDI in the 559 prescriptions， and binary judgment was performed on their output texts related to DDI. Taking the results judged by CRS as the gold standard， the confusion matrix was constructed and model performance indicators were calculated. The differences in sensitivity between any two large language models were compared， the relative sensitivity （RS） and its 95% confidence interval （CI） were calculated， and the results were visualized by a forest plot.

RESULTS

Wenxin yiyan achieved the highest sensitivity （74.8%） with the lowest risk of missed reports， whereas its specificity was only 40.8%. It had the lowest accuracy （53.3%） and presented a high risk of false positive reports. Tongyi qianwen obtained the highest accuracy （65.3%） and the optimal F1-score （0.587）. DeepSeek showed moderate performance in both sensitivity and specificity. Doubao and Kimi exhibited relatively low sensitivity （38.8% and 45.1%， respectively）， accompanied by high risks of missed reports. Pairwise comparison revealed that the sensitivity of Wenxin yiyan was significantly higher

than that of DeepSeek [

＝4.84，

＝0.028； RS＝1.124， 95%CI （1.018， 1.230）

]

. The sensitivity of Tongyi qianwen was significantly higher than that of Doubao [

＝4.77，

＝0.029； RS＝1.239， 95%CI （1.035， 1.443）

]

. DeepSeek had significantly higher sensitivity than Doubao [

＝37.65，

＜0.001； RS＝1.715， 95%CI （1.491， 1.939）

]

. No statistically significant differences in sensitivity were observed between Kimi versus Doubao and between Tongyi qianwen versus Kimi （

＞0.05）.

CONCLUSIONS

Wenxin yiyan is applicable for preliminary-screening scenarios with low tolerance for missed reports and requires combined manual review. Tongyi qianwen is suitable for auxiliary outpatient prescription review. DeepSeek can act as a transitional solution for medical institutions without dedicated prescription-review software. Doubao and Kimi are not recommended for independent application in prescription DDI audit.

关键词

Keywords

references

Aksoy N ， Ozturk N . A meta-analysis assessing the prevalence of drug-drug interactions among hospitalized patients [J ] . Pharmacoepidemiol Drug Saf ， 2023 ， 32 （ 12 ）： 1319 - 1330 .

World Health Organization . Patient safety：fact sheet [EB/OL ] .（ 2023-09-11 ）[ 2026-09-23 ] . https：//www.who.int/news-room/fact-sheets/detail/patient-safety https://www.who.int/news-room/fact-sheets/detail/patient-safety .

国家药品不良反应监测中心 . 国家药品不良反应监测年度报告：2025 年[EB/OL ] . （ 2026-04-27 ）[ 2026-09-23 ] . https：//www.cdr-adr.org.cn/center_news/202604/t20260- 427_51496.html https://www.cdr-adr.org.cn/center_news/202604/t20260-427_51496.html .

Li L ， Baker J ， Quirk R ， et al . Drug-drug interactions and actual harm to hospitalized patients：a multicentre study examining the prevalence pre- and post-electronic medication system implementation [J ] . Drug Saf ， 2024 ， 47 （ 6 ）： 557 - 569 .

国家卫生健康委办公厅 . 药事管理专业医疗质量控制指标：2025年版 [EB/OL ] .（ 2025-09-08 ）[ 2026-09-18 ] . https：//www.gov.cn/zhengce/zhengceku/202509/content_7039760.htm https://www.gov.cn/zhengce/zhengceku/202509/content_7039760.htm .

夏铮铮，李剑屏，颜海兰，等 . 合理用药监测系统在肿瘤专科医院的应用实践 [J ] . 肿瘤药学， 2022 ， 12 （ 5 ）： 649 - 655 .

Ray P P . ChatGPT：a comprehensive review on background，applications，key challenges，bias，ethics，limitations and future scope [J ] . Internet Things Cyber Phys Syst ， 2023 ， 3 ： 121 - 154 .

Thirunavukarasu A J ， Ting D S J ， Elangovan K ， et al . Large language models in medicine [J ] . Nat Med ， 2023 ， 29 （ 8 ）： 1930 - 1940 .

籍欣萌，昝红英，崔婷婷，等 . 中文医疗大模型综述：进展、评估与挑战 [J ] . 中文信息学报， 2024 ， 38 （ 11 ）： 1 - 12 .

牛露露，刘滔滔，黄天敏，等 . 评价DeepSeek与ChatGPT作为咨询工具在真实世界中回答药物-药物相互作用和超说明书用药的能力 [J ] . 中国现代应用药学， 2025 ， 42 （ 17 ）： 2929 - 2935 .

马红燕，周晓，张碧瑶，等 . DeepSeek大语言模型辅助处方合理性评估的实用性 [J ] . 医药导报， 2025 ， 44 （ 12 ）： 2057 - 2061 .

Mohammed M ， Khan M A ， Chaudhary V ， et al . Comparative performance of large language models and Drugs.com versus Lexicomp for antiseizure medication drug-drug interactions：a cross-sectional study with iterative prompting analysis [J ] . Explor Res Clin Soc Pharm ， 2026 ， 22 ： 100733 .

Domián B M ， Ashraf A R ， Fittler A T ， et al . Comparative evaluation of artificial intelligence platforms and drug interaction screening databases using real-world patient data [J ] . Explor Res Clin Soc Pharm ， 2025 ， 20 ： 100655 .

Wu Y G . Weighted McNemar’s test for the comparison of two screening tests in the presence of verification bias [J ] . Stat Med ， 2022 ， 41 （ 16 ）： 3149 - 3163 .

Hou B ， Mukherjee P ， Batheja V ， et al . One year on：assessing progress of multimodal large language model performance on RSNA 2024 case of the day questions [J ] . Radiology ， 2025 ， 316 （ 2 ）： e250617 .

Oehlert G W . A note on the delta method [J ] . Am Stat ， 1992 ， 46 （ 1 ）： 27 - 29 .

Sarkar S ， Baidya D K . Meta-analysis-interpretation of forest plots：a wood for the trees [J ] . Indian J Anaesth ， 2025 ， 69 （ 1 ）： 147 - 152 .

Bhati D ， Neha F ， Bandaru D S ， et al . Mapping the LLM landscape：a cross-family survey of architectures，alignment methods，and benchmark performance [J ] . AI ， 2026 ， 7 （ 4 ）： 142 .

Sicard J ， Montastruc F ， Achalme C ， et al . Can large language models detect drug-drug interactions leading to adverse drug reactions? [J ] . Ther Adv Drug Saf ， 2025 ， 16 ： 20420986251339358 .

Bakker T ， Klopotowska J E ， Dongelmans D A ， et al . The effect of computerised decision support alerts tailored to intensive care on the administration of high-risk drug combinations，and their monitoring：a cluster randomised stepped-wedge trial [J ] . Lancet ， 2024 ， 403 （ 10425 ）： 439 - 449 .

Hu H ， Wang L ， Chen L Q ， et al . Benchmarking large language models for drug combination alerts：achieving expert-level reliability via knowledge grounding and contextual reasoning [J ] . J Med Chem ， 2026 ， 69 （ 3 ）： 3477 - 3489 .

Xu K X ， Xu Y R ， Shi Q . Harnessing computational power for intelligent oncology in the age of large models：status，challenges，and prospects [J ] . Intell Oncol ， 2026 ， 2 （ 1 ）： 100038 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Prediction of the drug-drug interaction between sodium tanshinone ⅡA sulfonate and clopidogrel based on the PBPK model

Applications and prospects of prompt engineering in pharmaceutical popularization

Construction and application of anti-tumor drug prescription review decision-support system in a large general hospital

Analysis of the application status of prescription pre-review systems in Yunnan province

我院门诊处方中26 例高危药品用药错误的原因分析及防范措施

Related Author

QIAN Fengdan

SUN Mengru

ZHANG Lichao

ZHANG Die

ZHU Mingyang

QIAN Yan

FAN Xinyi

ZANG Jing

Related Institution

Dept. of Pharmacy， Shanghai Municipal Hospital of Traditional Chinese Medicine Affiliated to Shanghai University of Traditional Chinese Medicine

Dept. of Pharmacy，The Second Affiliated Hospital of Chongqing Medical University

Dept. of Pharmacy， Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine

Dept. of Pharmacy， Shanghai Eighth People’s Hospital

Lingyun Community Health Service Center， Xuhui District in Shanghai

⁰