Application of virtual reality technology in the teaching of orthognathic surgery in the era of artificial intelligence

Bao Jiahao; Cheng Mengjia; Yu Hongbo

doi:10.3760/cma.j.cn116021-20230403-01508

您当前的位置：

首页 >

文章列表页 >

Application of virtual reality technology in the teaching of orthognathic surgery in the era of artificial intelligence

Teaching Reform Special Column of Shanghai Jiao Tong University School of Medicine | 更新时间：2023-12-26

- Application of virtual reality technology in the teaching of orthognathic surgery in the era of artificial intelligence
- Chinese Journal of Medical Education Research Vol. 22, Issue 12, Pages: 1761-1766(2023)
- 作者机构：
  
  上海交通大学医学院附属第九人民医院口腔颅颌面科，上海交通大学口腔医学院，国家口腔医学中心，国家口腔疾病临床医学研究中心，上海市口腔医学重点实验室，上海市口腔医学研究所，上海 200011
- 作者简介：
  
  Yu Hongbo, Email: yhb3508@163.com
- 基金信息：
  
  National Natural Science Foundation of China(81571022);Shanghai Natural Science Foundation(23ZR1438100);The Multi-center Clinical Research Project of Shanghai Jiao Tong University School of Medicine(DLY201808)
- DOI：10.3760/cma.j.cn116021-20230403-01508
  CLC： R78
- Published：20 December 2023，
  
  Received：03 April 2023，
扫描看全文
鲍嘉豪, 程梦佳, 于洪波. 人工智能背景下虚拟现实技术在正颌外科教学中的应用现状研究[J]. 中华医学教育探索杂志, 2023, 22(12): 1761-1766.

Bao Jiahao, Cheng Mengjia, Yu Hongbo. Application of virtual reality technology in the teaching of orthognathic surgery in the era of artificial intelligence. [J]. Chinese Journal of Medical Education Research 22(12):1761-1766(2023)
鲍嘉豪, 程梦佳, 于洪波. 人工智能背景下虚拟现实技术在正颌外科教学中的应用现状研究[J]. 中华医学教育探索杂志, 2023, 22(12): 1761-1766. DOI： 10.3760/cma.j.cn116021-20230403-01508.

Bao Jiahao, Cheng Mengjia, Yu Hongbo. Application of virtual reality technology in the teaching of orthognathic surgery in the era of artificial intelligence. [J]. Chinese Journal of Medical Education Research 22(12):1761-1766(2023) DOI： 10.3760/cma.j.cn116021-20230403-01508.

摘要

正颌外科因手术设计和操作复杂，对医生的临床经验要求较高，传统教学模式难以满足复杂的正颌外科手术设计和操作的培训需求。虚拟现实（virtual reality，VR）技术为此提供了新的解决途径，但由于模型构建不精确、手术设计过程复杂、模拟手术内容单一、实时性与仿真性难以兼顾、缺乏自动化评价功能等问题限制了其推广应用。人工智能（artificial intelligence，AI）技术因其在数据挖掘、图像处理等方面的强大能力，为正颌外科临床和教学带来多重潜力，可有效弥补传统VR教学的不足，可应用于个性化模型构建、智能诊断、术后预测、虚拟手术仿真及个性化反馈和辅导等方面，有望推动正颌外科教学向更精准、高效的方向发展，促进医学与技术的融合，加速临床应用的转化。

Abstract

Owing to complex surgical planning and procedures

orthognathic surgery requires surgeons to possess a high level of clinical expertise. However

traditional teaching methods are difficult to meet the requirements for complex training on orthognathic surgical planning and operation. Virtual reality (VR) technology provides a new solution

but it is limited by problems like imprecise modeling

complex surgical design

single-simulated surgical contents

inability to integrate real time and simulation

and a lack of automatic evaluation function. Artificial intelligence (AI) is potent in data mining and image processing

which can address the limitations of traditional VR teaching

bringing multiple potential for clinical and educational application in orthognathic surgery. With research in personalized model construction

intelligent diagnostics

post-operative prediction

virtual surgical simulation

and personalized feedback and guidance

the fusion of "VR plus AI" holds promise for advancing orthognathic surgical teaching in a more precise and efficient direction

promoting the integration of medicine and technology

and speeding up the transformation of clinical applications.

关键词

正颌外科虚拟现实人工智能虚拟手术设计手术仿真

Keywords

Orthognathic surgeryVirtual realityArtificial intelligenceVirtual surgical planningSurgical simulation

references

Pulijala Y, Ma M, Pears M, et al. An innovative virtual reality training tool for orthognathic surgery [J]. Int J Oral Maxillofac Surg, 2018, 47(9): 1199-1205. DOI: 10.1016/j.ijom.2018.01.005http://dx.doi.org/10.1016/j.ijom.2018.01.005.

姜楠，刘尧，赵文丽，等. 数字化模型外科联合病例教学培训模式在正颌外科专科医师规培教学中的应用[J]. 中国药物与临床， 2021， 21（12）： 2170-2171. DOI: 10.11655/zgywylc2021.12.074http://dx.doi.org/10.11655/zgywylc2021.12.074.

王梓霖，郭陟永，刘一戈，等. 虚拟手术仿真培训系统在头颈肿瘤外科医师训练中的应用[J]. 肿瘤预防与治疗， 2021， 34（12）： 1131-1135. DOI: 10.3969/j.issn.1674-0904.2021.12.007http://dx.doi.org/10.3969/j.issn.1674-0904.2021.12.007.

Lungu AJ, Swinkels W, Claesen L, et al. A review on the applications of virtual reality, augmented reality and mixed reality in surgical simulation: an extension to different kinds of surgery [J]. Expert Rev Med Devices, 2021, 18(1): 47-62. DOI: 10.1080/17434440.2021.1860750http://dx.doi.org/10.1080/17434440.2021.1860750.

Ghaednia H, Fourman MS, Lans A, et al. Augmented and virtual reality in spine surgery, current applications and future potentials [J]. Spine J, 2021, 21(10): 1617-1625. DOI: 10.1016/j.spinee.2021.03.018http://dx.doi.org/10.1016/j.spinee.2021.03.018.

Tel A, Arboit L, De Martino M, et al. Systematic review of the software used for virtual surgical planning in craniomaxillofacial surgery over the last decade [J]. Int J Oral Maxillofac Surg, 2023, 52(7): 775-786. DOI: 10.1016/j.ijom.2022.11.011http://dx.doi.org/10.1016/j.ijom.2022.11.011.

谢福平，刘欢欢，陈嘉民，等. Proplan虚拟软件在正颌外科教学中的应用评价[J]. 上海口腔医学， 2018， 27（4）： 436-439. DOI: 10.19439/j.sjos.2018.04.022http://dx.doi.org/10.19439/j.sjos.2018.04.022.

颜光启，啜文钰，杨鸣良，等. 数字化技术在正颌外科及创伤外科临床教学中的应用[J]. 中国口腔颌面外科杂志， 2017， 15（5）： 463-465. DOI: 10.19438/j.cjoms.2017.05.017http://dx.doi.org/10.19438/j.cjoms.2017.05.017.

Gupta A, Cecil J, Pirela-Cruz M, et al. A virtual reality enhanced cyber-human framework for orthopedic surgical training [J]. IEEE Syst J, 2019, 13(3): 3501-3512. DOI: 10.1109/JSYST.2019.2896061http://dx.doi.org/10.1109/JSYST.2019.2896061.

Wu F, Chen X, Lin Y, et al. A virtual training system for maxillofacial surgery using advanced haptic feedback and immersive workbench [J]. Int J Med Robot, 2014, 10(1): 78-87. DOI: 10.1002/rcs.1514http://dx.doi.org/10.1002/rcs.1514.

张楚茜，回文宇，林艳萍，等. 用于临床技能培养的口腔颅颌面数字化虚拟手术平台的建立与应用[J]. 组织工程与重建外科杂志， 2019， 15（3）： 208-211. DOI: 10.3969/j.issn.1673-0364.2019.03.022http://dx.doi.org/10.3969/j.issn.1673-0364.2019.03.022.

Zaragoza-Siqueiros J, Medellin-Castillo HI, de la Garza-Camargo H, et al. An integrated haptic-enabled virtual reality system for orthognathic surgery planning [J]. Comput Methods Biomech Biomed Engin, 2019, 22(5): 499-517. DOI: 10.1080/10255842.2019.1566817http://dx.doi.org/10.1080/10255842.2019.1566817.

Medellin-Castillo HI, Zaragoza-Siqueiros J, Govea-Valladares EH, et al. Haptic-enabled virtual training in orthognathic surgery [J]. Virtual Real, 2021, 25(1): 53-67. DOI: 10.1007/s10055-020-00438-6http://dx.doi.org/10.1007/s10055-020-00438-6.

Zhao H, Cheng M, Huang J, et al. A virtual surgical prototype system based on gesture recognition for virtual surgical training in maxillofacial surgery [J]. Int J Comput Assist Radiol Surg, 2023, 18(5): 909-919. DOI: 10.1007/s11548-022-02790-1http://dx.doi.org/10.1007/s11548-022-02790-1.

Mirchi N, Bissonnette V, Yilmaz R, et al. The Virtual Operative Assistant: an explainable artificial intelligence tool for simulation-based training in surgery and medicine [J]. PLoS One, 2020, 15(2): e0229596. DOI: 10.1371/journal.pone.0229596http://dx.doi.org/10.1371/journal.pone.0229596.

Fazlollahi AM, Bakhaidar M, Alsayegh A, et al. Effect of artificial intelligence tutoring vs expert instruction on learning simulated surgical skills among medical students: a randomized clinical trial [J]. JAMA Netw Open, 2022, 5(2): e2149008.DOI: 10.1001/jamanetworkopen.2021.49008http://dx.doi.org/10.1001/jamanetworkopen.2021.49008.

Liu Q, Deng H, Lian C, et al. SkullEngine: a multi-stage CNN framework for collaborative CBCT image segmentation and landmark detection [J]. Mach Learn Med Imaging, 2021, 12966: 606-614. DOI: 10.1007/978-3-030-87589-3_62http://dx.doi.org/10.1007/978-3-030-87589-3_62.

Cui Z, Fang Y, Mei L, et al. A fully automatic AI system for tooth and alveolar bone segmentation from cone-beam CT images [J]. Nat Commun, 2022, 13(1): 2096. DOI: 10.1038/s41467-022-29637-2http://dx.doi.org/10.1038/s41467-022-29637-2.

刘琼，马婷婷，闵巧梅，等. 数字化颅颌面模型在正畸头影测量教学中的应用[J]. 中国高等医学教育， 2022（4）： 49-50. DOI: 10.3969/j.issn.1002-1701.2022.04.025http://dx.doi.org/10.3969/j.issn.1002-1701.2022.04.025.

何琴，卜艳，林光磊，等. Uceph头影测量软件在口腔正畸实验教学中的应用研究[J]. 科教导刊， 2022（16）： 101-103. DOI: 10.16400/j.cnki.kjdk.2022.16.032http://dx.doi.org/10.16400/j.cnki.kjdk.2022.16.032.

罗恩. 人工智能正颌外科的探索与临床初步应用[J]. 国际口腔医学杂志， 2022， 49（2）： 125-131. DOI: 10.7518/gjkq.2022041http://dx.doi.org/10.7518/gjkq.2022041.

Jeong SH, Yun JP, Yeom HG, et al. Deep learning based discrimination of soft tissue profiles requiring orthognathic surgery by facial photographs [J]. Sci Rep, 2020, 10(1): 16235. DOI: 10.1038/s41598-020-73287-7http://dx.doi.org/10.1038/s41598-020-73287-7.

Kim I, Misra D, Rodriguez L, et al. Malocclusion classification on 3D cone-beam CT craniofacial images using multi-channel deep learning models [J]. Annu Int Conf IEEE Eng Med Biol Soc, 2020, 2020: 1294-1298. DOI: 10.1109/embc44109.2020.9176672http://dx.doi.org/10.1109/embc44109.2020.9176672.

Cheng M, Zhang X, Wang J, et al. Prediction of orthognathic surgery plan from 3D cephalometric analysis via deep learning [J]. BMC Oral Health, 2023, 23(1): 161. DOI: 10.1186/s12903-023-02844-zhttp://dx.doi.org/10.1186/s12903-023-02844-z.

Knoops PGM, Borghi A, Breakey RWF, et al. Three-dimensional soft tissue prediction in orthognathic surgery: a clinical comparison of Dolphin, ProPlan CMF, and probabilistic finite element modelling [J]. Int J Oral Maxillofac Surg, 2019, 48(4): 511-518. DOI: 10.1016/j.ijom.2018.10.008http://dx.doi.org/10.1016/j.ijom.2018.10.008.

Kim D, Kuang T, Rodrigues YL, et al. A novel incremental simulation of facial changes following orthognathic surgery using FEM with realistic lip sliding effect [J]. Med Image Anal, 2021, 72: 102095. DOI: 10.1016/j.media.2021.102095http://dx.doi.org/10.1016/j.media.2021.102095.

Lampen N, Kim D, Fang X, et al. Deep learning for biomechanical modeling of facial tissue deformation in orthognathic surgical planning [J]. Int J Comput Assist Radiol Surg, 2022, 17(5): 945-952. DOI: 10.1007/s11548-022-02596-1http://dx.doi.org/10.1007/s11548-022-02596-1.

Ma L, Xiao D, Kim D, et al. Simulation of postoperative facial appearances via geometric deep learning for efficient orthognathic surgical planning [J]. IEEE Trans Med Imaging, 2023, 42(2): 336-345. DOI: 10.1109/tmi.2022.3180078http://dx.doi.org/10.1109/tmi.2022.3180078.

Cheng Q, Sun P, Yang C, et al. Forecasting and simulation of cutting force in virtual surgery based on particle filtering [J].Appl Intell, 2021, 51(4): 1934-1946. DOI: 10.1007/s10489-020-01910-1http://dx.doi.org/10.1007/s10489-020-01910-1.

Munawar A, Li Z, Kunjam P, et al. Virtual reality for synergistic surgical training and data generation [J]. Comput Methods Biomech Biomed Eng Imaging Vis, 2022, 10(4): 366-374. DOI: 10.1080/21681163.2021.1999331http://dx.doi.org/10.1080/21681163.2021.1999331.

Kirubarajan A, Young D, Khan S, et al. Artificial intelligence and surgical education: a systematic scoping review of interventions [J]. J Surg Educ, 2022, 79(2): 500-515. DOI: 10.1016/j.jsurg.2021.09.012http://dx.doi.org/10.1016/j.jsurg.2021.09.012.

Ahmidi N, Poddar P, Jones JD, et al. Automated objective surgical skill assessment in the operating room from unstructured tool motion in septoplasty [J]. Int J Comput Assist Radiol Surg, 2015, 10(6): 981-991. DOI: 10.1007/s11548-015-1194-1http://dx.doi.org/10.1007/s11548-015-1194-1.

Funke I, Mees ST, Weitz J, et al. Video-based surgical skill assessment using 3D convolutional neural networks [J]. Int J Comput Assist Radiol Surg, 2019, 14(7): 1217-1225. DOI: 10.1007/s11548-019-01995-1http://dx.doi.org/10.1007/s11548-019-01995-1.

Yilmaz R, Winkler-Schwartz A, Mirchi N, et al. Continuous monitoring of surgical bimanual expertise using deep neural networks in virtual reality simulation [J]. NPJ Digit Med, 2022, 5(1): 54. DOI: 10.1038/s41746-022-00596-8http://dx.doi.org/10.1038/s41746-022-00596-8.

Fekri P, Dargahi J, Zadeh M. Deep learning-based haptic guidance for surgical skills transfer [J]. Front Robot AI, 2020, 7: 586707. DOI: 10.3389/frobt.2020.586707http://dx.doi.org/10.3389/frobt.2020.586707.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Application of standardized family in pediatric clinical teaching

Cultivation of intelligent medical talents based on educational ecology

Current status of the application of artificial intelligence in clinical anesthesia teaching

Related Author

Yang Binbin

Zhu Yueling

Li Wei

Gao Zhigang

Hong Yunxia

Zhao Wenlong

Xu Qian

Fu Hailong

Related Institution

Department of General Surgery, The Children's Hospital, Zhejiang University School of Medicine, National Clinical Research Center for Child Health

Department of Traditional Chinese Medicine, The Children's Hospital, Zhejiang University School of Medicine, National Clinical Research Center for Child Health

Department of Clinical Laboratory, The Children's Hospital, Zhejiang University School of Medicine, National Clinical Research Center for Child Health

Education Office, The Children's Hospital, Zhejiang University School of Medicine, National Clinical Research Center for Child Health

College of Medical Informatics, Chongqing Medical University

⁰