空间双线性注意力网络识别溃疡性结肠炎与克罗恩病

戚婧; 阮广聪; 杨毅; 吴毅; 曹倩; 魏艳玲; 粘永健

doi:10.16016/j.2097-0927.202209187

您当前的位置：

首页 >

文章列表页 >

空间双线性注意力网络识别溃疡性结肠炎与克罗恩病

基础医学 | 更新时间：2026-01-09

- 空间双线性注意力网络识别溃疡性结肠炎与克罗恩病
- Identification of ulcerative colitis and Crohn's disease based on spatial and bilinear attention network
- 陆军军医大学学报 2023年45卷第3期页码：227-234
- 作者机构：
  
  1.400038 重庆, 陆军军医大学(第三军医大学)生物医学工程与影像医学系数字医学教研室
  2.400042 重庆, 陆军特色医学中心消化内科
  3.310016 杭州, 浙江大学医学院附属邵逸夫医院消化内科
- 作者简介：
  
  [粘永健，E-mail：yjnian@tmmu.edu.cn NIAN Yongjian, E-mail: yjnian@tmmu.edu.cn
- 基金信息：
  
  重庆市研究生科研创新项目(CYS22746)
- DOI：10.16016/j.2097-0927.202209187
  中图分类号： R319;R447;R574.62
- 收稿：2022-09-21，
  
  修回：2022-10-18，
  
  纸质出版：2023-02-15
- 稿件说明：
移动端阅览
戚婧, 阮广聪, 杨毅, 等. 空间双线性注意力网络识别溃疡性结肠炎与克罗恩病[J]. 陆军军医大学学报, 2023,45(3):227-234.

Jing QI, Guangcong RUAN, Yi YANG, et al. Identification of ulcerative colitis and Crohn's disease based on spatial and bilinear attention network[J]. Journal of Army Medical University, 2023, 45(3): 227-234.
戚婧, 阮广聪, 杨毅, 等. 空间双线性注意力网络识别溃疡性结肠炎与克罗恩病[J]. 陆军军医大学学报, 2023,45(3):227-234. DOI： 10.16016/j.2097-0927.202209187.

Jing QI, Guangcong RUAN, Yi YANG, et al. Identification of ulcerative colitis and Crohn's disease based on spatial and bilinear attention network[J]. Journal of Army Medical University, 2023, 45(3): 227-234. DOI： 10.16016/j.2097-0927.202209187.

摘要

目的

利用深度学习技术辅助内镜医师识别溃疡性结肠炎(ulcerative colitis

UC)与克罗恩病(Crohn's disease

CD)。

方法

收集2018年1月至2020年11月陆军特色医学中心消化内科与邵逸夫医院消化内科共1 576例受试者的内镜图像

包括CD、UC与正常三类共计34 300幅

并按照9:1的比例随机划分训练集与测试集

用于对网络进行训练与测试。在ResNet50基础上构建新颖的空间双线性深度网络(SABA-ResNet)

引入空间注意机制

通过膨胀卷积扩大感受野以联系上下文信息

并与普通卷积局部归纳特性相配合

自适应聚焦病变区域。利用双线性注意提高网络的特征表示能力

以二阶信息加权特征映射的通道信息

从而提高模型的分类性能。

结果

SABA-ResNet在测试集上对CD、UC和正常识别的总体准确率为92.67%(95%

: 91.91~93.37)

AUC分别为0.978(95%

: 0.972~0.983)、0.977(95%

: 0.971~0.982)和0.999(95%

: 0.998~1.000)

灵敏度分别为88.40%、89.07%、98.89%

特异性分别为95.49%、94.88%、98.93%

F1值分别为88.80%、89.01%和98.60%。消融实验与类激活映射图表明空间注意与双线性注意可帮助模型捕获更多病变区域的特征。

结论

所构建的网络将空间注意与双线性注意相结合

在CD、UC与正常的识别中取得了良好的性能

可以有效辅助内镜医师对UC与CD进行识别。

Abstract

Objective

To identify ulcerative colitis (UC) and Crohn's disease (CD) with aid of deep learning technology for endoscopists.

Methods

From January 2018 to November 2020

the endoscopic images of 1 576 subjects (including 34 300 CD

UC and normal images) were collected from the Department of Gastroenterology of Army Medical Center of PLA and Sir Run Run Shaw Hospital.The training set and test set were randomly divided according to the ratio of 9:1 to train and test the neural network.A novel spatial and bilinear deep network (SABA-ResNet) was constructed on the basis of ResNet50.The spatial attention mechanism was introduced

and the receptive field was expanded by dilated convolution to leverage contextual information

which was combined with the local induction of standard convolution to adaptively focus the lesion region.Bilinear attention was applied to improve the feature representation ability of the network

and the second-order information was used to weight the channel information of the feature map

so as to improve the classification performance of the model.

Results

The overall accuracy of SABA-ResNet for the recognition of CD

UC and normal tissues on the test set was 92.67%(95%

: 91.91~93.37)

the AUC value was 0.978(95%

: 0.972~0.983)

0.977(95%

: 0.971~0.982) and 0.999(95%

: 0.998~1.000)

the sensitivity was 88.40%

89.07% and 98.89%

the specificity was 95.49%

94.88% and 98.93%

and the F1 value was 88.80%

89.01% and 98.60%

respectively.The ablation experiment and the class activation map suggested that spatial attention and bilinear attention could help the model capture more features of the lesion region.

Conclusion

Our constructed network combines spatial attention and bilinear attention

achieves excellent performance in the recognition of CD

UC and normal tissue

and effectively assist endoscopists in the diagnosis of UC and CD.

关键词

Keywords

references

MCDOWELL C, FAROOQ U, HASEEB M. Inflammatory bowel disease[M]. Treasure Island: StatPearls Publishing, 2022.

MALIK T F, AURELIO D M. Extraintestinal manifestations of inflammatory bowel disease[M]. Treasure Island: StatPearls Publishing, 2022.

PARK J, CHEON J H. Incidence and prevalence of inflammatory bowel disease across Asia[J]. Yonsei Med J, 2021, 62(2): 99-108. DOI: 10.3349/ymj.2021.62.2.99.

FLYNN S, EISENSTEIN S. Inflammatory bowel disease presentation and diagnosis[J]. Surg Clin North Am, 2019, 99(6): 1051-1062. DOI: 10.1016/j.suc.2019.08.001.

SMOLANDER J, DEHMER M, EMMERT-STREIB F. Comparing deep belief networks with support vector machines for classifying gene expression data from complex disorders[J]. FEBS Open Bio, 2019, 9(7): 1232-1248. DOI: 10.1002/2211-5463.12652.

TONG Y R, LU K M, YANG Y Y, et al . Can natural language processing help differentiate inflammatory intestinal diseases in China? Models applying random forest and convolutional neural network approaches[J ] . BMC Med Inform Decis Mak, 2020, 20(1): 248. DOI: 10.1186/s12911-020-01277-w.

WANG L J, CHEN L P, WANG X Y, et al . Development of a convolutional neural network-based colonoscopy image assessment model for differentiating Crohn's disease and ulcerative colitis[J ] . Front Med, 2022, 9: 789862. DOI: 10.3389/fmed.2022.789862.

[YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL ] . 2015, arXiv: 1511.07122. https://arxiv.org/abs/1511.07122 https://arxiv.org/abs/1511.07122 .

SELVARAJU R R, COGSWELL M, DAS A, et al . Grad-CAM: visual explanations from deep networks via gradient-based localization[C ] . IEEE International Conference on Computer Vision, 2017: 618-626.

[VASWANI A, SHAZEER N,PARMAR N, et al . Attention is all You need[C ] //Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6000-6010. DOI: 10.5555/3295222.3295349 http://dx.doi.org/10.5555/3295222.3295349 ..

GUO M H, XU T X, LIU J J, et al . Attention mechanisms in computer vision: a survey[J ] . Comput Vis Media, 2022, 8(3): 331-368. DOI: 10.1007/s41095-022-0271-y.

[HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C ] //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018: 7132-7141. DOI: 10.1109/CVPR.2018.00745 http://dx.doi.org/10.1109/CVPR.2018.00745 .

[JADERBERG M, SIMONYAN K, ZISSERMAN A, et al . Spatial transformer networks[C ] //Proceedings of the 28th International Conference on Neural Information Processing Systems. New York: ACM, 2015: 2017-2025. DOI: 10.5555/2969442.2969465 http://dx.doi.org/10.5555/2969442.2969465 ..

[WOO S, PARK J, LEE J Y, et al . CBAM: convolutional block attention module[EB/OL ] . arXiv, 2018: 1807.06521. https://arxiv.org/abs/1807.06521 https://arxiv.org/abs/1807.06521 ..

[GAO Z L, XIE J T, WANG Q L, et al . Global second-order pooling convolutional networks[C ] //2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020: 3019-3028. DOI: 10.1109/CVPR.2019.00314 http://dx.doi.org/10.1109/CVPR.2019.00314 ..

[QIN Z Q, ZHANG P Y, WU F, et al . FcaNet: frequency channel attention networks[C ] //2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2022: 763-772. DOI: 10.1109/ICCV48922.2021.00082 http://dx.doi.org/10.1109/ICCV48922.2021.00082 ..

[LIN T Y, ROYCHOWDHURY A, MAJI S. Bilinear CNN models for fine-grained visual recognition[C ] //2015 IEEE International Conference on Computer Vision (ICCV). IEEE, 2016: 1449-1457. DOI: 10.1109/ICCV.2015.170 http://dx.doi.org/10.1109/ICCV.2015.170 .

[KIM J H, JUN J, ZHANG B T. Bilinear attention networks[C ] //Proceedings of the 32nd International Conference on Neural Information Processing Systems. New York: ACM, 2018: 1571-1581. DOI: 10.5555/3326943.3327087 http://dx.doi.org/10.5555/3326943.3327087 .

[FANG P F, ZHOU J M, ROY S, et al . Bilinear attention networks for person retrieval[C ] //2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2020: 8029-8038. DOI: 10.1109/ICCV.2019.00812 http://dx.doi.org/10.1109/ICCV.2019.00812 ..

XIAO B, YANG Z Y, QIU X M, et al . PAM-DenseNet: a deep convolutional neural network for computer-aided COVID-19 diagnosis[J ] . IEEE Trans Cybern, 2022, 52(11): 12163-12174. DOI: 10.1109/TCYB.2020.3042837.

ZHANG J P, XIE Y T, XIA Y, et al . Attention residual learning for skin lesion classification[J ] . IEEE Trans Med Imaging, 2019, 38(9): 2092-2103. DOI: 10.1109/TMI.2019.2893944.

XING X H, YUAN Y X, MENG M Q H. Zoom in lesions for better diagnosis: attention guided deformation network for WCE image classification[J]. IEEE Trans Med Imaging, 2020, 39(12): 4047-4059. DOI: 10.1109/TMI.2020.3010102.

SEYEDIAN S S, NOKHOSTIN F, MALAMIR M D. A review of the diagnosis, prevention, and treatment methods of inflammatory bowel disease[J]. J Med Life, 2019, 12(2): 113-122. DOI: 10.25122/jml-2018-0075.

RUAN G C, QI J, CHENG Y, et al . Development and validation of a deep neural network for accurate identification of endoscopic images from patients with ulcerative colitis and Crohn's disease[J ] . Front Med (Lausanne), 2022, 9(8): 854677. DOI: 10.3389/fmed.2022.854677.

[LI P H, XIE J T, WANG Q L, et al . Towards faster training of global covariance pooling networks by iterative matrix square root normalization[C ] //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018: 947-955. DOI: 10.1109/CVPR.2018.00105 http://dx.doi.org/10.1109/CVPR.2018.00105 ..

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

融合深度学习、临床与影像组学特征的综合模型提升非小细胞肺癌EGFR突变预测效能

基于CT影像的3D ResNet18模型在肺结核耐药性识别中的预测性能研究

深度学习和磁共振图像辅助脑胶质瘤基因分型的研究进展

基于深度学习的MRI影像女性尿控解剖元件及相关结构的自动分割

基于小波变换和CNN-Transformer的超声甲状腺结节分割算法研究