Objective：In order to achieve high-quality manual annotation of large-sample thyroid ultrasound images，we developed a semi-automatic semantic annotation software tool for multi-feature thyroid ultrasound images. In the study，we aimed to explore the reproducibility of doctors’ application of the thyroid ultrasound image annotation tool. Methods：Fifty cases of thyroid nodules in our hospital were randomly selected. Two ultrasound doctors independently used the annotation software tool to perform thyroid ultrasound image recognition. Two ultrasound doctors independently performed ultrasound images recognition through visual images.One ultra－sound doctor randomly used annotation software tools and visual images for image recognition，and then observer consistency analysis was performed. Results：In this study，two ultrasound doctors used annotation software tools to observe the differences between ultra－sound features and ultrasound grading（?资 value）：boundary 0.94，morphology 0.94，echo 0.63，internal structure 0.87，calcification 0.89，2015 American Thyroid Association（ATA） risk stratification was 0.84（P<0.01）. The difference between the two ultrasound doctors through the visual ultrasound image was as follows（?资 value）：boundary 0.75，morphology 0.80，echo 0.53，internal structure 0.62，calcification 1.00，ATA risk stratification 0.42（P<0.01）. The difference between the observers using the image annotation system and the visual ultrasound image was（?资 value）：boundary 1.00，morphology 1.00，echo 1.00，internal structure 0.88，calcification 1.00，ATA risk stratification 0.96（P<0.01）. Conclusion：The multi-feature thyroid ultrasound image semi-automatic semantic annota－tion software tool provides a standardized method for the diag－nosis of thyroid nodule ultrasound diagnosis，and provides high-quality training and verification data sets for its fine-grained in－telligent recognition research.