Validation of cervical vertebral maturation stages: Artificial intelligence vs human observer visual analysis

Abstract

Introduction: This study aimed to develop an artificial neural network (ANN) model for cervical vertebral maturation (CVM) analysis and validate the model's output with the results of human observers. Methods: A total of 647 lateral cephalograms were selected from patients with 10-30 years of chronological age (mean +/- standard deviation, 15.36 +/- 4.13 years). New software with a decision support system was developed for manual labeling of the dataset. A total of 26 points were marked on each radiograph. The CVM stages were saved on the basis of the final decision of the observer. Fifty-four image features were saved in text format. A new subset of 72 radiographs was created according to the classification result, and these 72 radiographs were visually evaluated by 4 observers. Weighted kappa (w kappa) and Cohen's kappa (c kappa) coefficients and percentage agreement were calculated to evaluate the compatibility of the results. Results: Intraobserver agreement ranges were as follows: w kappa = 0.92-0.98, c kappa = 0.65-0.85, and 70.8%-87.5%. Interobserver agreement ranges were as follows: w kappa = 0.76-0.92, c kappa = 0.4-0.65, and 50%-72.2%. Agreement between the ANN model and observers 1, 2, 3, and 4 were as follows: w kappa = 0.85 (c kappa = 0.52, 59.7%), w kappa = 0.8 (c kappa = 0.4, 50%), w kappa = 0.87 (c kappa = 0.55, 62.5%), and w kappa = 0.91 (c kappa = 0.53, 61.1%), respectively (P < 0.001). An average of 58.3% agreement was observed between the ANN model and the human observers. Conclusions: This study demonstrated that the developed ANN model performed close to, if not better than, human observers in CVM analysis. By generating new algorithms, automatic classification of CVM with artificial intelligence may replace conventional evaluation methods used in the future.