search
Contact Us
HOME

  • Crossref logo
  • Crossref Similarity Check logo
Journal Search Engine

to

Search Advanced Search Adode Reader(link)
Download PDF Export Citaion korean bibliography PMC previewer
ISSN : 1225-1577(Print)
ISSN : 2384-0900(Online)
The Korean Journal of Oral and Maxillofacial Pathology Vol.45 No.6 pp.189-196
DOI : https://doi.org/10.17779/KAOMP.2021.45.6.001

AlexNet in Determining Osteoporosis on Dental Panoramic Radiograph

Suyoung Bae1), In-Ja Song2), Hyongsuk Kim3), Shyam Adhikari3), Jae-Seo Lee4), Suk-Ja Yoon4), Ho-Gul Jeong5)*
1)School of Dentistry, Chonnam National University
2)Department of Nursing, Kwangju Women's University
3)Chonbuk National University Electronic engineering
4)Department of Oral and Maxillofacial Radiology, School of Dentistry, Dental Science Research Institute, Chonnam National University
5)InvisionLab Inc.
*Correspondence: Ho-Gul Jeong, InvisionLab Inc. Beobwon-ro, Songpa-gu, Seoul, Korea
Tel: +82-70-8693-1509 Email: rari98@naver.com
October 12, 2021 October 22, 2021 December 15, 2021

Abstract


This study was performed as a part of serial experiments of applying convolutional neural network(CNN) in determining osteoporosis on panoramic radiograph. The purpose of this study was to investigate how sensitively CNN determine osteoporosis on cropped panoramic radiograph. Panoramic radiographs from 1268 female patients(mean age 45.2 ± 21.1 yrs) were selected for this study. For the osteoporosis group, 633(mean age 72.2 ± 8.5 yrs) were selected, while for the normal group 635(mean age 28.3 ± 7.0 yrs). AlexNet was utilized as CNN in this study. A multiple-column CNN was designed with two rectangular regions of interest on the mandible inferior area. An occluding method was used to analyze the sensitive area in determining osteoporosis on AlexNet. Testing of AlexNet showed accuracy of 99% in determining osteoporosis on panoramic radiographs. AlexNet was sensitive at the area of cortical and cancellous bone of the mandible inferior area including adjacent soft tissue.


Osteoporosis , Panoramic Radiograph , Mandible , Neural network

치과용 파노라마방사선사진에서 AlexNet의 골다공증 판정

배 수영1), 송 인자2), 김 형석3), Shyam Adhikari3), 이 재서4), 윤 숙자4), 정 호걸5)*
1)전남대학교 치의학전문대학원
2)광주여자대학교 간호학과
3)전북대학교 전자공학부
4)전남대학교 치의학전문대학원 구강악안면방사선학교실, 치의학연구소
5)인비젼랩

초록

    Ⅰ. INTRODUCTION

    Osteoporosis is commonly identified as 'A disease characterized by low bone mass and microarchitectural deterioration of bone tissue, leading to enhanced bone fragility and a consequential increase in fracture risk'1,2).

    World Health Organization(WHO) study group meeting reported in 2004 that osteoporosis causes more than 8.9 million fractures annually worldwide. The risk for a wrist, hip or vertebral fractures is estimated to be 30-40% in developed countries. Osteoporosis also causes people to become bedridden with serious complications which might threaten life of the elders3).

    Most of the patients who suffer from osteoporosis have no feeling of the need to examinate their conditions because there is no symptom until fracture occurs. Fractures of patients with osteoporosis demand surgical treatment. Because the majority of patients who suffer from this disease are senior, they have trouble in daily lives and show a high ratio of death after surgery of fractures4). Dual-energy X-ray absorptiometry(DXA) and quantitative ultrasound(QUS) are effective in diagnosing osteoporosis. DXA is recognized as the most suitable measurement method to meet the diagnostic criteria of WHO3). However, these methods have a disadvantage of high price5). On the other hand, dental panoramic radiography is low priced and also the most routinely taken radiograph in the area of dental diagnosis6). According to recent studies it has been reported that the panoramic radiograph is an eligible way to diagnose osteoporosis 7,8,9).

    Artificial intelligence continues to develop. Experiments clarified that artificial intelligence can determine the nodule of the lungs as malignant or benign through CT, and the surgeons are using the automatic systems for low-invasive treatment10,11). Also in the field of diagnostic image reading, artificial intelligence using big data is held outstanding. Recently, a technology called deep learning or convolutional neural network(CNN) has been developed which layers the images given, extracts the characteristics and then reads the images based on the learned data12). AlexNet is a type of CNN, which was first introduced in the thesis by Alex Krizhevsky. After winning at the imageNet Large Scale Visual Recognition Challege in 201213), it has been applied to various fields of research. In the medical field, AlexNet effectively detected pathological changes in the brain on MRI14). AlexNet has been successfully applied in classifying subjects on images in many kinds of industrial fields15-18).

    This study was a part of serial experiments of applying CNN in determining osteoporosis on panoramic radiograph 6,12,19,20). The purpose of this study was to investigate the accuracy of the AlexNet in the diagnosis of osteoporosis and which area the AlexNet reacted sensitively when the region of interest(ROI) was limited to bilateral mandible inferior region.

    Ⅱ. MATERIALS and METHODS

    1. Study subjects

    This study was implemented by the approval of the Chonnam National University institutional review board(CNUDH-2017-014)19). This study was conducted with the female patients who visited the Chonnam National University Dental Hospital from 2008 to 2016. Panoramic radiographs from 1268(mean age 45.2±21.1 yrs) female patients that were clearly determined as normal or osteoporosis were selected as the study subjects. Panoramic radiographs were obtained with Kodak 8000C(Carestream Health Inc,. Rochester, NY).

    2. Interpretation of panoramic radiographs

    Panoramic radiographs were interpreted for determining osteoporosis patients by two oral and maxillofacial radiologists. Every radiograph was assessed with PiViewStar PACS(Infinitt, Seoul, Kodak) and LCD monitors(IF2105MP; WIDE, Seoul, Korea).

    The change of the mandible inferior cortical bone was categorized into three as C1, C2, and C3. C1 is in a normal state and the cortical bone is shaped even and sharp. C2 is referred as a state in which the coritcal bone is eroded mild to moderate, and there is semilunar defects. C3 is a severe state in which the porosity is seen clearly in the cortical bone. Among these, C2 and C3 has been interpreted to be osteoporotic, and even if it was C1, it was interpreted to be osteoporotic in the state which the bone was very thin21). The final interpretations were made by the agreement between the two oral and maxillofacial radiologists, and the patients in whom the interpretation between osteoporosis and normal was not clear or the two oral and maxillofacial radiologists disagreed in the interpretation were excluded from the study subjects19).

    Based on the selection criteria, 633 patients(mean age 72.2 ± 8.5 yrs) were interpreted as osteoporosis and 635 patients(mean age 28.3 ± 7.0 yrs) as normal.

    3. Structure of Multiple-column CNN

    Considering the interpretation of osteoporosis by the oral and maxillofacial radiologists in the limited region of bilateral mandible inferior area, the region of interest(ROI) was set in the size of a 400X200 pixels in the right and left of mandible inferior area. AlexNet was designed to have each line handle each ROI by setting two ROIs as input values. Each line was established by five convolutional layers, and max pooling layer and Relu active layer were located next to each convolutional layer. The fifth convolutional layer was connected with Softmax classifier by the fully connected layer. The result of Softmax output the same maximum probabilities together with normal or osteoporosis. Although studying the separated features of the two ROIs is allowed by the multiple row structures, the final decode has to be carried out by putting together the two ROIs6,11,19)(Fig. 1).

    4. Training and testing of CNN

    Out of a total of 1268 panoramic radiograph, 535 patients( mean age 28.6 ± 7.4 yrs) of the normal group and 533 patients(mean age 72.1 ± 8.7 yrs) of the osteoporosis group were used for training the AlexNet, and the panoramic radiograph of 100(mean age 26.6 ± 4.5 yrs) normal patients and 100(mean age 72.5 ± 7.2 yrs) osteoporosis patients were used for testing on AlexNet.

    5. Analysis of the sensitive area in the CNN

    To figure out the key part that distinguish between normal and osteoporosis, feature maps were designed to indicate which part the five convolutional layers actively reacted. To figure out whether the trained CNN refer to mandible inferior border as the criteria for classification, or if it includes the surrounding structures, a panoramic radiograph was covered with a 120X60 pixel sized black rectangular to observe the sensitivity. The output was checked by typing the panoramic radiograph that was partly covered with a black rectangular. If there was any change in the output, the part was remembered. When it was confirmed that there was no change after checking the output typed in a different part covered, it was opened to its original state. Parts that had changed were marked with a dot by repeating this method20,22).

    Ⅲ. RESULTS

    The panoramic radiograph of 535 patients(mean age 28.6 ± 7.4 yrs) for the normal group and 533 patients(mean age 72.1 ± 8.7 yrs) for the osteoporosis group were used for training AlexNet, and the panoramic radiograph of 100(mean age 26.6 ± 4.5 yrs) normal patients and 100(mean age 72.5 ± 7.2 yrs) osteoporosis patients were used for testing AlexNet. Among the normal group, AlexNet diagnosed 98(98%) as normal, and among the osteoporosis group, 100(100%) were diagnosed as osteoporosis. As a result, the sensitivity was 1, specificity 0.98, and accuracy 99%(Table 1). The right and left of the maximum active area of the mandible inferior area was identified on the feature maps(Fig.2). Visualization of the sensitive area was also conducted. The cortical and cancellous bone of mandible inferior including adjacent soft tissue was marked as the sensitive area(Fig. 3).

    Ⅳ. DISCUSSION

    This study was performed as a part of serial experiments of applying CNN in determining osteoporosis on panoramic radiograph6,12,19,20). This study aimed to investigate the accuracy of the AlexNet in the diagnosis of osteoporosis and which area the AlexNet reacted sensitively when ROI was limited to bilateral mandible inferior region. As a result, sensitivity was 1, specificity 0.98, and accuracy 99%. AlexNet was proved to be sensitive to the cortical and cancellous bone and adjacent soft tissue area by an occluding sensitivity analysis20,22)(Table 1, Fig. 3).

    Previous studies of our team showed the effectiveness of CNN in determining between normal and osteoporosis on panoramic radiograph. In the experiments of single and multple-column CNN with augmentation showed the area under the curve values of 0.9763~0.999119). Two experiments using different age sets with a wider ROI including maxilla and mandible showed error rate of 5.14% and 15.15% and sensitive area on cancellous bone of both jaws12,20). Two experiments using different digital imaging techniques of panoramic radiographs showed accuracy of 76.2% and 92.5%6). These studies showed the importance of study designs, especially the age of each group, number of ROI, anatomic structures included in ROI, and digital imaging techniques.

    The accuracy and sensitive area might be influenced by the ROI in this study. Oral and maxillofacial radiologists diagnose osteoporosis by interpretation at the mandibular inferior cortical bone of the panoramic radiographs. However, when inputting data from panoramic radiograph into artificial intelligence, it was very difficult to crop only mandible inferior cortical bone. The ROI was designed in a rectangular shape on the right and left mandible inferior including adjacent structures19,23-25)(Fig. 1).

    In this study, an occluding sensitivity analysis20,22) showed that cortical and cancellous bone and adjacent soft tissue area were sensitive. In cortical bone, it was assumed that artificial intelligence determined osteoporosis by a similar method as Oral and Maxillofacial radiologists did. Computer is much more precise in reading, while the human eyes have resolution power up to 75μm. It was reasonably acceptable that cancellous bone is sensitively reacted for its higher turnover rate than cortical bone. Though it is not easily accepted that soft tissue area was sensitively reacted, the difference of age of patients in both groups might explain it. The normal group for the testing consisted of 100 patients at the mean age of 26.6 ± 4.5 years, and the osteoporosis group 100 at the mean age of 72.5 ± 7.2 years(Fig 3).

    A study of deep CNN for screening osteoporosis on panoramic radiographs using a single column CNN showed that CNN was sensitive on cortical and cancellous bone and soft tissue. The results were very similar to those of this study26).

    Diagnosis of osteoporosis is approached by bone mass measurement or bone mineral density, which defines a fracture threshold. According to bone mineral density, four general diagnostic categories in women has been established: normal, low bone mass(osteopenia), osteoporosis, and severe osteoporosis(established osteoporosis)2).

    The number of patients with otseoporosis in South Korea increased by about 12% from 805,304 in 2013 to 906,631 in 2017. Most of the patients were diagnosed as osteoporosis after the age of 50, the death rate after femur fracture reached 15~20% within 1 year12).

    The most important requirement for the diagnosis technique of osteoporosis is its performance characteristics for fracture prediction. DXA and QUS are useful in diagnosing osteoporosis. DXA is most common validated technique applied to the hip, spine and forearm3). T score and Z score are used for the diagnosis of osteoporosis in DXA. T score is the value of comparing the maximum difference of bone density between a specific person and a young adult to show the absolute risk of fracture. Z score is the value of comparing the result of a specific person with the normal average of the same sex and the same age, it is also used in children and women before menopausal and men before their 50s2,3,27).

    Although the criteria of WHO2,3) can be applied in DXA, it is highly priced and needs installation space and a skilled engineer. As for QUS, although it is lower priced, it can only be applied to distal bones and is not applied to the criteria of the WHO. Since there are no symptoms of osteoporosis before fracture, the number of osteoporosis patients who are diagnosed by DXA or QUS is very low. However, as panoramic radiograph is routinely taken at dental clinics, it is easier for patients to access than the measuring bone mineral density by DXA. By determining osteoporosis on panoramic radiographs, dentists may be able to help patients to receive a further test for bone mineral density at the medical clinics appropriately.

    Previous researches have reported the usefulness of panoramic radiograph in diagnosis of osteoporosis7-9). A triage screening for osteoporosis and osteopenia by interpreting mandible inferior cortical bone on the panoramic radiograph showed that the sensitivity was up to 0.959). It was also recommended that patients with mandible inferior cortical bone less than 3mm thick take an additional osteoporosis diagnosis8).

    In conclusion, this study was performed as a part of serial experiments of applying convolutional neural network( CNN) in determining osteoporosis on panoramic ra diograph6,12,19,20), using two ROIs of the right and left mandible inferior area. The results showed a high accuracy rate of 99%. The cortical bone of mandible inferior area and adjacent cancellous bone and soft tissue were sensitively reacted on AlexNet.

    Figure

    KAOMP-45-6-189_F1.gif

    Data preparation and convolutional neural network for two ROIs on each panoramic radiograph. A. Right and left side columns were extracted as two ROIs, each 400 X 200 pixels in size. B. Multiple-column CNN for osteoporosis classification on a panoramic radiograph.

    KAOMP-45-6-189_F2.gif

    Feature maps of maximum activation of the fifth convolutional layer for the right and left of the mandible inferior area in the samples. A. Normal patients. B. Osteoporosis patients.

    KAOMP-45-6-189_F3.gif

    Sensitive area was marked as dots in a normal patient by occlusion sensitivity analysis for determination between normal and osteoporosis. A. The right and left of the mandible inferior area. B. The most sensitive area was marked red.

    Table

    Testing results of AlexNet in determining osteoporosis on multiple-column CNN

    Reference

    1. Peck WA: Consensus development conference: Diagnosis, prophylaxis and treatment of osteoporosis. Am J Med 1991;90:107-110.
    2. World Health Organization: Assessment of fracture risk and its application to screening for postmenopausal osteopososis. Report of a WHO Study Group, Geneva, World Health Organization, 1994 WHO Technical Report Series, No. 843.
    3. World Health Organization: WHO scientific group on the assessment of osteoporosis at primary health care level. Summary Meeting Report, Brussels, Belgium, World Health Organization 2007:1-17.
    4. Lee YK, Yoon BH, Koo K: Epidemiology of osteoporosis and osteoporotic fractures in South Korea. Endocrinol Metab 2013;28:90-93.
    5. Park YS: Diagnosis and treatment of osteoporosis. J Korean Med Assoc 2012;55:1083-1094.
    6. Lim YG, Lee EJ, Lee JS, Kang BC, Adhikari SP, Kim HS, Yoon SJ: Differences in osteoporosis readings on dental panoramic radiographs according to convolutional neural network test data. Korean J Oral Maxillofac Pathol 2019;43:103-109.
    7. Okabe S, Morimoto Y, Ansai T, Yoshioka I, Tanaka T, Taguchi A, Kito S, Wakasugi-Sato N, Oda M, Kuroiwa H, Ohba, Awano TS, Takata Y, Takehara T: Assessment of the relationship between the mandibular cortex on panoramic radiographs and the risk of bone fracture and vascular disease in 80-year-olds. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2008;106:433-442.
    8. Devlin H, Karayianni K, Mitsea A, Jacobs R, Lindh C, Stelt P, Marjanovic E, Adams J, Pavitt S, Horner K: Diagnosing osteoporosis by using dental panoramic radiographs: The OSTEODENT project. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2007;104:821-828.
    9. Taguchi A: Triage screening for osteoporosis in dental clinics using panoramic radiographs. Oral Dis 2010;16:316-327.
    10. Schwab K: The Fourth Industrial Revolution. Seoul, Korea: Newprint, 2017.
    11. Hua KL, Hsu CH, Hidayati SC, Cheng WH, Chen YJ: Computer-aided classification of lung nodules on computed tomography images via deep learning technique. Onco Targets Ther 2015;8:2015-2022.
    12. Kim JY, Lee JS, Kang BC, Kim HS: Effect of training and testing condition of convolutional neural network on evaluating osteoporosis. Korean J Oral Maxillofac Pathol 2019;43:73-80.
    13. Krizhevsky A, Sutskever I, Hinton GE: ImageNet classification with deep convolutional neural networks. Commun ACM 2017;60:84-90.
    14. Lu S, Lu Z, Zhang YD: Pathological brain detection based on AlexNet and transfer learning. J Comput Sci 2019;30:41-47.
    15. Park JH, Hwang KB, Park HM, Choi YK: Application of CNN for fish species classification. J Korean Inst Inf Commun Eng 2019;23:39-46.
    16. Azida Muhammad N, Ab Nasir A, Ibrahim Z, Sabri N: Evaluation of CNN, Alexnet and GoogleNet for fruit recognition. Indones J Electr Eng Comput Sci 2018;12:468-475.
    17. Lv M, Zhou G, He M, Chen A, Zhang W, Hu Y: Maize leaf disease identification based on feature enhancement and DMS-robust alexnet. IEEE Access 2020;8:57952-57966.
    18. Wang Y, Lv Y, Guo D, Zhang S, Jiao S: A novel multi-input AlexNet prediction model for oil and gas production. Math Probl in Eng 2018;1:1-9.
    19. Lee JS, Adhikari S, Liu L, Jeong HG, Kim HS, Yoon SJ: Osteoporosis detection in panoramic radiographs using a deep convolutional neural network-based computer-assisted diagnosis system: A preliminary study. Dentomaxillofac Radiol 2019;48:20170344.
    20. Lee SA, Park BJ, Kim H, Adhikari S, Lee EJ, Lee JS, Kang BC, Yoon SJ: Sensitive Area of Artificial Intelligence in Interpreting Osteoporosis on Panoramic Radiograph. J Oral Maxillofac Pathol 2021;45:59-65.
    21. Klemetti E, Kolmakov S, Kröger H: Pantomography in assessment of the osteoporosis risk group. Scand J Dent Res 1994;102:68-72.
    22. Zeiler MD, Fergus R: Visualization and understanding convolutional networks. In: Fleet D, Pajdla T, Schiele B, Tuytelaa T, editors. Computer Vision-ECCV, Springer, 2014:818-833.
    23. Kavitha MS, Asano A, Taguchi A, Kurita T, Sanada M: Diagnosis of osteoporosis from dental panoramic radiographs using the support vector machine method in a computer- aided system. BMC Med Imaging 2012;12:1471-2342.
    24. Arifin AZ, Asano A, Taguchi A, Nakamoto T, Ohtsuka M, Tsuda M, Kudo Y, Tanimoto K: Use of fuzzy neural network in diagnosing postmenopausal women with osteoporosis based on dental panoramic radiographs. J Adv Comput Intell Intell Inform 2007;11:1049-1058.
    25. Kavitha MS, Ganesh Kumar P, Park SY, Huh KH, Heo MS, Kurita T, Asano A, An SY, Chien SI: Automatic detection of osteoporosis based on hybrid genetic swarm fuzzy classifier approaches. Dentomaxillofac Radiol 2016;45:20160076.
    26. Lee KS, Jung SK, Ryu JJ, Shin SW, Choi JW: Evaluation of transfer learning with deep convolutional neural networks for screening osteoporosis in dental panoramic radiographs. J Clin Med 2020;9:392.
    27. Kim JG, Moon YW: Diagnosis of Osteoporosis. J Korean Hip Soc 2011;23:108-115.
    Copyright © Korean Association of Oral and Maxillofacial Pathology. All rights reserved.
    오늘하루 팝업창 안보기 닫기