Performance of Qure.ai automatic classifiers against a large annotated database of patients with diverse forms of tuberculosis
Autoři:
Eric Engle aff001; Andrei Gabrielian aff001; Alyssa Long aff001; Darrell E. Hurt aff001; Alex Rosenthal aff001
Působiště autorů:
Office of Cyber Infrastructure & Computational Biology, National Institute of Allergy and Infectious Disease, National Institutes of Health, Bethesda, MD, United States of America
aff001
Vyšlo v časopise:
PLoS ONE 15(1)
Kategorie:
Research Article
doi:
https://doi.org/10.1371/journal.pone.0224445
Souhrn
Availability of trained radiologists for fast processing of CXRs in regions burdened with tuberculosis always has been a challenge, affecting both timely diagnosis and patient monitoring. The paucity of annotated images of lungs of TB patients hampers attempts to apply data-oriented algorithms for research and clinical practices. The TB Portals Program database (TBPP, https://TBPortals.niaid.nih.gov) is a global collaboration curating a large collection of the most dangerous, hard-to-cure drug-resistant tuberculosis (DR-TB) patient cases. TBPP, with 1,179 (83%) DR-TB patient cases, is a unique collection that is well positioned as a testing ground for deep learning classifiers. As of January 2019, the TBPP database contains 1,538 CXRs, of which 346 (22.5%) are annotated by a radiologist and 104 (6.7%) by a pulmonologist–leaving 1,088 (70.7%) CXRs without annotations. The Qure.ai qXR artificial intelligence automated CXR interpretation tool, was blind-tested on the 346 radiologist-annotated CXRs from the TBPP database. Qure.ai qXR CXR predictions for cavity, nodule, pleural effusion, hilar lymphadenopathy was successfully matching human expert annotations. In addition, we tested the 12 Qure.ai classifiers to find whether they correlate with treatment success (information provided by treating physicians). Ten descriptors were found as significant: abnormal CXR (p = 0.0005), pleural effusion (p = 0.048), nodule (p = 0.0004), hilar lymphadenopathy (p = 0.0038), cavity (p = 0.0002), opacity (p = 0.0006), atelectasis (p = 0.0074), consolidation (p = 0.0004), indicator of TB disease (p = < .0001), and fibrosis (p = < .0001). We conclude that applying fully automated Qure.ai CXR analysis tool is useful for fast, accurate, uniform, large-scale CXR annotation assistance, as it performed well even for DR-TB cases that were not used for initial training. Testing artificial intelligence algorithms (encapsulating both machine learning and deep learning classifiers) on diverse data collections, such as TBPP, is critically important toward progressing to clinically adopted automatic assistants for medical data analysis.
Klíčová slova:
Deep learning – Extensively drug-resistant tuberculosis – Forecasting – Pleural effusion – Pulmonary imaging – Radiologists – Tuberculosis – Tuberculosis diagnosis and management
Zdroje
1. WHO Global tuberculosis report 2018, World Health Organization. http://www.who.int/tb/publications/global_report/en/
2. WHO World Health Organization. (2016). Chest radiography in tuberculosis detection: summary of current WHO recommendations and guidance on programmatic approaches. World Health Organization. http://www.who.int/iris/handle/10665/252424
3. WHO World Health Organization. (2013). Systematic screening for active tuberculosis: principles and recommendations. World Health Organization. http://www.who.int/iris/handle/10665/84971
4. Steingart KR, Henry M, Ng V, et al. Fluorescence versus conventional sputum smear microscopy for tuberculosis: a systematic review. Lancet Infect Dis 2006; 6: 570–81. doi: 10.1016/S1473-3099(06)70578-3 16931408
5. Dorman S E, Schumacher S G, Alland D, et al. Xpert MTB/RIF Ultra for detection of Mycobacterium tuberculosis and rifampicin resistance: a prospective multicentre diagnostic accuracy study. Lancet Infect Dis 2017; published online Nov 30. http://dx.doi.org/10.1016/S1473-3099(17)30691-6.
6. Folio Les R., Chest Imaging: An Algorithmic Approach to Learning, New York, Springer, 2012.
7. Pedrazzoli D, Lalli M, Boccia D, Houben R, Kranzer K. Can tuberculosis patients in resource-constrained settings afford chest radiography? European Respiratory Journal 2016; doi: 10.1183/13993003.01877–2016
8. ESR European Society of Radiology (ESR), and American College of Radiology (ACR). “European Society of Radiology (ESR) and American College of Radiology (ACR) Report of the 2015 Global Summit on Radiological Quality and Safety.” Insights into Imaging 7.4 (2016): 481–484. PMC. Web. 30 Aug. 2018.
9. Diagnostic Imaging, Radiologist sightings drop around the world, Jul. 14, 2003, http://www.diagnosticimaging.com/article/radiologist-sightings-drop-around-world
10. Henostroza German et al. “Chest Radiograph Reading and Recording System: Evaluation in Frontline Clinicians in Zambia.” BMC Infectious Diseases 16 (2016): 136. PMC. Web. 30 Aug. 2018. doi: 10.1186/s12879-016-1460-z 27005684
11. DeVries G. “Outreach programme of tuberculosis screening and strengthening care integration in Romania” In: 49th Union World Conference Abstract Book, October 2018, The Hague.
12. Zaidi S., Adam A., Azeemi KS., Habib SS., Madhani F., Safdar N., et al. “Identification of potential TB hot-spots through a mobile X-ray supported community-based mass screening program in Karachi, Pakistan” In: 49th Union World Conference Abstract Book, October 2018, The Hague.
13. Lovelace Jr. B. GE’s health unit wins first FDA clearance for A.I.-powered X-ray system. https://www.cnbc.com/2019/09/12/ges-health-unit-wins-first-fda-clearance-for-ai-powered-x-ray-system.html
14. Jaeger S, Karargyris A, Candemir S, Folio L, Siegelman J, Callaghan F, et al.: Automatic Tuberculosis Screening Using Chest Radiographs. IEEE Transactions on Medical Imaging 33:233–245. (2014). doi: 10.1109/TMI.2013.2284099 24108713
15. Heo SJ, Kim Y, Yun S, Lim SS, Kim J, Nam CM, et al. Deep Learning Algorithms with Demographic Information Help to Detect Tuberculosis in Chest Radiographs in Annual Workers' Health Examination Data, Int J Environ Res Public Health. 2019
16. Lakhani P, Sundaram B, Deep Learning at Chest Radiography: Automated Classification of Pulmonary Tuberculosis by Using Convolutional Neural Networks. Radiology. 2017
17. Hwang EJ, Park S, Jin KN, Kim JI, Choi SY, Lee JH, “Development and Validation of a Deep Learning-Based Automatic Detection Algorithm for Active Pulmonary Tuberculosis on Chest Radiographs.” Clin Infect Dis. 2018
18. Sivaramakrishnan R, Antani S, Candemir S, Xue Z, Abuya J, Kohli M, et al.”Comparing deep learning models for population screening using chest radiography.” Medical Imaging 2018: Computer-Aided Diagnosis. Vol. 10575. International Society for Optics and Photonics, 2018.
19. Dunnmon JA, Yi D, Langlotz CP, Ré C, Rubin DL, Lungren MP. Assessment of Convolutional Neural Networks for Automated Classification of Chest Radiographs. Radiology 2019 290:2, 537–544 doi: 10.1148/radiol.2018181422 30422093
20. Putha P., Tadepalli M., Jain S., Chiramal J., Nimmada T.R., Warier P. “Efficacy of deep learning for screening pulmonary tuberculosis” In: ESR 2018 Book of Abstracts, Insights Imaging (2018) 9(Suppl 1): 1. https://doi.org/10.1007/s13244-018-0603-8, March 2018, Vienna, Austria.
21. Putha P., Tadepalli M., Reddy B., Raj T., Chiramal J., Govil S., et al. “Can Artificial Intelligence Reliably Report Chest X-Rays?: Radiologist Validation of an Algorithm trained on 1.2 Million X-Rays” arXiv:1807.07455, July 2018.
22. Ridley E., “Is seeing believing in imaging artificial intelligence?”, Available at: https://www.auntminnie.com/index.aspx?sec=sup&sub=aic&pag=dis&ItemID=119347, Accessed January 18, 2019.
23. Singh R, Kalra MK, Nitiwarangkul C, Patti JA, Homayounieh F, et al. (2018) Deep learning in chest radiography: Detection of findings and presence of change. PLOS ONE 13(10): e0204155. doi: 10.1371/journal.pone.0204155 30286097
24. Irvin J., Rajpurkar P., Ko M., Yu Y., Ciurea-Ilcus S., Churt C., et al. “Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison” arXiv:1901.07031, 2019.
25. Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM. ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases. IEEE CVPR 2017.
26. Rajpurkar P, Irvin J, Ball RL, Zhu K, Yang B1 Mehta H, et al. Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists. PLoS Med. 2018.
27. Rosenthal A, Gabrielian A, Engle E, Hurt DE, Alexandru S, Crudu V, et al. “The TB Portals: An open-access, web-based platform for global drug-resistant tuberculosis data sharing and analysis”, J Clin Microbiol. 2017 Sep 13.
28. Ravimohan S., Kornfeld H., Weissman D., Bisson G.P. Tuberculosis and lung damage: from epidemiology to pathophysiology, European Respiratory Review 2018
29. Liauchuk V, Kovalev V, Kalinovsky A, Tarasau A, Gabrielian A, Rosenthal A. Examining the ability of convolutional neural networks to detect lesions in lung CT images, Proceedings of International Congress on Computer Assisted Radiology and Surgery. 2017.
30. Roman, V. How to Develop a Machine Learning Model from Scratch, Towards Data Science, https://towardsdatascience.com/machine-learning-general-process-8f1b510bd8af
31. Pasa F, Golkov V, Pfeiffer F, Cremers D, Pfeiffer D. Efficient Deep Network Architectures for Fast Chest X-Ray Tuberculosis Screening and Visualization. Sci Rep. 2019
32. Topalovic M, Das N, Burgel PR, Daenen M, Derom E, Haenebalcke C, Janssen R, Artificial intelligence outperforms pulmonologists in the interpretation of pulmonary function tests, Eur Respir J. 2019
33. Nielsen M., Neural Networks and Deep Learning, Chap 4, http://neuralnetworksanddeeplearning.com/chap4.html
34. The Economist, “Why Uber’s self-driving car killed a pedestrian” https://www.economist.com/the-economist-explains/2018/05/29/why-ubers-self-driving-car-killed-a-pedestrian
35. WHO Tuberculosis country profiles (2017), https://www.who.int/tb/country/data/profiles/en/
36. NIH Data Science Home / About BD2K, https://datascience.nih.gov/bd2k/about
37. Wilkinson MD, Dumontier M, Aalbersberg IJ, Appleton G, Axton M, Baak A, et al. 2016. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data 3:160018. doi: 10.1038/sdata.2016.18 26978244
Článek vyšel v časopise
PLOS One
2020 Číslo 1
- S diagnostikou Parkinsonovy nemoci může nově pomoci AI nástroj pro hodnocení mrkacího reflexu
- Proč při poslechu některé muziky prostě musíme tančit?
- Je libo čepici místo mozkového implantátu?
- Chůze do schodů pomáhá prodloužit život a vyhnout se srdečním chorobám
- Pomůže v budoucnu s triáží na pohotovostech umělá inteligence?
Nejčtenější v tomto čísle
- Severity of misophonia symptoms is associated with worse cognitive control when exposed to misophonia trigger sounds
- Chemical analysis of snus products from the United States and northern Europe
- Calcium dobesilate reduces VEGF signaling by interfering with heparan sulfate binding site and protects from vascular complications in diabetic mice
- Effect of Lactobacillus acidophilus D2/CSL (CECT 4529) supplementation in drinking water on chicken crop and caeca microbiome
Zvyšte si kvalifikaci online z pohodlí domova
Všechny kurzy