We evaluate commercial AI products using the TB REACH-CXR Archive, a de-identified evaluation database of 30,957 postero-anterior chest radiographs collected through 4 projects funded by the Stop TB Partnership’s TB REACH initiative between 2015-2020. These are privately held for the purposes of independent and external validation of commercial CAD products. These projects and analyses were funded by Global Affairs Canada.
Contact us to be evaluated using the Stop TB chest X-ray archive: email@example.com
Tuberculosis detection from chest x-rays for triaging in a high tuberculosis-burden setting: an evaluation of five artificial intelligence algorithms
We evaluated five commercially available AI algorithms for TB triage using 23,954 chest X-rays collected at TB screening centers in Bangladesh and Xpert MTB/RIF as the reference standard. All CAD outperformed experienced Bangladeshi radiologists and could halve the number of Xpert tests required while sensitivity remained greater than 90%. All algorithms performed worse in older age groups and those with a history of TB. Finally, we proposed a new evaluation framework for CAD to better inform selection of threshold scores.
This evaluation showed that CAD can be a highly accurate triage tool for TB.
A new resource on artificial intelligence powered computer automated detection software products for tuberculosis programmes and implementers
We conducted a landscape analysis to collect information from developers known to have, or soon to have, a CAD product for TB. We identified 27 CAD developers and 11 completed our survey with details about the certification, deployment, operational characteristics, input requirements, output format, pricing, and data privacy of their latest product version. For each response, a summary product profile was created based on the information provided and these were published on an open-access website: ai4hlth.org.
CAD products are constantly being improved and the site will continuously be updated to account for updates and new products. This unique online resource aims to inform the TB community about available CAD tools and enable TB programmes to identify the most suitable product to incorporate in interventions.
Using artificial intelligence to read chest radiographs for tuberculosis detection: A multi-site evaluation of the diagnostic accuracy of three deep learning systems
We conducted a retrospective evaluation of three CAD systems (CAD4TB, Lunit INSIGHT, and qXR) for detecting TB-associated abnormalities in chest radiographs from outpatients in Nepal and Cameroon. All 1196 individuals received a Xpert MTB/RIF assay and a CXR read by two groups of radiologists and CAD. Xpert was used as the reference standard. When matching the sensitivity of the radiologists, the specificities of the DL systems were significantly higher except for one.
Using CAD systems to read CXRs could reduce the number of Xpert MTB/RIF tests needed by 66% while maintaining sensitivity at 95% or better. CAD should be considered by TB programs where human resources are constrained, and automated technology is available.