06 Oct Evaluating deep learning approaches for AI-assisted lung ultrasound diagnosis: an international multi-center and multi-scanner study
Evaluating deep learning approaches for AI-assisted lung ultrasound diagnosis: an international multi-center and multi-scanner study
6 October 2025
Mario Muñoz, Xi Han, Jorge Camacho, Tiziano Perrone, Andrea Smargiassi, Riccardo Inchingolo, Yale Tung-Chen & Libertario Demi
Abstract
Lung ultrasound (LUS) interpretation is often subjective and operator-dependent, motivating the development of automated, artificial intelligence (AI)-based methods.
This international, multi-center study evaluated two distinct deep learning approaches for automated LUS severity scoring for pulmonary infections caused by COVID-19: a pre-trained classification model (CM) and a segmentation model based method (SM); assessing performance at video, exam, and prognostic levels.
Two datasets were analyzed: one comprising data from multiple scanners and another using data from a single scanner. Results showed that the SM achieved prognostic-level agreement with expert clinicians comparable to that of the CM. Furthermore, at the exam level, over 84% of examinations were classified with acceptable error (≤ 10 score difference) across both models and datasets, reaching both methods an agreement higher than 95% on the dataset acquired by a single scanner.
The obtained results demonstrate the potential of AI-assisted LUS for reliable prognostic assessment and highlight that image quality and acquisition technique are key factors in achieving consistent and generalizable model performance, as well as the potential for international clinical translations.
Materials and methods
This international multi-center and multi-scanner study was conducted as a collaborative effort between the Ultrasonic Systems and Technologies Group (GSTU) at the Spanish National Research Council (CSIC) in Madrid, Spain, and the ULTRa Lab group at the University of Trento in Trento, Italy.
The study involved the evaluation of two AI methods on a diverse dataset of LUS images and videos acquired from multiple centers.
The details of the study design, data acquisition, AI models, and evaluation methods are described in the following subsections.
Results
Figures 4 and 5 show the performance of the CM and SM methods at the video, examination, and prognostic levels for different threshold values in Dataset-1 and Dataset-2, respectively. The x-axis represents the threshold and the y-axis represents the accuracy, defined as the proportion of correctly classified videos or examinations. In both cases, a range of thresholds from 1 to 100% of the frames was tested. Table 2 summarize the results on both datasets.
Conclusions
In conclusion, this research provides strong evidence for the clinical utility of AI-powered LUS analysis, particularly for prognostic assessment, while simultaneously highlighting the importance of data quality and standardization for achieving reliable and generalizable results.
The similar performance between the segmentation-based method and the pre-trained classification model highlights the ability of both approaches to generalize effectively to patients and scanners excluded from the training phase.
The successful completion of this study, involving the analysis of multi-center data from diverse clinical settings, underscores the crucial role of international and multidisciplinary collaborations in advancing the field of AI for LUS. Such collaborations are essential not only for accessing diverse datasets, but also for promoting the exchange of expertise necessary to develop and validate AI methods that can truly benefit patients, contributing to earlier and more accurate diagnoses.
Sorry, the comment form is closed at this time.