Fidelity, Robustness, and Plausibility of Post-Hoc Explanations for Diabetic Retinopathy Grading: An Empirical Application of the Clinical-Explainable Artificial Intelligence Evaluation Framework

Abstract

Background: A recent critical review proposed a six-dimension Clinical-explainable artificial intelligence (XAI) Evaluation Framework (fidelity, robustness, plausibility, actionability, equity, and regulatory readiness) for judging whether XAI methods are clinically adequate, but the framework itself was not empirically tested. This study examined whether three of these dimensions can be measured from real post-hoc explanations of a diabetic retinopathy (DR) grading model, how Gradient-weighted Class Activation Mapping (Grad-CAM), Local Interpretable Model-agnostic Explanations (LIME), and SHapley Additive exPlanations (SHAP) compare under a matched protocol, and whether the comparison holds across training runs and architectures.

Methods: Compact convolutional neural networks (CNNs) were trained from scratch with five random seeds, and a ResNet-18 comparator with three seeds, on RetinaMNIST (1,600 fundus images at 64 × 64 pixels, five severity grades). Grad-CAM, LIME, and SHAP were applied to the same 40 test images (eight per grade) with the same three Gaussian noise realizations (standard deviation (SD): 0.05 on a 0–1 intensity scale) and fixed explainer seeds. Fidelity was measured with deletion/insertion curves compared against random and center-prior pixel rankings, together with model-randomization sanity checks for all three methods. Robustness was the Spearman correlation between clean and noisy explanation maps, at native resolution and after pooling every map to 4 × 4. A background-attention check compared each map’s share of attribution inside the fundus disk with that image’s own disk-area share. Results are given with 95% bootstrap confidence intervals (CIs).

Results: The primary classifier reached 52.0% accuracy and a quadratic weighted kappa (QWK) of 0.624 (five-seed mean 50.0±3.0% and 0.582 ± 0.046). Fidelity exceeded the random-ranking baseline for Grad-CAM (paired difference 0.045, 95% CI 0.033 to 0.058) and SHAP (0.038, 0.028 to 0.050) but not for LIME (0.003, −0.004 to 0.010). At native0 resolution, Grad-CAM explanations were the most stable under noise (Spearman ρ 0.786), followed by SHAP (0.254) and LIME (0.097); after pooling to 4 × 4, SHAP (0.739) was comparable to Grad-CAM (0.767), whereas LIME remained unstable (0.171). The noise changed the predicted class in 53% of perturbed images. Grad-CAM passed the model-randomization check (ρ −0.120); LIME (0.335) and SHAP (0.386) retained partial similarity to explanations of a randomized model. All three methods placed more attribution inside the fundus disk than expected from its area. These findings were consistent across two CNN training seeds; see Section “Consistency across seeds and architectures” for the ResNet-18 comparator.

Conclusion: Fidelity, robustness, and a background-attention check can be measured directly from a trained model, but their interpretation depends on controls: a random-ranking baseline showed that LIME’s fidelity was indistinguishable from chance, and a resolution-matched analysis showed that most of Grad-CAM’s robustness advantage over SHAP reflects its coarse spatial resolution. Because small input noise often changed the model’s prediction, explanation stability cannot be interpreted separately from classifier reliability. The background-attention check is not a measure of clinical plausibility, which requires lesion-level annotation or ophthalmologist review. Actionability, equity, and regulatory readiness could not be assessed with this benchmark and require deployment data, subgroup metadata, or documentation review.