Introduction to explainable deep learning in health care

Key takeaways:

• EHR integration has facilitated AI and deep learning applications in health care.

• Deep learning’s “opaque-box” nature complicates critical health care decision-making.

• Explainable AI enhances trust, transparency, and ethicality in AI decisions.

• Explainability allows stakeholders to understand AI operations, fostering patient trust.

• Lack of interpretability can lead to mistrust and ethical issues.

• Explainable models clarify decision-making for health care workers and patients.

• Techniques like saliency maps and rule extraction improve model interpretability.

• Trust between patients and clinicians is crucial for AI integration in health care.

• Addressing training data biases is essential for AI accuracy and fairness.

• Clear explanations support patient-centric care and informed health decisions.

Implementing electronic health care records (EHRs) in health care organizations makes required health care data much easier, making the application of artificial intelligence techniques in health care much more feasible.

As noted, there are many benefits of using deep learning in health care-related decision making; however, there are still huge barriers to its implementation. A major challenge, however, is that deep learning remains an opaque-box on how it derives the information and links it to its output. In severe disease diagnoses, probable readmission risk, or any application where the chance of misclassifying the patient is costly, hesitancy is experienced. Further, deep learning models are sensitive to both artificial and natural noise, making it even trickier to deploy them.

Another difficulty arises when using the so-called deep learning models, which are very difficult to explain due to their high degree of depth. To this end, explainability methods tackle this challenge by making these decisions understandable, building trust, avoiding risks, and enhancing ethicality. All such explanations are of different types, such as global, local, model-agnostic, and model-specific. One of the feature importance techniques is permutation feature importance, which is global and does not depend on the model: it determines the importance of the inputs in terms of the output. One basic requirement for moving these machine learning models into clinic decision support systems is the ability of the model to explain its decision.

Explainability in health care

While in the health care context, the explication refers to the ability of the AI system to provide a clear and easily understandable rationale for the operations and outcomes to all the stakeholders in the health sector. As simply as that, it encompasses the sense that the AI system makes the involved decision-making process understandable, in its bid to offer a particular diagnosis or treatment recommendation. In health care, explainability is important because patients are at the center and should be able to make an informed decision regarding their health. In addition, it has a significant role in developing trust from patients and clinicians, which is vital for implementing the AI-driven tools into the clinical setting. However, it’s crucial to acknowledge that explainability transcends mere technological aspects, giving rise to many medical, legal, ethical, and societal considerations that demand thorough exploration.

Lack of explainability in health care

One implication of not being able to interpret deep learning models is that it is detrimental to health care. It can result in model mistrust because the end user does not trust the predictions made by the particular model. This can be particularly disadvantageous in business organizations that deal primarily in the health care sector because wrong decision-making might lead to negative impacts that can even harm people’s lives.

Interpretability difficulties can also cause ethical problems; hence, explainable AI methods that enhance transparency and interpretability of deep learning models in health care settings are needed. These methods can enhance the confidence and reliability of the model and guarantee the correctness and ethicality of the model’s decisions.

Target audience of explainability

Explainable deep learning models

In health care, explainable deep learning models are machine learning models that can explain the reasons for decisions made by health care workers and patients. These models are less opaque-box in nature, and clinicians can explain why a model has arrived at a certain diagnosis or a treatment plan. This is important because engagement of the patient with the clinician enhances the credibility of the tools driven by artificial intelligence in clinical practice.

There are several methods for enhancing the interpretability of deep learning models in health care. These methods include:

• Saliency maps: Highlighting the most important features of an image that contributed to the model’s decision.

• Attention mechanisms: Focusing on the most relevant parts of the input data.

• Decision trees: Providing a step-by-step explanation of the model’s decision-making process.

• Rule extraction: Extracting human-readable rules from the model’s decision-making process.

These approaches aid physicians in gaining a deeper understanding of data-driven technologies and in assessing the model’s decision-making process. This evaluation allows them to determine whether to accept or reject predictions and recommendations before implementing any actions. Nevertheless, it is crucial to emphasize that explainability extends beyond a technological concern; it brings forth a range of medical, legal, ethical, and societal questions that necessitate thorough exploration.

Importance of explainable deep learning models

• Facilitating the establishment of trust between patients and clinicians is crucial for the successful integration of AI-driven tools into clinical practice.

• In health care, where incorrect diagnoses or treatment recommendations can have severe consequences, these models assist physicians in comprehending data-driven technologies. This aids in their ability to assess predictions and recommendations before implementing any actions.

• Detecting biases in the data used for model training is crucial for improving accuracy and fairness. As AI models rely on the quality of their training data, addressing biases in the data helps mitigate the risk of biased predictions and recommendations.

• Ensuring patient-centric care and informed decision-making is supported by providing clear and understandable explanations of the model’s decision-making process. This empowers patients to comprehend the rationale behind specific diagnoses or treatment recommendations, enabling them to make well-informed decisions about their health.

Methods to increase the interpretability of deep learning models

Some of the methods are:

• Permutation feature importance (PFI): A global method that measures the importance of each feature by permuting the values of that feature and measuring the decrease in model performance.

• Local interpretable model-agnostic explanations (LIME): A local method that explains the predictions of any classifier by approximating it with an interpretable model.

• SHapley Additive exPlanation (SHAP): A global method that assigns each feature an importance value for a particular prediction by computing the contribution of each feature to the difference between the prediction and the expected value.

• Class-activation mapping (CAM): A model-specific method that generates a heatmap of the input image to highlight the important regions for the classification decision.

• Gradient-weighted CAM: An extension of CAM that uses the gradient of the output class score concerning the feature maps to weigh each feature’s importance.

These methods are just a few examples of the many techniques developed to enhance the interpretability of deep learning models in health care. The choice of method depends on the specific problem and the type of model being used.

Taxonomy

Taxonomy for explainable deep learning models typically involves categorizing methods based on their approaches to explaining the decisions made by deep learning models.

Local vs. global explanations

• Local explanations focus on explaining a specific prediction made by the model. They help to understand how the model arrived at a particular decision for a given input. For instance, if a deep learning model is used to diagnose a patient, a local explanation would help to understand how the model arrived at its diagnosis for that particular patient.

• Global explanations provide a holistic view of the main factors influencing the model’s predictions. They help to understand how the model behaves in general. For example, an overview of the model’s feature importance is a global explanation.

Model agnostic vs. model-specific explanations

• Model agnostic explanations are techniques to explain machine learning model behavior without relying on internal workings. It can be used with any machine learning model, including random forests and deep neural networks. It is not restricted to a specific form of explanation; it supports various feature representations. SHAPley values and LIME are popular model-agnostic techniques.

• Model-specific explanations are tailored explanations for a specific machine learning model. They are designed for a particular model architecture, e.g., a specific convolutional network (CNN). It exploits the internal workings of the model, using a reverse engineering approach. It provides a deeper understanding of decisions by leveraging knowledge of the model’s internals, allowing for customization of explainable models. It has a drawback as it requires a thorough understanding of the model’s structure, potentially compromising performance due to model recreation.

Conclusion

Utilizing explainable deep learning models represents a promising strategy to improve the interpretability of machine learning models within the health care domain. These models facilitate a clearer understanding for physicians regarding data-driven technologies, aiding them in making well-informed decisions. A recent survey focused on the interpretability of deep learning models in health care serves as a comprehensive methodological guide, beneficial for future researchers and clinical practitioners.

Quiz

Test your knowledge from the quiz below: