MedExpMem: Adapting Experience Memory for Differential Diagnosis

Background

The article discusses a novel framework, MedExpMem, aimed at enhancing the diagnostic abilities of medical vision-language models (VLMs). Currently, VLMs lack the capability to differentiate between similar conditions as experienced physicians do due to their static nature. This limitation is addressed by introducing an experience memory framework that allows VLM-based diagnostic agents to learn from their own diagnostic errors and failures.

Key Points

MedExpMem operates on two main phases:

1. Initial Practice: In this phase, the agent encounters clinical cases where it makes mistakes or uncertain diagnoses.
2. Reflective Re-diagnosis: Here, the agent revisits its past misdiagnoses to refine understanding and learn key discriminators.

The framework constructs pairwise differential notes from these experiences, encoding critical information such as discriminative features, actionable decision rules, and reasoning error patterns. These notes are then used during future diagnosis by retrieving relevant memories for guiding differential reasoning.

Significance

MedExpMem's Impact on VLMs:

• Improves Accuracy: Evaluations show consistent accuracy improvements across different models, with the highest gain being 7.0%.
• Competitive Method: MedExpMem outperforms retrieval-augmented generation approaches by focusing on personalized learning through reflective re-diagnosis.
• Adaptability Beyond Parametric Learning: The method addresses complex medical needs that traditional parameteric learning cannot fully cover, making it a valuable addition to current VLMs.

Experience Memory Construction:

• Quality and Robustness: Analytical experiments validate the quality of experience stored in MedExpMem, ensuring its robustness.
• Diverse Models: The framework’s effectiveness is demonstrated across multiple scales and types of models, showcasing its broad applicability.

Conclusion

MedExpMem represents a significant advancement in training VLMs for medical diagnostics by leveraging experiential learning. Its ability to learn from mistakes and improve through reflective re-diagnosis positions it as a promising tool for enhancing the diagnostic capabilities of AI systems used in healthcare settings.