Graph-Constrained Policy Learning for Extreme Clinical Code Prediction 用于极端临床代码预测的图约束策略学习

TL;DR

• The paper introduces Graph-Constrained Policy Learning, reformulating ICD-10-CM code prediction as a finite-horizon decision process over a pruned hierarchical graph rather than flat multi-label classification.
• The proposed SFT-1+ model achieves superior performance on MIMIC-IV discharge summaries, improving micro-F1 by 0.044 and macro-F1 by 0.157 over the strongest flat baselines like CAML and LAAT.
• A single shared policy effectively matches the accuracy of a three-specialist cascade system while avoiding context-window overflow issues in 28-32% of test cases.
• Empirical analysis reveals that increasing supervised trajectory data is the primary driver of performance gains, whereas GRPO reinforcement learning offers no advantage over supervised fine-tuning with matched data.

Why It Matters

This approach addresses the critical bottleneck of rare code prediction in extreme multi-label classification tasks common in healthcare NLP. By leveraging the inherent structure of medical coding hierarchies, it provides a more robust and scalable alternative to independent scoring methods, which often fail to capture dependencies between related diagnoses.

Technical Details

• Methodology: The system uses a single language model to traverse a pruned ICD-10-CM hierarchy level-by-level, selecting valid child nodes until billable leaf codes are reached, ensuring structurally valid outputs.
• Benchmarking: Evaluated on MIMIC-IV discharge summaries against flat baselines (CAML, LAAT, PLM-ICD) across a curated 50-code subset and the full 15,761-code space.
• Performance Metrics: The best supervised policy (SFT-1+) achieved 0.709 micro-F1 on the subset and 0.527 micro-F1 on the full space, demonstrating significant improvements in macro-F1 for rare labels.
• Ablation Study: A factorial study compared architectures and training algorithms, confirming that supervised data volume is the key factor for improvement, while RL methods like GRPO did not enhance results beyond supervised continuation.

Industry Insight

Healthcare AI developers should prioritize hierarchical constraint integration over flat classification heads when dealing with large, sparse label spaces to improve rare event detection. Investing in high-quality supervised trajectory data yields higher returns than complex reinforcement learning strategies for this specific task. This method offers a computationally efficient alternative to cascaded models, reducing infrastructure costs associated with context-window management.

TL;DR

• 提出图约束策略学习框架，将ICD编码预测转化为在剪枝代码层级上的有限视界决策过程，解决极端多标签分类中的稀疏性问题。
• 单一语言模型通过逐层选择有效子节点生成结构合法的账单代码，显著优于CAML、LAAT等扁平化基线模型。
• 在MIMIC-IV数据集上，SFT-1+模型在全量15,761个代码空间中达到0.527 micro-F1，并避免了级联模型在部分长文本上的上下文溢出问题。
• 实验表明增加监督轨迹数据是提升性能的关键，而GRPO强化学习在此任务中相比监督微调未带来额外收益。

为什么值得看

本文针对医疗自然语言处理中极具挑战性的极端多标签分类问题，提供了一种结合知识图谱结构与强化学习思想的创新解决方案。其结果证明了简单的图约束策略学习在处理深层层级结构和罕见标签时，比复杂的级联或纯强化学习方法更有效，为医疗AI系统的落地提供了重要的技术参考。

技术解析

• 核心架构：采用图约束遍历策略（Graph-Constrained Traversal Policy），将ICD-10-CM代码层级视为有向无环图。模型作为单一策略网络，从根节点开始，在每个层级选择有效的子节点，直到到达可计费的叶节点，确保输出符合医学编码的层级结构。
• 性能表现：在MIMIC-IV出院摘要上进行评估。在精选的50代码子集上micro-F1达到0.709；在全量15,761代码空间上达到0.527 micro-F1和0.157 macro-F1的提升，有效缓解了罕见代码的训练信号不足问题。
• 对比优势：相比三专家级联系统，共享策略模型在28-32%的全空间测试笔记中避免了上下文窗口溢出，同时保持了相当的性能水平，展示了更高的工程效率和鲁棒性。
• 训练分析：通过控制变量法研究了架构、训练算法和数据预算的影响。结果显示，单纯增加监督微调（SFT）的轨迹数据能一致性地提高性能，而引入GRPO强化学习并未显示出相对于监督延续的优势。

行业启示

• 结构化先验的重要性：在医疗等强依赖领域知识的NLP任务中，显式地将领域本体（如ICD层级）融入模型决策过程（如图约束），比单纯依赖数据驱动的扁平分类更能提升罕见类别的表现和输出的合规性。
• 简化优于复杂：在极端多标签预测场景中，精心设计的监督策略学习可能比复杂的强化学习或多阶段级联架构更具性价比和稳定性，特别是在数据充足的情况下，简单的监督信号往往比奖励函数优化更有效。
• 工程落地考量：单一模型替代多模型级联不仅提升了性能一致性，还解决了长文本处理中的上下文限制问题，这对于部署资源受限或需要处理超长病历的临床AI系统具有重要的工程实践价值。