Auditable Emergency Triage for Maternal and Newborn Care in India
Noora Health replaced an opaque LLM-only emergency triage system with a two-stage pipeline combining LLM-based symptom extraction and a deterministic rule engine for maternal and newborn care queries The new architecture improved recall from 0.565 to 0.810 and F1 from 0.606 to 0.702, with structured rules driving most of the accuracy gains Decomposing triage into extraction and rule-based classification stages enabled full auditability, allowing clinicians to inspect and correct each decision st
Analysis
TL;DR
- Noora Health replaced an opaque LLM-only emergency triage system with a two-stage pipeline combining LLM-based symptom extraction and a deterministic rule engine for maternal and newborn care queries
- The new architecture improved recall from 0.565 to 0.810 and F1 from 0.606 to 0.702, with structured rules driving most of the accuracy gains
- Decomposing triage into extraction and rule-based classification stages enabled full auditability, allowing clinicians to inspect and correct each decision stage independently
- Since deployment, the system has processed 152,421 queries, flagged 18.7% as emergencies, maintained a 17.8% over-escalation rate, and seen clinicians add 48 new rules without costly re-evaluations
Why It Matters
This work demonstrates a practical pattern for deploying LLMs in high-stakes clinical settings: hybrid architectures that combine model-based extraction with deterministic reasoning can significantly outperform pure LLM approaches while remaining auditable and maintainable. For AI practitioners building healthcare systems, it provides a concrete blueprint for balancing automation with clinical oversight, showing that structured rules can complement LLMs rather than compete with them.
Technical Details
- Two-stage decomposition: An LLM extracts canonical symptoms and patient context from free-text queries using a clinician-authored vocabulary, then a deterministic rule engine applies a documented decision tree to classify emergencies
- Performance gains: Recall improved from 0.565 to 0.810 and F1 from 0.606 to 0.702; the shift from a flat list of danger signs to a structured decision tree was the primary driver of accuracy improvement
- Auditability by design: Each stage (mistranslation, symptom extraction, context inference, rule application) is independently inspectable, enabling clinicians to identify exactly where errors occur
- Operational impact: 152,421 queries triaged post-deployment, 28,535 flagged as emergencies (18.7%), 17.8% over-escalation rate with no increase in missed emergencies, and 48 new rules added by clinicians without regression testing
- Deployment context: Integrated into Noora Health's WhatsApp-based service handling 50,000+ medical queries monthly for maternal and newborn care in India
Industry Insight
- Hybrid LLM-plus-rules architectures should be the default consideration for any production healthcare AI system where auditability and regulatory compliance are required; pure end-to-end LLM approaches remain insufficiently transparent for clinical decision-making
- The 48 rules added by clinicians post-deployment without re-evaluation demonstrates that decoupling model logic from business rules creates a sustainable correction loop—organizations should design systems that allow domain experts to iterate on rules independently of ML engineering cycles
- The over-escalation rate of 17.8% with zero increase in missed emergencies suggests that in triage systems, optimizing for recall (minimizing false negatives) is ethically and clinically preferable to optimizing for precision, a principle that should guide evaluation metric selection in health AI deployments
Disclaimer: The above content is generated by AI and is for reference only.