I Built an AI Agent for NHS Cancer Waits. Its Best Feature Is the Question It Won’t Answer.
The author built an AI agent to address the lack of transparency in NHS cancer waiting times, inspired by the real-time tracking available in consumer services. The system explicitly refuses to provide individual patient forecasts, citing ethical concerns and the absence of predictive data in historical NHS records. Instead of prediction, the agent provides statistical probabilities based on historical performance, presenting data as "odds" (e.g., "two in three") rather than raw percentages to r
Analysis
TL;DR
- The author built an AI agent to address the lack of transparency in NHS cancer waiting times, inspired by the real-time tracking available in consumer services.
- The system explicitly refuses to provide individual patient forecasts, citing ethical concerns and the absence of predictive data in historical NHS records.
- Instead of prediction, the agent provides statistical probabilities based on historical performance, presenting data as "odds" (e.g., "two in three") rather than raw percentages to reduce anxiety.
- The architecture strictly separates computation from generation: Python handles all data processing and logic, while a lightweight LLM (Claude Haiku) is used solely for natural language formatting.
- An automated evaluation framework ensures factual accuracy and adherence to style guidelines, catching both model hallucinations and underlying code bugs.
Why It Matters
This case study demonstrates a responsible approach to deploying AI in high-stakes healthcare contexts, prioritizing honesty and clarity over speculative predictions. It highlights a robust engineering pattern where deterministic code manages data integrity and LLMs serve only as presentation layers, minimizing the risk of hallucination. This model offers a blueprint for building trustworthy AI agents in regulated industries where accuracy and ethical considerations are paramount.
Technical Details
- Data Pipeline: Raw NHS England data (over 1 million rows) is processed through a Bronze, Silver, and Gold refinement stage using pandas, resulting in a clean table with one row per trust per month.
- Architecture: A strict separation of concerns where Python computes all metrics (percentages, trends, comparisons) and generates plain-English phrases. The LLM receives pre-computed facts and is instructed not to introduce any new numbers.
- Model Selection: Uses Claude Haiku, selected for its cost-efficiency and speed, functioning purely as a text formatter rather than a reasoning engine.
- Evaluation Framework: An automated testing suite validates outputs against source data and style rules, flagging contradictions, unauthorized editorializing, or incorrect advice.
Industry Insight
- Trust Through Transparency: In sensitive domains, AI systems should clearly state their limitations (e.g., inability to predict individual outcomes) rather than attempting to guess, which builds user trust.
- Deterministic Pre-processing: Offloading complex logic and calculations to traditional code reduces the burden on LLMs, ensuring factual consistency and reducing token costs.
- Automated Quality Assurance: Implementing rigorous automated checks for both model output and backend logic is essential for maintaining reliability in production environments, especially when dealing with large volumes of generated content.
Disclaimer: The above content is generated by AI and is for reference only.