Google DeepMind released peer-reviewed evaluation data on Wednesday from three NHS trust pilots of a triage large language model, reporting clinician override rates below 8% on emergency referral suggestions and median inference latency of 1.4 seconds on on-premise hardware.

The pilots at trusts in London, Manchester and Leeds tested a fine-tuned model that reads structured symptom questionnaires and proposes urgency bands aligned with NHS 111 pathways. DeepMind stressed the system remained advisory: duty doctors retained final sign-off on every case.

Safety metrics

Across 142,000 simulated and live-shadow cases, the model missed zero time-critical cardiac presentations flagged by human nurses in the shadow cohort, but generated false escalations in 4.2% of low-acuity musculoskeletal complaints, increasing callback workload during overnight shifts.

DeepMind documented bias checks across age and ethnicity strata, reporting no statistically significant disparity in escalation rates after calibration, though sample sizes for some minority groups remained small enough that statisticians cautioned against definitive claims.

Procurement path

NHS England’s AI Lab said the results will feed a framework contract template requiring suppliers to publish override logs and maintain UK-hosted inference for patient data residency. The Medicines and Healthcare products Regulatory Agency has not classified the tool as a medical device yet, but expects a SaMD submission if trusts move from shadow mode to automated routing.

Workforce politics

UNISON branches at one participating trust raised concerns about deskilling of call handlers if metrics become performance targets. DeepMind’s contract includes a clause barring trusts from using model outputs as sole grounds for disciplinary action against staff.

Technical deployment

Pilots ran inference on NVIDIA H100 clusters hosted in NHS-approved data centres rather than public cloud, adding capital cost but satisfying data residency rules. DeepMind open-sourced evaluation scripts minus patient identifiers, allowing external auditors to replicate metrics on synthetic datasets.

Clinicians participating in Leeds reported the model’s plain-language summaries reduced handover time between night and day teams, though they asked for clearer audit trails when suggestions contradicted nurse intuition on paediatric fever cases.

Commercial rivals including Palantir and local start-ups offer competing triage dashboards; NHS procurement will compare total cost of ownership including clinician training hours before any national framework award.

Trust boards must approve any move from shadow mode to patient-facing deployment, a step none of the three sites has scheduled before spring. DeepMind said it would not seek exclusivity clauses, allowing NHS trusts to benchmark against other vendors in parallel pilots.

Calderdale and Humber integrated care board said it would publish a redacted evaluation summary for patient governors in December, including staffing hours spent reviewing model suggestions.