AI and digital tools promise to transform clinical workflows into efficient, learning health systems
Healthcare systems need defensible governance processes and independent clinical assessment of AI claims. But compliance alone doesn't answer the question that matters:
“What happens when this tool is put in the hands of clinicians, under real-world uncertainty and pressure, with patients at the receiving end?”
This isn't hypothetical. In our own research analysing 850 patient assessments at a major London teaching hospital, we found that among unplanned admissions, clinicians often recorded a single working diagnosis, and in the majority of these, no expression of uncertainty was documented. If clinicians already struggle to express uncertainty in routine practice, what happens when an AI system presents a confident recommendation?
Understanding these patterns – how clinicians actually think and document – is essential before deploying tools designed to influence their decisions.
It requires people who have practised at the bedside, published on AI evaluation methodology, and understand how human-AI interaction alters decision-making – a question we explored in depth in our book Thinking About Thinking.
