According to the team of researchers at King’s College London, UCL, King’s College Hospital NHS Foundation Trust and Guy’s and St Thomas’ NHS Foundation Trust, these results indicate the tool could be used to support clinical decision-making, monitoring in healthcare settings and to improve clinical trials.

The tool – called Foresight – is trained on existing healthcare data and uses a deep learning approach to recognise complex patterns in both the structured and unstructured data of electronic health records to produce insights and predictions.

It belongs to the same family of AI models as ChatGPT but, in contrast to ChatGPT which is trained on publicly available information, Foresight is trained on information from NHS electronic health records.

The study, published in The Lancet Digital Health, investigated the accuracy of Foresight’s medical predictions by comparing them to what actually happened to the patients as described in their records.

Using data from over 811,000 patients across King’s College Hospital NHS Foundation Trust, South London and Maudsley NHS Foundation Trust, and MIMIC-III – a publicly available dataset of patients from Beth Israel Deaconess Medical Center in the US, researchers trained three different models of Foresight.

Researchers extracted and processed the unstructured (free-text) and structured data (age, ethnicity and sex) within electronic health records using CogStack. The datasets were used to train Foresight and performance was measured by comparing its predictions on true outcomes in a smaller subset of these data. Foresight was trained on data under NHS and patient governance and inside the hospital NHS firewall.

Precision forecasting

When forecasting the next 10 possible disorders that could appear next in a patient timeline, the tests showed that Foresight correctly identified the next disorder 68 per cent and 76 per cent of the time in two UK NHS Trusts (King’s College Hospital NHS Foundation Trust and South London and Maudsley NHS Foundation Trust) and 88 per cent of the time in the US MIMIC-III dataset. Similarly, when forecasting the next new biomedical concept which could be a disorder, symptom, relapse or medication, the precision achieved by Foresight was 80 per cent, 81 per cent and 91 per cent respectively.

The accuracy of Foresight’s predictions was also assessed by clinicians. Five clinicians developed 34 mock patient timelines with simulated scenarios. When all five clinicians agreed on the forecasted medical event, the predictions Foresight provided were 93 per cent relevant, meaning they made sense from a clinical perspective.

Uses of Foresight

Foresight can be used for real-world risk forecasting, emulating trials and clinical research to study the progression of disorders, simulate interventions and lifestyle factors and educational purposes.

Professor Dobson, who is also supported by the NIHR BRC at University College London Hospitals NHS Foundation Trust (UCLH), added: “Foresight will improve with more real-world data, and we are now looking for more hospitals to be involved in developing Foresight 2, a more accurate language model.”

Co-author Professor James Teo, Joint Director for Data Science and Artificial Intelligence at King's College Hospital NHS Foundation Trust and Guy's & St Thomas' Hospital NHS Foundation Trust, and Professor of Neurology, King's College London said "Our study demonstrates the future potential of AI technologies in healthcare and how AI can be developed by NHS and universities in a safe regulated way for society's benefit; I am very proud of the support that our patients, our staff and our NHS provided CogStack along the way."

Professor Andrew Morris, Director of Health Data Research UK (HDR UK), said: “AI has immense potential to enable scientific discovery, support the prevention, diagnosis and treatment of disease, and improve care pathway management and education. This work demonstrates the potential that using electronic health records and the latest advancements in AI could deliver for improving clinical decision making. But this is all dependent on the quality and representativeness of data. Sustained investment in the UK's data infrastructure is needed to enable this in safe and secure ways to maintain privacy and anonymity of people's health data. 

"HDR UK aims to accelerate trustworthy data use. This study illustrates that the free-text portion of electronic health records, which are currently underutilised in research, hold vital information for understanding health outcomes.” 

Foresight is part of CogStack, an information retrieval and extraction platform developed by researchers at King’s College London, King’s College Hospital NHS Foundation Trust, Guy’s and St Thomas’ NHS Foundation Trust, South London and Maudsley NHS Foundation Trust, UCL and UCLH. It uses natural language processing to harness NHS electronic health record data to support clinical decision-making and health research.

This paper was part of a programme of work that received funding from the NHS AI Lab, National Institute for Health and Care Research and Health Data Research UK. It has infrastructure support from King’s College Hospital NHS Foundation Trust, NIHR Maudsley BRC and the London AI Centre for Value-Based Healthcare.

‘Foresight - Generative Pretrained Transformer (GPT) for Modelling of Patient Timelines using EHRs: A Retrospective Study’ was published in The Lancet Digital Health.

For more information, please contact Alex Booth or Franca Davenport, (Senior Communications and Engagement Manager, NIHR Maudsley BRC).