Artificial intelligence (AI) could potentially become a valuable tool for neurologists, particularly when it comes to pinpointing the location of a stroke within the brain. In a recent study, AI was able to analyze text from neurologic exams and health histories to identify brain abnormalities.
Image Credit: peterschreiber.media/Shutterstock.com
The research was published in Neurology® Clinical Practice, the official journal of the American Academy of Neurology. It focused on the huge language model known as generative pre-trained transformer 4 (GPT-4).
A stroke can lead to significant long-term disability or, in severe cases, death. Identifying the specific location of a stroke within the brain is crucial for predicting its long-term consequences, such as impairments in speech and language or the ability to move certain body parts. This knowledge is also vital for determining the most effective treatment strategy and assessing the overall prognosis for recovery.
Damage to the brain tissue from a stroke is called a lesion. To locate these lesions, healthcare professionals often conduct a neurologic exam, which is complemented by a review of the patient's health history. This exam includes evaluating symptoms, as well as conducting tests related to thinking and memory. Additionally, individuals who have experienced a stroke typically undergo brain scans, which play a crucial role in pinpointing the locations of the lesions.
Not everyone with stroke has access to brain scans or neurologists, so we wanted to determine whether GPT-4 could accurately locate brain lesions after stroke based on a person’s health history and a neurologic exam.
Jung-Hyun Lee, MD, Study Author, State University of New York (SUNY) Downstate Health Sciences University
Lee is also a member of the American Academy of Neurology.
In the study, researchers analyzed 46 published cases of individuals who had experienced a stroke. They collected textual data from the participants' health histories and neurological examinations.
This raw text was input into GPT-4, with researchers posing three specific questions to the model: (1) whether a participant had one or more lesions, (2) on which side of the brain the lesions were located, and (3) the specific region of the brain where the lesions were found. These questions were asked three times for each participant to ensure consistency. The responses generated by GPT-4 were then meticulously compared to the brain scans of each participant to assess the accuracy of the AI's analysis.
The researchers discovered that GPT-4 was capable of processing the textual data from the health histories and neurological exams to identify the presence and locations of lesions in the brains of many participants. It accurately determined which side of the brain the lesion was on and identified the specific brain region. However, it did not successfully identify lesions in the cerebellum and spinal cord.
For most participants, GPT-4 demonstrated the ability to accurately determine the side of the brain where lesions were located, achieving a sensitivity rate of 74 % and a specificity rate of 87 %. Sensitivity measures the proportion of actual positives correctly identified, while specificity measures the proportion of negatives correctly identified. Furthermore, GPT-4 identified the specific brain region affected with a sensitivity of 85 % and a specificity of 94 %.
When examining how frequently each participant's results from the three tests yielded the same result, the GPT-4 showed consistency in the number of brain lesions for 76 % of subjects. For 83 % of participants, it was consistent with the side of the brain, and for 87 % of participants, it was consistent with the regions of the brain.
However, when combining its responses to all three questions across all three times, GPT-4 only provided accurate answers for 41 % of the participants.
While not yet ready for use in the clinic, large language models such as generative pre-trained transformers have the potential not only to assist in locating lesions after stroke, they may also reduce health care disparities because they can function across different languages, and the potential for use is encouraging, especially due to the great need for improved health care in underserved areas across multiple countries where access to neurologic care is limited.
Jung-Hyun Lee MD, Study Author, State University of New York (SUNY) Downstate Health Sciences University
One limitation of the study, however, is that the accuracy of GPT-4 is dependent on the quality of the information that is provided. While researchers had detailed health histories and neurologic exam information for each participant, such information is not always available for everyone who has a stroke.
Source: https://www.aan.com/