Mass General Study Identifies an AI Model That Can Accurately Assess PTSD in Postpartum Women

BOSTON – A generative artificial intelligence (AI) model that can analyze the narrative accounts of women who have undergone recent childbirth has shown the ability to accurately screen for post-traumatic stress disorder (CB-PTSD), a study by Massachusetts General Hospital (MGH), a founding member of the Mass General Brigham healthcare system, has found.

By exploring the capabilities and shortcomings of several models from OpenAI, including ChatGPT, the researchers identified a version that offers rich insights into maternal mental health following traumatic childbirth.

The model can fit seamlessly into routine obstetric care and could potentially be harnessed to assess other mental health disorders. The results of the study were published in Scientific Reports.

“Evaluation of PTSD related to traumatic birth currently relies on extensive clinician evaluation, which fails to meet the urgent need for a rapid, low-cost assessment strategy,” says Sharon Dekel, PhD, director of MGH’s Postpartum Traumatic Stress Disorders Research Program, and senior author of the study.

“The use of brief patient narratives of childbirth analyzed by AI’s text-based computational methods could become an efficient, low-cost, and patient-friendly strategy for detecting CB-PTSD after a traumatic birth and with more research this tool may potentially aid in identifying women at risk for CB-PTSD before the condition fully develops.”

For an estimated eight million women a year globally, childbirth that is traumatic and/or medically complicated is expected to trigger post-traumatic stress disorder, a condition historically has been associated with military combat or severe sexual assault.

In recent years, childbirth has become acknowledged as a significant PTSD trigger which, if left untreated, can impair the health of both the mother and child and result in significant societal costs.

In previous studies, Dekel’s lab found evidence that brief psychological interventions delivered soon after traumatic childbirth can reduce maternal childbirth-related PTSD symptoms.

In their latest study, Dekel in collaboration with first author Alon Bartal, PhD, of Bar-Ilan University in Israel, investigated the effectiveness of artificial intelligence and related machine learning (ML) analysis strategies to detect CB-PTSD.

Specifically, they evaluated the performance of different large language models (LLMs) and variations of ChatGPT and their ability to extract novel insights from text-based data sets derived from the brief narrative descriptions by postpartum women of their childbirth experience.

As part of their work, the team collected short narrative accounts from 1,295 women who had recently given birth.

The study focused on an OpenAI model known as text-embeddings-ada-002, which converted narrative data from the personal accounts of women with and without probable CB-PTSD to a numerical format that was then analyzed by a trained machine learning algorithm developed by the team.

Researchers showed this model had superior performance in identifying postpartum traumatic stress compared to other ChatGPT and large language models, which are typically trained on huge volumes of data allowing them to understand, analyze and interpret natural language.

“The reliance of the ML model using childbirth narrative input from the Open AI model as its exclusive data source presents an efficient mechanism for data collection during the vulnerable postpartum period, demonstrating 85 percent sensitivity and 75 percent specificity in identifying CB-PTSD cases,” notes Dekel.

 “Moreover, the model we developed could potentially improve accessibility to CB-PTSD screening and diagnosis by fitting seamlessly into routine obstetric care and providing a foundation for commercial product development and mainstream adoption.”

Dekel, whose research program is dedicated to exploring women’s mental health following traumatic childbirth, underscores the clinical benefits of using a pre-trained large language model to assess potential PTSD in new mothers.

“Early intervention is essential to prevent the progression of this disorder to chronic stages, which can seriously complicate treatment,” the MGH investigator points out.

“Our unique approach could introduce an innovative and cost-effective screening strategy for identifying high-risk women and facilitating timely treatment. It may also holds promise for assessing other mental health disorders, and consequently improving patient outcomes.”

The emergence of artificial intelligence tools in health has been groundbreaking and has the potential to positively reshape the continuum of care. Mass General Brigham, as one of the nation’s top integrated academic health systems and largest innovation enterprises, is leading the way in conducting rigorous research on new and emerging technologies to inform the responsible incorporation of AI into care delivery, workforce support, and administrative processes. 

Dekel is a psychologist at MGH, and assistant professor of Psychology at Harvard Medical School. Bartal is an assistant professor of Information Systems at Bar-Ilan University in Israel. Co-authors in the Dekel Laboratory include Kathleen Jagodnik, PhD, a Harvard research fellow, and Sabrina Chan, a clinical research coordinator.

Dekel was supported by funds from the NIH (Eunice Kennedy Shriver National Institute of Child Health and Human Development, grants R01HD108619, R21HD109546, and R21HD100817).

About the Massachusetts General Hospital

Massachusetts General Hospital, founded in 1811, is the original and largest teaching hospital of Harvard Medical School. The Mass General Research Institute conducts the largest hospital-based research program in the nation, with annual research operations of more than $1 billion and comprises more than 9,500 researchers working across more than 30 institutes, centers and departments. MGH is a founding member of the Mass General Brigham healthcare system.