Research Spotlight: Tackling Prediction Uncertainty in Machine Learning for Healthcare

Michelle Chua, MD, and Synho Do, PhD, from the Department of Radiology at Massachusetts General Hospital, are the first and corresponding authors of a new perspective in Nature Biomedical Engineering, Tackling Prediction Uncertainty in Machine Learning for Healthcare.

What Question Were You Investigating?

Fully automated machine learning (ML) systems are expected to transform healthcare delivery by alleviating resource constraints and reducing overall healthcare costs

However, few ML models have been deployed in clinical practice and those being used in the clinic typically only provide decision-making support.

This reflects both high public expectations for healthcare delivery and high safety and ethical standards expected from medical professionals.

Unsafe prediction failure occurs when a ML system produces an erroneous prediction, and either fails to convey a lack of confidence in the prediction or deceitfully conveys a high level of confidence in the correctness of the prediction.

How can we develop “zero error” ML systems that do not make these mistakes?

What Was Your Approach?

We show how prediction uncertainty metrics may be utilized and calibrated to develop ML systems with zero tolerance for false negative and/or false positive errors.

What Are the Clinical Implications?

When faced with a difficult case, a junior physician who is feeling unconfident about a decision is expected to consult a more experienced colleague.

Safe and ethical ML systems must likewise be engineered to convey lack of confidence in the correctness of individual predictions, enabling referral to clinicians for further review and error correction.

About the Massachusetts General Hospital

Massachusetts General Hospital, founded in 1811, is the original and largest teaching hospital of Harvard Medical School. The Mass General Research Institute conducts the largest hospital-based research program in the nation, with annual research operations of more than $1 billion and comprises more than 9,500 researchers working across more than 30 institutes, centers and departments. In July 2022, Mass General was named #8 in the U.S. News & World Report list of "America’s Best Hospitals." MGH is a founding member of the Mass General Brigham healthcare system.