In the fog of war, medical decisions come quick and hot. Someone must be on hand to assess injuries and make rapid decisions about who to treat and in what order, who should be evacuated and—in worst-case scenarios—who can’t be saved. Could the best person for this job be an autonomous artificial intelligence? 

DARPA—the U.S. military’s research and development arm—recently called on experts in industry and academia to collaborate on a new decision-making tool that is “human off the loop”—in other words, completely autonomous. The In the Moment (ITM) project will focus first on small-unit battlefield triage but then will also aim to manage mass-casualty events. 

The goal for ITM is to examine “foundational questions” about this type of technology, including how to build an algorithm that humans can learn to trust with decisions of life and death, according to program manager Matt Turek. “We picked triage because it’s challenging,” he says. “It forces us to deal with difficult decisions, where humans often disagree about the right approach.” 

A central requirement will be to demonstrate that a system reliably produces “right” answers, in which algorithms’ answers compare favorably to those of human decision-makers. But there is also an opportunity for deeper questions. “Subject matter experts are going to disagree,” says Turek. So determining which humans should serve as models, and why, becomes a research frontier. “Studies have shown that when people evaluate which humans to trust, integrity and perceived benevolence are essential,” says Turek. Can such qualities be embedded into a machine?  

Triage is a task machine learning is well suited to.

The trust riddle aside, the In the Moment AIs must also reliably assess health and possible outcomes for a range of wounded soldiers. In that regard, many solutions will most likely build on existing health care projects. Autonomous risk prediction has been under development for more than a decade, says Michael R. Pinsky, a professor of critical care medicine at the University of Pittsburgh School of Medicine and a senior advisor in its Center for Military Medicine Research. Pinsky is a principal investigator and co-investigator in multiple federally funded projects developing tools that can foresee health outcomes in acute-care settings with startling clarity. 

Triage is a form of prediction based on a body of previous data, which is a task machine learning is very well suited to, Pinsky says. A project looking at emergency department patients who go into cardiac arrest, for instance, would explore medical records to find clear patterns. One current project, which uses only heart rate and other vital signs to predict which ER patients will remain stable, is right 80% of the time. “That’s better than you can do with any human method,” says Pinsky. 

A collaboration among Massachusetts General Hospital, MIT and Leiden University Medical Center in the Netherlands has produced another proof-of-concept tool in autonomous triage. Trained on a database of patients with gunshot wounds, the AI can assess a new shooting victim, accurately identify whether that person might soon go into shock and predict the need for massive transfusions or major surgery with a “high degree of certainty.” 

But not all AI tools are transparent in how they make decisions. Many are “black boxes”—that is, the logic of their predictions can’t be explained in a way that humans will understand. That might be all right for some uses, says Sara Gerke, assistant professor of law at Penn State Dickinson Law, who studies ethical and legal issues in health care AI. But for decisions about organ transplants or in traumatic injury triage—in which one person might have to die so that another can live—“you really want to have an interpretable model,” Gerke says. 

If such a triage tool is to be eventually rolled out to military teams, the ethical groundwork must be rock solid, she says. Research shows that “automation bias”—in which humans just follow what AI wants them to do—can be particularly common in emergency situations. Even if DARPA does its job, however, and the battlefield triage tool proves able to come up with reliable answers, the military will need to contend with weighty questions involving implementation. “For instance, if you use this tool, are you required to follow its decisions, even if you don’t agree with them?” asks Gerke.

Turek acknowledges that the ITM project is full of such “cross-disciplinary challenges.” He hopes it will bring together perspectives from both the Department of Defense and civilians of many stripes. Submissions are currently under review, and Turek expects to offer contracts to winning teams for work that will begin in early 2023. But ITM performing teams will have the opportunity to publish their research without any restrictions. “We can’t fund a lot of teams, but we want to share results with a large research community,” he says.