“Every time a patient in a U.S. hospital acquires an infection that they didn’t come in with, human misery and tens of thousands of dollars to the cost of a hospitalization follow,” he explains.

“No clinician wants to impose hospital-acquired infections on their patients. But clinicians are busy. They’re human. They’re imperfect. So they don’t always notice when they’ve just skipped a critical intended action step.”

That led to thinking about how artificial intelligence could be used to help detect and correct — in real time — deviations in essential clinical actions, like maintaining hand hygiene, which is a primary way to prevent hospital-acquired infections.

In 2015 CERC researchers, alongside graduate students and faculty in the AI Lab, began developing a system that detects whether someone used the alcohol hand dispenser that sits on the wall next to every hospital room entrance.

Arnold Milstein, MD, came to Stanford eight years ago with a simple assignment: Find out how to lower the national cost of producing great health care. Put another way, if we could find more affordable ways to deliver better care for conditions that consume the bulk of the country’s health care spending, more monies would be available for other ways to improve human well-being — like education and social services.

Milstein was ideally suited to the task. He spent two decades working to improve health care value in the private sector, after which he served as an advisor to Congress and the White House. In 2011 he created Stanford’s Clinical Excellence Research Center (CERC). It is the first university-based research center exclusively dedicated to discovering, testing, and disseminating cost-saving innovations in clinically excellent care.

One of CERC’s areas of emphasis is discovering how artificial intelligence (AI) can prevent inadvertent and costly failures in intended care delivery. This focus began with a call from Professor Fei-Fei Li, PhD, director of the Artificial Intelligence Lab in the Stanford School of Engineering.

“Our subsequent conversations sparked a decision to create a unique cross-school Partnership in AI-assisted Healthcare, which we call PAC. We imagined a world in which AI improves the performance of a broad range of human services that affect health,” Milstein says.

“We initially focused solely on health care in order to learn and make a difference before we expand our use of AI to improve performance across a broad range of health-affecting services,” adds Milstein, who turns to a favorite initial target: lowering the incidence of hospital-acquired conditions or HACs.

“Every time a patient in a U.S. hospital acquires an infection that they didn’t come in with, human misery and tens of thousands of dollars to the cost of a hospitalization follow,” he explains.

“No clinician wants to impose hospital-acquired infections on their patients. But clinicians are busy. They’re human. They’re imperfect. So they don’t always notice when they’ve just skipped a critical intended action step.”

That led to thinking about how artificial intelligence could be used to help detect and correct — in real time — deviations in essential clinical actions, like maintaining hand hygiene, which is a primary way to prevent hospital-acquired infections.

In 2015 CERC researchers, alongside graduate students and faculty in the AI Lab, began developing a system that detects whether someone used the alcohol hand dispenser that sits on the wall next to every hospital room entrance. Their system relies on computer vision, a rapidly progressing domain of artificial intelligence used in the automotive and other industries.

“If computer vision can detect when drivers initiate dangerous lane changes and safely control vehicular steering, can it similarly analyze motion to detect unintended deviations in important clinician behaviors or patient activities?” asked Milstein and Li’s research team in a New England Journal of Medicine article.

AI systems that take advantage of computer vision are relatively inexpensive. By using them, the team has shown it can achieve greater than 95 percent accuracy in detecting inadvertent omissions in the use of the hand sanitizer before staff enter patient rooms.

The vision of making excellent care more effective and efficient also targets behaviors that affect lifelong health trajectories. In collaboration with Stanford researchers in child development and pediatrics, the team is testing how computer vision can let mothers know if their eyes inadvertently drift to their smartphone screen instead of responsively returning their infant’s gaze.

The hope, Milstein says, is to unite technology and human care. “By mobilizing emerging science and technology from engineering, behavioral sciences, and medicine, Stanford can address a seemingly intractable national challenge to make affordable all forms of human caring that powerfully affect health.”

Their system relies on computer vision, a rapidly progressing domain of artificial intelligence used in the automotive and other industries.

“If computer vision can detect when drivers initiate dangerous lane changes and safely control vehicular steering, can it similarly analyze motion to detect unintended deviations in important clinician behaviors or patient activities?” asked Milstein and Li’s research team in a New England Journal of Medicine article.

AI systems that take advantage of computer vision are relatively inexpensive. By using them, the team has shown it can achieve greater than 95 percent accuracy in detecting inadvertent omissions in the use of the hand sanitizer before staff enter patient rooms.

The vision of making excellent care more effective and efficient also targets behaviors that affect lifelong health trajectories. In collaboration with Stanford researchers in child development and pediatrics, the team is testing how computer vision can let mothers know if their eyes inadvertently drift to their smartphone screen instead of responsively returning their infant’s gaze.

The hope, Milstein says, is to unite technology and human care. “By mobilizing emerging science and technology from engineering, behavioral sciences, and medicine, Stanford can address a seemingly intractable national challenge to make affordable all forms of human caring that powerfully affect health.”

Presence aims to ensure that patients, clinicians, funders, legislators, and other stakeholders are at the table as equitable and inclusive AI solutions are created and deployed in health care.

To that end, Presence presented two symposia during 2018. In April, Jonathan Chen, MD, assistant professor of biomedical informatics, was a leader of the first symposium, “Human Intelligence and Artificial Intelligence in Medicine,” which addressed augmented intelligence of humans and machines for diagnostics.

“I hold out hope that artificial intelligence and machine-learning algorithms will transform our experience, particularly if natural-language processing and video technology allow us to capture what is actually said and done in the exam room,” writes Abraham Verghese, MD, professor of medicine and founding faculty director of the Stanford Presence Center.

“The physician focuses on the patient and family, and if there is a screen in the room, it is to summarize or to share images with the patient; by the end of the visit, the progress notes and billing are done. But AI applications will help us only if we vet all of them for their unintended consequences. Technology that is not subject to such scrutiny doesn’t deserve our trust, nor should we ever allow it to be deeply integrated into our work,” Verghese continues in a May 2018 article that appeared in The New York Times Magazine.

That sentiment is behind a key focus for Presence, a center that emphasizes the value of the human connection in the high-wire balancing act between high tech and high touch.

Presence aims to ensure that patients, clinicians, funders, legislators, and other stakeholders are at the table as equitable and inclusive AI solutions are created and deployed in health care.

To that end, Presence presented two symposia during 2018. In April, Jonathan Chen, MD, assistant professor of biomedical informatics, was a leader of the first symposium, “Human Intelligence and Artificial Intelligence in Medicine,” which addressed augmented intelligence of humans and machines for diagnostics. The 350 physicians, business leaders, policymakers, social and behavioral scientists, venture capitalists, and political activists in attendance were challenged to determine how to ensure that humans are augmented by AI in defining and delivering compassionate services.

On that subject Verghese says, “Pitting humans against machines is not the point. Rather, how best to relevantly engage both for the sum to be greater than the parts should be the focus.”

“Machines do many things very well, but they really can’t do the caring work, so how do we augment the two preemptively, proactively, and equitably for the outcome that we all seek?” he asks.

The 350 physicians, business leaders, policymakers, social and behavioral scientists, venture capitalists, and political activists in attendance were challenged to determine how to ensure that humans are augmented by AI in defining and delivering compassionate services.

On that subject Verghese says, “Pitting humans against machines is not the point. Rather, how best to relevantly engage both for the sum to be greater than the parts should be the focus.”

“Machines do many things very well, but they really can’t do the caring work, so how do we augment the two preemptively, proactively, and equitably for the outcome that we all seek?” he asks.

“Artificial Intelligence in Medicine: Inclusion and Equity” was the second symposium in August, which drew 275 attendees from around the world. Presence executive director Sonoo Thadaney, MBA, co-chair of the National Academy of Medicine’s Working Group on AI in Healthcare, was one of the symposium leaders. Acknowledging the potential unintended consequences of AI in medicine, she examined how to prevent and manage the possible exacerbation of inequity and exclusion in health care.

Thadaney speaks of a huge inequity that looms depending on an individual’s circumstances, saying: “We cannot have a world where technology creates greater inequity such that those of us with privilege have access to second opinions and concierge physicians, and the rest of the planet ends up with medicine that is meted out with the efficiency and emptiness of fast food. We cannot afford a health care apartheid.”

The Gordon and Betty Moore Foundation and the Robert Wood Johnson Foundation support Presence by funding the symposia as well as another innovative program that began at the end of 2018: the AI in Medicine Inclusion & Equity (AiMIE) 2018 Seed Grants Program. The AiMIE program provides initial funding for projects seeking equitable and inclusive frameworks for AI in medicine.

Are there problems with data currently in repositories?
We’re starting to see an emphasis not just on putting the data into repositories but on actually doing a good job of it. The National Center for Biotechnology Information (NCBI) maintains most of the NIH repositories for experimental data, but it generally does no more than make sure that the forms are filled in. So NCBI databases contain lots of horrible stuff; for instance, some 25 percent of the metadata values that are supposed to be numeric don’t actually parse as numbers.

Making Large Data Easily Available Online
Several years ago, Mark Musen, MD, PhD, wrote: “The ultimate Big Data challenge lies not in the data, but in the metadata — the machine-readable descriptions that provide data about the data. It is not enough to simply put data online; data are not usable until they can be ‘explained’ in a manner that both humans and computers can process.”

Musen is a professor of biomedical informatics and director of the Stanford Center for Biomedical Informatics Research. He is also the head of CEDAR, the Center for Expanded Annotation and Retrieval, which helps researchers comply with requirements to archive their data so others can understand and use them. In a recent interview, Musen provided clarity about the problem of metadata.

Why is it a problem for researchers to comply with the requirement to publish their metadata?
The greatest challenge of this whole enterprise is the problem of “What’s in it for me?” We reward scientists for authoring journal articles and for creating PDFs, but we don’t have a system that recognizes the data contributions that scientists make. We need to change the culture so that when other investigators report secondary analyses of data, or when data sets are re-explored and then lead to new discoveries, there is a benefit to the original investigator other than being acknowledged in someone else’s paper. Currently, investigators don’t have the motivation to spend a lot of time making their experimental data easily available online, and they generally lack tools to enable them to do so in a standardized, reproducible fashion.

Are there problems with data currently in repositories?
We’re starting to see an emphasis not just on putting the data into repositories but on actually doing a good job of it. The National Center for Biotechnology Information (NCBI) maintains most of the NIH repositories for experimental data, but it generally does no more than make sure that the forms are filled in. So NCBI databases contain lots of horrible stuff; for instance, some 25 percent of the metadata values that are supposed to be numeric don’t actually parse as numbers.

Is this where CEDAR has a role to play?
Precisely. The idea of CEDAR is to make it easier and more attractive for investigators to publish their data because more science is going to come out of it if they do. CEDAR has a whole library of templates that correspond to “minimal information models” for describing different classes of experiments. And we have technology that makes it easy to fill in one of these templates to describe your particular experiment when you are ready to upload your data sets to a repository. By filling in the template, you create standardized, searchable metadata that future investigators will use to locate the data and to make sense of what you have done. Using a cache of metadata that it already has stored, CEDAR can make suggestions as you’re filling out a template to accelerate the process of creating the metadata in the first place.

How is CEDAR being used today?
CEDAR helps investigators put data sets online — with well-described metadata — that will allow future scientists to perform new analyses that may allow them to make new discoveries. Our collaborations with several large research consortia show that it’s not all that difficult for investigators to do a great job of annotating their data sets in a way that will benefit the entire scientific community. Immunologists in the Antibody Society use CEDAR to upload their data and metadata to repositories at the NIH. Scientists developing the Library of Integrated Network-Based Cellular Signatures use CEDAR in association with their own data coordinating and integration center. The Irish Health Research Board and the Dutch Clinical Funding Agency are evaluating using CEDAR to review proposed metadata before making funding decisions about new studies.

Is this where CEDAR has a role to play?
Precisely. The idea of CEDAR is to make it easier and more attractive for investigators to publish their data because more science is going to come out of it if they do. CEDAR has a whole library of templates that correspond to “minimal information models” for describing different classes of experiments. And we have technology that makes it easy to fill in one of these templates to describe your particular experiment when you are ready to upload your data sets to a repository. By filling in the template, you create standardized, searchable metadata that future investigators will use to locate the data and to make sense of what you have done. Using a cache of metadata that it already has stored, CEDAR can make suggestions as you’re filling out a template to accelerate the process of creating the metadata in the first place.

How is CEDAR being used today?
CEDAR helps investigators put data sets online — with well-described metadata — that will allow future scientists to perform new analyses that may allow them to make new discoveries. Our collaborations with several large research consortia show that it’s not all that difficult for investigators to do a great job of annotating their data sets in a way that will benefit the entire scientific community. Immunologists in the Antibody Society use CEDAR to upload their data and metadata to repositories at the NIH. Scientists developing the Library of Integrated Network-Based Cellular Signatures use CEDAR in association with their own data coordinating and integration center. The Irish Health Research Board and the Dutch Clinical Funding Agency are evaluating using CEDAR to review proposed metadata before making funding decisions about new studies.