The Clinical Informatics Group uses AI to improve how doctors and nurses identify and assess hospitalized patients at risk of deterioration.

Controversies around artificial intelligence (AI) and ChatGPT seem to be everywhere these days — from students using these technologies to cheat on tests to chatbots threatening to take away people’s jobs. But Stanford physicians are balancing the scale by using these technologies to innovate ways to improve patient care — and nowhere is that passion greater than in the Clinical Informatics Group in the hospital medicine division of the Department of Medicine.

These physicians are hospitalists who not only treat patients but also use their interest in computer science to conduct research, fine-tune operational workflow, and design medical education around the latest technologies. While these physicians have a wide range of interests and expertise, ultimately they all want to improve the quality and safety of hospital stays, as well as the overall delivery of health care.

Hospitalists and Research Are a Natural Match

The Clinical Informatics Group includes a robust team of researchers who collaborate with divisions and departments across Stanford University and Stanford Health Care. Pilot projects showing positive outcomes have led to improved patient care practices systemwide.

“As academic clinicians, we as hospitalists have interests and passions outside of practicing medicine, and for many that’s research,” explains Ashwin Nayak, MD, clinical assistant professor of hospital medicine. “Within research, informatics is a broad foundation that can be applied to different specialties and problems.”

Adds Ron Li, MD, medical informatics director for digital health, “As hospitalists, we are system thinkers. We are not focused on one specific disease but about the entire care journey for a patient who may have many complex issues during a hospital stay.”

Clinical informatics research projects are increasingly exploring the use of AI — specifically ChatGPT — in clinical practice.

Hospitalized patients with complex conditions are typically cared for by multiperson teams who assess large amounts of constantly changing data, making it challenging for the team to stay in sync. One recent research project, Clinical Deterioration Prediction & Prevention Using Artificial Intelligence, looked at how AI could be used to improve how doctors and nurses work together to identify patients whose condition could deteriorate in a hospital setting.

Explains Li, who is a clinical assistant professor of hospital medicine and biomedical informatics research, “We used AI to develop a collaborative huddle and checklist process, allowing doctors and nurses to better assess at-risk patients and work together to intervene more quickly.” Not only did the pilot project reduce deterioration events at Stanford Hospital by 20%, but also it won the 2023 Healthcare Information and Management Systems Society (HIMSS) Nicholas E. Davies Award of Excellence for using health information technology to substantially improve patient outcomes.

Large language model chatbots such as ChatGPT are a particular area of interest for Clinical Informatics Group members. A recently published study comparing the clinical notes written by ChatGPT versus Internal Medicine residents found the quality to be comparable. “This study shows one of the many time-saving applications of large language models that could help free up clinicians so they can focus more on patient care,” comments Nayak, who was first author of the study.

As academic clinicians, we as hospitalists have interests and passions outside of practicing medicine, and for many that’s research.

— Ashwin Nayak, MD, clinical assistant professor of hospital medicine 

Information Technology Drives Hospital Efficiency and Safety

“Informatics is the glue that underlies the operation of the modern hospital. Every step in a hospital’s workflow requires a computer or cellphone app,” notes Weihan Chu, MD, clinical assistant professor of hospital medicine and associate chief medical officer of Stanford Health Care Tri-Valley and medical informatics director, Stanford Health Care.

Chu works extensively with the Stanford IT department to represent the physician perspective in developing and updating content used in nearly 200 hospital workflows, from auto-populated content for doctor notes for greater accuracy to checklists for hospital-admitted patients to improve consistency and efficiency.

Even basic hospital operations can have complex workflows involving many different areas. Explains Chu, “A blood transfusion for a patient’s cardiac surgery involves many behind-the-scenes steps, from routing the request to a blood bank and getting it filled and picked up to the operating room notifying the blood bank if they need more blood. IT tools make this process seamless.”

Before there were computers there was paper. “When we used paper to track patient care, there wasn’t one easily referenced source of truth,” he notes. “You can’t have multiple people looking at and updating the same piece of paper at the same time. Ultimately, these IT tools help us better coordinate care and improve patient safety.”

The Role of Informatics in Medical Education

AI technology is moving so quickly and integrating into so many areas within health care that Clinical Informatics Group members are exploring how to incorporate training into the Stanford School of Medicine’s basic curriculum for medical students and physician assistants obtaining an MSPA degree.

“It’s not a question of ‘if’ we’re going to integrate formal teaching about AI into the curriculum for students, but ‘how’ and ‘when,’” says Jason Hom, MD, clinical associate professor of hospital medicine. “We want to make sure our students are fully prepared for what they encounter in their clinical rotations. And since practicing clinicians were trained in a pre-AI world, we’re looking at continuing medical education courses as well,” adds Hom, who also serves as course director, Practice of Medicine Year 2, at the Stanford School of Medicine.

Educators around the world are intrigued by ChatGPT’s performance capabilities. In a study published in the Journal of the American Medical Association Internal Medicine, several Clinical Informatics Group members found that ChatGPT performed well on answering free-form questions from Stanford School of Medicine clinical reasoning exams. The study, Chatbot vs. Medical Student Performance on Free-Response Clinical Reasoning Examinations, was co-first authored by clinical associate professor of hospital medicine Eric Strong, MD, and School of Medicine Associate Director for Evaluation and Scholarship Alicia DiGiammarino, along with co-senior authors Jonathan Chen, MD, PhD, assistant professor of hospital medicine, and Hom. Yingjie Weng, Andre Kumar, MD, MEd, and Poonam Hosamani, MD were also co-authors. “We have to ensure new MD and MSPA students have a minimum level of unassisted competency before integrating AI into their studies. And we have to ensure that students have a basic understanding of how these emerging models work and can be used and what their limitations/biases are,” says Hom.

While the debate over how best to integrate AI into health care continues, the uniquely human aspects of medical training become even more important. “Teaching how to build rapport with patients, how to compassionately tell patients about a cancer diagnosis, how to listen to a patient’s heart — these are irreplaceable aspects of the patient-clinician relationship that we can focus on in training,” explains Hom.

Stanford and Technology Go Hand in Hand

Li cites Stanford leadership’s strong support for the use of informatics to solve problems as instrumental in the success of the group’s projects. “At Stanford, it’s in our DNA to use technology in service of innovation. There’s the rich ecosystem we’ve developed with Silicon Valley companies and cross-pollination with local industry. Plus, we tend to attract faculty who are skilled both as informaticians and as physicians,” he says. One such faculty member is Jonathan Chen, who is also assistant professor of biomedical informatics research and is featured in “How to Endure in a Pandemic? Magic!”

Explains Li, who is a clinical assistant professor of hospital medicine and biomedical informatics research, “We used AI to develop a collaborative huddle and checklist process, allowing doctors and nurses to better assess at-risk patients and work together to intervene more quickly.” Not only did the pilot project reduce deterioration events at Stanford Hospital by 20%, but also it won the 2023 Healthcare Information and Management Systems Society (HIMSS) Nicholas E. Davies Award of Excellence for using health information technology to substantially improve patient outcomes.

Large language model chatbots such as ChatGPT are a particular area of interest for Clinical Informatics Group members. A recently published study comparing the clinical notes written by ChatGPT versus Internal Medicine residents found the quality to be comparable. “This study shows one of the many time-saving applications of large language models that could help free up clinicians so they can focus more on patient care,” comments Nayak, who was first author of the study.

As academic clinicians, we as hospitalists have interests and passions outside of practicing medicine, and for many that’s research.

— Ashwin Nayak, MD, clinical assistant professor of hospital medicine

Information Technology Drives Hospital Efficiency and Safety

“Informatics is the glue that underlies the operation of the modern hospital. Every step in a hospital’s workflow requires a computer or cellphone app,” notes Weihan Chu, MD, clinical assistant professor of hospital medicine and associate chief medical officer of Stanford Health Care Tri-Valley and medical informatics director, Stanford Health Care.

Chu works extensively with the Stanford IT department to represent the physician perspective in developing and updating content used in nearly 200 hospital workflows, from auto-populated content for doctor notes for greater accuracy to checklists for hospital-admitted patients to improve consistency and efficiency.

Even basic hospital operations can have complex workflows involving many different areas. Explains Chu, “A blood transfusion for a patient’s cardiac surgery involves many behind-the-scenes steps, from routing the request to a blood bank and getting it filled and picked up to the operating room notifying the blood bank if they need more blood. IT tools make this process seamless.”

Before there were computers there was paper. “When we used paper to track patient care, there wasn’t one easily referenced source of truth,” he notes. “You can’t have multiple people looking at and updating the same piece of paper at the same time. Ultimately, these IT tools help us better coordinate care and improve patient safety.”

The Role of Informatics in Medical Education

AI technology is moving so quickly and integrating into so many areas within health care that Clinical Informatics Group members are exploring how to incorporate training into the Stanford School of Medicine’s basic curriculum for medical students and physician assistants obtaining an MSPA degree.

“It’s not a question of ‘if’ we’re going to integrate formal teaching about AI into the curriculum for students, but ‘how’ and ‘when,’” says Jason Hom, MD, clinical associate professor of hospital medicine. “We want to make sure our students are fully prepared for what they encounter in their clinical rotations. And since practicing clinicians were trained in a pre-AI world, we’re looking at continuing medical education courses as well,” adds Hom, who also serves as course director, Practice of Medicine Year 2, at the Stanford School of Medicine.

Educators around the world are intrigued by ChatGPT’s performance capabilities. In a study published in the Journal of the American Medical Association Internal Medicine, several Clinical Informatics Group members found that ChatGPT performed well on answering free-form questions from Stanford School of Medicine clinical reasoning exams. The study, Chatbot vs. Medical Student Performance on Free-Response Clinical Reasoning Examinations, was co-first authored by clinical associate professor of hospital medicine Eric Strong, MD, and School of Medicine Associate Director for Evaluation and Scholarship Alicia DiGiammarino, along with co-senior authors Jonathan Chen, MD, PhD, assistant professor of hospital medicine, and Hom. Yingjie Weng, Andre Kumar, MD, MEd, and Poonam Hosamani, MD were also co-authors. “We have to ensure new MD and MSPA students have a minimum level of unassisted competency before integrating AI into their studies. And we have to ensure that students have a basic understanding of how these emerging models work and can be used and what their limitations/biases are,” says Hom.

While the debate over how best to integrate AI into health care continues, the uniquely human aspects of medical training become even more important. “Teaching how to build rapport with patients, how to compassionately tell patients about a cancer diagnosis, how to listen to a patient’s heart — these are irreplaceable aspects of the patient-clinician relationship that we can focus on in training,” explains Hom.

Stanford and Technology Go Hand in Hand

Li cites Stanford leadership’s strong support for the use of informatics to solve problems as instrumental in the success of the group’s projects. “At Stanford, it’s in our DNA to use technology in service of innovation. There’s the rich ecosystem we’ve developed with Silicon Valley companies and cross-pollination with local industry. Plus, we tend to attract faculty who are skilled both as informaticians and as physicians,” he says. One such faculty member is Jonathan Chen, who is also assistant professor of biomedical informatics research and is featured in “How to Endure in a Pandemic? Magic!”