Public Perceptions of Artificial Intelligence Use in Healthcare
Take Home Point: Patient and healthcare workers surveyed were generally accepting of the use of artificial intelligence (AI) in medicine. Respondents did express some concern about the potential impact of AI on the accuracy of medical decision-making, however.
Citation: Thornton N, Binesmael A, Horton T, et al. AI in health care: what do the public and NHS staff think? The Health Foundation. Published July 31, 2024. Accessed September 5, 2024. https://www.health.org.uk/publications/long-reads/ai-in-health-care-what-do-the-public-and-nhs-staff-think
Relevance: AI is encountering rapid adoption within healthcare for various applications. This study aimed to elucidate opinions regarding the implementation of AI in patient care.
Study Summary: This was a survey commissioned by the Health Foundation of nationally representative members of the public (aged 16 years and older) and National Health Service (NHS) staff members in the United Kingdom (UK) to learn more about attitudes toward AI. The authors administered an online survey and included a booster sample of 200 UK adults at risk of digital exclusion surveyed through computer-assisted telephone interviewing.
In all, 7,201 members of the UK public aged 16 years and older and 1,292 NHS staff members responded. 54% of the public responded with support for the use of AI for patient care in applications like diagnosing illness and recommending treatment. A greater proportion (61%) supported the use of AI for administrative purposes like sending letters or planning staffing.
Amongst the NHS staff surveyed, 76% were in favor of implementing AI for patient care and 81% for administrative purposes. The two potential disadvantages of AI which were most feared by the public were “that healthcare staff won’t question the AI system’s decision, even if it is wrong” (30%) and that “AI decisions might not be accurate enough, meaning that the wrong decisions could be made” (28%). Transparency in AI decision making seemed particularly important for ≥65-year-old participants. Interestingly, the proportion of individuals who felt that AI would negatively impact health outcomes did not differ meaningfully according to the age of respondents.
Editor’s Comments: This was a large survey of the perspectives of the public and healthcare workers in the UK–the main stakeholders–on the use of AI in patient care. The depersonalization of healthcare seems to be a particular concern raised in the study. It is critical that those who make policy decisions regarding the implementation of AI take the opinions of such stakeholders into consideration. This survey was limited to respondents in the UK. It is unclear how these opinions may be generalized to stakeholders in healthcare of other nations.
Changing Management of Toddler’s Fractures to Mirror Best Evidence
Take Home Point: In this quality improvement project, the intervention significantly increased the proportion of toddler’s fractures which were treated without cast immobilization.
Citation: Chen S, Holstein J, Samora J. Reducing Rigid Immobilization for Toddler’s Fractures: A Quality Improvement Initiative. Pediatr Qual Saf. 2024 Apr 3;9(2): e722. doi: 10.1097/pq9.0000000000000722
Relevance: Management of toddler’s fractures has evolved with increasing evidence that cast immobilization does not offer benefit over the use of a simple walking boot. The inherent stability of the fracture pattern confers little risk of displacement during the healing process and therefore, non-casting appears to be a safe, and often preferrable, treatment option.
Study Summary: This was a quality improvement (QI) project conducted at a tertiary care pediatric hospital in Columbus, Ohio, to increase the proportion of patients with toddler’s fractures treated without cast. The goal of the project was to change clinical practice by implementing evidence-based treatment for toddler’s fractures to decrease overall costs and complications and increase patient and family satisfaction without compromising patient outcomes. The intervention involved education for orthopedic surgery residents, emergency department (ED), and UC clinicians to ensure standardization of the process by reinforcing recognition of toddler’s fractures among all providers and emphasizing that cast immobilization was not necessary in the treatment of these stable injuries. Monthly unblinded compliance data were shared amongst the orthopedic attendings.
The authors noted that after the interventions were implemented, the average percent of patients with toddler’s fractures treated without rigid cast immobilization increased from the baseline of 45.6% to 90% (P ≤ 0.001). There was a shift in the percentage of patients with toddler’s fractures who were specifically treated in a boot during their first visit to the orthopedic clinics from 4.2% to 52% (P ≤ 0.001). There was also a decrease in the proportion of patients who required a 3-month follow-up visit from 93% to 65% (P ≤ 0.001). Additionally, by reducing the need for the follow-up visits, they reduced the need for follow-up radiographs and radiation exposure for patients from 65% to 13% (P ≤ 0.001).
Editor’s Comments: This QI project reinforces the importance of continuing education around current evidence. While non-cast immobilization of toddler’s fractures is gaining increasing supporting evidence for its safety, it is advisable for UC clinicians to consult with local orthopedics specialists to determine local practice preferences. As initial immobilization in UC centers usually involves non-circumferential splinting, this project will likely be of most value in guiding parents’ expectations for the possibility of non-casting at orthopedics follow-up.
Leveraging Artificial Intelligence in Chest Pain Triage
Take Home Point: Use of an artificial intelligence (AI) algorithm designed for ED triage led to significant reductions in ED length of stay (LOS) for patients admitted to the hospital and time until critical cardiac procedures in this quality improvement study.
Citation: Hinson J, Taylor R, Venkatesh A, et. al. Accelerated Chest Pain Treatment with Artificial Intelligence-Informed, Risk-Driven Triage. JAMA Intern Med. 2024 Jul 22: e243219. doi: 10.1001/jamainternmed.2024.3219.
Relevance: Chest pain remains a common presentation to ED with over 8 million presentations annually in the United States (US). However, only 6% of these chest pain presentations are related to life-threatening conditions. AI offers promise in accelerating identification of this small, but possibly critically ill minority.
Study Summary: This was a multisite quality improvement study, comparing treatment intervals for adult patients with chest pain before and after implementation of an artificial intelligence (AI)–informed, outcomes-driven decision support system for ED triage (TriageGO; Beckman Coulter). TriageGO used machine learning algorithms to estimate probabilities for critical care, emergency procedures, and hospital admission using demographics, arrival mode, vital signs, chief complaints, and active medical problems as predictors. The system then translated outcome probabilities to recommended acuity levels according to the emergency severity index (ESI) (ie, 1-5, with lower values indicating higher acuity). Downstream protocols for diagnosis and treatment of chest pain remained consistent before and after intervention. The authors performed adjusted analyses using median regression models to limit confounding factors.
The authors analyzed 12,147 adult ED visits (6,188 before and 5,959 after implementation of the AI system). They found that after implementation of the AI triage tool fewer patients were assigned to high acuity levels 1 or 2 (1,317 [22.1%] vs 1,708 [27.6%]) or mid-acuity level 3 (3,263 [54.8%] vs 4,086 [66.0%]), and more were assigned to lower acuity with ESI 4 or 5 (1,379 [23.1%] vs 394 [6.4%]) (χ24 = 771.6; P < .001). Median time to emergency cardiovascular procedures was reduced by 205.4 minutes (95% CI, 23.0-387.8 minutes), including cardiac catheterization (by 243.2 minutes; 95% CI, 43.7-442.7 minutes).
Editor’s Comments: The quality improvement project was limited as there was no randomization of patients as it was observational and therefore potential for confounding exists. The study also did not look at patient-oriented outcomes such as hospital length of stay or in-hospital or 30-day mortality. It was presumed that decreased time to cardiac procedures was for patient benefit, however, that is uncertain from this data and previous studies have shown that certain patients have worse outcomes if taken for emergent cardiac catheterization (eg, patients without acute coronary occlusion or ST-elevation myocardial infarction). The main value this AI offered from a chest pain triage utility standpoint that can be ascertained from the data presented definitively is that many patients with chest pain are overtriaged and given unnecessarily high ESI scores. This makes sense, given that over 90% of patients with chest pain presenting to the ED, do not have a serious diagnosis. While the use of this AI (and AI in general) for chest pain triage offers promise in more rapidly identifying critical patients with chest pain, it’s greater utility for healthcare resource utilization is likely that it correctly identifies patients with chest pain who do not require ESI 1-3 designation and can be seen safely in a less urgent fashion.
Many ED systems currently have an automatic chief complaint designation ESI protocol and assign any patient with chest pain an ESI of no lower than 2. It is clear, however, that AI can offer a more individualized triage score for chest pain. Further studies are required to examine how this affects outcomes and resource utilization rates as this project focused predominantly on ED LOS. It is conceivable that these uses of AI may be able to identify patients at ED triage in the future who can safely be diverted to UC centers as well.
Continuous Performance Feedback—Help or Hinderance?
Take Home Point: Prioritizing person-mediated feedback yielded superior outcomes compared to computer-mediated feedback in terms of improvements in performance, motivation, and engagement.
Citation: Giamos D, Doucet O, Léger P. Continuous Performance Feedback: Investigating the Effects of Feedback Content and Feedback Sources on Performance, Motivation to Improve Performance and Task Engagement. Journal Of Organizational Behavior Management. https://doi.org/10.1080/01608061.2023.2238029
Relevance: Performance, motivation, and engagement are critical aspects of effective team membership in UC. Feedback may be delivered in a variety of ways; it’s important for managers and team leaders to understand how deliver feedback to affect positive changes in these domains. This is especially important as poorly delivered feedback may be counterproductive and demoralizing, risking retention of staff.
Study Summary: This study was conducted to ascertain the effects of feedback content (ie, quantitative vs qualitative) and feedback source (ie, computer vs in-person), on subjects’ task performance, motivation to improve, and task engagement in the context of continuous performance feedback. The 36 participants aged 18-41 years (mean age = 24 years) were divided into 2 groups: quantitative feedback; or qualitative feedback. Participants in the quantitative feedback group received a numerical rating based on their performance in various tasks, those in the qualitative group did not receive any. All participants received qualitative feedback from different sources: a computer (ie, pop-ups), or a person (ie,, verbally delivered from a person).
The authors found that feedback content has a positive effect on performance, with participants in the quantitative group performing better than those who only received qualitative feedback. Secondly, participants had higher levels of performance, motivation to improve, and task engagement when they received continuous performance feedback from a person rather than a computer. Person-mediated feedback may have had a more powerful effect on outcomes because it is accompanied by affective and social cues.
Editor’s Comments: This study has many limitations including small sample size, limited diversity of participants, particularly in age, and simulated nature of the tasks. The study also only rated performance based on a single dimension of cognition, working memory and its setting in a laboratory is unable to recreate real-world work environments. However, the data do suggest that providing a combination of both quantitative feedback (eg, antibiotic prescribing rates, patients per hour, net promoter score) and qualitative feedback (eg, “patients don’t feel you explain their diagnosis clearly”) would likely be most effective. Furthermore, providing this feedback verbally rather than via e-mail or other digital communication seemed to improve engagement and motivation—2 highly desirable outcomes for managers. It is usually easier on supervisors, especially if overseeing a large and geographically dispersed staff, to provide feedback digitally. This paper suggests this convenience may come at significant cost and undermine the fundamental goals of providing feedback.
Can a Simple Nasal Spray Hold the Key to Curing the Common Cold?
Take Home Point: In this large UK based study, participants using nasal sprays or gels for respiratory illness treatment and prevention had shorter courses of illness and less antibiotic use. There was no difference in the frequency of respiratory infections (URI) between either nasal formulation or the control group. However, subjects randomized to receive health lifestyle education did have slightly fewer respiratory infections than the other groups.
Citation: Little P, Vennik J, Rumsby K, et. al. Nasal sprays and behavioral interventions compared with usual care for acute respiratory illness in primary care: a randomized, controlled, open-label, parallel-group trial. Lancet Respir Med. 2024 Aug;12(8):619-632. doi: 10.1016/S2213-2600(24)00140-1.
Relevance: Effective, low-cost, non-prescription interventions which can prevent or shorten the duration of symptoms of viral respiratory could have tremendous impact on population-level health metrics like healthcare utilization and missed workdays. There is some evidence that modifying the nasal environment which may hold promise in altering susceptibility to and recovery from viral URIs.
Study Summary: This was a randomized, controlled, open-label, parallel-group trial in UK based primary care study. Patients were included from both large and small general practice (GP) settings in both urban and rural locales. Participants were randomized to blocks of 4 trial groups (1:1:1:1). The 4 intervention groups were: usual care (ie, control group); gel-based nasal spray; saline nasal spray; or a digital intervention composed of educational content through a website which promoted physical activity and stress management. The gel-based spray was Vicks First Defense spray (Proctor and Gamble, Harrogate, UK), which contains a polymer and buffers nasal pH. The saline spray was Sterinase (Earol, Glasgow, UK), which had the method of delivery (a pump-action spray) identical to that of the gel-based spray without potential active excipients (eg, zinc or copper). The behavioral group had access to a health and stress management website and were provided with pedometers to help monitor activity.
The 332 GP practices participated over a 6-month period. From these practices, more than 13,00 participants with at least one co-morbidity were randomly assigned to either usual care (n=3,451), gel-based nasal spray (n=3,448), saline-based nasal spray (n=3,450), or the behavioral website intervention (n=3,450). The primary outcome was days of respiratory illness over a 6-month period. The investigators also compared proportion of patients who had a URI during the study, missed workdays, rates of adverse reactions, and rates of antibiotic use. The results were tracked by surveying the participants at the end of each month and again at the end of the entire 6-month study period. The participants were instructed to use the nasal spray at the first sign of illness or if they were concerned that they could have been exposed to someone with a contagious respiratory illness.
The authors found small but significant differences between groups in several measured outcomes. The control group and healthy lifestyle education groups had no significant difference in symptomatic days over the study period (8.2 vs. 7.4). The gel based nasal spray group had a significantly lower number of symptomatic days (6.4 days) as did the saline nasal spray group (6.5 days). Additionally, duration of illness in those who did have a URI was significantly lower in both the gel group (12.0 days) and saline nasal spray group (11.8 days) than the usual care group (15.1 days). Both nasal spray treatment groups also had small reductions in the proportion with prolonged respiratory illness (>2 weeks) and conversely slightly higher percentage with respiratory illnesses lasting <1 week.
The number of workdays lost was low in all groups (ie, <1 day on average) but slightly lower in the nasal spray groups. Both of the nasal spray intervention groups had significantly lower risk of receiving antibiotics during the study period (IRR=0.65 for the saline group and IRR=0.69 for the gel-based group. The group given access to exercise and stress reduction education interestingly also had significantly lower antibiotic use than the control group. Very low rates of adverse events were reported in any group, with the most common being headache reported by 7.8% of those patients using the nasal gel vs. 4.8% in the control group and 4.5% in the saline nasal spray group.
Editor’s Comments: There’s much to be said about this paper, and it’s a complex enough study on perhaps the most common issue we face in UC that it’s worth a detailed read. The use of randomization does limit the possibility of the many confounders that might introduce bias, however, there were differences in survey response rates between groups worth noting. Perhaps the most significant source of bias and questionably reliable data comes in the form of recall bias. Participants were asked monthly to recall their symptomatic days, which is difficult to have confidence in the accuracy of such an assessment. Furthermore, the similarity between the nasal gel and saline spray in many regards suggests much of the difference may be related to placebo effect rather than a specific mechanism of the sprays. Participants randomized to groups other than the control group may have been more inclined to take other measures to prevent or shorten duration of illness (eg, washing their hands more often after a possible exposure to a virus).
Perhaps most noteworthy is the observation that both nasal spray groups and the healthy lifestyle education group used between 25-35% fewer antibiotics during the study period. This points to the importance of patients who are sick feeling that they are actively doing something (or taking something) they believe will address their illness. The patients who were randomized to the nasal spray groups and healthy lifestyle education groups sought medical care less and reported less belief in the value of antibiotics for their illnesses. These are perhaps the most salient findings. Equipping patients with benign interventions which may help to shorten and reduce severity of symptoms and specific education about maintaining healthy habits may be largely a placebo effect, but this is a less risky placebo to provide for simple URIs than a prescription for non-indicated antibiotics. Consider using the data from this study with your patients to advocate for a nasal spray in lieu of an antibiotic. Afterall, you can point to this study demonstrating its efficacy–because the patient’s belief in your prescription and recommendations is likely what matters most.
Are Physician’s Still the Best at Early Diagnosis of Sepsis?
Take Home Point: In this study, physician gestalt during the first 15 minutes after arrival of ED patients outperformed usual screening tools and an AI tool in identifying sepsis among critically ill, undifferentiated medical patients.
Citation: Knack S, Scott N, Driver B, et. al. Early Physician Gestalt Versus Usual Screening Tools for the Prediction of Sepsis in Critically Ill Emergency Patients. Ann Emerg Med. 2024 Mar 25: S0196-0644(24)00099-4. doi: 10.1016/j.annemergmed.2024.02.009
Relevance: Early identification and treatment of sepsis improves clinical outcomes. There are several scoring tools used to identify patients with possible sepsis which have been used over the years. The relative accuracy of these tools compared to clinician gestalt is a topic of frequent debate, especially given the increasing prevalence of pop-up alerts and alarms from the electronic medical record (EMR) intending to warn about the possibility of sepsis.
Study Summary: This was a single center ED based prospective study, based in an academic urban hospital in Minneapolis, Minnesota. The authors recorded the initial treating physician’s ST-elevation myocardial infarction for sepsis in patients and compared them with screening tools, including SIRS, qSOFA, SOFA, and MEWS, which were calculated retrospectively. Additionally, a machine learning model for variable selection called Least Absolute Shrinkage and Selection Operator (LASSO) was also compared. The clinicians were physicians or senior residents who completed a visual analog scale (VAS) response indicating their suspicion of sepsis at ≤15 minutes and then again at 60 (+/-15) minutes after the patient presented to the ED.
The authors identified 275 patients (11%) out of the 2,484 patients screened who ultimately had a discharge diagnosis of sepsis. They found that physician gestalt within 15 minutes of ED arrival outperformed other sepsis screening methods, including the AI/machine learning algorithm. Physician gestalt remained superior to all scoring systems for up to 1 hour. The area under curve (AUC) for physician gestalt was 0.90 compared to 0.84 for LASSO (the AI algorithm) and 0.67 for SIRS, SOFA and qSOFA scores.
Editor’s Comments: This was an ED based study with emergency medicine physicians. It is unclear to what extent these results would be generalizable to UC and non-physician clinicians. Initial assessment of potentially critically ill patients relies on many pieces of data which are difficult to quantify (eg, the degree to which a patient appears acutely ill or toxic). Given the limited objective data points available in the first hour of patient presentation, it is unsurprising that an experienced emergency physician outperforms algorithms that rely on qualitative data points. This study suggests that in an ED setting physician judgment should trump score-based predictions of sepsis when it comes to treatment decisions. Further research about the ability of non-physician and UC clinicians in comparison to these models would be helpful to determine if such models may improve early recognition of sepsis in UC settings.
Ivan Koay MBChB, MRCS, FRNZCUC, MD, is an Urgent Care Physician and Medical Lead for Kings College Hospital Urgent Treatment Centre, London, United Kingdom. He is also the Convenor for the Ireland and UK Faculty of the Royal New Zealand College of Urgent Care.
Download the article PDF: Abstracts in Urgent Care – October 2024
