Can Large Language Models Help in Assessing Acuity of Patients Presenting to ED?
Take Home Point: Integration of large language models (LLMs) in the emergency department (ED) could enhance triage processes. This warrants further investigation particularly in the urgent care (UC) space.
Citation: Williams C, Zack T, Miao B, et. al. Use of a Large Language Model to Assess Clinical Acuity of Adults in the Emergency Department. JAMA Netw Open. 2024 May 1;7(5): e248895. doi: 10.1001/jamanetworkopen.2024.8895. PMID: 38713466; PMCID: PMC11077390.
Relevance: As we start to investigate the applications of artificial intelligence (AI) and LLMs specifically, it is useful to consider how they can improve efficiency without compromising patient safety.
Study Summary: This was a cross-sectional study using clinical details of all adults visiting the University of California, San Francisco (UCSF), ED with a documented Emergency Severity Index (ESI) acuity level (1-5) and a corresponding ED physician note created during the encounter. The authors queried chatGPT-4 (OpenAI model) to consider the clinical history of sets of 2 ED presentations and return a result to decide which patient in the pair had a higher-acuity presentation.
A sample of 10,000 pairs of presentations were compared and the authors found the LLM correctly inferred the higher acuity patient for 8,940 of 10,000 pairs, with an accuracy of 89% (95% confidence interval [CI], 0.89-0.90). Model performance improved as ED triage acuity scores became more extreme between pairs, with up to 100% accuracy when distinguishing between patients with immediate versus non-urgent acuity levels. Of note, a 500-patient visit subset was analyzed by both the LLM and a group of emergency physician participants. Between the two, the LLM’s accuracy (88%) was comparable and not statistically significantly different than that of the physicians (86%). The LLM’s only significant performance weakness identified was in distinguishing patients assigned a less urgent versus non-urgent acuity.
Editor’s Comments: There was no accounting for potential deterioration of patients within this study, an issue that is pertinent in the dynamic nature of ED/UC. Due to the pairwise nature of the study, it was not possible to reliably calculate model performance across different patient characteristics such as gender, race, and ethnicity. As most UC centers in the U.S. do not use ESI or similar triage or have nurses or other clinical staff for whom formal triage is within their scope of practice, the use of LLM holds great promise for triaging UC patients on arrival when this is beyond the scope of the front desk staff.
Improving Staff Engagement and Retention
Take Home Point: Working culture, physical working environment, pathways to care, and supportiveness of leadership represent the core areas of concern for ED workforce development to improve staff engagement and worker retention.
Citation: Daniels J, Robinson E, Jenkinson E, et al. Perceived barriers and opportunities to improve working conditions and staff retention in emergency departments: a qualitative study. Emerg Med J. 2024; 41:257–265
Relevance: Through the pandemic and beyond, record numbers of healthcare professionals have left clinical practice, with emergency medicine (EM) being the most affected specialty. UC has also faced severe staffing shortages, likely for similar reasons. Identifying the pain points for clinicians will enable organizations to improve retention that allows for ongoing quality of care provision. Slowing turnover is important not only to ensure ongoing UC center function, but also to mitigate stress for remaining staff associated with high rates of turnover.
Study Summary: This was a qualitative study involving online focus groups with ED staff (physicians, nurses, advanced care practitioners) of all levels of experience and professional backgrounds from across the United Kingdom (UK) to gain understanding of participant perspectives and views. Profession-specific focus group interviews were conducted online using a semi-structured topic guide with the contents recorded, transcribed, and stored securely. Directive content analysis was applied to the data to identify common themes from participant responses, using deductive codes to identify key concepts.
The authors used data from 33 participants of the initial 116 clinical staff who completed the eligibility consent form and survey. Four key themes were identified which included: “culture of blame and negativity,” “untenable working environments,” “compromised leadership” and “striving for support.” These issues were perceived to play a disproportionately influential role in participants’ ability to find their work sustainable. These also were the factors that most influenced their well-being and, importantly, their intention to leave. Leadership behavior and attitudes have a highly influential role across these themes and is unequivocally vital to workforce transformation; however, this is an area that has been largely neglected.
Editor’s Comments: The small size and low proportion of subjects completing the survey limits the data’s generalizability. The majority of the participants were female and Caucasian, with views of males and people of color less represented. The focus of the study on EM clinicians in the UK also limits generalizability to other healthcare systems and nations with alternate healthcare delivery structures. Regardless, there were common themes that emerged, and UC administrators would be wise to be mindful of the work environment of their centers if they truly hope to mitigate clinician burnout and turnover.
Is 4.5 hours from last known well time the appropriate upper limit for thrombolysis in patients with Ischemic Stroke?
Take Home Point: In this study, treating patients with large vessel occlusive ischemic strokes (CVA) between 4.5 and 24 hours of symptom onset with tenecteplase was shown to improve disability-free recovery, but resulted in higher rates of symptomatic intracranial hemorrhage (ICH).
Citation: Xiong Y, Campbell B, Schwamm L, et. al. Tenecteplase for Ischemic Stroke at 4.5 to 24 Hours without Thrombectomy. N Engl J Med. 2024 Jun 14. doi: 10.1056/NEJMoa2402980. PMID: 38884324.
Relevance: The present American Heart Association (AHA) guidelines for treatment of ischemic strokes are for the use of systemic thrombolytics (i.e. tPA) in patients without contraindications who were last observed to be at their baseline within 4.5 hours. Many patients present in a somewhat delayed fashion, limiting treatment options to mitigate long-term disability from ischemic CVA.
Study Summary: This was a phase 3, multicenter, prospective, open-label, randomized, blinded-outcome-assessment trial at 58 centers in China. Adult patients >18 years old who had stroke—including stroke on awakening and unwitnessed stroke—were recruited within 4.5 to 24 hours after the time that they were last known to be at their baseline. Patients were randomly assigned in a 1:1 ratio to receive systemic intravenous (IV) tenecteplase or standard medical treatment (control). The IV tenecteplase group received a bolus administered over a period 5 to 10 seconds at a dose of 0.25 mg per kilogram (maximum dose, 25 mg) immediately after randomization. The control group received antiplatelet therapy (i.e., standard medical treatment) at the discretion of the investigators. The primary outcome was the absence of disability (defined as a score of 0 or 1 on the modified Rankin scale) at 90 days.
The authors recruited 516 patients into the trial; 264 were assigned to receive tenecteplase, and 252 to receive standard medical treatment. They found percentage of patients who had no disability at 90 days was 33.0% in the tenecteplase group as compared with 24.2% in the standard-treatment group (relative rate, 1.37; 95% confidence interval [CI], 1.04 to 1.81; P=0.03). Symptomatic intracranial hemorrhage within 36 hours after treatment occurred in 8 patients (3.0%) in the tenecteplase group and in 2 patients (0.8%) in the standard-treatment group (relative rate, 3.82; 95% CI, 0.82 to 17.87). The incidence of other adverse events and serious adverse events did not differ substantially between the 2 groups. Four patients in the Tenecteplase group and five patients in the control group also underwent endovascular retrieval procedures.
Editor’s Comments: This study has a number of limitations and is not directly relevant to care provided in UC centers, however, it is important for UC clinicians to be aware of changes in how acute strokes may be treated. Among the limitations, this study was conducted in China where ischemic CVA is more often thrombotic rather than embolic (i.e. related to atrial fibrillation and/or structural heart disease). The window of 4.5 – 24 hours is large. It is unclear if patients who benefited received tPA at hour 5 or hour 23 from the data. Furthermore, these were only large vessel occlusive (LVO) ischemic CVA, which is the minority of cases. Many recent studies have shown that patients with LVO CVA often benefit from early endovascular intervention (i.e. “clot retrieval”), however, this not used as a comparator in this study. The important message for UC clinicians is really that the new meaningful “last known well” time that should prompt immediate ambulance transport for suspected stroke patients is now 24 hours. Additionally, it is important for UC clinicians to be aware of which hospitals in their area, if any, may offer endovascular interventions to ensure that patients who may potentially be candidates for endovascular intervention are referred to the most capable local facility.
Are Vital Sign Measurements Subject to Bias?
Take Home Point: While thought of as objective, vital signs data are affected by human factors (i.e., bias) and these biases may impact the care patients receive.
Citation: Kleinig O, To M, Ovenden C, et. al. Vital sign measurements demonstrate terminal digit bias and boundary effects. Emerg Med Australas. 2024 Feb 27. doi: 10.1111/1742-6723.14395.
Relevance: Vital signs are important data points for healthcare practitioners. However, the reliability of vitals depends on the practices of the humans recording them and the fidelity/agreement between actual recorded values and the values measured.
Study Summary: This was a retrospective cohort study of patients admitted to general medicine and acute medical units at a tertiary hospital in South Australia. All recorded values for selected vital signs (heart rate [HR], respiratory rate [RR], oxygen saturation [SpO2], and systolic blood pressure [SBP]) were collected from electronic medical records (EMR) over a 2-year period. The most common methods for recording vital signs in the hospital were observation for RR, non-invasive automatic blood pressure cuffs for SBP and HR, and associated pulse oximetry monitors for SpO2. Polynomial regression was used to determine underreporting of out-of-range (i.e., abnormal) values and overreporting of values ending in 0, 2, or 5.
Records for 15,734 individual visits were included in the study, including 11,746 unique patients. The authors noted a total of 749,941 HR, 644,600 RR, 757,726 SpO2 and 572,515 SBP measurements were recorded. They found HRs of 60 (P < 0.001) and 99 (P < 0.001)—each at the boundaries of the “normal” range—were over-recorded. Even numbers were 26.1% more likely to be recorded than odd numbers (P < 0.001). RR measurements demonstrated no boundary effects, including at 20, the upper boundary of normal (P > 0.1). SpO2 demonstrated a boundary effect at 95% (P = 0.003), corresponding to the lowest possible number prior to the threshold for escalation of monitoring. SBP recordings demonstrated terminal digit biases and boundary effects. Even numbers were 18.8% overrepresented (P < 0.001), and multiples of 5 were 34.9% overrepresented (P < 0.001).
Editor’s Comments: The study was conducted at a single center, therefore it is likely that the biases observed are contributed to by local factors and hospital policies. The study did not evaluate the clinical impact of these biases, nor did it assess whether clinicians may have been conscious of their behaviors. The study does highlight the human factors behind recording data and the implications of arbitrary vital sign cutoffs which compel certain actions. For example, in the hospital in the study, an SpO2 <95% mandated increased monitoring of patients which is burdensome for patients and clinicians. This speaks to the unexpected consequences of clinical policies which compel extra work for clinicians without rational justification.
This Afebrile Infant Has a Rash – Now What?
Take Home Point: In afebrile infants with pustules and/or vesicles, noninfectious etiologies were diagnosed two-thirds of the time and infection one-third of the time. Most of the infections were superficial and herpes simplex virus (HSV) was the culprit in <10% of cases.
Citation: Yun S, Cotton C, Faith EF, et al. Management of Pustules and Vesicles in Afebrile Infants ≤60 Days Evaluated by Dermatology. Pediatrics. 2024;154(1): e2023064364
Relevance: Vesicles and pustules are common in neonates and young infants. There are no specific evidence-based guidelines guiding the work-up of afebrile infants with this issue.
Study Summary: This was a multicenter, retrospective cohort study using data obtained from the electronic medical records (EMR) of children ≤60 days of age who received a pediatric dermatology consultation at 1 of 6 academic pediatric centers across the US. Afebrile infants who had skin lesions documented as pustules, vesicles, and/or bullae on manual review of medical records were included. Serious bacterial infection (SBI) was defined as bacteremia, urinary tract infection, or meningitis.
The authors identified 183 patients from their review. 73% of the patients were born at full-term. Of the 183 patients, 124 (67.8%) infants presented with pustules; 57 (31.1%) with vesicles; and 19 (10.4%) with bullae. Lesions most commonly were identified on the head (113 patients or 61.7%). In the EMR data, 83 (45.3%) patients had lesions on the trunk, 80 (43.7%) on the extremities, 47 (25.7%) in the diaper area, and 22 (12.0%) on skin folds. Also, 95 subjects (51.9%) had more than 1 affected site.
Forty of the 183 patients were evaluated by dermatology in the ED and 21 of these patients (52.5%) were admitted to the hospital. Seventy-one (38.8%) infants had infections etiologies and 122 (66.6%) non-infectious etiologies. Among the non-infectious etiologies, neonatal cephalic pustulosis was the most common (36 cases) with erythema toxicum neonatorum (18 cases), and irritant contact dermatitis (11 cases) being the other most common non-infectious diagnoses. No patient in this cohort was found to have an SBI was detected that could be attributed to a skin source. No cerebrospinal fluid (CSF) culture nor blood culture returned a pathogen. Among the infectious diagnoses, superficial gram-positive infections (35 cases) were the most common etiology. Nine of the 127 infants evaluated for HSV (7.1%) had positive confirmatory testing.
Editor’s Comments: This study has many important limitations. Notably, this was a retrospective study of infants <60 days of age who were afebrile and seen by a pediatric dermatologist in a tertiary care hospital. This would likely bias towards these subjects having more concerning rashes and/or the study population having more significant underlying medical issues than an average UC patient. It was assumed that if the patient did not return to the participating institution after discharge, they did not subsequently develop an SBI, herpes simplex virus infection, or other disseminated infection. Finally, it is critical to recognize that patients with measured or reported fever were excluded. This data offers some reassurance for the evaluation of younger afebrile infants that no patients in this cohort had CSF or bloodstream infection, however, should not be applied if there is a measured or reported fever.
Is Blood on a Urine Dip Useful in Risk Stratifying Pediatric Blunt Abdominal Trauma?
Take Home Point: Microscopic hematuria did not prove clinically useful as a marker for differentiating clinically important intra-abdominal injuries (ci-IAI) in this retrospective pediatric blunt abdominal trauma study.
Citation: Papillon S, Pennell C, Bauer S, et. al. Presence of Microscopic Hematuria Does Not Predict Clinically Important Intra-Abdominal Injury in Children. Pediatric Emergency Care. Publish Ahead of Print. doi: 10.1097/PEC.0000000000003210
Relevance: Traditional teaching and dogma in pediatric blunt abdominal trauma have suggested CT imaging is warranted if there is any degree of hematuria in children with blunt abdominal trauma. However, data from prior studies have called into question the significance of asymptomatic microscopic hematuria as its presence in isolation rarely is associated with injuries requiring intervention.
Study Summary: This was a retrospective chart review among children presenting with a blunt abdominal mechanism of injury at a single, level I pediatric trauma center in the United States. Data collected included patient demographic information, mechanism of injury, clinical symptoms, vital signs at presentation, physical examination findings, laboratory data, injury, and radiographic findings. The primary outcome was a composite end point termed “clinically important intra-abdominal injury” (ci-IAI) which occurred if patients required ≥2 nights admission, received blood or blood products, or required therapeutic angioembolization or surgery.
The authors identified 240 patients who presented with blunt abdominal trauma during the study period, of which 165 patients had complete accompanying urinalysis (UA) for the same visit. 45 patients had microscopic hematuria and 120 had a normal UA. Three patients with normal UA had ci-IAI, while 2 patients with microscopic hematuria had ci-IAI. The authors found that urinalysis as an added test for IAI resulted in many more false positives without identifying ci-IAI. All children with ci-IAI had either abnormal findings on physical exam and/or abnormal liver function tests (LFT) or pancreatic enzymes. No child with a ci-IAI had isolated microscopic hematuria.
Editor’s Comments: Although this study has interesting results, it has limited generalizability due to the small sample size and being conducted in a tertiary pediatric trauma setting. It is likely, for example, that these patients were much more significantly injured than would be expected to present to an UC center. While this does not offer definitive evidence of the inutility of UA for screening for renal injury after blunt trauma in children, the fact that no patient with ci-IAI had isolated microscopic hematuria coupled with the fact that over 25% of all patients had microscopic hematuria suggests that use of urine dip for risk stratification is much more likely to beget additional work-up without additional benefit.
COVID-19 Abstract
Attempting to Understand the Molecular Basis of COVID-19 Infection
Take Home Point: This mechanistic study provides insights into the dynamics of immune responses to exposure of the SARS-CoV-2 virus in previously unvaccinated and uninfected individuals. Interestingly, nearly half of the individuals exposed to SARS-CoV-2 who had no evidence of immunity still did not develop clinical or laboratory evidence of infection.
Citation: Lindeboom R, Worlock K, Dratva L, et. al. Human SARS-CoV-2 challenge uncovers local and systemic response dynamics. Nature. 2024 Jul;631(8019):189-198. doi: 10.1038/s41586-024-07575-x
Relevance: There remain ongoing key questions as to why certain individuals got infected with COVID-19, some more severe than others, while others did not.
Study Summary: This was a human SARS-CoV-2 challenge study of previously young adults who were seronegative for previous COVID infections. These healthy volunteers were intranasally inoculated with a wild-type pre-Alpha SARS-CoV-2 virus strain (SARS-CoV-2/human/GBR/484861/2020) in a controlled environment.
Following inoculation, 6 participants from the cohort developed a sustained infection as defined by at least 2 consecutive quantifiable viral load detections by nasal and/or throat PCR along with symptoms, 3 produced multiple sporadic and borderline-positive polymerase chain reaction (PCR) tests between day 1.5 and day 7 after inoculation, and 7 remained PCR-negative throughout the quarantine period, which indicated that these individuals successfully prevented the onset of a sustained or transient infection. In sustained infections, the authors observed global activation of interferon signaling that affected all circulating immune cells. There were higher levels of the protein HLA-DQA2 in multiple lineages of immune cells called antigen-presenting cells, both in the nasal mucosa and in blood in people who had transient or abortive infections. This suggests a non-typical role of this MHC II molecule in innate resistance to COVID-19 infection.
Editor’s Comments: This was a small sample size mechanistic study with little direct clinical utility. These immunology and molecular biology studies do, however, provide valuable insights into the intricacies and complexity of determining which exposed individuals develop clinical disease and how their immune systems respond.
Ivan Koay MBChB, MRCS, FRNZCUC, MD, is an Urgent Care Physician and Medical Lead Kings College Hospital Urgent Treatment Centre, London; Convenor Ireland and UK Faculty of the Royal New Zealand College of Urgent Care; and Independent Assessor European Reference Network, Andalusian Agency for Healthcare Quality.
