Changes in psychiatric documentation and treatment in primary care with artificial intelligence scribe use

Importance

Despite increasingly widespread use of artificial intelligence-driven ambient scribes in medicine, the extent to which they may impact clinician practice is not well-studied.

Objective

To characterize differences in documentation and treatment of psychiatric symptoms in primary care outpatient notes generated using ambient scribes.

Design

Case-control electronic health records

Setting

Primary care annual visit notes from the Massachusetts General and Brigham and Women’s Hospital systems between February 2023 and February 2023.

Participants

Random sample of 20,302 notes from 4 types of visits, matching 1:1 using sociodemographic and clinical features: those using an ambient scribe, those using a human scribe, those occurring during the same period without a scribe, and those occurring prior to scribe deployment

Exposure

Use of an artificial intelligence-driven ambient scribe

Main Outcome and Measures

Neuropsychiatric symptom documentation, in terms of estimated Research Domain Criteria, using a HIPAA-compliant large language model (GPT4o; gpt-4o-11-20); incident antidepressant prescriptions and diagnostic codes; referral for mental health follow-up.

Results

In the ambient scribe group, mean age was 48 (SD 14) years; 59% of notes reflected individuals of female sex, and 5.0% met criteria for moderate or greater depressive symptoms by PHQ-9. Estimated levels of RDoC symptomatology in all 6 domains were significantly greater in the ambient-scribed notes (p <.001 for all contrasts). In a logistic regression model, likelihood of a psychiatric intervention (referral, new diagnosis, or antidepressant prescription) was significantly lower among ambient-scribed visits compared to unscribed (aOR 0.83, 95% CI 0.72-0.95), but not for human-scribed compared to unscribed (aOR 1.01, 95% CI 0.87-1.17).

Conclusion and Relevance

In this case-control design examining outpatient primary care notes, we found that incorporation of artificial intelligence-driven ambient scribes in primary care was associated with greater levels of neuropsychiatric symptom documentation but lesser likelihood of acting on psychiatric symptoms. Further study will be required to determine whether these changes are associated with differential outcomes.

Trial registration

n/a

Key Points