Yale researchers’ machine-learning model identifies contributors to physician departure, which could help improve job satisfaction and stem costly turnover.
Physician turnover is disruptive to patients and costly to healthcare facilities and physicians alike. In a new study, Yale researchers used machine learning to reveal the factors — including the length of a physician’s tenure, their age, and the complexity of their cases — that can increase the risks of such turnover.
And, evaluating data from a large U.S. healthcare system over a nearly three-year period, they were able to predict, with 97% accuracy, the chances of physician departure.
The findings, researchers say, provide insights that can help healthcare systems intervene before physicians decide to leave in order to reduce turnover. The study was published in PLOS ONE.
While healthcare facilities typically use surveys to track physician burnout and job satisfaction, the new study used data from electronic health records (EHRs), which are used by the majority of U.S. physicians to track and manage patient information.
The problem with surveys, said Ted Melnick, associate professor of emergency medicine and co-senior author of the new study, is that physicians often feel burdened to respond. Consequently, response rates are often low. “And surveys can tell you what’s happening at that moment,” he added, “but not what’s happening the next day, the next month, or over the following year.”
Electronic health records, however, which in addition to collecting clinical patient data also generate work-related data continuously, offer an opportunity to observe physician behavioral patterns moment to moment and over long periods of time.
For the new study, the researchers used three years of de-identified EHR and physician data from a large New England healthcare system to determine whether they could take a three-month stretch of data and predict the likelihood of a physician’s departure within the following six months.
Specifically, data were collected monthly from 319 physicians representing 26 medical specialties over a 34-month period. Data included how much time physicians spent using EHRs; clinical productivity measures, such as patient volume and physician demand; and physician characteristics, including age and length of employment. Different portions of the data were used to train, validate, and test the machine learning model.
When tested, the model was able to predict whether a physician would depart with 97% accuracy, the researchers found. The sensitivity and specificity of the model, which show the proportion of the departure and non-departure months that were correctly classified were 64% and 79%, respectively. The model was also able to identify how strongly different variables contributed to turnover risk, how variables interacted with each other, and what variables changed when a physician transitioned from low risk of departure to high risk.
The details about what’s driving the prediction is what makes this approach particularly useful, researchers said. Through their approach, the researchers identified several variables that contributed to departure risk; the top four factors, they found, were how long the physician had been employed, their age, the complexity of their cases, and the demand for their services.
Whereas previous work enabled only analyses of linear relationships, the machine-learning model allowed researchers to observe the challenges facing physicians with more nuance. For example, risk of departure was highest for physicians more recently hired and those with longer tenures but lower for those with middling tenure lengths. Similarly, risk of departure was higher for those up to 44 years old, lower for physicians aged 45 to 64, and higher again for those 65 years old or older.
There were also interactions between variables. For instance, more time spent on EHR activities decreased the risk of departure for physicians who had been on the job for less than 10 years. But for those physicians employed longer, it increased the risk of departure.
The risk of physician departure shifted throughout the study period, which covered a 34-month span from 2018 to 2021 (a period that included the pandemic and the pre-pandemic world), researchers said. They also identified specific variables that changed when a physician switched from low to high risk of departure; the proportion of EHR inbox messages responded to by a team member other than the physician, physician demand, and patient volume, were the variables that changed the most when a physician’s risk flipped from low to high. COVID-19 waves were also linked to change in departure risk.
