If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
COVID redeployments resulted in short- but not long-term perception of skill decay.
•
Decreased case volumes had little long-term impact on trainee perception of skill.
•
Training interruptions are inevitable and may not impair competency attainment.
Abstract
Introduction
The COVID-19 pandemic immediately interrupted procedural training. The lasting impact of reduced caseloads and service redeployments on procedural-resident training has been underexplored. This longitudinal study investigated the long-term perspectives of skill decay after short breaks in training and implications for ensuring resident competency attainment.
Methods
Web-based cross-sectional surveys distributed immediately after (June 2020) compared to 1 y after (July 2021) COVID-19 redeployments at two tertiary academic medical centers of an integrated health system in New York. Participants included general surgery, surgical subspecialty, and anesthesiology residents and faculty.
Results
Fifty-five residents and 33 faculty completed the survey. Ninety-point nine percent of residents and 36.4% of faculty were redeployed to COVID-ICUs. Sixty-three-point seven percent of residents and 75.0% of faculty reported a reduction in resident technical skills in the short-term, with significantly less (45.5% of residents and 21.2% of faculty) reporting persistent reduction in technical skill after 1 y (P = 0.001, P < 0.001). Seventy-five percent of residents and 100% of faculty were confident residents would be able to practice independently at the conclusion of their training. Sixty-five-point five percent of residents and 63.6% of faculty felt that residents experienced a durable improvement in critical care skills. Residents also reported a positive long-term impact on professional core competencies at 1 y.
Conclusions
Longitudinal surveillance of residents after COVID-19 redeployments suggests washout of temporary skill decay and return of resident confidence upon resumption of traditional training. This may provide insight into the impact of other short-term training interruptions on resident skill and promote greater resident support upon training resumption to ensure competency attainment.
The SARS-CoV-2 (COVID-19) pandemic greatly impacted surgery and anesthesiology training. The New York metropolitan area was the epicenter during the initial stage of the pandemic. Many hospital operations were impacted, elective operations were postponed, and residents and faculty were redeployed. By August 2020, the Northwell Health system had treated more COVID-19 patients than any other healthcare system in the country.
Such shift in patient care altered training for procedural-based residents who depend on case numbers and skill development to achieve competency for graduation.
Skill decay is defined as the loss of trained skills or knowledge over an interval period of nonuse.
Whereas skill decay has been deeply studied in other vocations like aeronautics and firefighting, it remains relatively underexplored in surgical specialties.
It is accepted that technical skills may diminish during extended interruptions in operative training, such as years dedicated to research or academic development during residency.
However, studies addressing the impact of shorter interruptions on surgery and anesthesiology training and the subsequent long-term implications are sparse.
Our group previously published that as a result of redeployments to COVID-ICUs, residents and faculty perceived resident technical skill decay due to the interruption in their training; however, residents and faculty noted improvements in other areas, such as their ability to provide critical care.
The aim of this study is to evaluate the perceptions of a short-term interruption in procedural-based training on long-term skill decay and professional competencies in surgery and anesthesiology residents. Although short-term skill decay has been described, the lasting impact reduced caseloads and service redeployments have on resident training is unknown.
Understanding not only the perception of resident skill decay, but also the perceived impact on other professional competencies after 1 y will help inform residency program leadership on the unique needs of residents facing training interruptions moving forward.
Methods
In response to the rapid rise in COVID-19 cases, on March 15, 2020, New York State declared a moratorium on elective operations. General surgery, surgical-subspecialties and anesthesiology residency program directors at Northwell Health assembled a “Multidisciplinary Anesthesia/Surgical Trainees COVID Response Team (MASC)” to coordinate the structured redeployment of procedural residents to COVID-ICUs as previously described.
In June 2020, elective cases were resumed, redeployed residents were repatriated with their primary training services, and a postdeployment survey coinciding with the end of the academic year (June 2020) was conducted. After this initial redeployment, residents were not subsequently redeployed during future COVID-19 surges due to viral variants and remained in their usual clinical training environment throughout the study period. In July 2021, an IRB-exempt, 1-year follow-up survey was distributed to the same cohort of general surgery, surgical subspecialty, and anesthesiology residents and faculty. Surveys were emailed via the REDCap system which permits only one response per unique resident or faculty. Surveys were optional, and survey data were deidentified and analyzed in aggregate.
The follow-up survey replicated the content and structure of the initial survey.
The survey captured demographics, training level and specialty, objective data reflecting redeployment, as well as subjective perceptions of residents and faculty regarding the impact on residency training (including technical skill, critical care skills, and Accreditation Council on Graduate Medical Education (ACGME) core competencies of residents). The surveys were worded using inclusive terminology. As such, subjective questions aimed at assessing skill decay were applicable to all surgical specialty and anesthesiology program trainees in effort to assess their perception of loss of their respective, trained skills.
Survey data were analyzed in Statistical Package for Social Sciences software (SPSS). Two-sided P-values with alpha = 0.05 were used. Distribution characteristics were tabulated using percentages for categorical variables. Paired t-tests were used for analysis of case volumes and perceptions before and after the COVID-19 pandemic. Multivariable logistic regression was performed to evaluate for significant predictors of residency impact and skill decay.
Results
Resident responses
Fifty-five residents responded for an overall response rate of 37%. Resident characteristics are shown in Table. Overall, 90.9% of residents reported redeployment. Redeployment rate did not significantly differ between residency programs and residency class year.
Thirty-three faculty completed the survey representing three residency programs and nine subspecialties. Characteristics of faculty respondents are shown in Table. The overall redeployment rate of faculty respondents was 36.4%, with the greatest redeployment being from general surgery faculty (45.8%). Faculty reported working with residents less often during the pandemic, but reported working with residents similar to usual practice after COVID-19 redeployments (Fig. 1).
Fig. 1Frequency of faculty working with residents. Faculty reports on frequency working with residents assessed prior to COVID-19 redeployments, during COVID-19 redeployments, and after COVID-19 redeployments.
Case volumes were reduced by 90% for the 2-month period of redeployment. 63.7% of residents and 75.0% of faculty reported that residents experienced short-term reduction in technical skill. Significantly fewer residents (45.5%) and faculty (21.2%) reported persistent skill decay in residents after 1 y (P = 0.001 and P < 0.001, respectively) (Fig. 2). General surgery had the highest rate of reported short-term skill reduction at 68%. Skill reduction did not significantly differ between resident covariates, including age, sex, and residency program. Residents of R2 year at the time of redeployment were significantly less likely to report short-term skill reduction (P = 0.039, hazard ratio 0.086, confidence interval 0.008-0.878) relative to more senior residents; however, residency year was not predictive of persistent skill decay at 1-year follow-up. Similarly, resident specialty was not predictive of short-term or long-term perception of skill decay.
Fig. 2Recovery of resident technical skill decay. Resident and faculty perceptions of resident technical skill decay assessed within 1 mo and 1-year after repatriation from redeployments. ∗P = 0.001, ∗∗P < 0.001.
Residents who were concerned about fulfilling ACGME case requirements (44.0%) were more likely to endorse skill decay initially (P < 0.001); however, after 1 y, significantly fewer residents (16.4%) were concerned about achieving ACGME case requirements (P < 0.001). At 1 y follow-up, concern about achieving ACGME case requirements was no longer a significant predictor of perception of skill decay. Resident concerns regarding returning to the operating room with worse skills, skill acquisition to next clinical training year, and ability to practice independently were also significantly decreased after 1 y (P = 0.044, P < 0.001, and P = 0.024, respectively) (Fig. 3).
Fig. 3Decline in resident concerns over time. Resident concerns assessed within 1 mo and 1-year after repatriation from redeployments. ∗P < 0.001, ∗∗P = 0.044, ∗∗∗P = 0.001, ∗∗∗∗P = 0.024.
At 1-year follow-up, 100% of faculty were confident residents would be able to practice independently at the conclusion of their training. Faculty confidence levels trended towards being more confident in resident technical ability after 1 y compared to immediately after the end of redeployment (Fig. 4).
Fig. 4Faculty confidence in resident ability to practice independently and perform level appropriate operations. Faculty confidence in resident abilities assessed within 1 mo and 1-year after repatriation from redeployments. (A) Faculty confidence in resident ability to practice independently upon training completion. (B) Faculty confidence in resident ability to perform level-appropriate operations and/or procedures.
On initial survey, 65.9% of residents and 94.4% of faculty felt residents critical care skills improved as a result of COVID-related redeployments. At 1-year follow-up, 65.5% of residents and 63.6% of faculty felt that residents experienced a lasting improvement in critical care skills (Fig. 5).
Fig. 5Durability of resident critical care acquisition. Resident and faculty perceptions of resident critical care skill improvement assessed within 1 mo and 1-year after repatriation from redeployments.
At 1-year follow-up, residents were more likely to reflect positively on the impact on systems-based practice (61.8%), and either reflected positively or perceived no impact on other ACGME core competencies, with very few residents perceiving a negative impact (Fig. 6).
Fig. 6Resident perceptions of long-term impact on ACGME core competencies. Resident perceptions of the impact of COVID-19 redeployments on the six ACGME core competencies, assessed 1 y after repatriation from redeployments.
Compared to the previous survey administered immediately after repatriation (June 2020), survey engagement on the follow-up survey (assessed by participation through providing optional comments) was increased. Two major themes emerged: 1. Residents noted case reduction but were not as concerned about their overall case volume. One resident noted: “At the time it seemed a little overwhelming, but since I had 3 y of training remaining after the height, overall, I didn't feel it hurt me surgically.” Another resident noted: “I missed out on approximately 100 cases based on my quarterly average. That being said, I graduated with 1900 cases.” 2: Residents and faculty appreciated the unique experience that allowed trainees to mature and gain other skills: As one resident noted: “ …It did provide a unique experience which was beneficial in other ways, though hard to comprehend at the time.” A faculty member commented: “General maturation of the surgical resident was expedited so to manage complex critical care patients and procedures they required.”
Discussion
As a result of COVID-19 redeployments, many surgery and anesthesiology residents and faculty perceived a short-term reduction in resident technical skill but improvement in critical care and ACGME core competencies. However, 1-year after repatriation and return to their usual training environments, significantly fewer residents and faculty perceived a persistent reduction in resident technical skill and reported durable improvement in critical care skills and ACGME core competencies. Initially upon returning from redeployment, residents endorsed short-term skill decay owing in large part to reduced operative case numbers during that time.
Although skill decay is a phenomenon previously used to describe the loss of trained skills due to longer periods of nonuse, we have shown a significant number of faculty and residents believed residents perceived skill decay even with a shorter period of nonuse, as was the case after COVID-19 redeployments. Despite this short-term consequence, we found that most residents were able to overcome perceived deficits to regain their skills, rebuild confidence, and ultimately progress appropriately towards their graduation goals.
Previously, skill decay has been described in the context of surgical training for residents who pursue dedicated research experiences. In these circumstances with longer training interruptions, surveyed faculty noted that residents returned to clinical training with less technical surgical skill, required more instruction, and had less confidence.
Given the importance of skill retention for chief residents, short-term training interruptions just before graduation confers additional concern for the residents’ prospective ability to practice independently. Despite the loss in operative volume of more complex procedures for senior residents, residency class year was not predictive of skill decay or concern regarding case volume at 1-year follow-up. Interestingly, R2 residents were less likely to perceive short-term skill reduction, which was likely reflective of the greater time already scheduled in ICUs during that year, and therefore perceived less of a loss in operative cases. Overall, trainees of all levels may be reassured that skill development occurs on a continuum throughout residency training for both basic and complex procedures and persists into future practice as fellows and independent attending physicians.
Interruptions in training are not infrequent, and often impart greater stress on procedural residents and training faculty when it comes to ensuring resident progression, and ultimately, resident competency. Residents may experience short-term interruptions in training due to family, personal, or medical leave. Specifically, in a multicenter study on determinants of maternity leave for all residents, the average time away from training was 6 wk, though residents in surgical fields were likely to take less than 6 wk.
Of these residents, the most significant predictor of shorter leave duration was the desire not to extend residency training, with an additional reported concern of decline in clinical skills.
In another national survey including only surgical residents, over 40% took less than 2 wk of parental leave, and over 30% did not feel supported in taking leave.
Among the biggest obstacles reported to taking leave included perceived lack of support from faculty and peers, in addition to loss of education and training time. Furthermore, surgical trainees often bear children during dedicated research time, likely reflecting their concerns regarding avoidance of training extension and potential for skill decay.
This was corroborated in a national survey of surgery program directors, where 46% supported that research years were the best time for residents to have a child during residency.
The main perceived barrier to parental leave in procedural training is the compromise to attaining clinical competency. However, consistent with others, we learned in our experience through COVID-19 redeployments that there can be more flexibility in training programs to graduate proficient surgeons in situations that afford less than 48 typical operative weeks per year.
Short-term interruptions of any kind, from redeployments to other types of leave, and the concern for skill decay thereafter highlight the current need for better metrics to assess skill decay and the lack thereof within training programs. Furthermore, there are currently no focused interventions for identifying residents at risk for skill decay due to interruptions in training, nor are there established mechanisms to aid in reversing potential decay and restoring resident confidence to take necessary leave and return to their clinical training.
The ACGME created the Milestones project to establish the definition of competency in the domains of medical knowledge, patient care, and technical skills for each specialty. Milestones have served as a useful framework for program directors to assess a trainee's progress. However, there is no objective or standardized approach to the assessment of these skills or for remediation strategy when a deficiency is identified.
Currently, objective metrics for ensuring competency for procedural residents are focused on achievement of ACGME minimum case requirements; however, there are no current studies that support that a “number of procedures” based approach necessarily equates with competency.
Although time and case requirements have historically served to determine what makes a fully trained surgeon, these metrics lack more robust validation. For example, a resident that completes the required number of cases for a particular procedure may not be competent at performing that procedure independently, whereas another resident may achieve competency with fewer repetitions. At present, there is no formal external examination of technical performance or proficiency upon completion of residency for procedural residents, and thus case requirements and assessment by the program director (and clinical competency committees) are responsible for the evaluation of skill proficiency.
Recognizing the limitation of this system, growing attention in surgical education has focused on developing methods for competency-based evaluation, such as through “Entrustable Professional Activities” or EPA's. Through EPA's, core competencies are operationalized by applying them to clinical situations and used to evaluate resident capability.
Competency-based evaluation would not only offer a more reliable mechanism for evaluating residents, especially in procedural fields, but would also be less impacted by interruptions in resident training.
An individualized approach to ensuring resident competency may afford residents greater confidence in taking necessary leaves, and subsequently returning to clinical training. More importantly, competency-based evaluation may better identify residents at risk for skill decay and guide early intervention to ensure appropriate resident progression.
Recently, the ACGME changed rules for family leave, which may contribute to greater numbers of residents affected by training interruptions, and there continues to be a push towards supporting longer parental leave in line with other nonsurgical specialties.
Training interruptions are inevitable and largely unavoidable. The American Board of Surgery (ABS) has supported flexible approaches to training leaves and recent efforts have focused on changing the paradigm of surgical training away from a strict “number of cases” or “weeks in training” approach. Effort should focus on destigmatizing training interruptions and mitigating the impact of training gaps rather than discouraging or limiting these leaves of absences.
In the United Kingdom, a coordinated effort for supporting trainees and attending physicians returning from clinical leaves led to the development of the “Supported Return to Training” (SuppoRTT) program. SuppoRTT is aimed at increasing awareness of the difficulties faced by physicians returning to work after clinical leaves, and addresses the needed cultural shift away from discouraging work leaves or asking for help.
In the United States, improvements in general surgery training culture are being investigated in the Surgical Education Culture Optimization through Targeted Interventions Based on National Comparative Data—The SECOND Trial (NCT03739723).
ClinicalTrials.gov The surgical education culture optimization through targeted interventions based on national comparative data – the SECOND trial (SECOND) NCT03739723.
During the height of the pandemic, programs in the interventional arm have relied on these resources, and proposed additional solutions such as flexible calls and parental leave arrangements to address issues related to training leaves specifically.
Importantly, the impact of training interruptions will vary based on length of leave, circumstances surrounding training interruptions, and the individual, including level of training and personal circumstances. Acknowledging that temporary skill setbacks associated with training interruptions are inevitable, focus in training must shift to destigmatizing training leaves and creating a culture of support that encourages residents along their path to achieving individual competency.
Limitations of this study are inherent to the survey-based design. Residents may perceive changes in their skill level that may be influenced by the context of their training and may not necessarily reflect true change in skill level. To combat this, faculty surveys were utilized to offset self-reporting biases and corroborate the trends in perceived resident skill. Given that the follow-up survey was conducted a year later, recall bias may confound our findings. The timing of the follow-up survey may have also contributed to lower survey response rate, as two classes of affected residents had graduated at that time; however, senior residents still comprised a reasonable percentage of survey respondents (27%). Furthermore, the washout of resident concerns after 1 y may also be partially influenced by the current state of COVID-19 at the time of follow-up survey administration, as surgical residents were experiencing operative volumes near the prepandemic state.
Conclusions
Interruptions of surgery and anesthesiology training activities during COVID-19 redeployments provided a stimulus to acknowledging that training interruptions are unavoidable components of resident reality. Given that COVID-19 continues to have an impact on residency training, and the continued potential for other interruptions in training to occur, the implications for this study are paramount for training programs moving forward. Ensuring resident competency is complex and it is influenced by multiple factors including the methods of developing technical and cognitive skills and the means for assessing these skills. Thus, surgery and anesthesiology training needs more robust and improved mechanisms for evaluating resident competency and prospective ability to practice beyond graduation. During training, interruptions will occur not only during redeployments, but also from other forms of leave. Reduced case volume during the height of COVID-19 revealed that residents and faculty perceived little long-term impact on residents despite an elevated level of concern during the time of reduced volume—implying a likely similar case for residents affected by other short-term leaves. Renewed attention to competency attainment by residency programs may help combat the stigma associated with short-term gaps in training and support professional maturation in the setting of national, institutional, or individual sources of training interruptions.
Author Contributions
All authors contributed to the conception and design of the study. Colleen Nofi and Bailey Roberts contributed to the acquisition, analysis, and interpretation of the data. All authors contributed to drafting the article, critical revision, and final approval of the manuscript.
Disclosure
The authors have no related conflicts of interest to declare. The authors report no proprietary or commercial interest in any product mentioned or concept discussed in this article.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Declaration of Interests
None.
References
Lynam T.
Health care heroes see better days ahead. Northwell Health.
00853-8&id=gr1.jpg){kind=link}
00853-8&id=gr2.jpg){kind=link}
00853-8&id=gr3.jpg){kind=link}
00853-8&id=gr4.jpg){kind=link}
00853-8&id=gr5.jpg){kind=link}
00853-8&id=gr6.jpg){kind=link}