Maternal-fetal medicine fellow experience in the first year of training compared to interview expectations based on interview modality

Abstract

Background

Virtual interviews for Maternal-Fetal Medicine (MFM) fellowship have been feasible and successful with significant cost and time savings. However, there are concerns about an applicant’s ability to accurately evaluate a program virtually. The objective of this study was to evaluate MFM fellows’ experience in their fellowship training program at the completion of their first year compared to their expectations of the program based on their interview, comparing fellows who interviewed in-person to those who interviewed virtually.

Methods

A novel cross-sectional online survey was distributed to all first-year MFM fellows in the final class to undergo in-person interviews (2021), as well as those who were the first to interview virtually (2022) at accredited programs in the United States. Participants were asked whether their interview experience provided an accurate representation of their program, whether their interview experience allowed them to accurately assess their “fit,” how their expectation of the program prior to starting compared to their actual experience in their first year of training, and their overall satisfaction. Descriptive statistics and bivariate analysis were performed to compare groups.

Results

We received 48 responses in 2021, (37.5% response rate) and 48 responses in 2022 (36.1% response rate). There was no difference between the in-person and virtual groups in agreement that the interview experience provided an accurate representation of the program (p = 0.41). While there was a trend toward more fellows from the in-person group strongly agreeing that their interview experience allowed them to accurately assess their “fit” with the program as compared to the virtual group, there was no significant difference in ability to assess fit between groups (p = 0.06). There was no difference in expectation of the program prior to starting compared to the actual experience in the first year of fellowship (p = 0.18) or overall satisfaction with the fellowship program (p = 0.95).

Conclusions

MFM fellows’ expectations of their fellowship program matched their experience in the first year of training whether the fellow interviewed virtually or in person. These data suggest that virtual interviews are as effective as in-person interviews in providing realistic expectations of the program and thus support ongoing exclusive virtual interviews in our subspecialty.

Background

In the spring of 2020, the coronavirus disease 2019 (COVID-19) pandemic triggered an abrupt transition to virtual interviews for medical education training programs, including Maternal-Fetal Medicine (MFM) fellowship programs [1]. Prior work has shown that MFM fellowship programs successfully converted to virtual interviews and the virtual interview experience was overall positive for candidates and program directors [2, 3].

Several benefits of virtual interviews for fellowship training programs have been noted including decreased cost and decreased time away from residency training for travel [4,5,6]. These benefits improve equity in interview opportunities for applicants who may have otherwise been financially constrained or limited by their residency program in allotted time away from their training program to interview. However, concerns also exist regarding the applicant’s ability to learn about the culture and community of the program and culture and livability of the city/region through a virtual platform [7].

Given the likelihood of ongoing virtual interviews in medical education, the Coalition for Physician Accountability outlined a robust research agenda to address potential concerns associated with the continuation of virtual interviewing [8]. One of these concerns was whether a fellow’s actual experience once they began at a training program aligned with their expectations based on their virtual interview experience.

The objective of this study was to evaluate MFM fellows’ experience in their fellowship training program at the completion of their first year compared to their expectations of the program based on their interview. We compared the experience of the final class of fellows who interviewed in person in 2019, prior to the pandemic, with the first class of fellows who interviewed virtually in 2020 during the pandemic. Secondarily, we sought to identify how other sources of information about the fellowship program influenced MFM fellows’ perceptions of their training program.

Methods

A novel cross-sectional online survey was developed by members of the Society for Maternal-Fetal Medicine (SMFM) Fellowship Affairs Committee. This is a committee of the SMFM whose aim is to support, expand, and foster the overall training and development of MFM fellows in the United States. It consists of approximately 20 members, who hold faculty positions at MFM training programs, and each complete a 3-year term. The survey consisted of 13 questions and was formulated in accordance with the CHERRIES checklist [9]. Questions were formatted as multiple-choice, yes/no, or a 5-point Likert scale. The survey was voluntary, anonymous, and no answers could be linked to any individual respondent or program. All content in the survey was specifically designed for this study; no content had been validated by previous research. However, members of the Committee conducted survey face validation and content validation prior to distribution.

Fellows were asked to answer all survey questions regarding their experience with the fellowship program at which they matched for MFM fellowship. In the United States, MFM fellowship candidates register and rank programs they interview at through the National Residency Program Matching Program (NRMP). Fellowship programs also do the same for candidates that they interview. The NRMP then uses a computerized algorithm to match applicants to programs that also prefer them.

The distributed survey gathered baseline data on format of interview (in-person, hybrid, or virtual), geographic region of fellowship program, program position on fellow’s NRMP rank list, and whether prior to the interview the fellow had ever lived or visited the city of their program. Using a 5-point Likert scale, participants were asked whether their interview experience provided a comprehensive and accurate representation of their fellowship program, whether their interview experience allowed them to accurately assess their “fit” with the program, how their expectation of the program prior to starting compared to their actual experience in their first year of training, and their overall satisfaction with their program. Given virtual interviews do not allow fellows to visit the city of the program, fellows were asked how their expectation of the city compared to their actual experience living in the city.

To address the impact of other sources of information beyond the interview on fellows’ perception of their program, we asked fellows to evaluate the extent that these sources influenced their impression of the program prior to beginning their fellowship: program Facebook account, program Instagram account, program Twitter account, program-made videos (YouTube or other), program/department website, supplemental materials sent from program, recommendations from residency program director or other faculty mentors, recommendations from past co-residents/current fellows, prior personal experience with the program, experience at pre/post interview social event, experience during facility tour, experience during program overview presentation, experience during interviews, experience during casual interactions on interview day (including lunch), experience exploring the city/region during interview visit, and a “second look” re-visit to the program and/or city. The complete survey is available as Appendix A.

To allow a comparison between fellows who had an in-person interview experience and fellows who had a virtual interview experience, two classes of MFM fellows were included in this study. To capture the final in-person interview class, the survey was electronically distributed by SMFM to the SMFM list-serv of all first year MFM fellows on May 26, 2021, near the end of the first academic year of their fellowship training. Two reminder emails were sent on June 14, 2021 and June 25, 2021. The survey closed to responses on June 30, 2021. To capture the first virtually interviewed class, the same survey was electronically distributed by SMFM to the SMFM list-serv of all first year MFM fellows on June 8, 2022, near the end of the first academic year of their fellowship training. Two reminder emails were sent on June 21, 2022 and June 30, 2022. The survey closed to responses on July 1, 2022. The emails containing the survey link stated that the survey is optional, that completion of the survey is considered consent to participate in this study, and that the responses would be coded anonymously. The time window to complete the survey was selected so that the fellow had nearly one year of experience in their training program to adjust to their training environment, yet were still within a year of their interview to allow recollection of their interview perceptions. Survey data were collected and managed using Research Electronic Data Capture (REDCap, Nashville, TN) electronic data capture tools hosted at the University of Wisconsin-Madison [10, 11]. REDCap is a secure, web-based software platform designed to support data capture for research studies.

Descriptive statistics and bivariate analyses were performed to compare groups using chi-squared test, Student t test, or analysis of variance as appropriate. All tests were 2-sided and the significance level was set at P < 0.05. Analyses were performed using STATA/SE 17.0 (StataCorp LLC, College Station, TX). This study was determined to be exempt from institutional review board review by the University of Wisconsin-Madison.

Results

The survey was distributed to 128 first year fellows in 2021 and 133 first year fellows in 2022. We received 48 responses in 2021, corresponding to a 37.5% response rate, and 48 responses in 2022, corresponding to a 36.1% response rate. All the 2021 respondents reported they interviewed in person (now referred to as in-person group) and all the 2022 respondents reported they interviewed virtually (now referred to as virtual group). Zero respondents reported a hybrid in-person and virtual interview.

Baseline characteristics including geographic location of fellowship program, rank list position of program, and familiarity with the city of the program were similar between groups (Table 1). There was no difference between the in-person and virtual groups in level of agreement that the interview experience provided a comprehensive and accurate representation of the program (p = 0.41). While there was a trend toward more fellows from the in-person group strongly agreeing that their interview experience allowed them to accurately assess their “fit” with the program as compared to the virtual group, there overall was no significant difference in ability to assess “fit” between groups (p = 0.06). Additionally, there was no difference between groups in expectation of the city and lived experience in the city of the fellowship program (p = 0.47). There was no difference between the in-person and virtual groups in how the expectation of the program prior to starting compared to the actual experience in the first year of fellowship (p = 0.18) or overall satisfaction with the fellowship program (p = 0.95) (Table 2). When asking applications how their impression of their program compared to different sources of information (question 3), there was a significant difference between in-person and virtual groups for program-made videos and experiences with facility tours, casual interactions, and city/region exposure (p < 0.01), However, it is important to note that these differences were primarily related to whether the sources of information had any contribution to their impression of a program. Responses to question 13, the only open-ended question included in the survey, were insufficient to generate any specific themes or conclusions.

Table 1 Baseline characteristics of fellows

Table 2 Fellow perspective of interview experience after one year in fellowship program

The virtual interview group reported a greater influence from the program Instagram account (p < 0.01) and program-made videos (p < 0.01) on their impression of the program prior to beginning the fellowship compared to the in-person interview group. In contrast, the in-person interview group reported a greater influence from experience at the pre/post interview social event (p = 0.01), experience during facility tour (p < 0.01), experience during casual interactions on interview day (including lunch) (p < 0.01), and experience exploring the city/region during the interview visit (p < 0.01) compared to the virtual interview group (Table 3).

Table 3 Mean influence of source of information by interview type

The in-person and virtual interview groups both reported experience during interviews, experience during program overview presentation, recommendations from residency program director or other faculty mentors, and experience during program overview presentation within their five most influential sources of information prior to beginning fellowship. The in-person interview group also reported experience during casual interactions on interview day (including lunch) and experience during facility tour, while the virtual group reported recommendations from past co-residents/current fellows and program/department website as other influential sources of information.

Discussion

In this cross-sectional survey comparing MFM fellows’ experience in their first year of fellowship by in-person or virtual interviews, we found no difference in the actual experience in the fellowship program matching expectations whether the interview was in person or virtual. We also found no difference in satisfaction with the fellowship program between interview modalities. Overall, the interview experience, program overview presentation, and recommendations from faculty mentors remain highly influential on fellows’ perceptions of a program regardless of interview modality. However, in-person interviewees were also highly influenced by casual interactions on the interview day and the facility tour. In contrast, virtual interviewees were more highly influenced by recommendations from past co-residents/current fellows and the program/department website.

Our results are consistent with prior studies that have demonstrated that applicants have been satisfied with the virtual interview process and have felt able to accurately represent themselves and evaluate a program in a virtual interview [2, 12]. A recent study of Reproductive Endocrinology and Infertility fellowship applicants found that 90% of applicants were able to identify programs that would be a good fit and 73% were able to rank programs with a similar degree of confidence as they did for residency interviews [13]. In a single institution study of five Obstetrics and Gynecology fellowship programs, the authors found that 67.2% of applicants felt the virtual interview allowed them to determine if the program was the right “fit” for them and 82% felt comfortable ranking that institution based on their virtual interview experience [14]. A multi-institution study of five Obstetrics and Gynecology fellowships (MFM, Gynecologic Oncology, Complex Family Planning, Minimally Invasive Gynecologic Surgery, and Female Pelvic Medicine and Reconstructive Surgery) also showed similar results with 82% of applicants feeling confident in their rank list based on virtual interviews and 98% of applicants reporting that they received adequate program information [15]. Results were not as favorable in a multi-institution survey by Ding et al., evaluating mainly Maternal Fetal Medicine and Gynecologic Oncology fellowship applicants. Only 60% of surveyed applicants reported receiving necessary program information. However, the authors noted that their study was limited by a low response rate of 15% and that there was significant homogeneity among these participants [16].

This study provides novel data that after one year in the fellowship program, the lived experience in the program matches the expected experience whether that fellow interviewed in person or virtually. Additionally, MFM fellows are equally satisfied with their fellowship program regardless of interview modality. These results provide important data from the fellow’s perspective that can be incorporated by organizations developing recommendations and best practices regarding the optimal interview modality for fellowship programs following resolution of the COVID-19 pandemic.

These data also highlight the importance of supplementary videos, the program website, and social media in providing additional information for virtual candidates. While the interview day experience and recommendations from trusted mentors and colleagues remain most influential regardless of interview modality, these supplemental electronic materials are beneficial to further inform candidates about the program’s curriculum, facilities, and culture.

With resolution of the COVID-19 pandemic, medical education programs must decide whether to continue with the virtual interview format or return to in-person interviews. The recommendation from the Association of American Medical Colleges (AAMC) is to continue with virtual interviews due to the financial cost savings, time savings, applicant preference for virtual interviews, and environmental benefits [5]. Thus, ongoing longitudinal research will be necessary to follow the impact of mode of interview on NRMP match rates and fellow satisfaction as some programs may return to in-person interviews or develop a hybrid interview format.

Our study is strengthened by its unique long-term perspective of fellows’ experience after one year in their fellowship program rather than only evaluating the immediate perception of the virtual interview experience itself. We also uniquely sought to evaluate the impact of many other sources of information on the fellows’ perception of their program rather than only the interview experience itself, anticipating that some of these factors would be more influential in the virtually interviewed group.

Our study also has several limitations. Our response rate was overall low, which increases the risk of selection bias or nonresponse bias. It is possible that the trend seen with more fellows from the in-person group reporting that they were able to accurately assess “fit” could have been significant with a higher response rate. However, our response rate is the same or higher than many similar studies of candidate experience with virtual interviews which report rates of 14 − 46% [7, 13,14,15,16,17]. Additionally, our study sample was geographically diverse and similar between the two groups at baseline. Given our aim was to evaluate the actual experience in the fellowship program compared to interview expectations, our data are also subject to recall bias as fellows may not recall how they felt about their program at the time of the interview or may now report a different perception that has been influenced by their experience in the program. It is also important to note that our survey was conducted during the COVID-19 pandemic, and the unique impact on applicant stress, anxiety, and quality of life could be a confounding factor. Lastly, our data is limited to only our subspecialty which may limit generalizability to other medical training programs. However, since this data does not evaluate any clinical experiences, it may still be beneficial to programs outside of Maternal-Fetal Medicine.

Conclusions

Maternal-Fetal Medicine fellows’ expectations of their fellowship program matched their experience in the first year of training whether the fellow interviewed virtually or in person. Fellows reported no difference in satisfaction with their fellowship program, regardless of interview modality. These data suggest that virtual interviews are as effective as in-person interviews in providing realistic expectations of the program and thus support ongoing exclusive virtual interviews in our subspecialty.