Prospective assessment of learning curve and impact of intensive versus progressive training in colonoscopy competence among French residents

Abstract

Backgrounds

There are no existing data in the literature on the learning curve of French interns in colonoscopy or on the comparison between different frequencies of colonoscopy training modalities. We aimed to assess the number of procedures required for French residents in hepatogastroenterology to achieve competency in colonoscopy.

Methods

The primary outcome was achieving greater than 90% cecal intubation rate (CIR90) competency using the Learning Curve-Cumulative Summation (LC-CUSUM) method. Participants with over 80 procedures were categorized into intensive and progressive training groups. We compared the proportion of residents reaching competency, the number of colonoscopies to reach it, and the speed of competency.

Results

The study included 81 residents, totaling 6,259 procedures. 29 did more than 80 procedures: 12 in the progressive group and 17 in the intensive group. 204 colonoscopies were needed for reaching CIR90 competency (21% of residents). Achievement rates were similar across groups: 50% in the progressive and 65% in the intensive group (p = 0.50). LC competency was reached by 8.6% of residents after an average of 225 procedures, with no significant difference between groups (p = 0.21). Survival analysis showed no significant difference in the speed of competency acquisition between groups (p = 0.77 and p = 0.14, respectively). The Polyp Detection Rate (PDR) averaged 40%.

Conclusion

Given that only 21% reached CIR90, efforts are needed to increase the number of colonoscopies during training.

Clinical trial number

Not applicable.

Background

The acquisition and evaluation of competency in endoscopy stand as fundamental pillars in the assurance of quality care within hepatogastroenterology. Notably, colonoscopy plays an indispensable role in the detection and management of colorectal cancer, thus underscoring the necessity for rigorous training. In France, it is estimated that 93.1% of physicians board-certified in hepatogastroenterology routinely conduct colonoscopies as part of their practice [1], thereby underlining the importance of high-caliber, appropriate training [2]. Quality criteria and recommendations to delineate competency in colonoscopy have been published by the French Society of Digestive Endoscopy (SFED) and the European Society of Gastrointestinal Endoscopy (ESGE). Key among these is a cecal intubation rate (CIR) of at least 90% [3]. Moreover, the European Board Of Gastroenterology And Hepatology’s (EBGH) Blue Book [4] recommends the completion of a total of 200 colonoscopies during medical residency. Initial guidelines on the necessary procedure count for achieving competency – defined as a CIR exceeding 90% - were premised on expert consensus. Subsequent studies utilizing this competency indicator in colonoscopy have yielded divergent findings. This benchmark was reached post 233 procedures in the UK study by Ward et al. [5], and in over 90% of trainees following 500 procedures as per the US study by Spier et al. [6]. A recent survey [7] disclosed that a mere 35% of final-year French residents believed they had attained the 200 colonoscopies threshold during their residency, with no extant French data to ascertain if hepatogastroenterology residents meet the ESGE and SFED-set competency objectives. To address this lacuna, we introduced a digital form in 2017, enabling residents across four French subdivisions to record and describe the endoscopic procedures executed during their residency. Our study was thereby designed to appraise the training and the attainment of colonoscopy competency amongst French residents via responses to a self-administered online questionnaire.

Methods

Target population

This is a prospective, multicenter, observational study conducted among hepatogastroenterology residents in four different subdivisions in France (Montpellier-Nîmes – Aix-Marseille – Nice, and Paris). From May 2017 to April 2022, a digital data collection form with an individual questionnaire was made available (GoogleForm in supplements). On a voluntary basis, the instruction was to systematically fill out the questionnaire throughout their training course. Immediate evaluation after the end of each procedure was designed to minimize recall bias.

For each resident, their subdivision, gender, and exposure time to the questionnaire were recorded.

No patient information was collected. All information allowing the identification of residents was removed from the data. Informed consent for participation was obtained from all participants. The research was approved by the local ethics committee of the Montpellier University Hospital (IRB number IRB-MTP_2022_10_202201180) and the consent that was obtained from all of the participants.

Inclusion and exclusion criteria

All residents in training during the study period were eligible. Residents who completed less than 20 colonoscopy procedures on the questionnaire were excluded.

Parameters collected during colonoscopies

The following fields were collected for each colonoscopy: the procedure’s date, supervision by a senior trainer, progress without physical assistance from the senior (oral advice allowed), perception of ease of progress (using a Visual Analog Scale (VAS) with a slider from 0 (no difficulty) to 10 (maximum difficulty), embedded in the Form), and polyp detection. Progress without help was classified according to the segment reached: left colon, transverse colon, right colon, cecum, terminal ileum. A colonoscopy was considered complete if the trainee reached the cecum with visualization of the appendiceal orifice with no physical intervention from the trainer, indicating that the progression was classified as “cecum” or “terminal ileum”. The questionnaire used in this study was specifically developed for this research and has not been previously published. An English language version of the questionnaire is provided as a supplementary file.

Primary outcome measure

The primary aim of our study was to determine the average number of colonoscopies required to reach the threshold of cecal intubation competency measured by the moving average of cecal intubation rates. The calculation of the moving average is described below. Competency was defined by a maintained CIR ≥ 90% (CIR90).

Secondary outcome measures

We described the average number of colonoscopies required to achieve competency using a different statistical tool from the moving average: the LC-CUSUM score defining LC competency. This score is a cumulative statistical score calculated progressively as colonoscopy attempts are made. Subsequently, an analysis was conducted on questionnaires with at least 80 attempts, where we compared two groups: “intensive” and “progressive” residents. Residents were considered “intensive” if they had recorded at least 30 attempts over a 15 consecutive day period and progressive otherwise.

Finally, also among questionnaires with at least 80 attempts, we described the average polype detection rate (PDR) by analyzing the moving average of the detection rate among the last 20 attempts.

Statistical analysis

Descriptive

A flow chart was created to depict the selection of patients, along with the inclusion and exclusion criteria. Characteristics of the residents were described using frequencies and percentages for categorical data and means and standard deviations for continuous variables.

Moving average analysis

For each resident, a moving success average was calculated, corresponding to the average success rate among the last 20 attempts for each attempt [5]. We then plotted the average for all residents and the standard deviation as a function of the number of attempts. As competency is defined by CIR90, the estimation of the number of colonoscopies necessary to reach it was determined from this average performance curve.

LC-CUSUM analysis

We used the calculation method of LC-CUSUM performance as previously described [8]. This method is adapted to learning and sequential procedures [9]. The null hypothesis (H0) means that the process is not controlled (learning phase) while the alternative hypothesis (H1) means that the process is under control (competency acquired). It assumes the trainee is not proficient at the start of the follow-up. The trainee is not penalized by early failures by admitting a null limit. The LC-CUSUM signals when the procedure is acquired by crossing a previously defined threshold: H0 is rejected in favor of H1; the trainee is then competent. As long as the curve does not exceed the threshold H, the acquisition of competency is not demonstrated, and learning must continue. To determine the value of H, two parameters must be defined: the ideal failure rate (p0) and the unacceptable failure rate (p1). The chosen value varies according to the context, expert consensus, and the authors judgment. A description of LC-CUSUM is available in supplement.

In our case, the value of p0 was set at 0.05 (5%) and the value of p1 was set at 0.10 (10%) following current European recommendations [2]. The threshold score value H defining competency was calculated a priori and set at 1.75 since it was the score with the best performance to detect competency in a resident exposed to 200 colonoscopies for parameters p0 = 0.05 and p1 = 0.1. We determined the LC-CUSUM threshold of H = 1.75 through a simulation process using predefined criteria (p0 = 0.05 and p1 = 0.1) and generating a Receiver Operating Characteristic (ROC) curve. The threshold was selected based on the Youden Index, which provided the optimal balance between sensitivity and specificity, ensuring accurate detection of competency while minimizing false positives (cf. suppl). To compare the rate of competency and the number of colonoscopies to reach it between the intensive and progressive groups, we used Fisher’s and Mann-Whitney’s tests, respectively. The number of colonoscopies until competency was treated as a time-to-event variable. We used survival graph generated using the Kaplan Meier method to visually depict survival curves, illustrating the probability of not having reached competency. To identify any statistical differences between the two curves, the log-rank test was employed.

Results

In total, 140 residents registered on the platform, with 81 who had at least 20 recorded attempts being included. The characteristics of this resident’s population are summarized in Table 1. A total of 6259 procedures, including 3899 complete ones, were recorded. 52 (64.2%) residents came from the Paris subdivision, 17 (21%) from the Marseille subdivision, 8 (9.9%) from the Montpellier subdivision, and 4 (4.9%) from the Nice subdivision.

Table 1 Baseline characteristics of the students

18 residents participated in a dedicated endoscopy rotation and 29 performed more than 80 attempts (12 in the progressive group versus 17 in the intensive group) (Fig. 1). The average time exposed to the questionnaire was 1.11 years (± 0.72). The average number of declared colonoscopies was 77.3 (± 55.3) in the total population, 46.2 (± 17.8) in the progressive group, and 139.4 (± 52.7) in the intensive group.

Fig. 1

Flow-chart

No resident who declared less than 80 colonoscopies reached the CIR90 or LC-CUSUM competency threshold. In the total population, an average of 204 colonoscopies was required to achieve CIR90 (Fig. 2). Seventeen residents reached the CIR90 competency threshold, accounting for 20.99% of the total of 81 residents in the population.

Fig. 2

Moving average of success rate as a function of number of colonoscopies

The LC competency threshold was reached after a mean of 225 colonoscopies (Fig. 3). Seven residents reached the LC-CUSUM competency threshold (H = 1.75), which was 8.6% of the total 81 residents in the population.

Fig. 3

LC CUSUM average score as a function of number of colonoscopies

Among the residents who declared more than 80 colonoscopies (n = 29), 17 (58.62%) achieved CIR90; 6/12 (50%) in the progressive group and 11/17 (64.71%) in the intensive group (p = 0.47) (Fig. 4). For the 17 residents who reached CIR 90 competency, it occurred on average after 68.29 colonoscopies (± 45.11) in the general population, 57.83 colonoscopies (± 38.32) in the progressive group, and 74.00 colonoscopies (± 49.19) in the intensive group (p = 0.50).

Fig. 4

Moving average of success rate according to number of colonoscopies and group. The moving average is calculated over the nth attempt and the last 19 attempts, when it is possible

LC competency was achieved for 7 residents (24.14%), 3 (25%) in the progressive group and 4 (23%) in the intensive group (p = 1). The average number of colonoscopies to achieve it was 120.86 (± 69.36) in the general population, 80.67 (± 38.08) in the progressive group, and 151.00 (± 76.33) in the intensive group (p = 0.21)z; (Fig. 5). The average Polyp Detection Rate (PDR) was 40% and was not associated with the number of performed colonoscopies (p = 0.71) (Fig. 6).

Fig. 5

LC CUSUM average score as a function of number of colonoscopies and group

Fig. 6

Moving average of polyp detection rate as a function of number of attempts

Using survival analysis with log-rank test to compare the speed of competency acquisition based on the moving average of cecal intubation rate ≥ 90%, the median number to achieve competency was 100 colonoscopies in the intensive group and 104 in the progressive group (p = 0.77, log-rank test) (Fig. 7).

Fig. 7

Comparison of the speed of competency acquisition using CIR90 (moving average). Survival analysis

Similarly, when using the LC-CUSUM method to detect competency, the speed of competency acquisition was comparable between the two groups but only seven residents achieved competency with the predefined LC-CUSUM parameters (H = 1.75, p0 = 0.05, p1 = 0.1). No statistical difference probability of achieving competency was seen (p-value = 0.14, Log-rank test) (Fig. 8).

Fig. 8

Comparison of the speed of competency acquisition using LC-CUSUM method (H = 1.75, p0 = 0.05, p1 = 0.1). Survival analysis

Discussion

This study represents the first French investigation into the number of colonoscopies required for medical residents to achieve competence. Using two statistical methods, 58.2% of residents reached a cecal intubation rate (CIR) > 90% after an average of 204 procedures, whereas only 24.14% attained competence after 225 procedures according to the LC-CUSUM method. Among 84 residents who completed the questionnaire, 20.9% achieved competence using the CIR 90, while 8.6% did so using the LC-CUSUM. These low rates suggest two main issues: insufficient exposure to colonoscopy during training (median of 61 procedures) and potential underreporting due to the voluntary nature of the questionnaire. Even among the diligent subpopulation (29 residents), competence rates remained low (58.6% for CIR 90 and 24.1% for LC-CUSUM), indicating a need for increased colonoscopy training in French medical education which aligns with a 2017 national survey where nearly two-thirds of French gastroenterology students felt they had not performed enough colonoscopies during their course [7], with only 35% reporting reaching the 200 colonoscopies threshold.

Research indicates that competency in colonoscopy is tied to the total number of procedures and training intensity, but prolonged training interruptions did not significantly affect skill acquisition [5, 10]. This aligns with our findings, showing no significant difference in skill acquisition speed between progressive and intensive training groups, regardless of the statistical method. it is possible that the frequency of performing colonoscopies does not impact learning as much as the total number of procedures performed. However, our sample size is insufficient to draw definitive conclusions on this matter.

Our study revealed a mean polyp detection rate (PDR) of 40%, which appeared uncorrelated to the number of procedures. This suggests potential measurement bias, possibly influenced by trainer presence or endoscopy nurses. Despite limited data, this parameter is worth further investigation as a potential competence criterion in training.

The proportion of competent interns varied significantly with the evaluation method, being easier with the CIR 90 moving average than the more rigorous LC-CUSUM. The LC-CUSUM’s rigorous approach ensures sustained competence, not just sporadic success, offering a more reliable measure of long-term performance. While this evaluation method is the most rigorous and statistically precise for detecting competence, its stringent criteria resulted in identifying competence in only a small number of subjects in our study. We decided to define the ideal failure rate (p0) at 5% and the unacceptable failure rate (p1) at 10%, adhering to European endoscopy performance standards [2], which may explain lower competence rates compared to other studies.

Determining when a trainee achieves a sustainable competence level remains unresolved, with no consensus on the optimal statistical method. We used a moving average success rate with a block size of 20 as the primary outcome, balancing ease of calculation and feedback against its subjective parameters. The choice between using the LC-CUSUM method and the moving average success rate for detecting technical competence in colonoscopy presents distinct advantages and challenges. The LC-CUSUM method, while rigorous and statistically precise, may be overly complex and stringent for smaller cohorts with limited procedures, potentially leading to an underestimation of competence in such settings. This more stringent evaluation also results in fewer residents reaching the competency threshold, reflecting the challenges of sustaining consistent performance over time. On the other hand, the moving average success rate, although more straightforward and seemingly appealing for smaller sample sizes, carries the risk of falsely detecting competence due to its less stringent criteria. Balancing the need for a reliable and feasible assessment method remains a critical consideration, and the optimal approach may vary depending on the specific context and sample size of the training program. Comparative studies on the moving average and LC-CUSUM methods would be valuable but are currently lacking.

Despite varying evaluation methods, our study supports the recent recommendation of a 200-colonoscopy minimum threshold for achieving a > 90% CIR. However, our data collection had limitations, primarily self-declaration without senior validation, which could lead to overreporting success and underreporting failures. While self-reported CIR as a competency measure has limitations, it was chosen for its practicality in large-scale residency training. Comprehensive tools like GIECAT and ACE [11, 12], though robust, are less feasible in routine use due to required supervision. In contrast, the LC-CUSUM method offers a dynamic, real-time evaluation of competency acquisition over time. This approach not only provides an objective assessment of progress but also allows senior supervisors to tailor the intensity of training for residents facing greater challenges, ensuring targeted and adaptive educational support. Moreover, participation in portfolio completion was limited, with only 29 of 141 residents logging more than 80 procedures, potentially due to lack of obligation and systematic feedback, and network issues in operating rooms. To address the potential biases introduced by voluntary participation and incomplete data, we now have implemented an application (available offline) which allows external validation by senior trainers and mandated the completion of over 200 validated procedures for certification in France.

Other factors likely influence the speed of competency acquisition, including procedural context (inpatient vs. outpatient), case complexity (e.g., obesity, poor bowel preparation, therapeutic vs. diagnostic procedures), Boston bowel preparation score, withdrawal time and trainer involvement. However, systematically collecting these factors was limited by the voluntary nature of the questionnaire and the additional workload for trainees, who needed to balance questionnaire completion with their other responsibilities. A longer questionnaire resulted in low response rates, as it was not mandatory. To address this, we opted for a shorter questionnaire to improve participation, though this limitation should be considered when interpreting the findings.

Despite these limitations, data from the 29 diligent residents enabled reliable analyses of competence thresholds beyond 200 colonoscopies suggested in recent studies [5, 13, 14] and recommended by the Joint Advisory Group (JAG) on Gastrointestinal Endoscopy for obtaining a cecal intubation rate (CIR) > 90%.

We showed no difference in skill acquisition pace between intensive and progressive training. These findings represent the first objective data on endoscopy training challenges among French residents, confirming previously anecdotal sentiments.

This study focused on cecal intubation capabilities, yet the quality of a colonoscopy also involves adenoma detection rate, rectal retro-vision, and polypectomy success. Comprehensive skills assessment, as implemented in the UK and US training guidelines through Direct Observation of Procedural Skills (DOPS), as well as validated tools such as the Gastrointestinal Endoscopy Competency Assessment Tool (GIECAT) and the Assessment of Competency in Endoscopy (ACE), should be adopted to ensure patients receive care from well-trained professionals and to provide a multidimensional evaluation of competency [11, 12, 15, 16]. However, mastering cecal intubation is a prerequisite for more comprehensive competency accreditation in endoscopy. Given the low number of residents achieving competency, efforts must be made to increase the number of procedures performed during training in France. Notably, a reform was introduced in 2018, making a six-month rotation in an endoscopy unit mandatory for trainees. Combined with the integration of simulation-based training and structured feedback, this reform represents a promising step towards optimizing competency acquisition and ensuring high-quality endoscopy training nationwide.

Conclusion

This study highlights the restricted exposure to colonoscopy training among French medical residents, with a low rate of students achieving competency. This emphasizes the need for continued efforts to enhance colonoscopy learning in France. Moreover, determining the optimal methods for detecting competency remains a critical area for further investigation.