Game-based educational experience in clinical simulation and academic achievement in medical students: a retrospective study

Abstract

Background

In recent years, medical education has increasingly embraced gamification as a method for teaching clinical skills. In Peru, social isolation resulting from the COVID-19 pandemic forced universities and academic institutions to restructure their teaching methods, while there are no reports about the impact of this strategies in Peruvian students. In this study we evaluated the feasibility and patterns of use of a novel video game based didactic activity in undergraduate students from a School of Medicine in Peru.

Method

We conducted a retrospective pilot study in medical students who used the Full Code Medical Simulation platform. We retrieved scoring data obtained from this platform for selected cases of clinical courses with an appropriate number of users [clinical medicine (CM) I (7 cases), CM II (17 cases), surgery I (6 cases) and surgery II (6 cases)]. cases)]. We also evaluated patterns of use and the association between academic performance and the Full Code scores.

Results

A total of 590 students were included in the study. We found a direct correlation between the student’s course grade and Full Code score in all courses (CM I: p < 0.001, CM II: p < 0.05, Surgery I: p < 0.05 and Surgery II: p < 0.05). CM II course students who dedicated more time to completing cases received better grades (p < 0.05). The pattern of use of Full code were similar in students regardless their academic performance. In addition, students with higher academic performance were more likely to have higher scores in the platform (p < 0.001).

Conclusion

The use of gamification in clinical simulation was highly feasible in students of medicine regardless their academic performance. Prospective and interventional studies are needed to assess if the Full Code platform directly affect the learning outcomes.

Background

Simulation-based learning provides an opportunity to develop competencies, enabling students to overcome the constraints of real-world scenarios, and can serve as a potent method for cultivating intricate skills [1].

Currently, the educational process is based on cooperation between the teacher and the student to solve real problems. Knowledge assimilation can occur at any time while the physical time with professors is used to develop the specific skills. Therefore, the student is the protagonist of his learning process, with self-assessment and teamwork being fundamental components of this model [2].

Among educational tools, gamification has emerged as a powerful strategy in medical education for teaching clinical skills, especially in recent social isolation due to the COVID-19 pandemic [3,4,5]. Serious games not only have a design and technology that motivates and engages students in the experience but also combine psychological factors that help them develop their soft skills [6–7]. Serious games influence the adaptive neural plasticity of the brain, leading to functional [8] and structural changes [9].

One of the strengths of using serious games in medical education is that students can improve their diagnostic reasoning and problem-solving skills in clinical training through experiential learning [10]. Game-based education platforms not only serve as feedback on the essential tasks that students perform in their mannequin-based simulation courses but also allow students to adapt the learning process to their abilities and pace of study [11]. However, experience with clinical training games, such as serious games, is scarce in Latin America [12–13].

In Peru, social isolation due to the COVID-19 pandemic forced universities and academic Institutions to carry out a general restructuring of their teaching strategies, with 70% of universities not having previous experience in virtual courses [14]. This setting provides an interesting natural scenario that prove the value of virtual activities and strength the incorporation of TI technologies into modern teaching.

In the case of medical schools, clinical practices had to be suspended for undergraduate students. Therefore, research mentoring and complementary e-learning tools were improved to develop medical skills, such as the use of software for clinical decision-making practice. Students at the School of Medicine at the Universidad Privada San Juan Bautista (UPSJB) used the Full Code program to implement their skills in clinical courses. Therefore, this pilot study aimed to evaluate the feasibility to use of a novel video game based didactic activity and its association with the academical performance of undergraduate students from a School of Medicine in Peru.

Methods

Study design

A retrospective study was conducted in students registered in the second semester of the 2020 academic year. It involved students from the School of Medicine at the Universidad Privada San Juan Bautista (UPSJB) across three Peruvian cities: Lima, Ica, and Chincha.

Participants and sampling

During the COVID-19 pandemic, medical students from the sixth to twelfth cycles, specifically those enrolled in Clinical Medicine I, II, Surgery I, and II, were encouraged to utilize the Full Code platform as an additional learning resource. This platform provided access to a comprehensive library comprising over 150 clinical cases. Participation in utilizing the platform was voluntary. To qualify for inclusion in the study, students had to meet certain criteria: they needed to be at least 18 years of age, enrolled in the mentioned courses during the second semester of 2020, and maintain their enrollment throughout the semester without withdrawing.

Variables included in the study

Student achievement of graduation profile competencies was assessed using final course grades categorized into three levels for interpretation: 9–13 (not achieved), 14–16 (in progress), and 17–19 (achieved). Educational experience with simulated clinical cases on the Full Code platform was evaluated through platform-derived variables: global score (average score across completed cases), time spent per case (average time spent on each completed case), diagnosis time (ratio of total time spent to total diagnoses attempted, including both correct and incorrect), and percentage correct diagnoses [(number of correct diagnoses)/(total diagnoses attempted)] × 100. Additional data included university branch, gender, and enrolled courses.

Platform

Our institution employed Full Code, a virtual simulation software by Minerva Medical (https://fullcodemedical.com/). This internet-independent platform provided students with more than 150 realistic clinical cases to practice their clinical reasoning. Course instructors encouraged students to use Full Code as a supplementary tool for honing their clinical reasoning. Participation was voluntary. Each course offered a targeted set of cases aligned with its learning objectives, while students were free to explore additional cases within the study period. Upon completing a case, participants received a detailed, automatically generated performance report. This report assigns a score based on three weighted components: correct diagnosis (35%), correct disposition (20%), and individual actions taken (45%). Each action within a case is categorized as critical, recommended, neutral, unnecessary, or harmful, with corresponding positive or negative point values (except for neutral actions).

Data collection

The researchers designed a data collection sheet to set down the information obtained from the Full Code software records and the university databases. The completeness of all student performance tracking documents was thoroughly evaluated before the data were collected.

Data analysis

All numerical data were exported to Microsoft Excel and analyzed using SPSS Statistics, version 24.0 (IBM Corporation, Armonk, NY). Categorical variables are described in terms of absolute frequencies and percentages, while continuous variables are described as medians and ranges. The relationships between the number of times the student completed a clinical case, the number of clinical cases completed by the student, the time it took to complete the clinical cases, and the measures of the graduate’s skills were evaluated using Spearman’s rank correlation coefficient test, as well as the dispersion diagram with straight lines and adjustment curves. On the other hand, the comparisons of the performance of each student on Full Code and the levels of the competency measures of the graduate profile were analyzed through the nonparametric Kruskal‒Wallis H test. The analysis maintained the level of statistical significance at p < 0.05.

Ethical considerations

This study was reviewed and approved by the Ethics Committee of the Universidad Privada San Juan Bautista and the Universidad Internacional Iberoamericana. All the ethical principles of the Declaration of Helsinki and the standards of the code of ethics and deontology of the Medical College of Peru were considered. Anonymity was always respected during the execution of this study since the academic vice-rector’s office of the Universidad Privada San Juan Bautista provided us with blinded student data.

Results

General characteristics of students

In total, 590 out of 1452 enrolled students (40.6%) freely use the Full Code platform (Table S1). Among the students, 80.5% studied in Lima, 12% in Ica and 7.5% in Chincha. Regarding sex, 368 were women (62.4%). According to the final average of the subject (according to the vigesimal system 0 to 20), 21.7% of the students had a final average of 17 to 19 (medium level), 65.9% had a grade between 14 and 16 (medium level), and 12.4% had a grade below 13 (Table 1).

Table 1 Characteristics of the students who used the full code program from the 2021-II semester

Clinical cases developed in the full code

The analysis of the scores obtained in the use of the clinical simulation platform was carried out with the clinical cases with the greatest participation of the students according to the subject: six clinical cases for Surgery I, six cases for Surgery II, seven cases for Clinical Medicine I and 17 cases for Clinical Medicine II.

In total, 518 students developed some of the clinical cases, reaching a median of 4 clinical cases completed, and depending on the course, 4 (range: 1, 6) clinical cases in Surgery I, 1 (range: 1, 6) in Surgery II, 3 (range: 1, 7) in Clinical Medicine I, and 13 (range: 1, 16) in Clinical Medicine II.

Clinical simulation score according to the medical subject

The median global score was 81.7 (range: 1.8, 100.0), the time per clinical case was 16.4 min (range: 0.7, 469.2), the number of times a clinical case developed was 2.4 (range: 1, 21), and the median percentage of correct diagnoses was 85.1 (range: 0, 100.0).

Furthermore, the median global score for the performance of patients in Full Code was greater than 75% in all courses (surgery I: 81.5%, surgery II: 85.5%, clinical medicine I: 77.8%, and clinical medicine II: 80.7%). Regarding patterns of use of Full Code, the median time spent per clinical case was highly variable; the median time per case was 16 min in Surgery I, 14 min in Surgery II, 23 min in Clinical Medicine I, and 15.2 min in Clinical Medicine II. The median number of times a clinical case developed was 2 in the surgery course group and 3 in the clinical medicine course group. The median percentage of correct diagnoses was greater than 80%, except in Clinical Medicine I (Table 2).

Table 2 Pattern of use of full code platform in students of human medicine

Relationship between full code score and student performance

In general, the relationships between student performance (in terms of course grade) and the clinical simulation platform global score obtained during the development of the selected clinical cases are linear (r = 0.30); although this association is not strong, the results suggest that students with high scores have better academic performance. There was a direct relationship between the global Full Code score and course grade (Surgery I, Surgery II, Clinical Medicine I, and Clinical Medicine II), although this relationship was very low for Surgery I (r = 0.15) and Surgery II (r = 0.16) and low for Clinical Medicine I (r = 0.38) and Clinical Medicine II (r = 0.36).

The median global score was 71.1 (range: 17.0, 99.1) for students with grades between 9 and 13, 82.3 (range: 4.9, 100.0), and 82.5 (range: 1.8, 100.0) for those with grades between 14 and 16 and 17–19, respectively, which represented a significant difference (p < 0.05) between Clinical Medicine I and II. In Clinical Medicine I, the median scores were 58.5 (range: 30.8, 83.4), 77.6 (range: 34.5, 100.0) and 82.2 (range: 44.1, 98.9) in those with grades between 9 and 13, 14–16 and 17–19, respectively; in Clinical Medicine II, they were 65.9 (range: 17.0, 94.2), 82.4 (range: 10.5, 99.1), and 81.7 (range: 48.6, 97.6), respectively. (Table 3)

These results indicate that students with better academic performance are more likely to achieve also higher scores in the Full Code platform (Fig. 1). The patterns of use of Full code were similar between students with different academic performance (Table 3).

Fig. 1

Relationships between the global score and student performance in clinical simulation according to the surgery I (r=0.15), surgery II (r=0.16), clinical medicine I (r=0.38), and clinical medicine II (r=0.36) courses

Table 3 Full code scores, patterns of use and academic performance

Discussion

The COVID-19 pandemic has changed the landscape of education, mainly in health science programs, we conducted a retrospective pilot study to evaluate the feasibility to use of a novel video game based didactic activity for reasoning and clinical decision-making (Full Code) of undergraduate students from a School of Medicine in Peru.

The Accreditation Council for Graduate Medical Education (ACGME) of the United States defines the essential skills that every physician must have through six competencies. Gamification activities supports actively the developing of critical thinking and clinical decision-making, strengthening five essential competencies, such as practice-based learning and improvement, patient care and procedural skills, medical knowledge, interpersonal skills, and communication and professionalism [15]. In the particular academic scenario of Peru, the Ministry of Health (MINSA) has provided a profile of essential competencies for physicians in the country, which consists of 13 general competencies [16].

Our work has limitations. First, the use of the clinical simulation platform was voluntary and did not directly impact the students’ final grades since it was proposed as a complementary activity to the regular curriculum. Similarly, it is important to note that the study plan in relation to clinical competencies is developed in several semesters. On the other hand, under the current study design without a basal assessment, the main interpretation bias is related to correlation vs. causation issues [17]. It would be interesting to assess in a prospective evaluation the impact of similar platforms on the increase in academic performance and the assimilation of new knowledge.

Another limitation of our study is that it was conducted in a study center, and the population only received one intervention. Other studies should study whether it is possible to determine the effect of using the Full Code over a longer follow-up period. However, although our study included a large sample (n = 590), the study population was from a middle-income country. Most of the current literature on the feasibility of serious games/gamification has been conducted in high-income countries, which limits the applicability of the findings of this review to low- and middle-income countries. This is a critical gap in the evidence, as low- and middle-income countries bear the brunt of the global shortage of trained health workers [18].

Our study revealed a weak association between Full Code use and the presentation of better final grades in clinical courses. A review conducted by Zhonggen on the efficiency of “serious games” as a pedagogical strategy for teaching healthcare professionals showed that gamification was a more efficient technique than traditional teaching methods. Similarly, they found studies (n = 4) that used video games as a teaching technique and obtained results as efficiently as traditional teaching methods [19]. Regarding human medicine, several studies have shown positive effects of training with virtual patient simulators. Haubruck et al. showed that students who trained in chest tube insertion with the Touch Surgery app performed better than those who trained with traditional methods [20]. Similarly, Ghoman et al. trained 50 healthcare professionals (HCPs) using the RETAIN digital simulator and found that it effectively improved, maintained and helped reinforce the knowledge of neonatal resuscitation of professionals throughout their life [21].

Clinical simulation-based training is known to improve learners’ initial performance; however, skills deteriorate significantly after six months [22–23]. In this sense, a study by Hu et al. showed that coupling the use of a serious game improves knowledge retention in students’ short-term memory [24]. Another positive point of serious games is that students generally have favorable attitudes toward virtual platforms that train adaptive algorithms [25]. The Full Code is not only an immersive patient simulator that adds features found in modern computer games, such as a high-fidelity gaming environment but also opens up the possibility of teaching nontechnical skills that are important in high-risk environments, such as intensive care services [26]. Furthermore, the Full Code is an application that presents a score based on a competency rubric and a detailed report of the competencies to be developed with comments for each clinical case [27], which allows this technique to be easily coupled with the breadth of studies. Our study adds evidence of experience about the use of Full Code in students of human medicine. Currently, the studies evaluating Full Code are scarce.

In addition, it would be valuable to assess the contribution of teachers/educators in the implementation process of a simulation-based education course since their commitment to ongoing improvement is vital for fostering an effective and transformative learning environment. Simulation-based medical educators must be trained to fulfill multiple roles, including scenario architects, immersive facilitators, holistic evaluators, and constructive mentors [28–29]. Previous studies have shown that teachers’ guidance and commitment are crucial for motivating students and positively affecting their knowledge acquisition [30–31].

Finally, our work cannot determine whether there is direct causality between using the Full Code application and acquiring new knowledge in the clinical field. Mostly because our study lacked a control group, it is impossible to determine whether the new knowledge is due to the use of the application or due to familiarity with the questions asked in the Full Code clinical case program.

A game-based learning approach in clinical simulation could significantly enhance medical students’ educational experience and academic achievement. However, future research must evaluate whether the observed benefits are directly attributable to game-based elements or to other factors within the learning environment to fully understand their impact on knowledge acquisition. Thus, the completion of these research gaps will allow educators to better design and implement game-based learning strategies that will maximize medical students’ educational outcomes.

Conclusion

In conclusion, our pilot study suggests that the use of gamification in clinical simulation is highly feasible with similar patterns of use of the platform regardless the academical performance of students. Prospective studies are needed to evaluate the improvement of clinical skills with this platform.