Technology enhanced medical education using smart glasses for oral and dental examinations: an observational pilot study

Abstract

Background

The importance of dental knowledge in medical school education has been previously emphasized however, there have been limitation with adoption of a dental and oral examination component in the medical school curriculum, with hands-on training the least taught. Smart glasses have been used in a variety of applications and the objective of our study was to analyze the effectiveness of smart glasses use as a feedback tool in teaching and evaluating the oral exam performed by medical students.

Methods

The pilot study included ten medical students and a standardized patient. All ten medical students were provided with a didactic self-study online course on dental examinations and were arbitrarily assigned into two groups of five. One group was assigned to an intervention arm in which they performed an oral exam using the smart glasses and the other group performed the oral exam without the smart glasses. A preceptor supervised both groups and recorded his observations on a form. The students completed a questionnaire at the end of the study to discuss their experiences. The effectiveness of the smart glasses was reflected in a high summary score of the observations and the response to the questionnaire reflected the use of the smart glasses as a feedback tool.

Results

Our pilot study demonstrated feasibility of using the smart glasses as an effective tool for learning oral and dental examination. The student feedback was more favorable in the intervention group.

Conclusion

Our pilot study demonstrated that smart glasses were an effective tool to enhance medical education of the oral and dental examination performed by medical students. This technology can be explored further to conduct other innovative medical education projects.

Clinical trial number

Not applicable.

Background

According to the World Health Organization, good oral health allows a person to perform essential functions, like eating and speaking, and maintain self-confidence [1]. Oral diseases are ubiquitous and can have serious health and socio-economic consequences affecting the quality of life [2]. Dental caries are the most common chronic infectious disease in the world [3]. Oral diseases cause two-thirds of US adults to miss school or work due to emergency dental appointments [4]. Like in many other areas of healthcare, people from disadvantaged backgrounds are more disproportionately affected by oral diseases [1].

Oral health is closely entwined with overall health and awareness of this increased after the publication of “Oral Health in America: A Report of the Surgeon General” in the year 2000 [4]. One of the points made in this report was that many systemic diseases, such as diabetes and HIV, can have oral health manifestations [4, 5]. Conversely, poor oral health can cause other health issues, such as cardiac disease. Since more than a third of the United States population does not have dental insurance, many individuals visit the emergency department or are seen by their primary care physician for dental concerns [6]. For these reasons, it is important for medical providers to be familiar with the oral health exam.

In 2008, the Association of American Medical Colleges (AAMC) published a report titled “Report IX Contemporary Issues in Medicine: Oral Health Education for Medical and Dental Students” that emphasized the importance of adding oral health training to the medical school curriculum [7]. One of the necessary skills listed for medical students was an ability to “perform head and neck examination that includes recognition of caries, periodontal disease, dental erosion from eating disorders, cleft palate and other anomalies, mucosal changes, indications of oral cancer.” However, a survey of U.S. medical schools showed that the majority offer minimal oral health education, with hands-on training being the least taught [5]. This gap is reflected in medical students’ reduced confidence in performing the oral health exam [8].

There are established oral health curricula that are easily accessible, such as the Smiles for Life curriculum. However, this was designed as a lecture only format, which may limit its effectiveness as compared to a hands-on mode of learning [8]. There is data about incorporating oral health curricula into medical training [5]. However, there are still only a small number of reports on the assessment and feedback of this learning, especially of the hands-on training component [8].

One of the key components of hands-on learning is real-time feedback and the ability to ask questions and smart glasses can help facilitate this. Smart glasses, which are wearable technology devices, are capable of video recording and communication via voice commands and have been tested in various health-related applications [9, 10]. Some of the benefits in medical education can be attributed to its live video-teleconferencing (VTC) with the option of instantaneous feedback for the trainee [11]. With this technology, trainees can broadcast their “point of view” (POV) to supervising trainers and vice-versa [11]. Some current uses for smart glasses in medical education are using the device to evaluate suturing skills, vaginal delivery, cardiopulmonary resuscitation, and variety of surgical procedures [11,12,13,14]. To the authors knowledge there has been no report on using smart glasses for interprofessional education of teaching oral examinations to medical students. It is our intention to use this device as a workplace-based assessment tool while the student performs an oral exam. Our pilot study’s objective was to analyze the utility of using smart glasses in teaching and evaluating the oral exam performed by medical students.

Methods

Study approval was obtained by the Institutional Review Board at the Albert Einstein College of Medicine # 2022–14,393 and was conducted in accordance with the Declaration of Helsinki. This was a feasibility study using a limited number of participants to evaluate the effectiveness of the use of smart glasses on medical students and to assess the logistics of performing such a study. The primary aim of the study was to evaluate the effectiveness of smart glasses in learning to perform an oral examination. The secondary aim was to determine if the smart glasses can be used as an effective assessment and feedback tool. Ten final year medical students were selected based on their willingness to volunteer. After verbally agreeing to volunteer for the study, they were given detailed instructions and provided with a written informed consent document. One dental resident was selected as the standardized patient. The medical students were arbitrarily assigned in 1:1 ratio to a control group that performed the procedure without the smart glasses or to an intervention group that performed the procedure wearing the smart glasses.

Fig. 1

All ten medical students were provided with a didactic self-study online course on oral examinations from the Smiles for Life” ™ curriculum in a Microsoft PowerPoint presentation [6]. The location for conducting the practical procedure was a simulation clinic at the medical school. Personal Protective equipment (PPE) was worn by the examiner-students and the materials used were a disposable wooden tongue depressor and disposable 2 × 2 gauze pads. Before each session, the medical students were advised to review the presentation. On the day of the study, the consent form was signed, and questions about the pre-viewed presentation, and the case report form (CRF) (Appendix 1) were answered. The Principal Investigator (PI) explained the procedure and demonstrated the oral examination for all the medical students to observe. The PI also served as the preceptor for the study. In the control group, the PI stood beside the student examiner and patient as the scores were recorded on the Case report form (CRF). The preceptor directly observed the student performing the oral exam on the patient and feedback was provided based on this observation. As the student performed the examination, they called out the structure they observed as listed in Appendix 1. If the structure, such as the buccal mucosa or hard palate was also observed by the preceptor during the examination, a ‘1’ was marked as the read score. If the structure was not observed by the preceptor a score of ‘0’ was entered in the CRF and if structure was only partially observed, a score of ‘2’ was entered as it was unable to be determine. On a different day, the five students in the intervention group performed the same procedure wearing the smart glasses Vuzix 4000(Vizix, Rochester NY) while the preceptor evaluated the students remotely, on a laptop computer that was situated outside the examination room (Fig. 1). The smart glasses selected have a high quality camera, voice commands, a long battery life, Wi-Fi and Bluetooth capabilities as well as two way, video conferencing capabilities. During the procedure, the PI provided immediate feedback via voice commands through the laptop computer and smart glasses headset and the read and unread scores were recorded in the CRF. For example, if the preceptor did not observe the floor of the mouth on the laptop screen, he instructed the student via the teleconferencing feature to tell the patient to lift their tongue so that he could see the structure more clearly.

Immediately after the study on the same day, the Client Satisfaction Questionnaire, CSQ-8 was distributed to the students asking about their learning experience. The CSQ-8 was modified for our study with the word ˜service”, replaced with the word “feedback’” and only questions 1, 2,5,6,7 from the standard instrument were used. (Appendix 2)

No formal comparison between the control and the intervention arms were intended as this was a feasibility study. An overall > 80% complete participation within each group based on completeness of forms and questionnaires implied satisfactory feasibility to proceed with the main study. The scores from the CRF were transferred to a Microsoft Excel spreadsheet for analysis using the SAS statistical software package.

Results

A Descriptive summary is provided to characterize the participants’ impression and selected study outcomes (see Table 1).

Table 1 Overall distribution of participants impression and extent of reading

The total score from Appendix 1 was used for the primary outcome of this study, where a high mean/median score indicated increased effectiveness of using the smart glasses to learn how to perform an oral exam i.e., the effectiveness of smart glasses in learning to perform an oral exam were reflected in a corresponding high summary score, i.e., mean / median total score. The summary results of questionnaires From Appendix 2, were used to determine if the students that used the smart glasses valued the feedback.

The pilot study achieved an overall 100% completeness; therefore, feasibility of smart glasses was demonstrated in this pilot study (> 80% completeness as an a priori specified criterion in the IRB approved study protocol). One of the main study outcomes, the read score, which was described earlier, was markedly higher for the Smart Glasses arm (median = 10.0, IQR: 10.0,10.0) as compared to the control arm (median = 4.0, IQR: 4.0,5.0). This result concurred with our hypothesis. In addition, the second outcome of interest, the unread score was significantly higher for the control arm (median = 3.0, IQR: 2.0, 4.0) as compared to the smart glasses arm (median = 0.0, IQR: 0.0, 0.0). This result was also concurrent with our hypothesis (see Table 2). We observed no statistically significant difference in the extent of participant impression/satisfaction between the two arms (see Table 3).

Table 2 Association between score and intervention

Table 3 Association between impression/satisfaction and intervention

Several of the comments from the medical students in the intervention group are as follows:

The negative opinions were as follows:

“Have the glasses positioned prior to starting the oral exam to have an ideal view”.

“Sometimes hard to tell what the person (instructor) watching the video can see but otherwise a very helpful device”.

“If the lens alignment was more centered, it would be ideal so that the instructor and student can have the same perspective”.

Little disorienting at first but quick to get used to. Would take practice to be able to use it without a proper training”.

“I think it would be very helpful once some of the technical difficulties are resolved. I think it could make patients uncomfortable though but as a learning device”.

The positive opinions were as follows:

“I felt very comfortable receiving live feedback as I was doing the oral exam”

Very fun to use! Super helpful that an attending/preceptor can see what you’re seeing and give real time feedback.

“I think it’s very helpful”.

Discussion

This pilot study strongly demonstrated feasibility of using smart glasses for oral exam education and a promising result regarding the effectiveness of the smart glasses, despite the small size of the pilot study. The read score was higher in the intervention arm than in the control arm which demonstrated effectiveness of the smart glasses. The student feedback from the CSQ-8 questionnaire also demonstrated a more favorable outcome for the smart glasses group. Overall, our result suggests that a well-designed pivotal study can be carried out to definitively establish the effectiveness of the smart glasses as an educational instrument to enhance oral and dental care education within a medical school curriculum.

The smart glasses used in this study features a high-quality camera, voice commands, a long battery life, Wi-Fi and Bluetooth capabilities as well as two way, video conferencing applications. This same device has been used previously in another medical education study with positive results [15]. The medical students in our study expressed the device to be effective, interactive and useful however, not without some initial minor challenges with the technology. These opinions were similar to a pervious study using the same device [15].

As medical education is leaning towards standardized competency-based teaching and assessments [16] a device such as this would be a useful tool to provide immediate feedback as the leaner moves towards the different stage of skill development. As an assessment method, this device can also be used for direct observation of clinical skills without being too intrusive on the student doctor-patient interaction.

Smart glasses can be used to record an encounter or procedure and played back at a later time for review. This recording can be used to provide additional feedback and archived to add to the student’s or resident’s case log. Another use of the video recording feature is that it can be used in a blended-class teaching method in which a pre-recorded video of a procedure is shown to the students and followed by the student performing the same procedure in real time. This technique can also save time on demonstrations and reduce the cognitive load of class content [17].

Several limitations of the study include the small sample size coupled with the assessment set-up. Participants might be reluctant to report any of their negative impressions for fear of the possibility of being traced back or a desire to please. Therefore, this part of the result is likely to be biased. It would have been more desirable and rigorous to involve a “neutral” preceptor, i.e., an individual who is not directly involved with the study and blinded to the working hypotheses of the study, in the assessment process of the medical students rather than the PI to rule out the possibility of bias in favor of the smart glasses and a general conflict of interest. Another limitation of the study was that it was not controlled and not randomized. This could have led to a selection bias since the participants were found on a voluntary basis and were likely more interested in technology than the general medical student population. However, in spite of these limitations it did address Level 1 of the Kirkpatrick model, [18] which is the ‘reaction’ of the learners to this teaching method which was favorable and satisfactory. The positive reaction of our participants are similar to those in the Sirdhar et al. study, [16] which showed that medical students enjoyed using smart glasses technology. A future study using a more robust study design such as a randomized controlled trial would help address the other levels of the Kirkpatrick model.

Overall, the smart glasses show promise in enhancing hands on medical education curriculums and should be explored further.

Conclusion

This feasibility study of using smart glasses to enhance medical education of oral and dental examinations was well received by medical students and revealed positive results. This information can be used to design a more robust study to overcome the limitations of this pilot project as well as conduct other innovative medical education projects using this technology.