From unconsciously biased to bias awareness: a single site case study of the effectiveness of community-based implicit bias education amongst medical students

Abstract

Objective

Community-based education places students’ education in a community context and exposes students to patients’ social and environmental circumstances. Studies that evaluate the effectiveness of community-based education on bias awareness regarding migrants are limited. This study answered the following questions: (1) How do students’ knowledge, attitude, and skills regarding their cultural competencies, including ethnocultural empathy and implicit biases, change during community-based implicit bias education? (2) What is the perception of students towards this type of education?

Method

This single site case study used a pre-survey and post-survey consisting of the psychometric instrument Implicit Association Test and the questionnaire Scale of Ethnocultural Empathy as quantitative methods. These were combined with focus group interviews and qualitative analysis of reflection reports as qualitative methods. This study was conducted amongst third-year students of the bachelor of medicine at a medical faculty in the Netherlands. Thirty-five students completed the pre-survey and twenty-one students completed the post-survey. Thirty-eight students gave consent to analyze their reflection report. Twenty-three students took part in the group interviews.

Results and conclusion

The quantitative analysis showed that the students’ scores on the ethnocultural empathy scale decreased. The biases for skin color and ethnicity reduced after the educational intervention based on the Implicit Association Test. The qualitative analysis showed that community-based implicit bias education increased students’ knowledge about the network of non-profit governmental organizations in the domain of migrant health in the Netherlands, the needs of migrants, and the barriers migrants face in the healthcare system. Students’ self-reported attitudes changed after the educational intervention, since their awareness increased of, for instance, the factors that play a role in migrant healthcare and the differences in epidemiology between patients with and without a migration background.

Clinical trial number

Not applicable.

Background

Bias and discrimination are current and important problems in healthcare. Discrimination is: “the unjust or prejudicial treatment of different categories of people, especially on the grounds of ethnicity, age, sex, or disability”. Bias is: “Tendency to favour or dislike a person or thing, especially as a result of a preconceived opinion” [1]. In the Netherlands, 27% of the population experiences discrimination [2]. Causes of discrimination are, amongst other causes, prejudices (negative feelings) and stereotypes (characteristics that are prescribed to a certain group of people) [3]. These biases and stereotypes affect our behavior, especially when these are implicit or unconscious. Therefore, these biases can lead to unequal treatment of different groups of people. American research shows that discrimination, biases, and stereotypes play a major role in healthcare [4]. A European study showed that discrimination is also a problem in healthcare in the Netherlands, as 2.7% of the Dutch respondents stated that they felt discriminated by general practitioners based on their ethnicity or income [5]. These percentages of perceived discrimination are probably underestimations. Research shows that biases and discrimination lead to inequality: worse healthcare provider-patient communication, worse treatment advices, less therapy compliance and worse healthcare outcomes [6]. Discrimination is thus a current and important problem in healthcare.

Education that helps students develop cultural competency and become aware of their implicit biases is a crucial step in the fight against discrimination. This kind of education aims to improve equity and social justice and care for patients from all backgrounds. In this study, cultural competencies are defined as knowledge, skills, and attitudes students need to work with a diverse group of patients, in this case specifically patients with a migration background.

An important aspect of cultural competencies is bias awareness. In recent years, many studies have been conducted on educational methods to teach students’ bias awareness. For example, storytelling can improve students’ attitude towards stigmatized groups because this helps them to take different perspectives which leads to improvement of the attitude towards the out-group, the group with different characteristics than the group to whom students identify themselves [7]. A video intervention about institutional racism and white privileges was effective for improving general ‘race awareness’ and the recognition of white privileges and racism [8]. Furthermore, mental imaging of counter stereotypes reduces implicit biases more than half [9]. Another study showed that fourteen mechanisms can be distinguished to take into account in the educational methods for teaching anti-prejudice. Amongst these mechanisms are the provision of information and the involvement of students with respect from both sides, teachers and students. Other mechanisms are the emphasis of commonality and differences between groups, the focus on the needs of and the situations of local communities by spending time with them, the arrangement of appropriate contact between students and society, and the evaluation of learning outcomes with a pre-and a post-survey [10].

As shown in the previous paragraph, in recent years, many educational interventions to increase bias awareness have thus been designed and evaluated. The roots of this pedagogical paradigm are based on frameworks of cultural competencies which are often used in medical education [11,12,13]. A promising intervention that has been barely described in the literature and which has not yet been evaluated thoroughly is community-based education. Community-based education is: “Learning activities that use the community extensively as a learning environment, in which not only students but also teachers, members of the community, and representatives of other sectors are actively engaged throughout the educational experience” [14, p.8]. The limited research that exists shows that community-based education helps students to recognize and mitigate unconscious biases [15]. Furthermore, community-based education increases students’ knowledge of health inequities [16] and their willingness to work with the underserved [17].

The aim of this study is to provide insight into the knowledge, attitudes, and skills medical students acquire during community-based implicit bias education including the development of ethnocultural empathy and reduction of implicit biases. Furthermore, it provides insight into the perceptions of students regarding this type of education. These insights can be used to design community-based implicit bias education in an effective way. By creating bias awareness amongst (future) healthcare professionals, equality of healthcare can be improved, and better healthcare can be provided to a diverse patient population. The research questions are: (1) How do students’ knowledge, attitude, and skills regarding their cultural competencies, including ethnocultural empathy and implicit biases, change during community-based implicit bias education? (2) What is the perception of students towards this type of education?

Method

Design

This study employed a single site case study research design. The context of this single site case study was the bachelor of medicine at the University of Amsterdam in the Netherlands and the course was the elective module Cultural Diversity in Practice. Multiple sources of quantitative and qualitative data were used. The changes in knowledge, attitudes and skills and students’ perceptions were analyzed by semi-structured focus groups and qualitative analysis of reflection reports. To assess the ethnocultural empathy and implicit biases, before and after the intervention, the implicit biases were assessed by the psychometrical instrument Implicit Association Test (IAT) and the ethnocultural empathy was assessed by the Scale of Ethnocultural Empathy (SEE).

Data collection

Context

At the beginning of the third year of the medical curriculum, an elective course module is integrated that aims to prepare students for their clerkships. We designed as part of this course an elective module in which students complete an e-learning before they have a plenary introductory meeting in which they do experiential, active learning activities. They visit a non-profit organization in the domain of migrant health and exchange their experiences in the plenary closing meeting. In addition, they write a reflection report. We designed the educational intervention based on the teaching approach flipped classroom and the previously mentioned mechanisms. Sing et al. defined the flipped classroom as a reversion of the learning process of the traditional classroom in which students acquire knowledge before class and deepen and apply this knowledge during class so that higher-order learning goals can be achieved more easily [18, 19].

The learning goals of this elective course are: (1) apply previously acquired knowledge and skills in a new situation, (2) show responsibility and independence in the execution of the assignment, (3) work together in a professional way with fellow students and other stakeholders, (4) reflect on the experience acquired in this assignment, (5) reflect on their own way of thinking and behavior in relation to this assignment and practical experience, (6) shows accountability to the supervisor in the form of a report, (7) review self-reflection based on the judgement of others, (8) show accountability to fellow students in the form of a presentation about the results, (9) be open to feedback on your report, presentation, and behavior.

Figure 1 presents the procedure. Before the plenary kick-off meeting, students had to complete an e-learning. In the e-learning (pre-existing, not specifically designed for this course), students acquired knowledge independently and applied this knowledge in the following educational activities. The e-learning focused on working with patients with a migration background. At the beginning of the plenary kick-off meeting, students completed the pre-survey. Then, students did the invisible knapsack exercise. The invisible knapsack exercise aims to approach aspects of life from different perspectives and thus creating awareness about diversity and privileges. The exercise consists of fifteen statements and when the statement applies to the student they go two steps forward. An example of a statement is as follows: If one of your parents has got an university degree or an university of applied sciences degree. A study by Cory stated that it is necessary to unpack the ‘invisible knapsack’ to enhance social justice [20]. At the end of the plenary kick-off meeting, students received instructions for the small-group exercise. After the plenary kick-off meeting, students had to choose one migrant and refugee health service for the small group exercise, they formulated interview questions and conducted an interview with someone from the non-governmental organization (NGO) or a refugee. They could choose between Doctors of the World (a human rights organization, providing emergency and long-term medical care), a general physician who works in an asylum seeker center, the foundation Abadi (supporting asylum seekers living in asylum seekers centers with activities), and a refugee who completed an integration trajectory at the Amsterdam UMC as a nurse. After the small-group exercise, students attended a plenary closing meeting. First, they participated in focus groups to reflect and exchange experiences. At the end of the plenary closing meeting, students completed the post-survey. After the plenary closing meeting, they wrote a report in which they reflected on their scores on the IAT, the interview with the NGO or refugee, and the acquired knowledge, skills, and attitudes. In total, this course took eight weeks.

Fig. 1

Context of the research

Ethics

The Dutch Society for Medical Education (NVMO) granted ethical approval for this study (case number 2022.1.4). We can confirm that all procedures were carried out in conformity with the applicable guidelines and regulations. The information letter was posted on the electronic learning platform. At the start of the first class, the participants were also given an information letter on paper and they got the possibility to sign the informed consent. The pseudonymization of the data ensured privacy.

Sampling and recruitment

The study population consisted of third-year students of the bachelor of medicine at a medical faculty in the Netherlands. The inclusion criterion was that students had to take part in the elective module Cultural Diversity in Practice, a course that prepares students for their clerkship at the end of their third year. Medical students who did not fulfill this criterion were excluded from this study. We used purposeful sampling to recruit the participants. The students were informed about this study via a message posted on the electronic learning platform two weeks before the course started. Jeanine Suurmond and Janique Oudbier provided this elective course to the students. They are therefore teachers of the students who participated in this elective course and took part in this study. We would like to note that we are aware of the fact that the elective nature of the course could have led to self-selection bias. The focus groups, the completion of the IAT and the SEE were part of the class. However, the students were informed that they did have an option to decline to participate in this study without having any consequences. The study took place from April to June 2023.

Materials

Interview guide

The students were divided randomly in two groups during the plenary closing session. One focus group included eleven students and the other twelve. The focus groups were moderated by the main researcher (M.J. Oudbier) and J.L. Suurmond. The focus group interviews were conducted in Dutch and consisted of one session of approximately one hour. After the analysis, the quotes of the participants were translated to English. We tried to translate as literally as possible to ensure that the meaning did not change. The data was transcribed verbatim by a student-assistant. We ensured consistency between the focus groups by means of an interview guide, which we developed for this study. The interview guide is added to this article as supplementary file 2. The following questions were included in this guide: (1) To which degree did the educational intervention contribute to the acquisition of the different cultural competencies?, (2) In which ways has the educational intervention contributed to the acquisition of cultural competencies?, (3) To which degree contributes the acquisition of these cultural competencies to your professional development as a physician?, (4) Which competencies would you like to develop further? The interview guide is added to the supplementary material.

Reflection reports

At the end of the course, students individually wrote a reflection report. In this report they had to reflect on the following aspects: (1) experiences of the interview with the NGO or refugee and outcomes of the interview, (2) scores on IAT, (3) the way the acquired insights might affect their care to migrants, (4) the acquired competencies and the competencies they would like to develop further. Thirty-eight student reports were analyzed.

Implicit association test

To assess the effectiveness of the educational intervention on the implicit bias, the psychometric instrument IAT was conducted before and after the intervention, in April and June. The IAT is developed by Greenwald, McGhee, and Schwartz in 1998 and measures the association between constructs in the memory [21]. The students completed the Dutch version of the test and they could chose to complete the test for ethnicity or for skin color. The effect is calculated by the difference in reaction time between compatible conditions and incompatible conditions. Compatible conditions are words that are highly associated with each other (such as flower and positive) and incompatible conditions are words that are slightly associated with each other (such as insects and negative). The assumption is that the task is easier with compatible conditions than with incompatible conditions and because of time pressure it is hard to consciously regulate this process. The conditions in the ethnicity test are Moroccan and Dutch and the conditions in the skin color test are Black and White. The IAT is often used because of its predictive validity that significantly exceeds that of self-reported measures [21]. Students self-reported their own scores on the IAT and we have compared the scores before and after the educational intervention.

Scale of ethnocultural empathy

To assess the effectiveness of the educational intervention on empathy towards different ethnic groups, the SEE was conducted [22]. This is a standardized questionnaire which consists of 31 items measured on a 5-point Likert scale (from strongly disagree to strongly agree). This scale consists of four sub scales: empathic feeling and expression, empathic perspective taking, acceptance of cultural differences, and empathic awareness. Examples of items are: I share the anger of those who face injustice because of their racial and ethnic background. It is easy for me to understand what it would feel like to be a person of another racial or ethnic background other than my own. Cronbach’s Alpha for the whole scale was 0.91 and for the sub scales it was respectively 0.89, 0.75, 0.73, 0.76. A confirmatory factor analysis provided evidence for the generalizability [22].

Survey

The survey conducted pre and post intervention consists of demographic questions, the IAT and the SEE. The survey is added to supplementary file 3.

Data analysis

Qualitative

The framework analysis was used to deductively analyze the reflection reports and the focus group transcripts. A thematic framework is composed based on the literature and the aims of the study combined with the themes that are expressed by the participants. The cultural competency framework as developed by Suurmond et al. was used as the starting point [23]. This framework distinguishes knowledge (e.g. knowledge of effects of refugee hood on health), attitudes (e.g. awareness of one’s own prejudices and tendency to stereotype), and skills (e.g. ability to explain what can be expected from healthcare) regarding cultural competencies. This ‘index’ of codes was systematically applied on all data. The next steps were charting, mapping, and interpretation in which the data was structured, abstracted, and summarized [24]. If codes did not fit in the framework, emergent codes were added to the framework. We made use of the qualitative software analysis tool MaxQDA.

Our research team, which included two educational scientists, a principal educator, a vice-dean of the faculty, and two students, discussed the findings in order to gain credibility. We included participants’ quotes to bolster credibility. To make sure we understood the information correctly and to enhance confirmability, we questioned the participants for clarification. We discussed the findings of the analysis in the team till consensus was reached to improve confirmability. Focus groups were conducted until saturation was reached to improve the dependability. We have provided as much detail as possible in our description of the participants, research methods, and procedures to maximize transferability. Data saturation were reached when new focus groups did not produce new information compared to the previous ones.

Reflexivity

We would also like to mention the researchers’ positionality. The research team consists of three woman and one man. They and their parents are all born in the Netherlands. S.M. Peerdeman is member of the board of the Amsterdam UMC and the vice-dean of the faculty of medicine of the University of Amsterdam. J.L. Suurmond is associate professor at the Amsterdam UMC. M.J. Oudbier is a PhD associate at the Amsterdam UMC. T.B.B. Boerboom is an education policy director at the University of Utrecht. This study is part of a Comenius fellowship, which aimed to better integrate social accountability in medical education. All members of the research team were involved in other research concerning social accountability in medical education and have stressed the importance of this topic.

Quantitative

We used descriptive statistics to assess demographic data, the score on the IAT and the score on the SEE. We made a frequency distribution of the scores on the separate items of the SEE. A paired t-test was conducted to assess whether the pre and post scores on the scales of the SEE were significantly different. The q-q plot shows that the data is normally distributed, which is an assumption of the t-test. Because of the small number of participants we have not tested whether these scores differ based on gender or ethnicity.

Results

Participant demographics

Sixty students registered themselves for this course. Thirty-eight students gave consent to analyze their reflection report and twenty-three students took part in the focus groups during class. The IAT and the SEE were conducted at the beginning of the plenary kick-off meeting (pre-survey) and approximately six weeks later at the end of the plenary closing meeting (post-survey). Thirty-five students completed the pre-survey and twenty-one students completed the post-survey (see Table 1).

Table 1 Participant demographics

The characteristics of the participants showed that the age was comparable for both groups, pre and post. The groups slightly differed based on gender. Almost all participants were born in the Netherlands. Other countries of birth of the mother were Italy, Peru, Egypt, Turkey, Suriname, Morocco, Pakistan, Vietnam, Japan, Somalia, Afghanistan, and Curacao. Other countries of birth of the father were Egypt, Turkey, Suriname, Morocco, Pakistan, Afghanistan, Vietnam, and Somalia. In the Netherlands, the operationalization of ethnicity is based on students’ and parents’ country of birth, because this represents students’ identity.

Perception of acquired knowledge, attitudes, and skills

The qualitative analysis of the focus groups and the reflection reports showed several main themes and sub themes. The main themes are knowledge and attitudes. Skills was not a main theme that came forward in the analysis, because no data supported a change in skills. Table 2 presents the acquired knowledge and attitudes. The qualitative analysis also revealed several sub themes. The acquired knowledge consists of the sub themes: (1) Barriers migrants face in the healthcare system, (2) Needs of migrants, (3) Network of NGOs in the domain of migrant healthcare in the Netherlands. Acquired attitude consists of the sub themes: (4) Awareness of factors that play a role in migrant healthcare, (5) Awareness of differences in epidemiology and manifestation of diseases in migrants, (6) Awareness of differences in healthcare expectations between patients with and without a migration background, (7) Awareness of differences in healthcare expectations between patients with and without a migration background, (8) Awareness of strengths and points of improvement of migrant healthcare.

Table 2 Acquired knowledge and attitudes of cultural competency

As mentioned in Table 2, the reflection reports and group interviews showed that one of the changes in the students’ attitude is the awareness of differences in healthcare expectations between patients with and without a migration background. It is good to notice that a second-generation migrant student said that these healthcare differences not only exist in people who have recently migrated to the Netherlands, but also in second-generation migrants. “They are born here, they are raised here and they think it is strange that they get no treatment. If they are ill they are used to get advice soon, to get medicines or even a boost of vitamins and that is here not the case” (focus group 1). To acquire cultural competency skills students should apply the acquired knowledge in practice and gain experience themselves with the communication with migrants. “The cultural competencies which I would like to further develop are the skills a physician should have to deliver adequate healthcare to migrants. I think you will develop these specific communication skills further when you treat patients in your profession as a physician” (reflection report 32). However, students also stated that to master these skills many training experiences are required and therefore it is best to acquire as much knowledge as possible in this short elective course. “It is a stepping stone to your work as a physician. …. A seed is planted, on which you can grow” (focus group 2).

Perception of ethnocultural empathy

The qualitative analysis showed that participants got insight into the position and needs of migrants and the barriers with which they are coping. They became aware of the importance of providing language support and showing respect for and understanding of cultural differences. I became aware of the fact that an interpreter or an interpreter phone is insufficient to create a good physician-patient connection or to really understand the patient. The norms and values are also different in different cultures. Now I have understanding of the fact that they need a culture coach (focus group 2). Students who had a multicultural background mentioned that their background helped them to take the perspective of migrants, show understanding of cultural differences, and form biases less quickly. Because of the fact that I have a migration background myself and my social network is diverse, I expect to be able to show a better understanding of people with a migration background who come to my consultations (reflection report 8).

Perception of implicit bias

The analysis of the reflection reports showed that most students who had a preference according to the IAT did not think that this score would affect their work as a physician. Almost all students who had a neutral score on the IAT thought that they had biases even though the test showed no preference. Many students mentioned that they thought it is unavoidable to have biases, but that it is about becoming aware of these biases. In the current society, it is unavoidable to develop biases. However, it is of importance how we deal with these biases. Becoming aware of these biases is one of the most important steps for creating equal changes in healthcare (reflection report 1). Some students mentioned that there could be a learning curve, because they have made the test twice. Several students who scored neutral on the IAT concluded that they do not have biases.

Perception of bias awareness education

The analysis of the focus groups and the reflection reports also showed several findings regarding the perception of this type of education. Many students mentioned that every healthcare provider should be educated about migrant healthcare. Students have chosen this elective course, because in the bachelor curriculum there is insufficient focus on migrants. We learn which diseases are more common amongst certain population groups, but we do not learn how to cope with cultural sensitivity (reflection report 2). Students see the relevance of this type of education because of the increasing number of migrants in the Netherlands and they see the value of the acquired knowledge and attitudes. I am glad I have chosen this elective course, because it prepared me better for my future career as a physician (reflection report 13). I do not think I am more comfortable in the way of skills and communicating with the patient, but I know that there are resources available such as an interpreter or a cultural mediator, which make the provision of healthcare easier (focus group 2). Students are impressed of the activities the NGOs organize and the intrinsic motivation of the volunteers. The way the people of the organization set their own time, money and energy aside to help others who go through the same process as they went through is admirable (reflection report 3). Students would like to further develop their cultural competencies by, for instance, acquiring more knowledge about different cultures. This way, they will learn how to be culture sensitive and to cope with cultural differences in order to provide every patient the best care possible. It is important to further develop my cultural competencies, so I can provide all patients, regardless of differences in culture and background, the best care possible (reflection report 14). Students consider advocacy on micro level as a part of their role as physician. If physicians have a passion for certain aspects of healthcare which they would like to advocate for on a macro or meso level they recognize the possibilities of physicians to advocate for change and their function as a role model. For instance, in the domain of preventive medicine and planetary health. In addition, students mentioned that applying the acquired knowledge might be difficult.

Development of students’ ethnocultural empathy

An important aspect of student attitude regarding migrant health is ethnocultural empathy. The quantitative analysis of the SEE showed that the students’ scores on all of the four subscales of the SEE decreased. For the outcome variable empathic feeling and expression, this means that students scored themselves lower on aspects such as seeking opportunities to speak with individuals of other racial or ethnic backgrounds about their experiences. For the outcome variable empathic perspective taking, this means that students scored themselves lower on aspects such as knowing what it feels like to be the only person of a certain race or ethnicity in a group of people. For the outcome variable acceptance of cultural difference, this means that students scored themselves lower on aspects such as understanding why people want to keep their indigenous racial or ethnic cultural traditions instead of trying to fit into the mainstream. For the outcome variable empathic awareness, this means that students scored themselves lower on aspects such as recognizing that the media often portrays people based on racial or ethnic stereotypes. Table 1 in the Appendix shows the means and the standard deviations of the separate items of the SEE. Table 3 in the Appendix shows the frequency and percentages per score on the separate items of the SEE. Figure 1 to Fig. 8 in the Appendix show the frequency distributions of the separate items of the SEE. The paired sample t-test did not show any significant changes. Table 3 shows the means and standard deviations of the outcome variables.

Table 3 Descriptive statistics ethnocultural empathy scale

Development of students’ implicit bias

Another important aspect of student attitude regarding migrant health is implicit bias. Table 4 shows the frequency distribution of the IAT pre and post the educational intervention. These frequency distributions show the percentage of people who have a slight, mediocre or strong preference for either a light skin color or a dark skin color and a Moroccan ethnicity or a Dutch ethnicity or have a neutral score. Several students completed both tests, skin color and ethnicity. In the post-survey less students had a mediocre or strong preference for a dark or a light skin color or a Dutch or Moroccan ethnicity (pre: 40.48% vs. post: 7.41%).

Table 4 Scores IAT

Discussion

Based on a literature search in the medical educational databases MEDLINE, PsycINFO, PubMed, Web of Science, and Scopus there is no similar research in this domain. Therefore, we have chosen to reflect on methodologies and explain the reasons for our results. In our study, the students showed an increase in self/reported knowledge and awareness about migrant health. However, they showed a decrease in their scores on the ESS. This incongruence could be explained by the stages of learning model, which distinguishes four stages of learning: unconscious incompetence, conscious incompetence, conscious competence, and unconscious competence [25]. These stages range from being unaware of a knowledge or skills gap to unconsciously performing a skill or recalling knowledge. The fact that the scores were lower at the post-test than at the pre-test might be explained by students shifting from unconscious incompetence to conscious incompetence due to the educational intervention. The qualitative analysis of the reflection reports and the focus groups showed that students acquired knowledge and awareness and realized what they do not know about cultures that are different from their own and the aspects that they were unaware of and how this affects their empathy towards different ethnic groups. They became aware of their knowledge or skills gap regarding migrant health. Students also started to realize the significance of knowledge and skills to effectively communicate with migrants. This is an essential step to teach students further skills and knowledge in this area of healthcare. To ensure this shift takes place, it is important to collaborate with a non-profit organization on migrant healthcare or a refugee who can share authentic stories.

The IAT can serve two aims, namely a metric of bias and a tool to elicit reflection and discussion. Sukhera et al. conducted a meta-narrative review on the use of the IAT in health professions education [26]. The authors found that on the one hand, studies have a positivist epistemological viewpoint and state that the test should be conducted before and after the intervention to assess the effectiveness of an educational intervention [27, 28]. On the other hand, studies consider the test as a catalytic reagent to facilitate teaching and learning in order to identify strategies to teach bias [29,30,31]. In this study, the two approaches were combined. The test was used to assess the effectiveness of the educational intervention and it was used to start a reflection on the meaning of the scores, implicit biases, and the effect of implicit biases on quality of healthcare. Despite the strengths of this test, there are also some points of criticism on this test that should be addressed. Uhlmann, Brescoll, and Paluck concluded that the test assesses egalitarian negative associations (oppression, mistreatment, and victimization of groups in society) instead of reflecting a person’s authentic negative associations [32]. The associations that are assessed by the test could be based on empathy instead of prejudice. When external explanations are endorsed, the associations are based on empathy and as a result people will have a positive attitude towards low status groups. This means that assessed negative associations are not always an indication of prejudice and do not always have to be a bad outcome [33]. However, this also works the other way around. The test can be trained or manipulated. When the test shows that someone has no implicit biases this does not mention automatically that someone has no implicit biases. We addressed these points of criticism in the reflection on the test during the plenary kick-off meeting.

A strength of this study is the rich data that is collected by combining several research methods. The quantitative data enabled us to assess the effectiveness of the educational intervention regarding ethnocultural empathy and implicit biases. The qualitative data gave us insight into the self-reported knowledge and attitude acquisition and students’ perceptions. This way we were able to get a comprehensive picture of the development of students during community-based medical education to increase bias awareness and their perceptions. Another strength of this study is the practice-oriented focus. The learning environment enabled us to apply insights retrieved from previous literate into practice. We designed community-based implicit bias education, we taught this course to our students, and we conducted a scientific study on this course.

We are aware of the fact that this course was an elective and our study population is therefore restricted to the students who decided to take part in this course and to a single university. This group of students is generally highly interested in bias awareness and cultural competencies. The results might thus be more positive than when all medical students would have participated. Furthermore, community-based implicit bias education might have more impact if all students receive this type of education and not only the students who already have affinity with this topic. However, we should notice that this elective was one of the first electives that was fully booked and previous years this elective was also overbooked. This could indicate the need that is expressed amongst medical students to learn more about this topic. However, a number of other issues can be at play here such as scheduling availability and workload in comparison to other electives. The small sample size and the self-selection bias necessitate a larger study involving the entire class in the future, which could offer more robust insights into the bias awareness development in this educational program and further clarify the impact of community-based education on cultural competence and its underlying factors. Another limitation might be that we have not conducted a confirmatory factor analysis for the ESS.

Carter, Onyeador, and Lewis stated that it is important for organizations to combine anti-bias training with other initiatives regarding diversity and inclusion, such as a designated diversity officer and mentoring programs, to yield meaningful change [34]. At our faculty we currently have two diversity officers and we have two student diversity officers. We also have a mentoring program that is part of the UvA diversity policy [35].

Although the quantitative analysis showed that the students’ scores on the ethnocultural empathy scale decreased, the biases for skin color and ethnicity reduced after the educational intervention based on the Implicit Association Test. The qualitative analysis showed that students acquired important knowledge and awareness in several domains: community-based implicit bias education increased students’ knowledge about the network of non-profit governmental organizations in the domain of migrant health in the Netherlands, the needs of migrants, and the barriers migrants face in the healthcare system. Students’ self-reported attitudes changed after the educational intervention, since their awareness increased of, for instance, the factors that play a role in migrant healthcare and the differences in epidemiology between patients with and without a migration background.

Based on the analysis of the data, we can provide some recommendations to further improve community-based bias awareness education. First of all, this study showed that there is a need to learn about migrant health amongst medical students. Therefore, we commend that bias awareness education should be mandatory to make sure all students learn about this topic, because every physician provides care to a diverse patient population. An introductory course on bias awareness, such as this one, can teach students knowledge and awareness and provides them with tools to deal with a diverse patient population. However, further education is needed to enable them to develop skills in this domain. Secondly, during the focus groups students stated that it is important to involve experience experts in this type of education, such as people who have migrated or patients who have experienced bias from care providers. This makes talking with a diverse group of patients and taking their perspective even more tangible than interviewing someone who works at a non-profit organization for migrant care. Our last recommendation is to integrate bias awareness education throughout the whole curriculum. Students stated in the focus groups that migrant health is complex and that it is advisable to start teaching students about this already in the beginning of the curriculum. Students should be supported to develop their competencies from unconscious incompetent to conscious competent. This way all medical students will be prepared for their future profession, to deliver care to a diverse patient population, and to work towards an equal healthcare system.