Introduction: Simulation-based teaching and assessment are integral to education in the health professions, with simulated patients (SP) being a widely accepted strategy. Ensuring high-quality SP role-portrayal is crucial for the authenticity and standardization of assessments, particularly in high-stakes exams like the Swiss Federal Licensing Examination (FLE).

The study assesses the quality of SP role-portrayal over consecutive instances of the Swiss FLE. We hypothesized that the quality of role-portrayal improves over time.

Methods: The study employed the FAIR OSCE instrument to assess SP role-portrayal in five consecutive FLE exams from 2016 to 2021. The instrument, developed between 2011 and 2014, includes four categories: introduction, delivery of information, portrayal, and others. Data analysis involved retrospective examination of FAIR OSCE ratings, calculating item scores, and overall mean scores for each exam year.

Results: The study involved 37 SP educators observing 1803 SP-candidate interactions across five exam sites. Results demonstrated a continuous improvement in SP role-portrayal over the five-year period, with significant differences between 2016 and subsequent years. The overall mean scores of SP role-play ratings increased steadily, indicating a positive trend in SP performance.

Discussion: The findings supported the hypothesis of continuous improvement of SP role-portrayal within the Swiss FLE. The quality of role portrayal not only improved consistently but also maintained a very high level, with no items on the FAIR OSCE instrument rated “do not agree” more than 5% of the time. This suggests that SPs role play aligned well with case scripts, reflecting the intended authenticity and standardization of assessments.

Limitations were acknowledged, including potential bias in local SP educators rating their own SPs and the study’s sole focus on SP role-portrayal. While the findings contribute to understanding SP effectiveness in standardized, high-stakes clinical exams, the study did not scrutinize other potential sources of variance.

Conclusion: In conclusion, the research demonstrated a continuous improvement and high quality of SP role-portrayal in the Swiss FLE over five years. Well-trained SPs, assessed using the FAIR OSCE instrument, play a crucial role in maintaining the standardized and high-quality nature of clinical skills exams in a high-stakes context. Further research could explore additional factors influencing overall exam quality and address potential biases in SP educator ratings.