During the academic year of 2019-2020, the Practicum Foundation conducted a pilot study to investigate the effectiveness of Practicum Script as a clinical reasoning training tool in undergraduate teaching and assessment. This international multicentre study, known as Practicum Universities, was developed as a partnership between the Practicum Foundation and the European Board of Medical Assessors (EBMA) and consisted of the application of the clinical reasoning simulator Practicum Script in the subject of Internal Medicine to senior medical students across more than 20 top medical universities in Europe and America.
The students develop their clinical reasoning skills in self-regulated learning and, while interacting with peers, improve the quality of care that they can provide
The development of decision-making skills is limited in the medical school curriculum. The Royal College of Physicians and Surgeons of Canada, the Accreditation Council for Graduate Medical Education in the United States and the new General Medical Council’s Good Medical Practice in the UK, among other relevant agents, have called for the implementation of specific methods for evaluation of the clinical reasoning competency of medical trainees in formative and summative assessments. To this end, Practicum Script facilitates reflective practice in undergraduate medical education.
In light of this ongoing concern, more than 2,500 medical students across eleven European and American countries have experimented Practicum Script in 2019 and 2020. This pilot program consisted of the simulation of 20 clinical cases based on prevalent conditions in Internal Medicine. For each case, the students were asked to generate hypotheses and justify them by identifying pertinent positive and negative findings in each case. They were then required to report, in five different clinical scenarios, how new data would have affected their original hypotheses. The students were able to observe the agreement between their responses and those of the experts, along with the experts’ rationales for each decision and literature-based clinical evidence.
Practicum Script proposes an approach to daily professional dilemmas in Internal Medicine to bring the student closer to clinical practice without the pressure of treating patients in person. The students develop their clinical reasoning skills in self-regulated learning and, while interacting with peers, improve the quality of care that they can provide. They face clinical challenges based on real cases with diagnosis, treatment and investigations in three types of settings, i.e., clinical wards, outpatient clinics and emergency departments. The students can practice – repetitively and independently – their decision-making skills.
Most students invited to answer a survey at the end of the course have rated Practicum Script and its methodology as “excellent”: 82.1% would recommend the incorporation of Practicum Script into their medical schools and 89.8% defined their overall satisfaction with the training experience as “satisfactory” or “very satisfactory”. The students also valued Practicum Script in terms of broadening differential diagnoses, application of theoretical knowledge into real-life situations and ease of use. Additionally, all medical schools involved in the pilot project expressed interest in continuing working with Practicum Script.
Cognitive diagnostic modelling
Practicum Universities assessed the feasibility of integrating Practicum Script as a methodology for longitudinal purposes, based on the above-mentioned level of satisfaction of educators and medical students and analysis of participants’ results. It also explored new methods that could support the use of Practicum Script for summative assessment and the application of cognitive diagnostic models to help students, educators and institutional stakeholders to identify knowledge/reasoning gaps at an early stage.
Cognitive diagnostic models have been applied to provide fine-grained information about students’ learning strengths and weaknesses. These models offer the possibility of more qualitative feedback for specific clinical reasoning skills (e.g., interpretation, comparison, prediction, inference, knowledge application, etc.) and better alignment with a competency-based curriculum.
The results of this unique research will be published shortly. In the meantime, the team in charge of the psychometric analysis confirms that reliability based on Cronbach’s alpha internal consistency coefficient and its 95% confidence interval has been excellent for hypothesis generation and hypothesis argumentation. Also, multidimensional confirmatory factor analysis indicate adequate goodness-of-fit indices.