Two years from lockdown, the Association for Medical Education in Europe (AMEE) will celebrate its 50th anniversary conference in the French city of Lyon, located at the confluence of the rivers Rhône and Saône, and we will be there by the end of August to present the final results of the multicentre pilot experience of the clinical reasoning training programme Practicum Script for medical students, known as . The key message? Practicum Script can contribute to improve care decisions of future physicians, by helping users to develop effective thinking skills to deal with complexity and ambiguity in clinical practice.
the study is one of the pioneers in the use of cognitive diagnostic modelling, a novel multidimensional psychometric paradigm
In 2019 and 2020, the Practicum Foundation conducted an international study among 2,500 students to investigate the utility of the simulation-based programme Practicum Script as a clinical reasoning training methodology in undergraduate teaching and assessment. Today, the findings show that Practicum Script helps students strengthen their thinking skills and their ability to manage uncertainty during decision making, with evidence of acceptability, reliability and validity. Furthermore, the study is one of the pioneers in the use of cognitive diagnostic modelling (CDM), a novel multidimensional psychometric paradigm, which has led these results to be presented at AMEE 2022.
This oral session will take place on the morning of Monday 29 August. So far, the model, developed in alliance with the European Board of Medical Assessors, the Imperial College London and dozens of renowned specialist from all over the world, has been implemented by around 35 universities in Europe, the US, Latin America and Singapore. The standard approach covers the final three years of the degree, with an adapted formative proposal for each academic year, and a summative exam for final-year medical students. Under the motto of “Redefining health professions education together”, this conference means a unique opportunity to get to know the final results of Practicum Universities.
Grounded in the dual process theory, Practicum Script is the first simulator to include heuristics as part of the student’s learning process. For this project, the assessment material consisted of 20 internal medicine clinical cases, drawn from real patients, and validated by an international panel of experts. For each clinical case, students were asked to generate hypotheses in a ‘free-text’ format, assign them a likelihood and justify them. Subsequently, they had to report, in 5 different clinical scenarios, how new data might affect their assumptions. Feedback blended the opinions of experts with evidence-based medicine.
The point was to allow students to practice their problem-solving skills repetitively, longitudinally, and autonomously, without any risk for patient safety. In short, Practicum Script targets the entire reasoning process rather than a specific outcome, measuring the way participants use know-how to solve daily situations beyond the statistical patient. And the inclusion of CDM would enable more qualitative and meaningful feedback for learners and teachers regarding specific clinical reasoning skills, such as concentration, identification, association and integration of key data, inference, prediction, active use of knowledge, critical judgment, metacognition and reinterrogation of signs.
Psychometric properties and perceived benefits
In the opinion of the psychometrician of the study, the professor at the University of Maastricht Carlos Collares, “instead of a single line, we should conceptualize clinical reasoning as a 3D space”. In this case, over a total of 2,457 final year medical students from 21 medical schools in 10 European and American countries, results showed “Cronbach’s alpha coefficients were excellent for hypothesis generation (alpha = 0.909, 95% CI = 0.902-0.916) and hypothesis argumentation (alpha = 0.926, 95% CI = 0.920-0.931)”. Also, cognitive diagnostic modelling analysis of the knowledge application items resulted in a good absolute test fit for RMSEA (0.0497) and SRMR (0.0893) indices.
In total, 1,502 volunteer participants (61%) completed all cases, and 89,8% of respondents to a survey rated the experience as “excellent” or “good”. They highlighted that Practicum Script triggered differential diagnosis and made them understand that more than one valid solution is possible in real clinical practice. The levels of satisfaction and perceived benefits by the medical educators were also outstanding. In fact, a vast majority of them keep collaborating with the Practicum Foundation and they are thankful for being able to use a more qualitative approach which allows them to do a much better job in terms of remediation.
These conclusions lead to a promising future area of research to unravel the underlying cognitive processes involved in doctor’s thinking.