Does Simulation Make a Difference?

It’s a difficult question… the answer complicated. If you want the quick answer, skip to the bottom. Otherwise, enjoy the read

Simulation is becoming an important part of medical education. In the US, agencies such as the AHRQ have devoted entire funding blocks to simulation research to try to determine the effect of simulation on patient safety and physician skill.

It is generally accepted that simulation enhances learner engagement and allows for repetition of skills in a non threatening environment. It also allows for the opportunity to showcase rare clinical events with relative ease. Despite all of these seemingly positive attributes, evidence for simulation in education was/is somewhat lacking.

I had the privilege of attending a session at IMSH  by Dr David Cook (Mayo Clinic Medical School) and Dr Rose Halata (University of British Columbia). Their group has published the largest study to date outlining the evidence for simulation ( JAMA Vol 306 Sep 7 2011). This is the first large meta-analysis published on the topic. They scanned over 10,000 articles narrowing the search to 609 studies involving over 35,000 participants (students, nurses, physicians, paramedics) with the goal of answering the following question:

 To what extent are simulation technologies associated with improved outcomes versus no intervention?

The results are interesting. About 4% of the studies showed little or no benefit to using simulation with the remaining 96% of studies showed varying degrees of positive effects. The most marked effects were noted in the areas of: knowledge, time to complete a task, overall result (known as “product skills”), and behaviors. Of the 32 studies that looked into the effect on patient care, the results were mixed  – still showing modest improvement, but 2 studies showing a negative effect. This is somewhat surprising, but is likely confounded by the large heterogeneity (difference) found between the study groups and less defined outcome measures.

It is quite impressive to see the effect that simulation has on the above noted areas. Rarely does a meta-analysis yield such striking results. It is even more impressive if one considers the variables that are in play,and the nature of the participants and their learning experiences. Despite these confounding variables, simulation statistically outperformed the other interventions.

 Although this study is affected by the same biases of any meta-analysis such as study size bias, study selection bias and Simpson’s Paradox, it should be applauded for its sheer scope. Clearly determining the effect of simulation on learning is a monumental task.

What is quite clear, however, is that simulation is an inherently effective method of teaching many groups of learners and that it is here to stay. As more literature becomes available, this distinction will become even more clear.

So….

Does simulation make a difference?  YES

Stay tuned for further updates …..

Kish