Task force readies OSTATS for test
Carol R. Hutchinson, MD, right observes William Barrett, MD, working with sawbones in pilot of the Objective Structured Assessment Technical Skills
More than 20 orthopaedic residents from Chicago area hospitals will undergo an intensive day of didactic training, hands-on training and hands-on testing in the Orthopaedic Learning Center in January, as part of a study designed to test the effectiveness of skills courses offered in the facility.
The study, approved by the Academy's Board of Directors last year, is designed to provide controlled scientific investigation that documents the acquisition and retention of surgical skills techniques. It is being organized and monitored by members of the Academy's Task Force on Educational Effectiveness, who recently participated in the pilot of the Objective Structured Assessment of Technical Skills (OSTATS) that will be used when the 24 senior orthopaedic residents participate in the study's first phase on Jan. 23, 1999.
The task force spent part of its day at the hands-on task of experimenting with cadaver knees and sawbones to figure out which of the two learning tools was best suited for the tests the residents would be taking during the study. The task force also determined what preparation the cadaver knees and sawbones would need to undergo in advance of the pilot to make the testing uniform for each participant. With 100 people-including residents, examiners and assistants-participating in some form or another in the 12-hour testing day, the task force also spent considerable time talking logistics.
Task force chairman James Kasser, MD, said, "Our objectives were met. We wanted to work with the cadavers and sawbones and see if we could reduce the lab experience to something that people could be tested on."
On the day of the study, each resident will be asked to perform several of the steps involved in a total knee arthroplasty procedure and in an ACL reconstruction within a given time frame (pre-test.) The residents then will be divided into two groups to receive instruction on either the total knee or ACL procedure. Following the instruction period, the entire group will reconvene in the lab and be given the post-test on both the total knee and ACL procedures.
The OSTATS testing process for this study was designed by Carol R. Hutchison, MD, MEd, director of the University of Toronto Surgical Skills Center at the Mount Sinai Hospital. She also is the principal investigator for the study, which will eventually measure whether participants demonstrate an improvement in surgical skills after taking an OCL course. The study also will evaluate the effect of didactic training on skills.
Kasser said the Academy has been interested in finding a way to measure the effectiveness of the OCL for some time. "There are a number of people interested in the effectiveness of skills education within the Academy," he said. "And a number of people who are interested in credentialing. This test should help answer questions for both those groups by helping us to judge whether a person can do the procedure and whether additional skills can be acquired through performing that procedure."
The task force plans to run a second, smaller pilot program with a few residents in late fall and then make any necessary adjustments before the January testing.
In the second phase of the study, the OSTATS will be used in a standard OLC course. Fifteen orthopaedic surgeons attending a knee course for total knee arthroplasty and 15 attending a knee course for ACL reconstruction will participate in OSTATS before and after the course. This should determine whether OSTATS evaluation is valid and practical for assessment of OCL courses, according to Dr. Hutchison.
The study is being funded by $70,000 from the Academy's Council on Education and a $52,000 grant from the Orthopaedic Research and Education Foundation.