OSAE provides CME, MOCŪ opportunities
Exams are reliable tests of knowledge; valuable tools for Maintenance of Certification
By Paul Tornetta III, MD
Each year, hundreds of orthopaedic surgeons use the AAOS Orthopaedic Self-Assessment Examination (OSAE) as part of a life-long learning program. In addition to providing necessary continuing medical education (CME) credits that enable surgeons to meet licensing and certification requirements, the OSAE also helps AAOS members sharpen their clinical skills and keep up with the latest outcome data, technologies and surgical developments.
(From left) Gregory J. Schmeling, MD, and Robert F. Ostrum, MD, members of the Evaluation Committee, review test questions in developing the next Orthopaedic Self-Assesment Examination.
The Evaluation Committee develops the OSAE and the Orthopaedic In-Training Examination (OITE) based on test development and scoring specifications in 12 orthopaedic topic areas such as foot/ankle, trauma, sports medicine and pediatric orthopaedics.
2005 OSAE results
More than 900 orthopaedic surgeons participated in the first scoring for the 2005 OSAE. An analysis of the items in the examination indicates that the degree of difficulty and the degree of reliability for the 2005 OSAE were consistent with previous years’ examinations. The reliability of a test measures how consistently each item scores. Test questions should be appropriate for the group and the material being covered. The Committee’s goal is an examination with a reliability score between .88 and .90. The 2005 OSAE had a reliability score of .89, which indicates 89 percent consistency in examination scores.
An examination’s degree of difficulty provides an understanding of the participant group’s knowledge. A higher number indicates the examination is less difficult. The Examination Committee aims for a degree of difficulty ranging from .67 to .71 in OSAEs. The 2005 OSAE had a .68 degree of difficulty. About half of the OSAE questions are derived from content found in Orthopaedic Knowledge Update 8 (OKU 8). The remaining questions are written based on other orthopaedic literature and expert consensus.
Special interest examinations
The Academy also offers 10 orthopaedic special interest examinations (OSIEs), which currently are self-scored. With the impending Maintenance of Certification (MOCŪ) requirements, the American Board of Orthopaedic Surgery (ABOS) will require orthopaedic surgeons to complete a scored self-assessment program that provides at least 20 CME credits. The Academy has asked the ABOS if it will accept two OSIEs, each providing 10 CME credits and scored by AAOS, as fulfilling the MOC requirement.
The Evaluation Committee is considering a proposal to offer these examinations in either a scored or self-scored format, and may allow users to complete the examinations online for immediate scoring and feedback.
New exams being developed
The Academy is developing two new OSIEs for release in 2007 and 2008. The first is linked to the Academy’s Essentials of Musculoskeletal Care and Essentials of Musculoskeletal Imaging textbooks. Thomas R. Johnson, MD, is the chair of the item-writing subcommittee for this project. The six-part examination will focus on physical examination, common fractures and dislocations, degenerative problems, sports medicine, pediatric orthopaedics and imaging interpretation.
A second item-writing subcommittee, chaired by Tye Ouzounian, MD, will be preparing an examination on orthopaedic practice management. Some of the content for this exam includes financial management, practice governance, personnel and compliance issues, as well as comprehensive and medical liability insurance coverage.
The committee is also reconsidering the test development and scoring specifications that serve as blueprints for the OSAE and OITE examinations. The specifications, which help determine how many questions are included in each content area, are set by how the specialty of orthopaedic surgery is defined.
As the field of orthopaedic surgery changes, examinations also need to change to reflect new practices and new techniques. The Evaluation Committee last undertook a review of the OSAE and OITE test development and scoring specifications in 1997. The Committee is currently considering whether an update is necessary and, if so, will begin the study later this year and through early 2007.
Paul Tornetta III, MD, is chair of the AAOS Evaluations Committee. He can be reached at firstname.lastname@example.org.
Participate in the 2005 OSAE
There are several ways that AAOS members can participate in the 2005 OSAE for CME credits:
• Complete the OKU Home Study Program and gain 50 CME credits by reading OKU 8, completing the 2005 OSAE and scoring above chance level.
• Complete the 2005 OSAE and gain 20 CME credits by using the self-scored “Fast-Track” format.
• Complete the 2005 OSAE and gain 20 CME credits by participating in scoring by the AAOS.
There are three different scoring participation deadlines in 2006. For more information, call AAOS customer service at (800) 626-6726.