By Khaled J. Saleh, MD, MSC, FRCSC, and Michael J. Goldberg, MD
Many countries around the world have joint registries. These efforts have enabled researchers to examine long-term orthopaedic outcomes on a population basis. Orthopaedic outcomes research in the United States is hampered for a variety of reasons,including the peculiarities of the U.S. health care infrastructure, the absence of a truly unique patient identifier and the costs entailed with large-scale data collection efforts.
In 1997 the AAOS established an outcomes initiative. Following a three-year planning and development process, the Musculoskeletal Outcomes Data Evaluation and Management System (MODEMS) was rolled out to the fellowship. MODEMS was envisioned as the premiere program for the collection and analysis of functional outcomes information on musculoskeletal conditions in the United States.
Its purpose was to improve the quality of care for patients who have musculoskeletal disorders by developing an information system that would make it possible to analyze the outcomes of musculoskeletal care and to provide participating physicians with valid and reliable methods of evaluating functional outcomes and patient satisfaction. The key elements were a portfolio of validated patient-based functional questionnaires that were developed by the AAOS (e.g. DASH, lower extremity module, spine module, pediatrics questionnaire), a generic health assessment tool—the SF36—and a list of comorbidities.
While thousands of AAOS members expressed interest and several hundred participated, the project itself was not sustainable. The outcomes tools, however, have continued in wide use, mainly for research purposes.
Common pitfalls highlighted
Recently, the hip and knee portions of the MODEMS data set were analyzed. The analysis revealed important information regarding this model project and how it might inform those who would undertake large-scale data collection initiatives in the future.
Of the 2,419 hip and knee records available at base line, 31 percent were excluded due to data quality issues and an additional 47 percent were excluded because there were only baseline data without postoperative follow up data. Indeed, of the entire group, only 150 patients had data for a baseline visit as well as for first and second postoperative visits. Even in this small cohort, however, the value of hip and knee replacement was apparent from the SF36 general health scores.
The failure to establish a long-term, valid data base of patients undergoing hip and knee replacement was disheartening. However, the analysis showed that the MODEMS experience simply highlighted common pitfalls in outcomes research and in establishing joint registries. For example, success requires the development of clear, simple and quickly completed data collection forms and high quality controls must be in place for both data collection and data entry.
Challenge of collecting patient data
We must find ways to solve the substantial challenge of patient follow up. To promote adherence, it is important to clearly articulate the study protocol for both the physician and the participating patient. The burden of patient data collection for the surgeon and the clinical office staff—both in terms of time and money—must not be overlooked.
Although efforts should always be made to collect information from routine administrative data, perceptions of health and actual functional performance can only be collected from the patients themselves. There must be a thorough and thoughtful planning process that clearly outlines the goals of the data base as a first step; establishes inclusive eligibility criteria, including a limited set of clearly defined variables that match the purpose of the data base; and accepts as a rule that at the end of the day, it is better to have more patients than to have more variables.
The experience with MODEMS is not dissimilar from other data base initiatives that did not meet early expectations. One often hears that the problem was the lack of technology. Although electronic medical records and Web-based systems are helpful for data collection, there are also important psychological factors that are often overlooked. These include willingness of physicians to share data and to incorporate data collection into the fabric of the medical office visit, and a requirement of the registry to make the value of such data collection clearly beneficial to the participating surgeon.
The AAOS is now embarking upon the American Joint Replacement Registry (AJRR). Through lessons learned from MODEMS and others, the institution of large-scale data collection projects will be successful. The AJRR can meet the needs of the practicing orthopaedic surgeon and answer questions that will enhance the care of our patients.
Khaled J. Saleh, MD, MSc, FRCSC, is an Associate Professor in the Department of Orthopaedic Surgery, University of Minnesota School of Medicine; Adjunct Associate Professor, Division of Health Services Research and Policy, University of Minnesota School of Public Health; Director of Orthopaedic Research, VAMC; Director of Adult Reconstruction Fellowship, University of Minnesota; and a member of the AAOS Evidence-Based Practice Committee. He can be reached at firstname.lastname@example.org.
Michael J. Goldberg, MD, is the chair of the AAOS Evidence-Based Practice Committee, a member of the Pediatric Orthopaedic Society of North America, and the AAOS representative to the Physician’s Consortium for Performance Improvement. He can be reached at email@example.com.