October 2003 Bulletin

Knowing is not enough

Implementation of performance measures to improve quality

By Michael J. Goldberg, MD and
Belinda Duszynski

“Knowing is not enough; we must apply.
Willing is not enough; we must do.”
– Goethe

Walking into her doctor’s office, Michelle has complete confidence that Dr. Smith will be able to help her with her knee pain. Dr. Smith is a well-respected orthopaedic surgeon, who sees patients with knee pain every day, and will surely know which tests are required to make an accurate diagnosis and will recommend the appropriate treatment.

After seeing six patients and following a morning knee replacement, Dr. Smith enters the exam room to meet Michelle for the first time. He diagnoses her knee pain as osteoarthritis and recommends a course of NSAIDS and activity modification. A typical day at the office, but Dr. Smith admits that due to his increased workload and family obligations, he has not had time to read the last few editions of the JBJS and has not been able to attend the most recent CME course on OA of the knee. Dr. Smith admits that although he is aware that all potential NSAID patients should be assessed for GI and renal risk factors, he failed to assess Michelle prior to starting her on NSAIDS.

Bridging the quality chasm: The role of performance measurement

Patients “should be able to count on receiving the care that meets their needs and is based on the best scientific knowledge. Yet there is strong evidence that this frequently is not the case.”1 The Institute of Medicine’s (IOM) 2001 Report “Crossing the Quality Chasm” clearly identifies that there is a gap between knowledge and practice. Disparities between what is known to be “best practice” and what is actually practiced underscores the urgent need to improve clinical performance and the quality of care and reduce medical errors.

Analyzing the evidence base serves as the cornerstone for developing the Academy’s clinical practice guidelines and the Improving Musculoskeletal Care in America (IMCA) modules. What is the level of evidence for particular treatments? What is the strength of the recommendations for particular therapies?

As previous articles addressing clinical quality improvement have pointed out, two critical aspects of clinical guideline development are (1) rating the level of evidence; and (2) assigning the strength of recommendations for appropriate care with respect to a particular condition. Clinical practice guidelines assist physicians in deciding whether or not a particular process of care is likely to result in better patient outcomes. Both the levels of evidence and strength of recommendations are of utmost importance to the development of performance measures.

Performance measures indicate whether and how often a specific targeted process or outcome of care actually occurs in a physician’s practice. Developers of measures derive them from those aspects of care that are widely accepted as based in solid evidence, especially those for which there is the highest level of evidence and strongest recommendation. For this reason, evidence-based clinical practice guidelines serve as the basis for development of performance measures.

Based on the AAOS Clinical Guideline on OA Knee, use of NSAIDs and appropriate activity modification are “A” recommendations, and are therefore, solidly supported by the evidence. Dr. Smith is right on in his initial treatment of Michelle, however the guideline clearly recommends assessment of risk factors for GI or renal toxicity, which Dr. Smith failed to perform. Would Dr. Smith have omitted the NSAID risk assessment had he made use of a prospective data collection flow sheet, attached to Michelle’s chart, which reminds him to conduct the assessment?

Would it be beneficial to quality improvement efforts to provide physicians with data on these specific aspects of care? Ultimately, the goal of performance measurement is provision of meaningful data to physicians to facilitate quality improvement or enhancement. Providing physicians with credible data about their practice performance in relation to established evidence-based clinical recommendations holds promise for facilitating quality improvement and reducing medical errors. Making Dr. Smith aware of how often he omits the NSAID risk assessment may encourage improvement in his practice.

For successful performance measurement, it is critical that physicians trust that the measures are based on the strongest evidence, are current, and have clinical relevance. Only with such confidence will performance measures have the capacity to meet society’s demands for significant and sustained improvement.

Where the rubber meets the road: Implementation of the clinical performance measures on osteoarthritis of the knee

The Centers for Medicare and Medicaid Services (CMS), acknowledging the importance of implementation of physician-developed performance measures for quality improvement in the primary care setting, approached the Physician's Consortium for Performance Improvement (Consortium) and relevant specialty societies about developing chronic disease physician performance measures for the Doctor's Office Quality (DOQ) Project.

The CMS DOQ project goal is to develop a model for measurement and improvement of quality of care for chronic disease and preventive services at the level of the individual physician/medical office. This model will include performance measures in three areas: 1) clinical quality (measures to be provided by the Consortium); 2) systems of care (measures currently being developed by National Committee for Quality Assurance); and 3) patient experience of care [Consumer Assessment of Health Plans (CAHPS-like measures)].

On September 23, 2002, the technical expert panel for the DOQ project identified osteoarthritis as one of the chronic conditions to be included in the DOQ pilot. The Consortium and the AAOS, as lead organization, in collaboration with other organizations with osteoarthritis expertise, formed a work group, identified osteoarthritis of the knee as the condition on which to focus, and worked to identify the measures to be used by CMS.

The performance measures found in the final measurement set have been developed in agreement with existing evidence-based clinical practice guidelines from the AAOS, American College of Rheumatology and the American Geriatric Society. The measurement set includes the essential elements, as identified in the literature, that need to be assessed in treating patients with osteoarthritis of the knee.

The DOQ project will include vehicles for measurement, educational materials to encourage improvement where warranted, and incentives, including CME credits. CMS intends to first pilot the DOQ project in three states over a two-year period in the offices of at least 100 generalist physicians. Three Quality Improvement Organizations (QIOs formerly PROs) will administer the pilot beginning in July 2003.

Had Dr. Smith made use of a prospective data collection flow sheet, which is included in the new OA knee performance measurement set (easily attachable directly to Michelle’s chart), would he have remembered to conduct the NSAID risk assessment? Would it be beneficial to Dr. Smith’s quality improvement efforts to have the data on his performance related to how often he conducts the NSAID risk assessment? Such systemic changes have great potential for significantly improving the quality of care and reducing medical errors in Dr. Smith’s practice, and yours as well.

Physicians, of course, worry about the use and misuse of performance measurement data. The AAOS and Consortium had this in mind and clearly stated that data collection, related to all Consortium measures, is intended for quality improvement and not for public accountability. Nevertheless, data collection and public reporting are a reality and may soon become a requirement. Thus, implementation of data collection instruments (e.g. OA knee clinical performance measurement flow sheet) developed by physician-colleagues, based on evidence, and able to be seamlessly incorporated into daily practice is the best method to measure what we are doing.

Excerpts from the OA of the Knee Core Physician Performance Measurement Set have been included in this Bulletin; the complete document is available on the AAOS Evidence-Based Practice Committee's (EBPC) website. AAOS members are encouraged to review, share, use, and encourage usage of these measures by any physicians who treat OA Knee.

Additional information about the Consortium is available on the web

For additional information about the EBPC or AAOS clinical quality improvement activities, please contact Belinda Duszynski, EBPC staff liaison, at duszynski@aaos.org

1. Institute of Medicine, ”Crossing the Quality Chasm: A New Health System for the 21st Century,” National Academy Press, 2001.

Michael J. Goldberg, MD, is the chair of the AAOS Evidence-Based Practice Committee, a member of the Pediatric Orthopaedic Society of North America and the AAOS representative to the Physician’s Consortium for Performance Improvement. He can be reached at MGoldberg@tufts-nemc.org Belinda Duszynski is the staff liaison to the Evidence-Based Practice Committee. She can be contacted at duszynski@aaos.org

Home Previous Page