Hospital Performance Measures May Not Accurately Reflect Quality Of Care Or Predict Patient Outcomes

A comparison of hospitals with high and low Medicare performance measures found little difference in the rate of death for three common conditions at the hospitals, indicating that the measures may not accurately reflect patient outcomes, according to a study in the December 13 issue of JAMA.

In the United States, quality of care delivered in hospitals is often variable. Because it is assumed that measuring quality of care is a key component in improving care, quality measurement has an increasingly prominent role in quality improvement, according to background information in the article. These measures can provide an incentive to improve the quality of the care delivered and to influence consumer choice of hospitals and health care plans. While some research has documented an association between higher adherence to care guidelines and better outcomes of patients who receive that care, to date there has been limited evidence demonstrating that hospitals that perform better on process measures also have better overall quality.

Rachel M. Werner, M.D., Ph.D., of the Philadelphia Veterans Affairs Medical Center, Philadelphia, and Eric T. Bradlow, Ph.D., of the University of Pennsylvania, Philadelphia, conducted a study to determine whether certain quality measures are correlated with and predictive of hospitals' risk-adjusted death rates. The researchers analyzed data from Hospital Compare, a website of the Centers for Medicare & Medicaid Services (CMS) that reports results of hospital performance measures. This study included data on hospital care between Jan. 1 and Dec. 31, 2004, for heart attack, heart failure, and pneumonia at acute care hospitals included on the Hospital Compare website. Ten process performance measures were compared with hospital risk-adjusted death rates, which were measured using Medicare Part A claims data. A total of 3,657 acute care hospitals were included in the study based on their performance reported in Hospital Compare.

Across all heart attack performance measures, the absolute reduction in risk-adjusted death rates between hospitals performing in the 25th percentile vs. those performing in the 75th percentile was 0.005 for inpatient death, 0.006 for 30-day death, and 0.012 for death at 1-year. For the heart failure performance measures, the absolute death reduction was smaller, ranging from 0.001 for inpatient death to 0.002 for 1-year death. For the pneumonia performance measures, the absolute reduction in death ranged from 0.001 for 30-day death to 0.005 for inpatient death.

"Our study suggests that in the case of hospital performance, the CMS's current set of performance measures are not tightly linked to patient outcomes. These findings should not undermine current efforts to improve health care quality through performance measurement and reporting. However, attention should be focused on finding measures of health care quality that are more tightly linked to patient outcomes. Only then will performance measurement live up to expectations for improving health care quality," the authors conclude.

(JAMA. 2006;296:2694-2702.)

Dr. Werner was supported by a career development award from the Department of Veterans Affairs. Please see the article for additional information, including other authors, author contributions and affiliations, financial disclosures, funding and support, etc.

Commentary: Performance Measures and Clinical Outcomes

In an accompanying commentary, Susan D. Horn, Ph.D., of the Institute for Clinical Outcomes Research, Salt Lake City, discusses the findings concerning hospital performance measures.

Dr. Horn notes, "The results of this study raise questions about the appropriateness of using Hospital Compare performance measures as the basis either for pay-for-performance systems or for consumers to identify better-quality hospitals. If performance measures are not strongly associated with better outcomes, why should clinicians and health care centers be required to collect and submit the data, and why would payers and consumers want to act on them""

"As the study by Werner and Bradlow illustrates, current simplistic process measures based on randomized controlled trials (RCTs) do not necessarily provide a meaningful basis for consumers to choose one clinician or hospital over another, or for clinicians or hospitals to improve their outcomes. In the real world where multiple clinical variables and patient factors affect outcomes, RCTs and comprehensive observational studies both have a role to play in improving patient care: the effects of RCTs in clinical practice can be examined in observational studies and observational studies can be progenitors [originators] for new RCTs. Patients, physicians, and policymakers will all benefit from efforts to evaluate rigorously and further understand the relationship between performance measures and clinical outcomes."

(JAMA. 2006;296:2731-2732.)

Dr. Horn reports that she in an employee, officer, shareholder, and founder of International Severity Information Systems Inc., which provides products and services to facilitate studies on practice-based evidence for clinical improvement.

###

Contact: Judi Cheary
JAMA and Archives Journals