Over the past several years, the Centers for Medicare & Medicaid Services (CMS) have reported hospital quality measures on the Hospital Compare web site, providing the public with a way to compare hospital performance. Hospital readmission rates in particular have gained substantial attention from policymakers and health-care providers because of their high frequency and significant costs. Among the most prominent of these are 30-day hospital readmission rates for specific conditions, including heart failure (HF), pneumonia, and myocardial infarction (MI or heart attack), which are risk-adjusted to account for differences in patients’ health status. Higher-than-average 30-day readmission rates not only result in poor report cards for low-performing hospitals, but can also result in substantial financial penalties.

Now, new research finds that these condition-specific readmissions measures may not accurately or fairly reflect hospital quality. Led by researchers at the Smith Center for Outcomes Research in Cardiology at Beth Israel Deaconess Medical Center (BIDMC) and publishing online today in the Annals of Internal Medicine, the study found significant differences in hospitals’ performance when readmissions were assessed for non-Medicare patients and for conditions other than those currently reported, showing that when these additional factors are taken into account, half of hospitals would be subject to a change in their financial penalty status.

“As it currently exists, the Medicare public reporting system offers an incomplete picture,” said senior author Robert W. Yeh, MD, MSc, director of the Smith Center for Outcomes Research in Cardiology at BIDMC. “Significant attention has been given to hospitals’ overall performance as determined by the public reporting of a small number of specific conditions and patient populations. It’s a little bit like issuing a final grade based on a few homework assignments and not a full semester’s worth of work.”

Calculations known as excess readmission ratios (ERRs) examine hospitals’ readmissions for HF, heart attack, and pneumonia among Medicare beneficiaries. These ERRs are currently used to assess care quality and determine hospital payments.

“We wanted to know if hospitals that reported high readmission rates for fee-for-service Medicare patients would also have high readmission rates for patients with other payers, including Medicaid and private insurance,” said first author Neel Butala, MD, MBA, an investigator at the Smith Center at BIDMC and cardiology fellow at Massachusetts General Hospital. “We also wanted to find out if hospitals with high readmission rates for the three specified conditions—heart failure, pneumonia, and heart attack—had similarly high readmission rates for other unreported conditions.”

To answer these questions, they reviewed data from the Healthcare Cost and Utilization Project’s Nationwide Readmission Database. The researchers focused on more than 2,100 hospital observations in 2013 and 2014 to investigate whether 30-day readmission measures for publicly reported conditions (HF, MI, and pneumonia) among Medicare patients reflect hospital performance on readmissions more broadly in an all-payer national sample. They found that 29% of hospitals currently being penalized for readmissions would no longer incur a penalty if unreported conditions were used as the basis of the calculations. The findings revealed that the difference was even greater when examining non-Medicare readmission rates: 40% of penalized hospitals would no longer be issued penalties if—instead of Medicare patients—performance was based on readmission rates for non-Medicare patients hospitalized for HF, pneumonia, or heart attack.

“This tells us that similar hospital or patient characteristics may influence readmissions more than similar disease conditions and suggests that efforts to prevent readmissions may be more successful by targeting hospital-wide processes rather than condition-specific processes,” said Butala.

“Our results suggest that although performance on readmission may reflect broader institutional characteristics, the concordance of each hospital’s performance across different conditions varies substantially,” said Yeh, who is also an associate professor of medicine at Harvard Medical School. “It goes to show that Medicare metrics alone may not be the final word on hospital quality for readmissions.”