Where The 4 Top Hospital Rating Groups Get Their Data

Data is king. At no other time in history have we had the amount of data available to us to make informed decisions – both as consumers and as professionals. With advanced computer software, crunching those numbers has never been easier. One of the fields benefiting from these advances is healthcare, with everyone from doctors, hospital administrators, insurance companies, pharmaceutical developers, and patients using data to predict outcomes, make choices, and evaluate success. What we do with all this data is at least as important as what that data is, but for today, we focus first on where the top 4 hospital rating groups get their data.

Well, let’s make that 5 rating groups. The first and most important source of data used by all four rating systems is the information collected by the Centers for Medicare and Medicaid Services, or CMS. This massive database contains publicly-available, redacted data on patient demographics, diagnoses, outcomes, procedures, devices, and many, many other topics. Anyone can download this data in its raw form but CMS also provides individuals access to this information in a user-friendly format through Hospital Compare, a searchable website geared toward the healthcare consumer (making it our 5^th rating group). Thanks to this availability of data, groups interested in evaluating and rating hospitals could access this data and not have to collect their own. And that’s exactly what happens.

All four of the top hospital ratings groups start with the CMS data. Millions of reports covering all aspects of healthcare at over 4,000 Medicare/Medicaid certified hospitals include everything from medical, financial, and patient experience data. (No personal information is accessible, per stringent HIPAA regulations.) Specialized reports about hospital infections come from the CDC’s National Healthcare Safety Network, and some supplemental data is drawn from The Joint Commission and the Veteran’s Health Administration. CMS provides the list of data sources here.

Each rating group adds their own particular twist to the CMS data to make their rating unique, and hopefully, more beneficial to the consumer. One basic difference is the idea of ranking vs. rating. Two of the groups rate hospitals, giving them a standalone score. The other two rank the hospitals, giving them a score and then placing them in a list from “best” to “worst” as compared to the other hospitals. Let’s look at what each of the top 4 groups adds to the mix to make their rating different.

The Leapfrog Group’s Hospital Safety Grade |  Rating    The Hospital Safety Grade is calculated with additional information from the group’s proprietary survey in addition to CMS data. About 2,500 hospitals receive grades from A to F, with percentage scores for subcategories. Only hospitals with all data available are given grades. 

US News & World Report Best Hospitals | Ranking    These ranks are calculated with additional information from the AHA survey as well as certification data from various professional accrediting associations. Hospitals receive higher scores for being certified by these professional associations. For those measures which are rated by the medical community (“reputation measures”), the results from the proprietary survey sent to the members of Doximity (an online medical professional networking site) are used. The top ranked hospitals are drawn from a pool of 4,716 qualifying institutions.

Healthgrades |  Ranking    Healthgrades provides information about individual hospitals. These scores are calculated solely from the CMS Medicare Provider Analysis and Review (MedPAR) database and are based entirely on risk-adjusted mortality and in-hospital complications for 33 specific conditions or procedures. Where information is missing, state inpatient databases are used to fill in those gaps. Depending on the condition or procedure, the number of evaluated hospitals ranges from 628 to 4,192.

Consumer Reports Hospitals  |  Rating    The Consumer Reports rating is the only one of the four explored today that provide reports only to paying subscribers. They pull their data from CMS (the only scoring for which Consumer Reports uses data they themselves did not collect). They supplement this data with the Society of Thoracic Surgeons Adult Cardiac Thoracic Database, which helps them generate scores for heart-related conditions. Over 4,000 hospitals have been rated.

This post serves only as a description of data sources. Each of the groups mentioned publishes their methodologies, available as links in our previous post.

The topic of hospital ratings is certainly a controversial one. Any time raw data is used to draw conclusions, those conclusions can be questioned – regardless of how well-defended those choices are. It is up to the consumer to understand how they were reached and make their own conclusions. 

For an infographic containing the material in this post, please click the link below.