Since the Nursing Home Compare Five Star rating program was first launched in 2008, providers have approached it with a mix of trepidation, envy and awe. Using a set of complex statistical models, CMS awards between one and five stars to every nursing facility, with the goal of helping consumers make more informed and better care decisions. Unfortunately, its reliability is somewhat suspect, as most providers experience fluctuation in their star rating. According to a five-year analysis prepared by Abt Associates, only 3% of providers had the same star rating over the entire period, although 28% ended up with the same rating as when the program started. Almost 40% moved up or down one star over the five year period, while 20% moved two stars. Recently, CMS announced changes to the quality measure content and scoring methodology, costing about 8% of nursing facilities their five-star rating and creating unhappiness in the provider ranks:
LeadingAge Supports Transparency, But Still Finds Flaws in the CMS 5-Star Nursing Home Rating System
New Government Rating System Forces Nearly 1 in 3 Skilled Nursing Centers to Lose Coveted Quality Star
CMS’s confusing kick in the teeth
To understand what happened, first we need to understand how the score is calculated. You can read the CMS Technical Users Guide, or just follow along:
Health Inspections Five Star Rating
The health inspection rating is calculated from the number, severity and scope of deficiencies in the past 3 annual surveys, along with deficiencies from the last 36 months of complaint surveys. There is an aging component, as well, so that more recent surveys are weighted more than older ones. CMS then stratifies the star rating so that the lower 20% of nursing homes in each state receive one star, the middle 70% receive between 2 and 4 stars, and only the top 10% receive 5 stars. The actual state-by-state distribution looks like this:
Staffing Five Star Rating
The staffing rating is based on reported staffing during the more recent annual survey, and looks at RN staffing and total nursing personnel (licensed nurses and nurse aides) adjusted by case-mix based on the RUG-III group. The star rating is determined according to the grid below:
And the state-by-state distribution looks like this:
Alaska is excluded from the chart above. Why? Because 100% of Alaskan facilities received five stars for staffing.
Quality Measures Five Star Rating
The quality measure star rating is based on how the facility performs during the previous 3 quarters (9 months) on 11 of the 18 quality measures currently calculated by CMS:
Long-Stay Residents:
• Percent of residents whose need for help with activities of daily living has increased
• Percent of high risk residents with pressure ulcers (sores)
• Percent of residents who have/had a catheter inserted and left in their bladder*
• Percent of residents who were physically restrained
• Percent of residents with a urinary tract infection
• Percent of residents who self-report moderate to severe pain*
• Percent of residents experiencing one or more falls with major injury
• Percent of residents who received an antipsychotic medication
Short-stay residents:
• Percent of residents with pressure ulcers (sores) that are new or worsened*
• Percent of residents who self-report moderate to severe pain
• Percent of residents who newly received an antipsychotic medication
* Risk adjusted
The scoring methodology is somewhat complex, but essentially point values are assigned for each measure based on the nursing home’s performance compared to other nursing homes in the same state. Each quality measure is given equal weight, so the individual scores are simply summed. The total score is then applied to a scale to determine the star rating.
The current star distribution for quality measures, by state, looks like this:
Overall Five Star Rating
With the complicated math behind, calculating the overall star score is easy. Begin with the health inspection rating. Next, add one star if the staffing rating is a four or five stars *and* the staffing rating is higher than the health inspection rating. Subtract one star if the staffing rating is one star. Finally, add one star if the quality measure rating is five stars, or subtract one star if the QM rating is one star.
Special note: If a facility has a one star health inspection rating, its maximum overall rating is limited to two stars. Also, if the facility is in the Special Focus Program, its overall star rating cannot be more than three stars.
Why was the Recent Rebasing Done and Why are Providers Unhappy?
Since the fundamental methodology is intended to stratify facilities relatively evenly, but the actual scoring is based on a known point system, a gradual drift towards higher star ratings occurred over the past five years. Part of this is probably due to overall care improvements, and part of this is probably due to providers gaming (or at least selectively improving) parts of the system.
There is also a slight issue of state imbalance. Consider the District of Columbia:
More than 50% of DC facilities hold a five star rating. Does this mean that DC providers provide phenomenal care? Perhaps, though it’s unlikely they provide significantly better care overall than providers in many other states. So how did so many providers get this coveted rating? As you can see above, the majority of DC providers received either four or five stars for staffing and five stars for QMs, which would lead to either one or two stars added to the health inspection rating. This points to a core problem with the five star system: CMS selectively stratifies measures by state, attempting to account for differences in state Medicaid policy and surveyor quirks, but this ends up causing some parts of the measure to reflect best-in-a-state results and others to reflect best-in-the-country results. Since the methodology isn’t standardized, it creates a real problem when comparing the five star rating of facilities located in different states.
What Does This Mean for Providers
CMS has clearly stated that it will continue to update measures and rebase scoring to promote continual improvement in nursing homes. While providers are understandably frustrated, this is simply the reality of healthcare today. Complaining about being ‘kicked in the teeth’ only furthers the perception that the nursing home field is behind the times. Instead, providers should see this as an opportunity to excel, not only in Nursing Home Compare rankings, but in service and care as well.
Understanding the methodology behind the five star rating system, as well as the state-level distribution, is a crucial first step to responding to this recent change. It should be clear that the health inspection score is the biggest driver of the overall score, and to reach a five star rating, a provider must be at or below the state average for survey deficiencies. Thus, it is critical, now more than ever, to build effective systems that embed quality processes into daily work. Customer service is also key, as most complaint surveys begin from unhappy residents or family members. Second, staffing hour cutoffs make it easy to compare current staffing metrics to the star rating system. For some facilities, adding slightly more staffing hours is an easy way to improve the star rating. Third, since quality measures are equally weighted, facilities can determine what measures are close to a higher point cut-off and work to improve those specific measures.
