Important Updates to the QHP Enrollee Survey and the Quality Reporting System

The Centers for Medicare and Medicaid Services (CMS) has released new technical guidance for the Quality Rating System (QRS) and the QHP Enrollee Survey for 2016. This Technical Guidance is available along with the 2016 Measure Technical Specifications and the 2016 version of the QHP Enrollee Survey here.

Below are the key differences between the 2015 Beta Test and the new 2016 guidance:

• For 2015, QHP issuers were given the option to include enrollees from identical QHPs offered outside the Marketplace. For 2016, enrollees in QHPs that are offered outside the Marketplace (off-Marketplace) and non-QHPs are not included.
• For 2015, QHP issuers were required to collect and submit validated data for 29 of the 43 measures in the QRS measure set. For 2016, QHP issuers are required to collect and submit validated data for all 43 measures in the QRS measure set. The additional measures require 2 years of data so they can now be submitted in 2016.
• In 2016, Marketplaces are required to publicly display QHP quality rating information for the first time, making the ratings available for consumers for the Open Enrollment Period for 2017.
• The 2015 Beta Test Guidance did not provide marketing guidelines. Moving forward, a QHP issuer that elects to include 2016 QHP quality rating information in its marketing materials must do so in accordance with the information included in this new section.

Other notable highlights:

• QHP Enrollee Survey data will be case-mix adjusted. CMS will use the 2014 and 2015 test data to determine the factors that will be used and will release that information in subsequent guidance.
• CMS will use a maximum of 28 measures (omitting one of the 29 collected – Relative Resource Use) for the Beta Test scoring. For 2016, even though issuers are required to collect and submit data for all 43 measures, CMS anticipates using the same rating methodology as the 2015 Beta Test. So only the 28 measures will be used. They will conduct further analyses on the additional measures to inform refinements in 2017.
• Since 2015 was a test year, plans cannot market the star ratings they receive from the Beta Test.
• CMS will standardize all measure scores by calculating national percentile ranks, based on one national, all-product reference group. CMS intends to provide the national percentile ranks each year so issuers can calculate their own standardized measure scores. CMS will determine score cut points to create the rating categories based on the 2015 Beta Test data.

Proposed QHP Enrollee Survey Changes

CMS posted updated proposals regarding the 2016 survey administration. The updates were largely based on feedback received from the most recent public comment period.

Below is a summary of the comments they received and how they have been addressed by CMS:

Revise cover letter. Several comments suggested revisions to the cover letter. There was concern about the instructions for those that switched health plans and the introduction to the intent of the survey. We have made wording updates to clarify both points.

Cognitive test all new survey questions and make results publically available. Many groups highlighted the importance of cognitive testing of questions, especially those that are new to CAHPS. We have conducted three rounds of cognitive testing of all new items and all items were also included in the psychometric testing after the field test.

Revise question wording. There were multiple recommendations for revisions to item wording for core CAHPS questions that have been previously validated in cognitive testing and field testing. We are not making changes to items from the CAHPS Health Plan core questionnaire in order to allow CMS, policy makers, and health plans to make direct comparisons between their QHP and Medicaid populations.

Allow the option for additional questions. Stakeholders advocated for the opportunity to add a small number of custom questions to assist in cultural and linguistic reporting. CMS will consider these questions and other revisions in the future; however because of the rounds of psychometric and beta testing already done we will not include additional questions at this time to ensure that we are able to implement the 2016 QHP Enrollee Survey in appropriate timeframes.

Revise provider questions to assess quality of provider network. Commenters recommended that questions about a personal doctor and provider communication be replaced with questions about the plan’s provider network. We are not making these changes to the current QHP Enrollee Survey because these items are all standard CAHPS Health Plan questionnaire. We may consider questions about provider networks in future QHP surveys.

Revise the response scales. Some commenters recommended changing the question “Would you recommend this plan to your friends and family?” to “How likely is it you would recommend your health plan to a friend or family member?” with a scale from 0 to 10 to allow for calculation of a Net Promoter Score. We have made this change. In addition, there was also a concern with the Never/Sometimes/Usually/Always scale used and recommended a yes/no scale instead. We did not make this change, as the QHP response scale needs to be consistent with other CAHPS surveys.

Revision of invasive questions. One stakeholder group noted that prevention questions about smoking, aspirin use, and cholesterol may be too invasive and respondents may not answer honestly. These questions are National Committee for Quality Assurance (NCQA) Healthcare Effectiveness Data and Information Set (HEDIS) items and have been previously tested. They are also included in the Quality Rating System (QRS) measures and must be included in this implementation.

Survey length. Stakeholders expressed their concern about the length of the survey; in particular they recommended shortening the “About You” section. We recognize the length of the survey, but there are many factors that affect response rates. Previous research has shown that the effect of additional questions up to even 95 questions had minimal effect on response rates. Additionally, many of the items included in the “About You” section are required for the aspirin use, tobacco cessation, and flu shot measures that are included in the Quality Rating System. Furthermore, other CAHPS surveys are of similar length. For example, the ACO CAHPS questionnaire is currently 80 questions.

Clarify the survey methodology and ensure participants do not get both the Marketplace and the QHP Enrollee Survey. There was a recommendation to clarify the survey methodology and indicate whether the survey sample will be selected to ensure the same person does not get both the Marketplace and the QHP survey. There is no way to ensure that because the samples are drawn by different organizations.

Metal-level analysis. Stakeholders recommended the inclusion of analysis by metal level. We do have this information and will include it in appropriate analyses.

Assess if experience differences exist for those who were previously uninsured. Stakeholders suggested that CMS assess whether experience differs across previous insurance status. We plan perform this analysis using a question on the survey that asks whether the enrollee had insurance during the previous year.

Paperless survey option. It was recommended to include a paperless survey option. We included an internet survey option in the 2014 psychometric field test and 2015 beta test. We plan to continue offering this option and will expand internet surveys to include Spanish in the 2016 Implementation.

The 2016 version of the survey tools were recently posted here. The updated version of the survey contains 85 questions, compared to 76 on the 2015 Beta Test version.

Contact SPH Analytics for additional resources and information about the QHP Enrollee Survey and the Quality Rating System.

Note: Information provided above is based on interpretation of information posted, as of September 2015, on the CMS websites noted. Opinions and recommendations expressed above are those of SPH Analytics and not to be regarded as views or opinions expressed by CMS or other governing authorities.