A little stamp can make a difference

Background

The basic fundamentals of survey research are well known. Draw a representative random sample. Make sure that the source file includes all eligible members, and does not systematically exclude some populations.  With a large, representative, random sample, based on historical response rates, there will be enough respondents to accurately represent the target population.

These survey results can then be used to reliably compare health plans and guide activities to improve the member experience. This logic chain anchors the importance of the member experience across several important measurement initiatives.

However, healthcare is changing and life remains busy for all involved, including those that receive healthcare. Consequently, as some are aware, response rates from most member populations have been slowly declining over time. SPH Analytics carefully tracks the response rates of all of our clients and examines them annually across each population of insured members. We take pride in consistently having among the highest response rates for insured populations in the survey research industry.

This also means we remain alert to opportunities to strengthen member response rates. Recently, we ran a carefully constructed test to improve member response rates, with a specific successful result.

Test

Standard protocols provide the member with prepaid postage on the return envelope as a courtesy and convenience when completing a mail survey, and to increase the likelihood of responding to the survey.  These are called business reply envelopes (BRE).  The typical BRE has a small box that states “NO POSTAGE NECESSARY IF MAILED IN THE UNITED STATES” where the stamp would usually go.

The purpose of the test was to see if using a standard postage stamp (Stamp), similar to those used to send cards and letters, would increase member response rates. Two groups of 500 members each were randomly selected to receive either a BRE (the Control group) or Stamp return envelope (the Enhanced group).  This was done for two populations, Medicaid and Commercial health plan members.  (Note: This was done in conjunction with a SPH Analytics client, who has graciously agreed to share these results.)

Results

We checked to make sure there were no differences by either age or gender between the BRE and Stamp member samples in the Medicaid or Commercial populations.  There were none.  As has been long recognized, responding members to both the BRE Control and Stamp Enhancement were generally older than the members that were sent the survey ( i.e., older members are generally more responsive to health plan surveys, and were so in this test).

Overall, Medicaid members were more likely to respond when the BRE contained a stamp while commercial group members were not impacted by the presence of a stampBoth member populations, Medicaid and Commercial, responded more often when a stamp was placed on the return envelope. The improvement was minor among Commercial members (< 1%). There was a meaningful improvement in the response rate of the Medicaid member population.  Medicaid members receiving the Stamp Enhancement responded much more often compared to the Medicaid BRE Control group. There was a net gain of 3.6 percentage points in the response rate, or a 26.5% increase, of Medicaid members receiving the Stamp Enhancement compared in the BRE Control group.

A stamp on the BRE encourages more Medicaid females to respond, but did not affect commercial group members the same wayInterestingly, this increase was entirely associated with female Medicaid members, which even exceeded the response rate of Commercial female members in both groups. Medicaid males in the Stamp Enhancement group responded similar to Commercial males in both groups, but were slightly less likely than males in the Medicaid Control group to participate when the Stamp was present.

Summary

The use of a postage stamp on the return envelope resulted in a substantially improved response rate among Medicaid members.  Since this gain was exclusive to female Medicaid members, and historically the female population has long been accepted as the primary health care decision-maker, there is also the exciting possibility that these survey results will more effectively guide improvement and provide more meaningful comparisons of health plan results on behalf of the primary health care decision-makers in the Medicaid population.