This memo outlines findings from a digital test that aimed to determine if consumers would be more likely to start the enrollment process when exposed to information about the fine for not having coverage and, if so, what fine language was most effective for motivating consumers to enroll.
Between January 23 and February 15, 2015, consumers who used the Get Covered America Calculator to get personalized estimates of financial help and premiums for marketplace plans in their area saw one of four possible results if their inputs suggested they were likely not eligible for Medicaid and they might face a fine for not having health insurance in 2015:
- No mention of the fine. Consumers only saw what they might expect to pay for premiums, without getting any additional information about the fine.
- Personalized fine amount. Consumers saw the estimated dollar amount of the fine they might have to pay for being uninsured in 2015, based on their Calculator inputs (ZIP code, income, number of adults and children in household who need insurance, and total household size).1
- Detailed fine language. Consumers saw that they might face a fine for being uninsured, and that the fine would be “$325 or 2% of income (whichever is more).”
- Simple fine language. Consumers saw that they might face a fine if they did not get health insurance by February 15, but the amount of the fine was not specified.
The fine conditions were assigned at random. After consumers saw the Calculator output, they had the option to click an “Enroll Now” button to start the enrollment process. The goal of the test was to determine if consumers were more likely to start the enrollment process when exposed to information about the fine and, if so, what fine language was most effective for motivating consumers to enroll.
The results of the fine test were inconclusive. Overall, the test did not provide sufficient evidence that consumers were more or less likely to start the enrollment process when they were shown information about the fine alongside their premium estimate compared to not seeing any fine information.
Although the test was not conclusive, the Calculator data collected during the course of the test provides additional insight into how consumers used the tool to make enrollment decisions.
- Information about premiums was a much larger driver for consumers to enroll than information about the fine. Across all fine conditions, consumers were more likely to click through to the enrollment form the lower the premium estimate was. The rate at which Calculator users went on to start the enrollment process among all submissions that resulted in a premium estimate ranged from 38 percent for consumers with plan options costing less than $50 down to 9 percent for those facing higher premium costs ($500 or more), a difference of 29 percentage points. The overall difference in click-throughs to the enrollment form between the best and worst performing fine condition, however, was less than 2 percentage points.
- Most consumers who used the Calculator saw premium estimates that were less than $100, and they were twice as likely to start the enrollment process as consumers who received higher estimates. Among all consumers who received a premium estimate between January 1, 2015 and February 15, 2015, half (51 percent) saw an estimate for the lowest priced plan in their area that was less than $100. Of those consumers, 36 percent went on to the enrollment form, versus only 18 percent of those whose lowest premium estimate was $100 or more.
- Continue testing fine language in OE3. Although the results from OE2 were inconclusive, the fine for not having health insurance in 2016 will increase to $695 or 2.5 percent of income (whichever is higher), so details about the fine might be more motivating. Starting the test earlier during open enrollment will also allow more data to be collected, making it more likely that any effects will be identifiable.
- Continue to prioritize messages around financial affordability while exploring messages for consumers who face higher prices. Since most Calculator users saw premium estimates below $100 — and since this aligns with data from FFM states showing that 69 percent of consumers who enrolled in marketplace plans during OE1 with a tax credit paid $100 or less2 — it is important to ensure that consumers know that they could qualify for low-cost plans. However, given the steep drop-off in the share of consumers who continued with the enrollment process once the lowest premium increased beyond $100, it will also be important to explain the value of paying more for health insurance to those who face higher costs.
- Continue to message around the fine. Although including information about the fine in the Calculator output did not increase the likelihood that consumers would start the enrollment process, including fine information did not hurt. Given previous evidence from email tests showing that consumers are more likely to take action when told about the fine and surveys showing that consumers generally underestimate the amount of the fine, outreach efforts should continue to mention the fine to consumers.
Enrollment form click-throughs by fine condition
Figure 1 below shows the click-through rates to the enrollment form by fine condition. Overall, being exposed to information about the fine did not make consumers more or less likely to start the enrollment process compared to the control condition (see the appendix for examples of each condition). Since the effect of the fine language could be different if consumers faced lower or higher cost plans, click-through rates to the enrollment form were also broken out depending on if consumers saw a premium estimate for less than $200 or not.
Figure 1. Percent of consumers going on to enrollment form by fine condition and premium amount.
None of the differences in click-through rates to the enrollment form are statistically significant at conventional p = 0.05 levels. Nevertheless, there was a trend toward the fine language performing better relative to the control group among consumers who faced higher premium costs.
To better examine the relationship between premium amount and the effect of the fine language, Figure 2 shows the percent of Calculator users who went on to the enrollment form by fine condition. While there are no statistically significant patterns, it is possible that personalized fine language is most effective among consumers facing higher premium costs. Therefore, further testing should help provide more clarity about the effect of comparing premium estimates and potential fine amounts side-by-side.
Figure 2. Percent of consumers going on to enrollment form by fine condition and premium amount.
Enrollment form click-throughs by Calculator premium estimates
Slightly more than half of all consumers who used the Calculator and received a premium estimate were shown a plan costing less than $100. Another 30 percent of consumers received an estimate for a plan costing between $100 and $250. Only 5 percent of all Calculator submissions returned a premium estimate of $500 or more. However, only 9 percent of those who received the highest premium estimates started the enrollment process compared to 36 percent of those receiving premium estimates below $100.
|Lowest Calculator Premium Output||
Percent of Submissions
Percent Starting Enrollment Process
As shown in Figure 3 below, the click-through rate to the enrollment form steadily decreased as the lowest premium increased, with consumers seeing premiums less than $100 twice as likely to start the enrollment process as those who received premium estimates of $100 or more.
Figure 3. Estimated percent of Calculator submissions that led to the consumer starting the enrollment process, among those receiving a minimum plan estimate between $0 and $1,000 per month. Black dots represent Calculator submissions, with dots on top indicating that the consumer clicked through to the enrollment form and those below the line indicating no click-through. Estimates are smoothed using Loess regression.
All findings described here were calculated using data from the Get Covered Calculator API generated between January 23, 2015 and February 15, 2015, which corresponds to the start of the study through the end of OE2.
Results are limited to Calculator users who inputted their information via GetCoveredAmerica.org and received a premium estimate from the Calculator. The analysis excludes submissions whose information suggested that the users were below the Medicaid eligibility threshold and therefore were not shown premium estimates for marketplace plans. Users who received a premium estimate generally received a range from the lowest-cost bronze plan to the second-lowest-cost silver plan.3
All premium estimates mentioned here refer to the lowest estimate that a consumer saw. The fine test results exclude any submissions where the user was deemed to be exempt from the fine for not having health insurance in 2015, since these users would not see any fine language regardless of what condition they were assigned. The analysis excludes duplicate entries and any Calculator submissions with unusually large household sizes (more than 10 people) or incomes (higher than $5 million).
The final sample used for analysis included 29,519 unique Calculator submissions.
Appendix: Examples of fine language
The screenshots below show the four possible Calculator outputs for a particular scenario. The premium and fine estimates are for a two-person household in Florida (ZIP code 33133) with an annual income of $30,000 seeking insurance for a 34-year-old adult and a 12-year-old child.
Control (no mention of fine):
Personalized fine amount:
Detailed fine language:
Simple fine language:
1 The Affordable Care Act sets the fine for not having coverage in 2015 at either $325 per adult and $162.50 per child without coverage, or 2 percent of income above the federal tax filing thresholds (whichever is greater). The Calculator methodology calculates the user’s household income as a percentage of the Federal Poverty Level (FPL) using the income and total household size provided. If the user is deemed to be exempt from the fine based on this information, no information about the fine is shown. The fine calculation assumes that all household members looking for insurance are without coverage. The size of the fine is capped, which the Calculator takes into account.↩
2 Amy Burke, Arpit Misra, and Steven Sheingold, “Premium Affordability, Competition, and Choice in the Health Insurance Marketplace, 2014. ASPE Research Brief, June 18, 2014. Available online at http://aspe.hhs.gov/HEALTH/REPORTS/2014/PREMIUMS/2014MKTPLACEPREMBRF.PDF.↩
3 More information about the Calculator methodology is available here: https://s3.amazonaws.com/assets.getcoveredamerica.org/wp-content/uploads/Get-Covered-Calculator-Methdology_2015.pdf.↩