May 14, 2021
The skill of the chase 2:  Do chaser postcards help to amplify the utility of an earlier voter contact?

Abstract: This study tested the efficacy of postcards that follow up on a successful voter contact made by a campaign in increasing voter turnout. Six state legislative campaigns in 4 states provided SDAN with a list of people that they had successfully contacted and identified as supporters through phonebanking or canvassing. A randomly chosen half of these voters received a “Get Out the Vote” (GOTV) postcard shortly before the election providing them with information about how to vote and, for contacts in states besides Florida1, reminding them of the contact they had with the campaign. In the analysis, the raw numbers trended in the expected direction, with more voters in the postcard chaser condition voting than voters who did not receive a postcard, though this difference was not statistically significant (p = 0.923). Interestingly, the postcards had a similar effect as in SDAN’s prior 2019 pilot study, with some campaigns completely unaffected, some experiencing backlash, and some experiencing a small boost in turnout (though none were statistically significant). Small sample size, high turnout tendencies, and messaging requirements in Florida were all limitations of this study.

Objective: This study explored the efficacy of following up a successful voter contact with a handwritten GOTV postcard reminding voters of the previous contact as a way to increase turnout.

Background: SDAN has run a few studies that suggest that using handwritten postcards to “chase” an effective voter outreach attempt may be more effective than using handwritten postcards as a standalone tactic (see the voter registration chase studies we conducted with VPC here and here). In order to test this chaser approach in a voter mobilization context, SDAN ran a pilot study in 2019. Overall, that study did not find GOTV chasers to have a statistically significant impact on voter turnout in that sample. However, the sample was small and it is possible we might have been able to see a statistically significant effect in a larger study.In order to expand on the pilot findings, SDAN ran a larger study in 2020 with twice the candidates and states represented. The larger pilot study investigated the same question: Do voters who have a successful phone or canvass contact with a campaign, as defined by the voter identifying as a supporter, vote at higher rates when they receive a GOTV chaser postcard reminding them of the earlier voter contact?

Specifics: Six state legislative candidates across four states (Florida, Georgia, North Carolina, and Wisconsin) worked with SDAN to chase supporters, identified via canvassing and phonebanking, with handwritten GOTV postcards. As in the first pilot, campaigns gave SDAN contact information for voters who had expressed support for the candidate (identified as leaning towards or definitely supporting the candidate). SDAN randomly chose half of the supporters contacted by campaigns to receive a handwritten GOTV postcard shortly before the general election. The postcard message reminded voters about the contact they had with the campaign (in all states but Florida) and provided them with information about how to vote. The other half did not receive a handwritten postcard.

Sister District volunteers completed the postcards and sent them to an in-state partner by October 20, 2020, who then mailed the postcards locally by October 24th. It is estimated that postcards arrived at homes from October 27-November 2, 2020. Overall, 3,538 voters in the analysis received postcards and 3,535 voters were enrolled in the control condition, for a total of 7,073.

 Basic Takeaways:

  • People in the postcard chaser condition voted at a slightly higher rate than control participants, but this difference was not statistically significant (p = 0.923).
  • Effects did vary by candidate, with three candidates seeing a boost in voting rate from postcards, one candidate seeing almost no difference in voting rate, and two candidates seeing a slight decrease in voting rate.
    • However, none of the interactions were statistically significant (ps = 0.232 or more).
  • This sample had a high voter turnout baseline (the control group turned out at 89.17%), indicating that this sample did not have much to benefit from any intervention. This study should be run again with voters who are less likely to vote (low-mid turnout propensity voters), in order to see more voter turnout movement based on the postcard intervention.

Key findings:

  • People in the postcard chaser condition voted at a slightly higher rate than control participants, but this difference was not statistically significant.
    • Voters who received a chaser postcard turned out at a rate of 89.47%, while people who did not receive a chaser postcard turned out to vote at a rate of 89.17%. This difference of 0.29% was not statistically significant (p = 0.923). This indicates that, as in the pilot study, receiving a chaser postcard was associated with a small increase in voter turnout, but it was not statistically meaningful in this sample.
  • Effects varied by candidate, with some candidates seeing an increase in voter turnout, some seeing a slight backlash, and some seeing no effect.
    • However,  none of the interactions were statistically significant (ps = 0.232 and above). This indicates that, even if postcards behaved differently for these candidates, it was not enough as to be statistically meaningful. We saw similar non-significant effects in the 2019 pilot. A graph of the current interaction below shows the trends in the different campaigns.
    • This provides additional evidence that postcards behave differently based on the specific campaign, but it does not give us additional information about why that may be.

Caveats and considerations

  • Study used a unique sample of voters. All of the voters in this study lived in the specific districts being targeted and had explicitly expressed their support for the candidate the postcard supported. This group of voters is not generally representative of the larger electorate.
  • This sample was composed of high turnout voters. A usual GOTV sample includes people who are more likely than not to vote, but may need a push to get the polls. This sample voted at 89%+ in both conditions, indicating that this sample was actually a base voter sample. The median voter turnout score of voters targeted was 96.4, indicating that this sample had very little room to move (i.e., be influenced by the intervention). The study should be replicated in a sample of lower-mid turnout propensity voters to determine if this tactic works better on voters who need more encouragement to get to the polls.
  • This study was statistically underpowered. This study was planned to expand on the original pilot, but the inclusion criteria limited our ability to adequately power the sample. It is difficult to obtain a large enough sample size with this recruitment method (just using people already contacted by a campaign). Future studies should include paid phonebanks or canvasses to increase the statistical power.

Contributions and Future Directions:

This study helps to continue the investigation into chaser postcarding. As postcarding continues to be popular among volunteers and economical for campaigns (as volunteers shoulder the lion’s share of the cost), determining best practices for postcarding remains an important goal. The results of both of SDAN’s GOTV chaser studies to date were underwhelming, but the data suggests that the sample was not truly a GOTV sample. This means that this tactic should be replicated in a low-mid turnout propensity sample with more room to move in their turnout likelihood. Further, a larger sample size is necessary to truly detect an effect. This means that tactics to increase the number of contacts and identified campaign supporters, such as paid phonebanks and canvasses, should be explored.

If you’re interested in reading more about this study, a longer report is here.

1 Due to in-kind donation laws in Florida, we were unable to mention specific candidates in the postcards. Since this is a departure from the intentions of the treatment to reactivate the campaign contact, we conducted the same analyses on a restricted sample composed just of the GA, NC, and WI candidates. It displayed a similar pattern of results, and condition and interactions between condition and candidate were similarly non-significant.

SDAN’s commitment: It is SDAN’s intention to provide as much context as possible to allow for the nuanced interpretation of our data. SDAN’s convention is to contextualize effects by reporting p values, confidence intervals, and effect sizes for all models tested (these items may be in the longer report linked in the blog). Additionally, SDAN always differentiates between planned and exploratory analyses and a priori and post hoc tests, and reports the results of all planned analyses regardless of statistical significance. If you have questions about these findings please email Mallory.