Abstract: Sister District Action Network (SDAN) and the Voter Participation Center (VPC) conducted an experiment in April-May 2020 directly comparing the efficacy of postcards and letters as chaser communications to improve the response rate to voter registration forms. Half of our registration targets in Pennsylvania received a partially handwritten postcard or a partially handwritten letter approximately 5-7 days after receiving a VPC voter registration form in the mail, and half of targets did not receive a partially handwritten chaser communication (control group).  All targets were tracked to see if they registered to vote via VPC’s effort. Results indicate that both conditions elicited some degree of backlash in this sample, with fewer targets registering after receiving either type of partially handwritten material. Further, we found no difference between letters and postcards, indicating that, when postcards and letters contain the same information, they may not have different effects.

Objective: This experiment explored whether chaser handwritten mail might increase response rate to an earlier official-looking voter registration form; and further, whether targets who received a partially-handwritten letter might register at a higher rate than those who received a very similar partially handwritten postcard (or vice versa).

Background: Writing postcards and letters have become increasingly popular among volunteers, but the efficacy of these partially or fully handwritten voter communications is not well or consistently established. SDAN has conducted several experiments to establish best practices for postcarding, and has amassed a few results that indicate that postcarding may be useful as a “chaser” tactic that amplifies a previous voter contact (see here and here). Additionally, partially handwritten letters have also amassed some reported evidence from special elections that sending partially handwritten letters to voters increases voter turnout. At the time this study was conducted, no studies had directly compared whether postcards or letters might work better or equally in terms of encouraging people to register to vote.

Specifics: SDAN ran this study with VPC. VPC provided SDAN with a list of voter registration targets in Pennsylvania that were receiving voter registration mail (forms) from VPC. This list was composed of young people, people of color, and unmarried women. The list was pre-randomized into half control (who did not receive a postcard or letter to chase their voter registration form) and half handwritten communication groups. The handwritten communication group was further divided into approximately half postcard targets and half letter targets. Overall, this amounted to 70,046 in the control condition where folks did not receive a postcard or a letter, 34,970 in the letter condition, and 34,984 in the postcard condition.

In order to standardize the messaging across the letter and the postcards, volunteers purchased postcards created by SDAN that featured the printed content of the letter on the front of the postcard, with room for the handwritten message on the back. This allowed for a direct comparison of the two mediums when content is held relatively constant. SDAN recruited volunteer teams to handwrite the handwritten portions of the postcards and letters and mail them directly to Pennsylvania voters on April 27, 2020.

Basic Takeaways:

• In this study, receiving a partially handwritten communication had a negative impact on voter registration rate.
• The letter condition had the lowest registration rate (3.87%), followed by the postcard condition (3.91%). The control condition, where targets received no communication, had the highest registration rate (4.15%).
• Though more people registered after receiving a postcard than after receiving a letter, the difference between the efficacy of letters and postcards is not statistically significant in this sample. This means that there is no evidence that letters outperformed postcards or vice versa.

1) Partially handwritten chasers did not work in this sample. 

• Receiving a partially handwritten communication (postcard or letter) made people less likely to register to vote. Voter registration targets assigned to receive a partially handwritten communication (both postcards and letters combined) registered to vote at a statistically significantly lower rate (p = 0.046) than targets in the control condition who did not receive a postcard or letter to chase their voter registration form. This indicates that being assigned to receive a partially handwritten chaser communication was actually associated with significant backlash among potential registrants.
  • People who received a partially handwritten communication had approximately 5.6% lower odds of returning the voter registration form compared to controls who received no follow up to the registration form.

2) People who received a letter chaser registered less often than people who received no chase communication in this sample.

• Both the letters and postcards performed worse compared to no chaser communication, but only the backlash in the letter condition approached statistical significance.
  • When looking at the conditions separately, the letter condition performed marginally worse than the control condition, meaning that receiving a chaser letter was associated with marginal backlash compared to not receiving a communication (p = 0.058).
  • Postcards did not perform significantly different from controls (p = 0.179), but their odds ratio also indicates that fewer people registered in the postcard condition compared to the control condition.
  • This means that fewer people voted in both treatment conditions than in the control condition, and this was especially the case in the letter condition.
  • Folks who received a letter were approximately 6.6% less likely to register to vote than targets who did not receive a letter.

3)  Letters and postcards did not perform significantly differently from each other. 

• When we compared the letter and postcard conditions to each other, we did not see a significant difference in registration rate between the two types of communication (p = 0.629). In other words, a similar amount of voters registered after receiving a postcard as after receiving a letter.

Caveats and considerations: 

• Study focused on a niche group of voters. This study focused on a very specific group of voter registration targets (members of the rising American electorate who were receiving voter registration mail from VPC), and as such these results should not be widely generalized.
• Limitations in knowledge of true contact rate. We were only able to account for the returned mail and the unassigned/never sent addresses when considering contact rate, but is likely that the real contact rate is lower than estimated (e.g., voters didn’t read it, voters threw out the mail as opposed to returning it, mail was not received but not successfully returned, etc).
• Study was statistically underpowered. This study  was underpowered to detect all of the effects. This means that there were not enough people in the sample for us to be able to detect some of the results.

Contributions and Future Directions:

This study continues SDAN’s exploration into the most effective uses of postcarding. Unlike previous postcard chaser studies, we observed unexplained backlash in this study. It may be related to the pandemic, or it may be related to an increased level of societal strife in general. But this study’s results provide a cautionary tale that postcards and letters may not be behaving the same way this year as they have in the past.

Additionally, this study was the first to directly compare postcards and letters bearing almost identical content. The results indicate no evidence that letters perform differently than postcards. Both letters and postcard chasers underperformed compared to the control group, and postcards and letters performed at similar rates. Since postcards are cheaper than letters, this initial evidence suggests that postcards may be a good choice.

However, the results of this study are not conclusive and would need to be replicated in order to further explore how letters and postcards may compare to each other as a tactic to boost voter registration.

If you’re interested in reading more about this study, a longer report is here.

SDAN’s commitment: It is SDAN’s intention to provide as much context as possible to allow for the nuanced interpretation of our data. SDAN’s convention is to contextualize effects by reporting p values, confidence intervals, and effect sizes for all models tested (these items may be in the longer report linked in the blog). Additionally, SDAN always differentiates between planned and exploratory analyses and a priori and post hoc tests, and reports the results of all planned analyses regardless of statistical significance. If you have questions about these findings please email Mallory.