Gaby Goldstein, SDAN Director of Research;
Mallory Roman, SDAN Associate Director of Research
Objective:
We ran a randomized controlled trial to determine if the receipt of a handwritten postcard a week after the receipt of an official voter registration form increased the odds of people completing & returning the voter registration form, as compared to the odds of return among people who did not receive a postcard.
Background:
Handwritten postcarding has become an incredibly popular voter outreach tactic among progressive volunteers, with volunteer postcarding groups all over the country. However, there is still very little research on the efficacy of postcarding, especially looking beyond voter turnout effects to effects on registration or candidate choice.
Specifics:
Sister District Action Network (SDAN) partnered with the Voter Participation Center (VPC) to send handwritten postcards encouraging registration to eligible, unregistered individuals living in Michigan, Pennsylvania, and Arizona in September 2018 (September Study). These postcards were sent in conjunction with 3 of VPC’s voter registration programs, which sent pre-filled official voter registration forms to all of the people included in this experiment. This study was a replication of our March 2018 voter registration postcarding experiment, except that the present study design included chaser postcards only, and targeted recipients of 2 additional VPC programs.
Takeaways:
1. Marginally Significant Replication of March Study Findings.
In the September Study, receiving a handwritten chaser postcard after receipt of a pre-filled voter registration form had a positive impact on the odds that a person would return their materials. But unlike the March Study, the effect was only marginally statistically significant (p=0.071), after controlling for individual-level variables like age, gender and race/ethnicity. On its own, these data cannot be generalized outside of this Study, but as a replication of the March Study it generally appears to confirm the March findings that chaser postcards boost registration form return rates.
- Unregistered individuals had 12% higher odds of returning the completed form when they were sent a chaser postcard along with the form, compared to the odds of returning the registration form after receiving the form alone (without a postcard) (p=0.071).
2. Postcards Have A Significant Effect on Movers.
The postcard chaser appears to work particularly well among people who had been registered before and just recently moved to their new address (“Movers”). This population was not included in the March study.
- Unregistered Movers had 18% higher odds of returning the completed form when they were sent a chaser postcard along with the pre-filled form, compared to the odds of unregistered Movers returning the registration form after receiving the form alone (without a postcard). This finding did achieve statistical significance (p = 0.017) after controlling for individual-level variables like age, gender and race/ethnicity.
3. Basic Takeaway.
The results of the two studies taken together indicate that postcarding in conjunction with the voter registration forms reliably produces an increase in the gross amount of registered voters. However, that effect is likely small due to the powerful influence of other factors like age and gender, and may or may not rise to statistical significance depending on the number of people in the study.
Caveats:
- Studies Underpowered. The September Study was statistically underpowered, which means we didn’t have enough people in the experiment to be able to rely on the findings. However, since this study is a replication of the March Study, we can have more clarity about the reliability of the effect of postcards. Taken together, the March + September Studies indicate that receipt of a handwritten chaser postcard has a small effect, which could be meaningful in a close race, and might have a larger effect among more motivated voters.
- Differences Between March and September Studies. There are a few reasons that we might expect differences between the results of the March Study and the replication September Study. First, there are far fewer treatment participants (people who got a postcard) in the September Study than in the March Study, and as noted above, the March Study is statistically underpowered. This suggests that the effect of postcarding itself is fairly small, as it takes a very large sample to be able to detect a statistically significant effect. Second, the September Study included different populations in the experiment and used different postcard messaging than in the March study. These factors indicate that more research is warranted, but the September study indicates some encouraging reliability to the March findings.
Contributions and Future Directions
This study helps to establish reliability for SDAN’s voter registration postcarding findings from March, and further suggests that postcarding as an add-on tactic for another voter registration effort may provide a meaningful bump in voter registration. For future research, we will build on the statistically significant finding in the Movers group, and will continue to look at groups like Movers to determine how to best reengage folks who have been registered previously and others who may be more likely to register.
A more detailed report of these findings can be found here.
Please donate to support our work at SDAN!
SDAN’s commitment: It is SDAN’s intention to provide as much context as possible to allow for the nuanced interpretation of our data. SDAN’s convention is to contextualize effects by reporting p values, confidence intervals, and standardized/contextualized effect sizes for all models tested. Additionally, SDAN always differentiates between planned and exploratory analyses and a priori and post hoc tests, and reports the results of all planned analyses regardless of statistical significance. The findings reported here were informed by similar analyses in previous studies that were peer-reviewed a subset of the Sister District Data and Research team composed of senior-level statisticians called the Quantitative Advisory Committee. If you are interested in joining the Quantitative Advisory Committee please email Mallory.