Gaby Goldstein, SDAN Director of Research;
Mallory Roman, SDAN Associate Director of Research
Objective
We ran two randomized controlled trials in the Fall of 2018 to determine if receiving a handwritten postcard encouraging the target to vote increased the odds of voters turning out to vote for in the 2018 general election in Pennsylvania and Michigan, and further sought to determine if those odds differed based on the postcard message.
Background
Social pressure, or referring in some way to the fact that individual voting records are public, has been one of the most effective messaging tactics in direct mail for many years, but it can be difficult to use in postcards due to the need for personalization. A common form is positive social pressure, where you thank voters for past voting behavior, but research by Costas Panagopoulos(1) suggests that expressing gratitude to voters is effective with or without social pressure. This research tests that idea with handwritten postcards.
Specifics:
In Pennsylvania, Sister District Action Network (SDAN) partnered with the Voter Participation Center (VPC) to send postcards to Pennsylvania voters already receiving “get out the vote” (GOTV) mail. In Michigan, SDAN sent postcards to GOTV voters identified by a Democratic state senate campaign. Voters were sorted into 3 conditions: control, where voters received no contact from SDAN; gratitude, where voters received a postcard expressing gratitude to them for being informed voters and reading the postcard; and social pressure, where voters received a postcard expressing that whether or not people vote is public record. Postcards were sent to arrive approximately 1–2 days before the 2018 general election.
Takeaways:
1. Receiving a Postcard Did Not Increase Odds of Actually Voting.
In both the Pennsylvania and Michigan samples, receiving a handwritten postcard during GOTV did not significantly increase the odds that a person would vote (all p values ≥ 0.466). This means that postcarding did not have a statistically meaningful effect in either the PA or MI samples. It is possible we do not see meaningful effects because: 1) the study is underpowered and the effect of GOTV postcarding is relatively small and undetectable at this power, 2) the sample of voters in the study is not representative of the broader voting population, and 3) people in the study voted at much higher rates than is typical.
2. Basic Takeaway.
The results of the studies, when taken together, indicate that GOTV postcarding was not effective in increasing voter turnout in these samples of voters. However, because the studies are underpowered and the samples were skewed, these results should be replicated to more reliable conclusions about the value of postcarding as a GOTV tactic.
Caveats:
- Studies Underpowered. The studies are statistically underpowered, which means we didn’t have enough people in the experiment to be able to rely on the findings. The studies would need to be replicated with a larger sample of people to get a better picture of whether encouraging people to vote using handwritten postcards has an effect on actually turning those people out to vote.
- Non-Representative Populations. The samples in these two studies are not representative of the nationwide electorate. For instance, the Pennsylvania sample is highly racially skewed, with only 50 white voters. The Michigan sample was also racially skewed in the other direction.
- Unexpected Behavior. The Michigan sample voted at much higher rates than would be expected of a traditional GOTV sample (94%+ in all conditions compared to about 53% of voters nationwide) even though the average Targetsmart (a national voter file) voter turnout propensity score, a measure of likelihood to vote, in the sample was around 59 and the median turnout score was 63. The 2018 midterms saw the highest midterm election turnout in over 50 years, indicating that folks were more highly motivated to vote than usual. It was also a very noisy election environment, with multiple voter touches that we are unable to control for beyond the random assignment of voters to condition. Both of these factors may have inspired higher than average turnout.
Contributions and future directions:
This study helped to shed a little light on the utility of GOTV postcarding but there is a large amount of research on messaging and targeting still to be done. Importantly, future studies would focus on more representative samples of GOTV voters that could be generalized outside of the context of a study.
A more detailed report of the Pennsylvania findings can be found here and a more detailed report of the Michigan findings can be found here. A combined report of both studies can be found here.
Please donate to support our work at SDAN!
SDAN’s commitment: It is SDAN’s intention to provide as much context as possible to allow for the nuanced interpretation of our data. SDAN’s convention is to contextualize effects by reporting p values, confidence intervals, and standardized/contextualized effect sizes for all models tested. Additionally, SDAN always differentiates between planned and exploratory analyses and a priori and post hoc tests, and reports the results of all planned analyses regardless of statistical significance. These findings were informed by similar analyses in previous studies that were peer-reviewed a subset of the Sister District Data and Research team composed of senior-level statisticians called the Quantitative Advisory Committee. If you are interested in joining the Quantitative Advisory Committee please email Mallory.
References
Panagopoulos, C. (2011). Thank you for voting: Gratitude expression and voter mobilization. The Journal of Politics, 73(3), 707–717.