Abstract
We sent GOTV postcards to voters in North Carolina and Texas to see if receiving a postcard at all might boost turnout, and to see if turnout was different for people who received a postcard sent from in-state as opposed to a postcard sent from out of state. The results indicate that GOTV postcards did increase turnout, and that receiving a postcard sent from in-state increased turnout to a larger degree than receiving a postcard sent from out of state (though the difference was not statistically significant). Further, this study’s results indicate that postcarding may be particularly useful in quieter electoral environments, like primary elections.
Objective
This experiment explored whether receipt of a handwritten postcard encouraging voting may be associated with higher rates of voting in the North Carolina and Texas 2020 primary elections on March 3, 2020, and further, whether receipt of a postcard with an in-state postmark may be associated with higher rates of voting than either receipt of a postcard with an out of state postmark or not receiving a postcard (control).
Background
Handwritten postcarding is a popular volunteer activity, but research about its efficacy in boosting turnout has been mixed. This study investigated handwritten postcards as a GOTV tactic in general, as well as whether the postmark location affected turnout. Campaigns and advocacy organizations often ask volunteers to handwrite postcards to voters and to mail them in bulk back to the campaign or organization to mail to targeted voters locally. If postmark location does not influence postcard efficacy, this extra step for volunteers costs thousands of dollars a year that could perhaps be better spent elsewhere. However, if postmark location does influence postcard efficacy, then it’s important to understand the magnitude of its effect so that campaigns and organizations can make more data-driven spending decisions.
Specifics
SDAN generated a list of registered voters in a number of competitive state legislative districts in North Carolina and Texas who had TargetSmart partisanship scores of 80+, indicating a high likelihood of voting for Democrats, and TargetSmart primary voter turnout propensity scores between 25-60, indicating low-mid likelihood of voting in the primary election. Each list was randomized and the first 15,000 voters on each state’s lists were chosen as subjects. These subjects were then randomized into 3 conditions: control (no postcard), in-state (received a postcard sent from inside the target state the voter is registered in), and out of state (received a postcard sent from outside the target state). This resulted in a total sample of 30,000 voters. Sister District volunteers hand wrote informational, nonpartisan GOTV postcards to the voters assigned to the treatment conditions. Volunteers writing postcards for the in-state condition mailed to in-state partners on February 18, 2020 and were mailed locally by in-state partners on February 22, 2020. Volunteers writing postcards for the out of state condition mailed from their location outside of the target state on February 20, 2020. It is assumed that all postcards reached voter mailboxes from February 24-26, 2020.
Takeaways
1) Receiving a postcard made people more likely to vote.
The results indicate that voters who received postcards were statistically significantly more likely to vote in the 2020 primary election than voters who did not receive postcards (p = 0.008). Note: statistical significance is an indication that something (e.g. receiving a postcard) is meaningfully different from something else (e.g. not receiving a postcard) in a way that can be statistically detected.
2) The in-state postcards had a larger effect on voting than out of state postcards.
When looking at the 3 conditions separately, in-state postcards were statistically significantly more effective in producing turnout than no postcards (control condition; p = 0.005) and out of state postcards were marginally more effective in producing turnout than no postcards (control condition; p = 0.071). This means that people who received in-state postcards voted at higher rates than people who received out of state postcards. However, people who received out of state postcards still turned out at higher rates than control participants (+1.25% more than controls), indicating that out of state postcards did have some practical effect on voter turnout — just not as much as in-state postcards, which boosted turnout by 2.17% compared to controls.
3) But when we directly tested the difference between receiving an in-state postcard vs. an out of state postcard, that difference was not statistically significant (p=0.328).
When directly comparing the turnout in the in- and out of state conditions (instead of to the control condition that received no postcard), the p-value indicates that the difference between the two treatment groups is not large enough to be statistically significant in this test (p = 0.328). This indicates that the increased efficacy from sending postcards in-state, as opposed to from out of the state, is not so different in effect size that the difference is statistically meaningful. However, this particular test was underpowered, and in a larger sample size, we may have been able to detect a significant effect. This suggests that we aren’t able to draw conclusions about the increased efficacy of in-state postcards.
4) Basic takeaways
- This study indicates that GOTV postcards encouraging the targeted voters to vote in the 2020 primary election boosted turnout by 1.71% compared to controls, a result that achieved statistical significance.
- This study suggests that sending postcards to voters from in-state has a bigger effect size than sending postcards to voters from out of state. Postcards sent from in-state boosted turnout significantly compared to controls (not receiving a postcard; receipt of in-state postcard boosted voting by 2.17% over controls; p = 0.005), while postcards sent from out of state only boosted turnout a marginal amount (by 1.25%) over controls (no postcards; p = 0.071). If the goal is to turnout as many voters as possible, sending from in-state makes sense as long as the cost is not prohibitive.
- In-state postcard recipients voted 0.92% more than out of state postcard recipients. However, this difference was not so large as to be statistically meaningful in this test. This means that we can’t make a completely confident statement that in-state postcards are more effective than out of state cards, regardless of their larger effect size.
- In-state postmarks should be explored further, to determine if the increase in efficacy from in-state postcarding is worth the expense, considering that out of state postcarding did not perform worse in a statistically meaningful way and is cheaper.
Caveats:
- Study focused on a niche group of voters. This study specifically looked at the primary voting behavior of individuals in a handful of competitive state house and senate districts in North Carolina and Texas who had partisanship scores of 80+ and turnout scores of 25-60. This is a particular group of voters and these results may not apply to voters who are from other states (or even less competitive districts in the same states), who are less partisan, or who have higher or lower turnout scores than included here.
- Primaries are quieter than general elections. It’s important to note that tactics are expected to have larger effects during quieter elections compared to noisier elections. For that reason, it is more appropriate to generalize these results to primary elections than to general elections.
- Out of state postcard test is underpowered. We do not have adequate statistical power to fully reveal the out of state postcard condition efficacy compared to controls. This again suggests that this study should be replicated, this time with the caveat that it is replicated in a larger group of voters to reach adequate power for this comparison. All other tests were adequately powered.
Contributions and Future Directions:
This study helps to clarify how effective GOTV postcards are in motivating low-mid propensity Democratic leaning voters to vote in a presidential primary election. The results suggest that primaries may be particularly useful times to postcard, as our previous postcarding results undertaken during the general cycle reveal smaller effect sizes than those revealed in this study. Further, this study tested the assumption that sending postcards to voters locally is superior to sending postcards to voters from other areas. The results provide some evidence that this may be the case (the larger effect size and statistical significance compared to the control condition), but also that this boost may not be significantly different from postcards sent from out of state (the non-significant direct comparison in efficacy between the two treatment conditions). This study would benefit from replication to clarify the relative efficacy of the two treatment conditions, but initial evidence suggests that sending postcards locally may provide a small, statistically nonsignificant boost.
If you’re interested in reading more about this study, a longer report is here.
SDAN’s commitment: It is SDAN’s intention to provide as much context as possible to allow for the nuanced interpretation of our data. SDAN’s convention is to contextualize effects by reporting p values, confidence intervals, and effect sizes for all models tested (these items may be in the longer report linked in the blog). Additionally, SDAN always differentiates between planned and exploratory analyses and a priori and post hoc tests, and reports the results of all planned analyses regardless of statistical significance. If you have questions about these findings please email Mallory.