Abstract: Sister District Action Network (SDAN) and the Mississippi NAACP (MS NAACP) partnered to conduct an experiment around GOTV text messaging during the days leading up to the general election in Mississippi on November 5, 2019. The study targeted predominantly Black / African-American voters whose prior voting history indicated they would be unlikely to vote in the 2019 general election. The MS NAACP sent GOTV text messages to half of the voter targets, and then all targets were tracked to determine if they voted in the 2019 general election. The results indicate that people were less likely to vote in the 2019 general election when they received a GOTV text message, and this was especially true for people who received the persuasion message as opposed to the shortened message.
Objective: This experiment explored whether Mississippi voters who received a GOTV text message from the Mississippi NAACP (MS NAACP) asking them to vote in the 2019 general election had a higher rate of turning out to vote than voters who did not receive a text message; and further, whether voters who received a message with persuasion content voted at a higher rate than those who received a shorter / simpler GOTV text.
Background: Text messaging has become a popular political tactic in recent elections, and a good amount of industry research has tested messaging, timing, and warmth of contact (warm contacts are people who gave the organization their information with the knowledge they may be contacted). Less is known about whether text messaging efficacy may vary by the presumed race or regional location of the recipient. Further, industry research generally suggests that GOTV messaging should be information-forward and should minimize persuasion about candidates and parties. However, persuasion messages generally do not focus on why the act of voting itself is important. This study sought to look at the effect of GOTV texting in a relatively high profile odd-year election among Black / African-American low propensity voters, and further, to test if using persuasion messaging around the act of voting itself may boost turnout.
Specifics: SDAN ran this study with MS NAACP. MS NAACP generated a list of registered voters in almost all state legislative districts in Mississippi who were considered non-white, low propensity voters. SDAN matched that list to the TargetSmart voter file to obtain cell phone numbers for as many targets as possible. “Low propensity” voters were defined as voters who had a turnout score of 75 or below for off year elections, who voted in 2016 but not since, who were newly registered, or who had voted in 2008 and 2018 but not in the elections between those years. Further, voters needed to have a cell phone number listed to be included in the sample. This yielded a list of 31,844 voters. The list was randomized and subjects were randomly assigned to two conditions: Control (no text message) and GOTV text (received a text message). MS NAACP volunteers and staff send text messages to people in the GOTV text condition on November 3, 2019.
While sending the messages, the texting platform suggested that a shorter message would increase response rates. As a result, MS NAACP switched to a shorter message that did not include persuasion content. This was not a planned experimental factor, meaning results are exploratory and cannot be generalized, as people were not randomly assigned to conditions, but it does allow for analyses comparing the efficacy of the two different messages. See Appendix for the text of the messages sent.
It is important to note that 465 people assigned to be text messaged were unable to be messaged for various reasons (e.g., because the number is disconnected, the number was accidentally changed in a record, the number is actually a landline or fax machine and can’t be text messaged, etc). This put 16,387 people in the no message control group in the message type exploratory analysis, while there are only 15,922 people assigned to the no message control condition.
1) Receiving a text message made people less likely to vote. Voters assigned to the text message condition were less likely to vote in the 2019 general election than voters who did not receive text messages. However, this result was not statistically significant (p = 0.121). This indicates that being assigned to the texting condition had no significant effect on voters’ behavior, although, in practice, was associated with decreased voter turnout compared to the control condition.
Caveats and considerations:
Contributions and Future Directions:
This study helps to clarify the utility of GOTV text messaging to low propensity Black / African-American voters in Mississippi ahead of an off-year Gubernatorial election. It also helps to clarify the utility of persuasion messaging around voting itself, as opposed to around candidates or parties. The results provide some evidence that GOTV text messaging may not be a good tactic to mobilize these voters. Further GOTV text messaging tests are needed to clarify these effects and to determine if one would expect backlash in a more high profile election, like a Presidential election. A larger group of targets would be needed to be adequately powered to test another non-persuasion short message.
If you’re interested in reading more about this study, a longer report is here.
SDAN’s commitment: It is SDAN’s intention to provide as much context as possible to allow for the nuanced interpretation of our data. SDAN’s convention is to contextualize effects by reporting p values, confidence intervals, and effect sizes for all models tested (these items may be in the longer report linked in the blog). Additionally, SDAN always differentiates between planned and exploratory analyses and a priori and post hoc tests, and reports the results of all planned analyses regardless of statistical significance. If you have questions about these findings please email Mallory.