May 18, 2021
You don’t have to tell me twice: Additional text message reminders for confirming volunteer attendance for Mobilize event

Abstract: This study tested the efficacy of sending an additional personal text message reminder to volunteers who have RSVP’d for an event in boosting event turnout. Using an event on April 29, 2021, SDAN randomized the RSVP list to either receive a personal text message in addition to the automated text message sent by Mobilize (treatment) or not (control). Volunteers assigned to receive personal messages did not attend at higher rates than volunteers who did not receive personal messages. Further, people who had already confirmed and people who had not already confirmed behaved similarly in their attendance regardless of treatment. Results indicate that the auto-generated Mobilize text message reminder is sufficient and an additional personal text message reminding RSVP’s of the event does not result in additional attendance.

Objective: This study explored the efficacy of following up automated email and text message event reminders from Mobilize with a personal text message reminder sent to RSVPs the day of the event as a way to increase attendance to the event.

Background: Organizer knowledge and previous SDAN research suggests that reminding volunteers about events they have RSVP’d for increases event attendance. However, services like Mobilize and EventBrite have automatic reminders for events, and in the case of Mobilize at least, even sends RSVPs a confirmation/reminder text message the day before the event. But organizers are unsure if these automated reminders sent by an app or website replace the need for personal confirmation text messages. Further, it is unclear if people who confirmed attendance via the automated reminders from Mobilize (or another service) need additional reminders. This pilot study explored both of these questions.

Specifics: This study was a randomized controlled trial (RCT) designed by SDAN. SDAN chose a fundraising event with a large RSVP list for the pilot. The inclusion criteria for the study were RSVPing for the event and listing a phone number with which one could be sent a text message. All RSVPs that met inclusion criteria were enrolled in the study for a total sample of 293. All of the RSVPs were sent email reminders and a text message reminder automatically by Mobilize. A randomly chosen half of the sample received an additional text message from a real person the morning of the event on April 29, 2021. Confirmations and event attendance were tracked via Mobilize.

Basic Takeaways:

  • People who RSVP’d attended the event at almost the same rate, regardless of condition.
    • Event attendance was only 0.38% more in the treatment condition than in the control condition, and this difference was not statistically significant (p = 0.947)
  • Effects did not vary based on confirmation status.
    • People who already confirmed attendance and people who had not confirmed their plan to attend the event ended up attending at about the same rate regardless of condition, and any difference was not statistically significant (p = 0.253).
  • Though unrelated to the intervention, people who confirmed their RSVP for the event attended the event at a significantly higher rate than people who did not confirm their attendance (p<0.001). This is a useful finding for organizers, as it demonstrates that confirmation via the Mobilize system is a strong indicator that a volunteer will actually attend.
  • Even though this study was intended as a pilot, it is almost adequately powered at its current n. The results suggest that there is no effect here to explore further.

Key findings:

  • RSVPs who received an additional personal text message attended at almost the same rate as RSVPs who did not receive an additional text.
    • RSVPs who received an additional personal text message attended the event at a rate of 56.16%, while RSVPs who did not receive the additional message attended at a rate of 55.78%. This difference of 0.38% was not statistically significant (p = 0.947). Further, the turnout numbers were almost identical with 82 attendees in both conditions and 65 and 64 RSVPs who failed to show up in the control and text conditions respectively. This indicates that, unlike in the confirmation study SDAN ran in 2018 where confirmation tactics increased attendance by at least 51% in all conditions, there was no effect from the additional text message in this study.

  • Effects of the text message did not vary based on confirmation status.
    • A moderation model that included an interaction between confirmation status and condition found no differences (p = 0.253). Both RSVPs who confirmed and RSVPs who did not confirm attended at similar rates.
    • However, there is a main effect of confirmation status that was unrelated to the additional text message, such that people who confirmed their RSVP were more likely to actually attend the event (p < 0.001).

Caveats and considerations

  • The sample was unique. All of the people in this study had previously RSVP’d for the event, meaning they are Sister District volunteers or people who are aware of Sister District events. Further, only people who shared their phone numbers were included, and these people may be categorically different from people who don’t provide a phone number. This likely means the results do not fully generalize to all volunteer groups or even to all politically-oriented volunteer groups.
  • This was a single study for a single type of event. This study was designed as a pilot study, but based on sample size analyses using the confirmation study estimate for texting, this study is close to adequately powered. As such, we won’t be exploring the topic further. However, no single study can be the final word on a topic, and it should be noted that this tactic may have worked differently for a different audience or different event type.

Contributions and Future Directions:

This pilot suggests that sending an additional, personal text message reminder about an event people have RSVP’d to does not boost event attendance. It appears that, unlike situations in which only one text message and/or one email reminder is sent (like in our previous confirmation study), a second, additional personal text reminder does not boost turnout. This means that this volunteer labor can be saved: The automated reminder message from Mobilize appears to be adequate.

Future directions for others interested in this research may include trying this tactic for a different event type, like a phonebank, or attempting this tactic in different audiences, like non-political volunteers.

If you’re interested in reading more about this study, a longer report is here.

SDAN’s commitment: It is SDAN’s intention to provide as much context as possible to allow for the nuanced interpretation of our data. SDAN’s convention is to contextualize effects by reporting p values, confidence intervals, and effect sizes for all models tested (these items may be in the longer report linked in the blog). Additionally, SDAN always differentiates between planned and exploratory analyses and a priori and post hoc tests, and reports the results of all planned analyses regardless of statistical significance. If you have questions about these findings please email Mallory.