During the COVID-19 pandemic many traditional methods of data collection, such as intercept surveys or focus groups, are not feasible. This paper proposes that establishing community panels through SMS text messages may be a useful method during the pandemic, by describing a case study of how an innovative SMS text message community panel was used for the “Shisha No Thanks” project to collect data from young adults of Arabic-speaking background about their attitudes on the harms of waterpipe smoking. Participants were asked to complete an initial recruitment survey, and then subsequently sent 1 survey question per week. The study recruited 133 participants to the SMS text message community panel and the mean response rate for each question was 73.0% (97.1/133) (range 76/133 [57.1%] to 112/133 [84.2%]). The SMS text message community panel approach is not suited for all populations, nor for all types of inquiry, particularly due to limitations of the type of responses that it allows and the required access to mobile devices. However, it is a rapid method for data collection, and therefore during the COVID-19 pandemic, it can provide service providers and policymakers with timely information to inform public health responses. In addition, this method negates the need for in-person interactions and allows for longitudinal data collection. It may be useful in supplementing other community needs assessment activities, and may be particularly relevant for people who are considered to be more difficult to reach, particularly young people, culturally and linguistically diverse communities, and other groups that might otherwise be missed by traditional methods.JMIR Form Res 2021;5(11):e28929
There is a high level of interest in communities’ experiences and needs during the COVID-19 pandemic [- ], and as people’s lived experiences have been heterogeneous, more information is needed to understand what different subgroups within populations have been through. This is especially true of minority populations who are underrepresented in mainstream conversations. Traditional methods of collecting participant data, such as intercept surveys or focus groups, are not feasible during a pandemic due to physical distancing requirements.
Establishing community panels that provide data through SMS text messaging is a potential method that could be used during the COVID-19 pandemic to provide information for support services, policy planning, and research studies. SMS text message community panels allow longitudinal data collection and involve the recruitment of a sample of the community to form the community panel, and then sending a small number of survey questions via SMS text messages to the panel participants at regular short time intervals. Panel participants respond to the survey questions by sending a short response back via an SMS text message. This method allows tailoring of language to different community groups, and is particularly suitable to younger people who have been socialized to communicate using this channel. SMS text message surveys have been trialed in health research studies for data collection and have been found to be user-friendly and produce reasonable response rates [- ].
While there are other noncontact methods of data collection that are useful during the COVID-19 pandemic, such as online surveys [- ] and online focus groups [ ], we propose that SMS text message community panels could be an additional useful tool. This report highlights an example of an SMS text message community panel, and then discusses how the approach could be used during the pandemic.
The “Shisha No Thanks” SMS Text Message Community Panel
“Shisha No Thanks” is a co-design project that aims to raise awareness about the harms of waterpipe (shisha) smoking among young adults (aged 18-35 years) of Arabic-speaking background in Sydney, New South Wales, Australia [, ]. To evaluate the campaign, a community panel using SMS text message–delivered survey questions was established to identify changes in attitudes about the harms of waterpipe smoking. The survey questions were specifically designed for this study to measure awareness of project messages and attitudes toward risks of waterpipe smoking, and questions were adapted from the Cancer Institute NSW Tobacco Tracking Survey [ ] and the Syrian Center for Tobacco Studies Narghile-Waterpipe Users Survey [ ]. As 94% of young people (18-29-year olds) have smartphones [ ], SMS text message–delivered survey questions were considered to be a useful way of engaging the target audience.
Recruitment advertisements directed people to an online recruitment survey which was built using Qualtrics software. The recruitment survey had information about the study, and then asked for demographic information, a mobile phone number, and consent to participate in the study. People who completed the online recruitment survey were then added to the SMS text message community panel database.
To recruit and retain participants, financial reimbursements were provided. The recruitment material explained to participants that they would be compensated for their time with e-gift cards valued at AUD 50 (US $37) each, which would be sent via SMS text messages at 3 different stages of the study if they answered 75% or more of the questions. To provide context, AUD 50 (US $37) is equivalent to 6.5% of the minimum weekly wage in this country .
SMS Text Message Survey Development
SMS text message community panel members were then sent 1 survey question per week. In total, the study survey consisted of 22 questions—a set of 8 questions that were asked at the beginning of the study period, 6 other questions, and then the initial set of 8 questions were asked again. This was designed for longitudinal follow-up of the cohort, to detect changes in attitudes and awareness about the harms of waterpipe smoking before and after the project.
The survey questions were also set up using Qualtrics, which allows for questions to be sent via SMS text messages. While there are many tools available for SMS text message surveys, Qualtrics was selected for this study, as it is a platform available through research institution licensing, allows for secure data hosting arrangements, and provides a user-friendly process to build SMS text message surveys (as it uses the same interface as the one used to build online surveys). It is worthwhile to note that SMS text message distribution in Qualtrics is an add-on feature, and not part of its standard license.
Each question was set up as an individual survey, and an identification number was assigned to each participant to match his/her responses throughout the study. The survey questions were designed specifically for SMS text messages, with short and concise questions that can be responded to using either multiple choice or short-text answers. Study participants were able to participate in English or Arabic (seeand for examples of the survey questions in English and Arabic).
Questions from the Shisha No Thanks SMS text message survey (English language version).
How would you rate smoking shisha compared to cigarettes considering its health effects?
- Less harmful
- More harmful
- Don’t know
What’s the main reason(s) you smoke shisha (in a few words):
Have you recently talked to someone (eg, family or friend) about the harms of smoking shisha?
C. Don’t know
Questions from the Shisha No Thanks SMS text message survey (Arabic language version).
Recruited Community Panel
The study was able to recruit 133 participants for the SMS text message community panel. This was roughly equivalent to the sample size that the project had planned for (n=100 paired responses for each response, anticipating that not all participants would respond to every question).
Community panel participants were recruited through the local community partner’s communication channels, including email newsletters and social media pages; through active local community champions who shared the recruitment survey link with their own networks via email, SMS text message, or in person; and through printed flyers with the survey link at community events, such as tertiary education open days. The recruitment survey was available on a tablet device for participants to complete at these events. Local community partners and active community champions were provided with all the recruitment materials that were used to promote the online recruitment survey.
The research team perceived the following factors to be influential in the recruitment and retainment of community panel participants to the study: nature of the study by reducing the burden of participation, participants’ age range, close engagement with the community during recruitment, participants being financially compensated for their time, and providing Arabic translations for individuals who do not speak English or prefer to participate in Arabic. Although standard SMS text messaging rates applied to the participants to answer each question, this did not hinder their response rate.
The SMS text message community panel participants’ age ranged from 18 to 35 years (mean 25.8 [SD 5.1]), with 64.7% (86/133) being female. In terms of language spoken at home, 12/133 (9.0%) spoke only Arabic, while 87/133 (65.4%) spoke English and Arabic. These demographics were consistent with the target group the research was designed to study. Only 5/133 (3.8%) participants opted to complete the survey in Arabic.
The SMS text message community panel participants received questions on their phone via SMS text messages. To respond to the question, they sent their response by replying to the same number via SMS text message and typing in either a multiple-choice response or a short text. The mean response rate for individual survey questions was 73.0% (97.1/133) (range 76/133 [57.1%] to 112/133 [84.2%]). This response rate is comparable to the rates reported in other studies using SMS text message surveys [- ]. shows the response rates for each question that was asked before and after the project. Response rates for 6 out of 7 questions were lower for the second round.
|Questiona||Participants who responded (N=133)|
|First round (before project), n (%)||Second round (after project), n (%)|
|Q1||101 (75.9)||89 (66.9)|
|Q3||105 (78.9)||87 (65.4)|
|Q4||103 (77.4)||87 (65.4)|
|Q5||112 (84.2)||85 (63.9)|
|Q6||106 (79.7)||93 (69.9)|
|Q7||76 (57.1)||93 (69.9)|
|Q8||105 (78.9)||87 (65.4)|
aQ2 has not been included, as it was only sent to participants who answered yes to Q1.
Study Design Challenges
The main challenge encountered was related to the Arabic translation of participant material, including the participant information and consent form, online recruitment survey, and the SMS text message survey questions. An accredited translator translated these materials from English to Arabic. To check for accuracy and content, an Arabic-speaking researcher on the evaluation team compared the translated version with the original English version. However, during the initial recruitment phase, some Arabic-speaking participants who chose to complete the online recruitment survey in Arabic informally reported to the project officer that the participant information and consent form included complex research terminology that were difficult for the general community to understand in Arabic. To rectify this, the participant materials were re-translated using a different translation service and reviewed by 5 Arabic-speaking community members.
Potential Uses and Benefits of SMS Text Message Community Panels During COVID-19
The SMS text message community panel is a feasible approach that overcomes many barriers to data collection during a pandemic. SMS text message community panels allow for noncontact data collection, which is an important attribute during the COVID-19 pandemic, with physical distancing and isolation being key behavioral strategies in preventing COVID-19 spread. SMS text message community panels are potentially able to include people who are more difficult to reach using other data collection methods, such as people from culturally and linguistically diverse backgrounds and differing levels of language proficiency, young people, people who live in rural and remote locations, and people with no fixed address. The method allows for timely data processing, as the data are automatically populated into digital format for analysis, which is particularly pertinent during COVID-19 as situations change quickly. This approach could be used for relatively quick data collection on needs, perceptions, and self-reported behaviors in the context of COVID-19. It is not intended to be a replacement for disease surveillance activities, but represents a potentially important additional method of collecting data.
SMS Text Message Community Panels in Comparison to Other Data Collection Methods
In comparison to online surveys, SMS text message surveys are more specifically tailored to mobile phones. Participants respond to SMS text message survey questions in the same phone app on which they receive the questions, and the app is built into the functionality of mobile phones. By contrast, for online surveys, participants are required to click on a link in an email or social media post, which takes them to a web browser application to complete the survey. While this is a small obstacle, removing any obstacle is beneficial for improving response rates. In addition, SMS text message enables people with mobile phones (not smartphones) to participate.
Online surveys are usually developed as 1 survey with numerous questions. SMS text message community panels send only 1 or 2 questions per week, which means that participants only require a short amount of time to respond to the question(s) each week. In this way, SMS text message community panel surveys are more beneficial for measuring repeated measures at short time intervals. As an example, questions could collect data on how people are feeling during each week, and track changes in relation to situational changes (eg, small outbreaks, changes in lockdown policies, or vaccine rollout announcements).
An important benefit of using SMS text message is that it does not rely on proprietary messaging platforms, such as Facebook or WhatsApp. These proprietary platforms may have privacy or data governance implications. SMS text messages also allow a degree of anonymity as participants can be identified only as their phone number, unlike proprietary platforms, which automatically display names and personal information.
The SMS text message survey method is not appropriate to address all areas of research, particularly those that require more in-depth and detailed inquiry. Closed questions are generally limited to simple ordinal or categorical responses, and responses to open questions are limited to 160 characters before they are split up into multiple messages.
Considerations for Use
Recruitment and Participant Demographics
The experience of the “Shisha No Thanks” project in recruiting SMS text message community panel members demonstrated that recruitment to an already engaged community is effective. Recruitment by texting random mobile numbers with invitations to an SMS text message panel may not be effective. The researchers propose that recruitment to SMS text message community panels should be through channels where people have already established an interest or relationship (eg, signed up to community organization’s database), or through regular recruitment methods (eg, advertisements in newsletters, social media, and personal networks). In addition, this method of data collection is most suitable for recruiting participants from demographic groups who are confident with using SMS text message technology, and frequently use their mobile phone, such as the young people (18-35-year olds) who were the focus of the “Shisha No Thanks” project.
Incentives to Participate
In the “Shisha No Thanks” project, participants reported that reimbursements acted as reminders for people to respond to the survey. Distributing reimbursements using the same platform as was used for the survey questions also facilitated this approach. During COVID-19, easily redeemable reimbursements may be equally important, given the disruptions and other challenges people face.
Data Privacy and Security
In the “Shisha No Thanks” project, data collected were in a nonidentifiable format, and the platform used to create the SMS text message survey used firewall-protected systems and passwords to protect the data. However, SMS text message technology does not use end-to-end message encryption, and so SMS text messages do carry a risk of unauthorized access to the data. Therefore, this method of data collection may not be suitable for sensitive data. Despite the security limitations of SMS text message technology, it is worth noting that studies have found most people do not have privacy or security concerns with using SMS text messages [, ].
Not everyone will have high levels of literacy, health literacy, or digital literacy. As with all surveys, careful and considered design of the survey questions can help reduce some of these barriers. Enabling the option for people to participate in the “Shisha No Thanks” study in English or Arabic presented a range of challenges including ensuring accurate translation and difficulties in ensuring non-Roman characters displayed correctly when sent via SMS text message. However, the ability to address these challenges illustrates that if inclusiveness is meaningfully considered in design, SMS text message community panels can broaden participation when compared with cross-sectional surveys, as they can be more specifically tailored to the subgroup of interest.
An important consideration is whether the cost of SMS text message would be a potential barrier for participation. If this method is intended to be used with socioeconomically disadvantaged groups, then this issue needs to be investigated, and it may be important to use toll-free response numbers.
This paper presents a case study demonstrating how SMS text message community panels were used for the “Shisha No Thanks” project. Being only 1 case study, there are several limitations in our understanding about how SMS text message community panels could work for other research, particularly in the COVID-19 context. The “Shisha No Thanks” project featured substantial community participation and engagement, which may have contributed to its ability to recruit and retain participants to the SMS text message community panel. It is unclear how important this initial engagement with the community is to the success of SMS text message community panels. In addition, it is not known how important reimbursements were to the recruitment and retention success of the “Shisha No Thanks” SMS text message community panel. As reimbursement practices vary substantially between studies , establishing SMS text message community panels in the future with reduced or no reimbursements would demonstrate whether substantial financial reimbursements are an essential component for this method of data collection. Finally, while we did notice some attrition during the study, further research is needed to identify how long people would be willing to be engaged on such SMS text message community panels.
The authors thank Dr Ross MacKenzie for his valuable comments on this paper. LC receives support from the Prevention Research Support Program, funded by the New South Wales Ministry of Health. The “Shisha No Thanks” project was funded by the Cancer Institute NSW.
Conflicts of Interest
- Holmes EA, O'Connor RC, Perry VH, Tracey I, Wessely S, Arseneault L, et al. Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science. Lancet Psychiatry 2020 Jun;7(6):547-560 [FREE Full text] [CrossRef] [Medline]
- Cao W, Fang Z, Hou G, Han M, Xu X, Dong J, et al. The psychological impact of the COVID-19 epidemic on college students in China. Psychiatry Res 2020 May;287:112934 [FREE Full text] [CrossRef] [Medline]
- Armitage R, Nellums LB. COVID-19 and the consequences of isolating the elderly. Lancet Public Health 2020 May;5(5):e256 [FREE Full text] [CrossRef] [Medline]
- Liem A, Wang C, Wariyanti Y, Latkin CA, Hall BJ. The neglected health of international migrant workers in the COVID-19 epidemic. Lancet Psychiatry 2020 Apr;7(4):e20 [FREE Full text] [CrossRef] [Medline]
- Premkumar A, Lovecchio FC, Stepan JG, Kahlenberg CA, Blevins JL, Albert TJ, et al. A Novel Mobile Phone Text Messaging Platform Improves Collection of Patient-Reported Post-operative Pain and Opioid Use Following Orthopedic Surgery. HSS J 2019 Feb 22;15(1):37-41 [FREE Full text] [CrossRef] [Medline]
- Lee SSS, Xin X, Lee WP, Sim EJ, Tan B, Bien MPG, et al. The feasibility of using SMS as a health survey tool: an exploratory study in patients with rheumatoid arthritis. Int J Med Inform 2013 May;82(5):427-434. [CrossRef] [Medline]
- Säve-Söderbergh M, Toljander J, Mattisson I, Åkesson A, Simonsson M. Drinking water consumption patterns among adults-SMS as a novel tool for collection of repeated self-reported water consumption. J Expo Sci Environ Epidemiol 2018 Mar 14;28(2):131-139 [FREE Full text] [CrossRef] [Medline]
- Kuntsche E, Robert B. Short message service (SMS) technology in alcohol research--a feasibility study. Alcohol Alcohol 2009 May 29;44(4):423-428. [CrossRef] [Medline]
- Chang T, Gossa W, Sharp A, Rowe Z, Kohatsu L, Cobb EM, et al. Text messaging as a community-based survey tool: a pilot study. BMC Public Health 2014 Sep 08;14(1):936 [FREE Full text] [CrossRef] [Medline]
- Geldsetzer P. Use of Rapid Online Surveys to Assess People's Perceptions During Infectious Disease Outbreaks: A Cross-sectional Survey on COVID-19. J Med Internet Res 2020 Apr 02;22(4):e18790 [FREE Full text] [CrossRef] [Medline]
- Wang C, Pan R, Wan X, Tan Y, Xu L, Ho CS, et al. Immediate Psychological Responses and Associated Factors during the Initial Stage of the 2019 Coronavirus Disease (COVID-19) Epidemic among the General Population in China. Int J Environ Res Public Health 2020 Mar 06;17(5):1729 [FREE Full text] [CrossRef] [Medline]
- Kwok KO, Li KK, Chan HHH, Yi YY, Tang A, Wei WI, et al. Community Responses during Early Phase of COVID-19 Epidemic, Hong Kong. Emerg Infect Dis 2020 Jul;26(7):1575-1579 [FREE Full text] [CrossRef] [Medline]
- McFadden SM, Malik AA, Aguolu OG, Willebrand KS, Omer SB. Perceptions of the adult US population regarding the novel coronavirus outbreak. PLoS One 2020 Apr 17;15(4):e0231808 [FREE Full text] [CrossRef] [Medline]
- Dos Santos Marques IC, Theiss LM, Johnson CY, McLin E, Ruf BA, Vickers SM, et al. Implementation of virtual focus groups for qualitative data collection in a global pandemic. Am J Surg 2021 May;221(5):918-922 [FREE Full text] [CrossRef] [Medline]
- Shisha No Thanks! Raising the awareness of the harms of waterpipe smoking in young people from Arabic speaking backgrounds. Centre for Primary Health Care and Equity, UNSW. 2020. URL: https://cphce.unsw.edu.au/shisha-no-thanks-raising-awareness-harms-waterpipe-smoking-young-people-arabic-speaking-backgrounds [accessed 2021-08-08]
- ShishaNoThanks. URL: https://www.shishanothanks.org.au/ [accessed 2021-10-14]
- Cancer Institute NSW. Cancer Institute Tobacco Tracking Survey (CITTS). URL: https://www.cancer.nsw.gov.au/research-and-data/cancer-data-and-statistics/request-unlinked-unit-record-data-for-research/cancer-institute-tobacco-tracking-survey-citts [accessed 2021-10-14]
- Syrian Center for Tobacco Studies, University of Memphis Center for Community Health. Narghile-Waterpipe users Survey. URL: http://www.scts-sy.org/en/instruments [accessed 2021-10-14]
- Yellow Social Media Report 2018. Part One - Consumers. 2018. URL: https://www.yellow.com.au/social-media-report/ [accessed 2019-02-05]
- FairWork Ombudsman. Minimum Wages. URL: https://www.fairwork.gov.au/tools-and-resources/fact-sheets/minimum-workplace-entitlements/minimum-wages [accessed 2021-08-08]
- Giguere R, Brown IW, Balán IC, Dolezal C, Ho T, Sheinfil A, et al. Are participants concerned about privacy and security when using short message service to report product adherence in a rectal microbicide trial? J Am Med Inform Assoc 2018 Apr 01;25(4):393-400 [FREE Full text] [CrossRef] [Medline]
- Dechand S, Naiakshina A, Danilova A, Smith M. In Encryption We Don’t Trust: The Effect of End-to-End Encryption to the Masses on User Perception. 2019 Presented at: 2019 IEEE European Symposium on Security and Privacy (EuroS&P); June 17-19, 2019; Stockholm, Sweden p. 401-415. [CrossRef]
- Fry CL, Ritter A, Baldwin S, Bowen KJ, Gardiner P, Holt T, et al. Paying research participants: a study of current practices in Australia. J Med Ethics 2005 Sep 01;31(9):542-547 [FREE Full text] [CrossRef] [Medline]
Edited by G Eysenbach; submitted 18.03.21; peer-reviewed by S Brabyn, A Sudaryanto; comments to author 30.07.21; revised version received 09.08.21; accepted 19.09.21; published 03.11.21Copyright
©Lilian Chan, Nouhad El-Haddad, Becky Freeman, Blythe J O'Hara, Lisa Woodland, Ben Harris-Roxas. Originally published in JMIR Formative Research (https://formative.jmir.org), 03.11.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.