Published on in Vol 5, No 3 (2021): March

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/18460, first published .
System Architecture for "Support Through Mobile Messaging and Digital Health Technology for Diabetes" (SuMMiT-D): Design and Performance in Pilot and Randomized Controlled Feasibility Studies

System Architecture for "Support Through Mobile Messaging and Digital Health Technology for Diabetes" (SuMMiT-D): Design and Performance in Pilot and Randomized Controlled Feasibility Studies

System Architecture for "Support Through Mobile Messaging and Digital Health Technology for Diabetes" (SuMMiT-D): Design and Performance in Pilot and Randomized Controlled Feasibility Studies

Original Paper

1Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, United Kingdom

2Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom

Corresponding Author:

Yuan Chi, BEng, PhD

Institute of Biomedical Engineering

Department of Engineering Science

University of Oxford

Old Road Campus Research Building

Headington

Oxford, OX3 7DQ

United Kingdom

Phone: 44 01865 617675

Email: yuan.chi@eng.ox.ac.uk


Related ArticleThis is a corrected version. See correction statement in: https://formative.jmir.org/2021/4/e29451/

Background: Diabetes is a highly prevalent long-term condition with high morbidity and mortality rates. People with diabetes commonly worry about their diabetes medicines and do not always take them regularly as prescribed. This can lead to poor diabetes control. The Support Through Mobile Messaging and Digital Health Technology for Diabetes (SuMMiT-D) study aims to deliver brief messages as tailored interventions to support people with type 2 diabetes in better use of their diabetes medicines and to improve treatment adherence and health outcomes.

Objective: This paper describes the overall architecture of a tailored intervention delivery system used in the pilot and randomized controlled feasibility studies of SuMMiT-D and reports its performance.

Methods: The SuMMiT-D system includes several platforms and resources to recruit participants and deliver messages as tailored interventions. Its core component is called the clinical system and is responsible for interacting with the participants by receiving and sending SMS text messages from and to them. The personalization and tailoring of brief messages for each participant is based on a list of built-in commands that they can use.

Results: For the pilot study, a total of 48 participants were recruited; they had a median age of 64 years (first quartile, third quartile [Q1, Q3: 54.5, 69]). For the feasibility study, a total of 209 participants were recruited and randomly assigned to either the control or intervention group; they had a median age of 65 years (Q1, Q3: 56, 71), with 41.1% (86/209) being female. The participants used the SuMMiT-D system for up to 6 months (26 weeks) and had a wide range of different interactions with the SuMMiT-D system while tailored interventions were being delivered. For both studies, we had low withdrawal rates: only 4.2% and 5.3% for the pilot and feasibility studies, respectively.

Conclusions: A system was developed to successfully deliver brief messages as tailored health interventions to more than 250 people with type 2 diabetes via SMS text messages. On the basis of the low withdrawal rates and positive feedback received, it can be inferred that the SuMMiT-D system is robust, user-friendly, useful, and positive for most participants. From the two studies, we found that online recruitment was more efficient than recruitment via postal mail; a regular SMS text reminder (eg, every 4 weeks) can potentially increase the participants’ interactions with the system.

Trial Registration: ISRCTN Registry ISRCTN13404264; http://www.isrctn.com/ISRCTN13404264

JMIR Form Res 2021;5(3):e18460

doi:10.2196/18460

Keywords



Diabetes is one of the most prevalent long-term conditions affecting the world population [1]. Currently, 4.7 million people in the United Kingdom have diabetes; more than 5 million people will have diabetes by 2025. Among people with diabetes, approximately 90% have type 2 diabetes [2]. People with type 2 diabetes are at high risk of developing serious complications, including cardiovascular disease, stroke, and chronic kidney disease, which in turn lead to an increase in the cost and resources needed for health care [1,3,4]. To avoid related comorbidities and complications, treatments with proven efficacy are needed alongside facilitating patients in the self-management of their condition [5-7].

People with diabetes commonly worry about their medicines and do not always take them regularly as prescribed [8]. This can lead to poor diabetes control. To support people with type 2 diabetes in better use of medicines, a diverse range of care programs are available. Many such services are standardized according to available evidence and based on guidelines; however, not all people equally benefit from such a one-size-fits-all approach [9-11]. People with poorly controlled diabetes benefit mostly from intensive, provider-driven management, whereas people with adequate glucose levels might maintain these levels independently. A more patient-centered approach is becoming a preferred strategy for improving patient outcomes, where the care is tailored according to individual patient needs and preferences [12-15].

Delivering brief messages that can address a wide range of concerns at a wide scale via digital health systems, in addition to usual care, is a promising approach to develop tailored interventions to improve treatment adherence and health outcomes [16-19]. It has been shown to be both effective and of low cost in improving health in other health conditions, including in reducing cardiovascular risk, lowering blood pressure, and stopping smoking [20-22]. In addition, as tailored interventions are more personally relevant for recipients, they would have a higher chance of being noticed, read, understood, and acted on [23].

This paper describes a SMS-based system developed for and evaluated in the Support Through Mobile Messaging and Digital Health Technology for Diabetes (SuMMiT-D) study. Through mobile health technology integrated with clinical care, this system delivers automated, tailored brief messages to support people with type 2 diabetes in effectively using medicines. The system has currently been used by more than 250 patients with type 2 diabetes across the United Kingdom and is planned to be used by another 1000 patients in the near future. The system is designed to be user-friendly: it is based on carefully developed SMS messages aimed at a large target population and accommodates users without previous experience of digital technologies. The SuMMiT-D system incorporates algorithms that identify messages to be tailored according to the user’s response and characteristics (eg, medical history). This paper addresses an existing gap in the literature: the design of systems to deliver brief messages in the health care delivery system. The messages are tailored and personalized interventions to support people with type 2 diabetes. This paper presents the technical methodology used in the pilot and feasibility studies and reports on the system performance. A paper reporting the protocol for the SuMMiT-D feasibility study [24] has been published separately.


SuMMiT-D

SuMMiT-D is a program of work intended to develop and test the delivery of tailored and personalized brief messages, implemented in SMS text messages. Although implemented in a study, it is a demonstration of how this could be achieved when integrated within the National Health Service (NHS). The aim of the SuMMiT-D system is to provide enhanced and optimal support for people with type 2 diabetes. The program of work reported here consisted of two stages: a formative work package based around a pilot study to develop and test the system and a randomized controlled trial to evaluate feasibility. For the pilot study, 48 participants were recruited to evaluate and assess the system. During this phase, participants were asked to use the SuMMiT-D system for 13 weeks, with the possibility of continuing their participation for up to 26 weeks. This stage aimed to understand the feasibility of the technical aspects of the study.

For the second phase, a feasibility study, 209 participants were recruited. This study evaluated the use of the system for a total of 6 months (26 weeks). Its purpose was to understand the participant recruitment rate and timing as well as the feasibility of integrating the developed system directly with general practitioner (GP) systems. This work was completed in October 2019. The developed methodology will be used for the main trail in 2020.

The SuMMiT-D System

Figure 1 shows the system architecture designed for the SuMMiT-D study. The design work required the coordination of the following platforms and resources: Sentry, an online-based recruitment system developed and managed by the Primary Care Clinical Trials Unit, University of Oxford; the mobile phones used by the participants to send and receive SMS messages; a web interface accessible by the system users (ie, system administrators and research staff) to manage the clinical system; and Esendex [25], an SMS engine provider that enabled delivery and receipt of text messages to and from participants. The system was also designed to integrate alerts and personalization of messages based on primary care electronic health record (EHR) data. Platforms and protocols were required to meet the standards of data and security management [26]. Operating procedures were designed to achieve secure and independent management of the system as well as the storage of data.

Figure 1. Overview of systems developed for the Support through Mobile Messaging and Digital Health Technology for Diabetes study. Sentry: Secure entry; SuMMiT-D: Support Through Mobile Messaging and Digital Health Technology for Diabetes.
View this figure
Clinical System

The back end (ie, the data access layer that provides access to data stored in a database) of the clinical system was developed using the CakePHP web framework [27], and the associated database tables were developed using the relational database management system MySQL [28], whereas the front end (ie, the presentation layer that formats and delivers information for further processing or display) was developed using three core technologies of the WWW [29]: HTML [30], Cascading Style Sheets [31], and Javascript [32]. These open-source technologies were chosen because of ease of use, fast iteration, and adherence to communication standards. The core component of the clinical system was implemented using CakePHP. The CakePHP web framework [27] has several advantages: it is an open-source platform that supports the PHP programming language [33]; it requires almost no preconfigurations, as most of the settings and options are auto-detected; it implements a model-view-controller pattern, which divides the developing user interfaces into 3 interconnected elements: model, view, and controller; it offers a wide range of built-in plug-ins (ie, software components that add specific features to an existing computer program) to enable customization; and it supports fast connectivity with database systems. All these characteristics make it particularly suitable for the reliable and extendible implementation of web-based services.

To meet both the trial and the NHS security and privacy requirements and to allow clinical implementation once the whole trial ends, the clinical system has been deployed in a server managed by the Oxford University Hospitals NHS Foundation Trust. The server is located behind the firewall managed by the local information management and technology team at the Oxford University Hospitals NHS Foundation Trust and hosted within the NHS Health and Social Care Network [34]. Access to this system is password protected and available only to registered users. All data (ie, database tables and participants’ GP records) are stored separately on another internal server, which is inaccessible directly from the internet and communicates with the main front-end server via a password-protected, Transport Layer Security (TLS) [35] encrypted channel.

Figure 2 shows the architecture of the clinical system with various key components that have been developed. The various components of the system include the following: Access Management, Data Sharing Management, Receiver and Sender Management, Random Message Selection Management, Reminder Management, Interaction Management.

Figure 2. Support through Mobile Messaging and Digital Health Technology for Diabetes architecture. Sentry: Secure entry; SuMMiT-D: Support Through Mobile Messaging and Digital Health Technology for Diabetes.
View this figure

Participants can interact with the SuMMiT-D system by sending SMS messages from their mobile phones to a virtual mobile number (provided by Esendex) using one of the predefined commands. Esendex provides two ways of interacting with their SMS engines: (1) using the toll-free number 0800 or (2) using a virtual mobile number. For using the toll-free number 0800, each command needs to start with a specified keyword (eg, DIA) for Esendex to identify the system with which the user wants to interact. In using a virtual mobile number, a virtual number is exclusively associated with one system, so that the routing keyword is not required. In this instance, the user bears the cost of sending SMS messages. Although the latter method represents a cost to the user, it has the following advantages: the cost of sending a text message is acceptable to participants; text messages from participants will always reach the SuMMiT-D system; and if the former method is used and the specified keyword has been forgotten or mistyped, the delivery of messages to the SuMMiT-D system will fail. System users (ie, health care professionals, research staff, and system administrators) engage with the clinical system using an internet-enabled computing device (eg, a smartphone, a tablet, or a computer).

Interaction Management

The Interaction Management module determines how the implemented algorithms in the SuMMiT-D system tailor messages according to each individual’s preferences. Each time a participant sends a text via Esendex, the system (1) retrieves the original text from Esendex via the receiver or sender management; (2) parses and interprets the text referring to preferences; and (3) generates a response to be pushed back, for Esendex to finally send it to the corresponding participant.

The study participants can send predefined commands to the SuMMiT-D system. A command in the context of text messages refers to one or several words corresponding to one or more actions on the SuMMiT-D receiver end. In the SuMMiT-D system, the commands correspond to a list of settings that participants can change according to their preferences, for example, when to receive messages in a day and like or dislike the last received message to express interest in more or fewer messages of the same kind in the future. Other possible interactions include initial registration with the system, pausing messages for X weeks, restarting from being paused, completely stopping receiving messages, reviewing, and rating received messages, and requiring help instructions on how to interact with the system. The full list of commands is listed in Table 1.

Table 1. List of commands available in the Support through Mobile Messaging and Digital Health Technology for Diabetes system and the corresponding descriptions.
Study and commandsDescription
Common to both studies
Register namesRegister with the SuMMiT-D system
StartStart receiving messages again
StopStop receiving messages completely (Need to contact the SuMMiT-D study team to restart the messages)
AMReceive messages in AM (from 9 AM to 12 PM)
PMReceive messages in PM (from 12 PM to 6 PM)
Help, Help 1, Help 2Help messages for using the list of commands
Pilot
Less or MoreReceive fewer or more messages similar to the last received one
Like or DislikeLike or Dislike the last received message
Easy or HardThe last received message is easy or hard to understand
Useful or Not UsefulThe last received message is useful or not useful
Help 3Help messages for using the list of commands
Feasibility
Like or DislikeReceive fewer or more messages similar to the last received one
Reminder Management

Besides instantaneous interactions, a participant also received scheduled text messages during the study. The reminder management module schedules text reminders. During the pilot study, to encourage the participants to review and rate the randomly selected messages sent to them, a prompt text message was sent to each participant every week (ie, a reminder to use the commands Like or Dislike, Easy or Hard, and Useful or Not Useful). For the pilot study, after 4 weeks of participation, a reminder was sent on how to use the commands More or Less; whereas for the feasibility study, 4 weeks after randomization, a reminder was sent on how to use the commands Like or Dislike. In addition, at the beginning and in the middle of both studies, a participant would receive a text reminder on how to use the Help commands. Other types of engagement messages were a happy birthday text message and a reminder on the day of world diabetes day celebration (14th November).

Random Selection Management

The SuMMiT-D study participants received a number of brief messages sent periodically. Two types of brief SMS messages were used: one type is based on behavior change techniques (BCTs), and the other type is based on general lifestyle advice [36-38].

During the pilot study, in which all participants belonged to one group, these messages were sent three times a week. For every five continuous BCT-based messages, one general lifestyle message was sent.

During the feasibility study, participants were randomized to either the control or intervention groups. For those in the control group, only control messages were sent at a frequency of approximately once a month. The control messages here refer to general messages that are irrelevant to medicine management. For those in the intervention group, two types of messages (BCT and general lifestyle) were sent at a frequency of 3 messages per week (1 general lifestyle message, for every 2 continuous BCT-based messages). The increased frequency of lifestyle messages was based on the feedback of the participants in the pilot study. Examples of SMS text messages used in the SuMMiT-D feasibility study are shown in Table 2.

Table 2. Examples of SMS text messages used in the Support through Mobile Messaging and Digital Health Technology for Diabetes feasibility study.
Message categoryExample messages
Behavior change technique
1.4 Action PlanningPlan when, where and how you are going to take your medication
2.3 SelfmonitoringFind a way to split your tablets into days so you notice when you have forgotten to take your tablets.
7.1 Prompts or cuesIt can be difficult to remember to take your tablets. Why not set an alarm to remind you to take them?
G Health care system related concernsLots of questions? Check who the best person to see might be.
General lifestyle
SignpostingWant to hear what other people with diabetes think? You can hear different people discuss their experiences at healthtalk online.

For the pilot study, there were a total of 157 unique BCT messages used, and they were from 30 different BCT groups, whereas there were 35 unique general lifestyle messages. On the basis of the feedback (ie, the message ratings) of the participants in the pilot study, the messages used in the feasibility study have been edited accordingly: there were a total of 170 unique BCT messages used, and they were from 30 different BCT groups, whereas there were 35 unique general lifestyle messages and 6 unique control messages. More detailed information on the messages used in the pilot and feasibility studies can be found in the studies by Farmer et al [24] and Bartlett et al [39].

The algorithm below describes the methodology for sending these SMS messages:

Suppose there are t different groups of BCTs as: {B1, B2, ..., Bt}, with the i-th (1≤it) BCT group having ni different SMS texts, that is, . This results in n unique BCT SMS texts, where . There are also m different general lifestyle SMS texts . As a result, there are n+m unique SMS messages in the system, which can be represented as {B1, B2, ..., Bt, G}. The SMS messages sent to each participant follow this rule: for each week (ie, 7 days), 3 messages are sent, and they are sent with an even time gap; in addition, after continuously sending q BCT-based SMS messages, 1 general lifestyle-based message should be sent (q=5 for the pilot study and q=2 for the feasibility study according to the participants’ feedback from the pilot study).

The SMS messages sent to each participant were randomly selected as follows: for each participant, all t different BCT groups of messages have the same probability. To ensure that a different message was received each time, the system excluded all the previously sent messages and the most recently used BCT group and selected a message from the remaining messages and BCT groups. The same approach was used when sending general lifestyle-based SMS messages. The above process is enabled only for those participants who were scheduled to receive a message and only within the time slot selected by each participant.

In addition to the above process, the SuMMiT-D system tailors and personalizes the messages for each participant. Tailoring and personalization were performed according to the text responses from each participant. In the first instance, upon receiving Pause X, Start, and STOP from a participant, messages could be paused for X weeks, restarted, or completely stopped, respectively.

Upon receiving More and Like from a participant in the pilot and feasibility studies, respectively, after she or he receives a BCT-based message (ie, , where ), another message from the same BCT group (ie, , where ) is scheduled to be sent to this participant after 2 weeks. In addition, all the messages in the same BCT group (ie, ) would have doubled the chance (ie, increase 100% probability) to be selected and sent to this participant in the future. Upon receiving Dislike from a participant in the feasibility study, after she or he receives a BCT-based message (ie, , where ,), all the messages in the same BCT group (ie, ) would have halved the chance (ie, decrease 50% probability) to be selected and sent to this participant in the future. For details, please refer to the pseudocode listed in Multimedia Appendix 1.

Upon receiving AM and PM from a participant, the message delivery time slot to this participant was changed accordingly.

Receiver or Sender Management

The purpose of this component is to send and receive messages to and from participants. This management component employs a third-party mobile messaging system (Esendex [25]) to act as a bridge between the participants and the clinical system. To dispatch SMS messages to the participants’ mobile phones, they are sent to the Esendex system’s application program interface (API) by the clinical system. The information exchanged between the 2 systems includes the target mobile number, the message context to be sent, a unique message ID, and a key code to authorize this message to be sent. This communication is established over an encrypted HTTP channel over TLS [35]. After 5 hours, the clinical system again connects to Esendex’s API and then checks the delivery status of the message using the unique message ID, which is returned by Esendex during the first connection. This process allows tracking of the delivery status of every sent message. Each message delivery status (ie, Delivered, Sent, Failed, Expired, Cancelled, and Partially Delivered) is stored by the clinical system, along with the corresponding message information.

For messages whose delivery fails (ie, a delivery status of Failed, Expired, Cancelled, and Partially Delivered), another attempt is scheduled immediately. After further 2 failed redelivery attempts, an automatic email is sent to the system users, whose role is to be either administrators or researchers, to notify this failed delivery and suggest the possibility that the participant may have changed this mobile number.

The clinical system checks Esendex’s API at an interval of every 30 seconds. This very short period was selected to provide a sense of timeliness of the response to (potential) participants and to maintain a high engagement level.

Data Sharing Management

The eligible candidates were screened and then invited to the SuMMiT-D study (full details of the recruitment process have been reported elsewhere [24]). To participate in any SuMMiT-D study, patients had to express their interest with a text message (ie, Register names) to one of the virtual numbers provided by Esendex. Once the text was received, an automatic reply was sent back. This handshake process guaranteed that the mobile number was valid. For the pilot study, the recruitment process was completed over the phone, the paper consent form was sent and returned via post, and personal information was directly entered into the SuMMiT-D system via a web interface. For the feasibility study, to add another level of security and automation, the recruitment process was performed electronically via a system called Sentry (Secure entry) [40]. The consent form could then be completed electronically. Once the recruitment process for a participant was completed, all personal details were securely stored in REDCap [41]. To enable data sharing of personal details required to be shared with the clinical system, data were pushed from the Sentry system to the clinical system via HTTPs [42]. During this data transaction, the mobile number that the participant used to express interest was used as the matching key. The following personal details were shared by Sentry and stored in the clinical system: first name, last name, mobile number, GP practice name, date of birth, preferred name, gender, NHS number, whether using a smartphone, and participant study ID. As Sentry also implemented the randomization algorithm for determining the control or intervention group, it shared also the randomization result so that the SuMMiT-D system would know whether a participant data that had been pushed from Sentry to the clinical system belonged to the control or intervention group.

Access Management

System users can log in to the SuMMiT-D system using their personal usernames and passwords. Users are categorized according to 3 levels of privileges that determine their access rights to the information and functions available. The system administrators (Admin) get access to the entire system and all its functionalities; the study researchers (Researcher) get full access to the information of participants and GP practices; and the research nurses, trail manager, and coordinator (User) get only a read-only view of the information of participants and GP practices.

The web pages that a system user can access depend on the privilege level, which is determined by the log-in credentials. Table 3 provides the main views that a user can access according to their privilege level and a brief description for each of those views.

Table 3. Privilege levels and the corresponding functionality available at each of these privilege levels.
FunctionalityPrivilege level
AdminResearcherUser
  • Add system users (admin, researcher, user)
  • List of system users
  • Detailed view or edit of system users
ab
  • Add participants
  • Add GPc practices
  • Detailed edit of participants
  • Detailed edit of GP practices
  • List of sent or received SMS
  • Detailed view of sent or received SMS
  • List of participants
  • Search participants
  • List of GP practices
  • Detailed view of participants
  • Detailed view of GP practices
  • Detailed view or edit of own details

aFunction is available at this level.

bFunction not available at this level.

cGP: general practitioner.

Development Approach

The SuMMiT-D system was developed in collaboration with health care professionals, the study team, the software development team, and a group of patient representatives. Both test GP practices and patients were set up during the early development process, so that the system could be evaluated for its reliability and to allow software bugs to be identified and fixed.

Defined Metrics

To evaluate the performance of the SuMMiT-D system, the following 3 metrics were defined: response rate, keyword percentage, and rate versus prompt.

Response Rate

To determine whether a participant was active in the study, the response rate of SMS messages for each participant was calculated according to:

(1)

where, for the pilot study,

(2)

whereas, for the feasibility study,

(3)
Keyword Percentage

In the pilot study, to evaluate the usability of a keyword-based system, a fictitious keyword was selected. Participants were asked to submit all commands after using the DIA keyword. The keyword (DIA) percentage is calculated according to:

(4)
Rate Versus Prompt

In the pilot study, to review whether the prompt messages could improve the total number of responses (ie, rating) received, the ratio of number of response messages to the number of prompt messages is calculated as:

(5)

Where rating is calculated according to equation 2.


Pilot Study

For the pilot study, 48 participants were recruited, with a median age of 64 years (first quartile and third quartile [Q1, Q3: 54.5, 69]). Among the 48 participants, 19 (40%) were from the Greater Manchester region and 29 (60%) were from the Thames Valley region.

Duration of Study and Admission Process

The first participant expressed interest on April 24, 2018, and the last participant expressed interest on August 24, 2018. The first participant started to use the SuMMiT-D system on May 8, 2018, whereas the last participant started to use the system on September 17, 2018. The first participant stopped using the SuMMiT-D system (ie, either finished or withdrew from the study) on August 11, 2018, whereas the last one stopped on January 31, 2019.

An admission process duration refers to how long (in days) it took the participants from expressing interest (ie, sending the Register command to register with the system) to fully start the SuMMiT-D study (ie, to begin receiving regularly sent messages). The admission process duration here has a median value of 21.07 days (Q1, Q3: 17.31, 24.93).

Two participants withdrew from the study at days 23 and 67 (withdrawal rate of 4%). At 3 months (13 weeks) after using the system, 19% (9/48) participants chose to stop, whereas the remaining 77% (37/48) decided to continue.

In addition, a total of 11 STOP commands have been received, which are in line with the results above: 2 participants withdrew and 9 stopped at 3 months after using the system. Only 2 Pause X commands were received: one chose to pause the messages for 2 weeks and the other paused for 4 weeks.

SMS Message Response Rate of Participants

The metric here is defined as the response rate, which is calculated according to equations 1 and 2. The median response rate was 71% (Q1, Q3: 53%, 111%). This actual response rate is lower than what might have been expected, as there was a weekly reminder (ie, the prompt message encouraging participants to rate received messages).

Participants Including the Keyword DIA in Their Responses

Both toll-free and virtual numbers are available for SMS-based studies to interact with participants. The first approach generally shares the same toll-free number across different systems and, thus, requires the use of a keyword to identify the SMS texts received from users. However, the additional keyword may represent a challenge for some participants, as they may forget to add the keyword at the start of a message, and as a result, such a message will be missed at the corresponding system end completely.

The metric here is defined as the keyword percentage, which is calculated according to equation 4. The median keyword percentage was 93% (Q1, Q3: 87%, 98%). The histogram for the keyword percentage is shown in Multimedia Appendix 2. Although the keyword (DIA) was used quite frequently, some messages would have still not been received on the SuMMiT-D system side.

Prompt Messages Versus Message Responses

A weekly reminder was sent to the participants in the pilot study in the form of a prompt message asking to rate the received messages. To review whether this could improve the total number of responses (ie, rating) received, the metric rate versus prompt is calculated as described above equation 5. The rate versus prompt has a median value of 280% (Q1, Q3: 215%, 434%). The rate versus prompt is also lower than expected, which is in line with the results shown above.

In addition, for each BCT message, the total number rated against the total number prompted is summarized and shown in Figure 3, with a quadratic polynomial curve fitted to the data points. From the fitted curve, prompt messages can potentially improve the total number of responses (ie, rating) received.

Figure 3. The total number of rated versus prompted for each behavioral change therapy message for participants in the pilot study.
View this figure
Comparison Among the Preference Groups of Messages Responses

To compare which preference groups (ie, Like or Dislike, Easy or Hard, Useful or Not Useful, and More or Less) are mostly used by participants in the pilot study, the total numbers of Like or Dislike, Easy or Hard, Useful or Not Useful, and Less or More were counted. The results are shown in Figure 4. From the results, it can be concluded that the vast majority of the BCT messages sent in the pilot study received a positive rating (ie, Like and Useful).

Figure 4. Different preference groups among the SMS message response rating for participants in the pilot study. A: Preference group: Dislike vs Like; B: Preference group: Not useful vs Useful.
View this figure

Feasibility Study

For the feasibility study, a total of 209 participants (with a median age of 65 years, Q1, Q3: 56, 71; with 86/209, 41.2% female) were recruited and randomized. Of these, 106 were randomized into the control group, whereas the remaining 103 were randomized into the intervention group. Of the 209 participants, 52 (24.8%) were from the Greater Manchester area and 157 (75.1%) were from the Thames Valley area.

Duration of Study and Admission Process

The first participant expressed interest on December 3, 2018, and the last one expressed interest on March 20, 2019. The first participant started to use the SuMMiT-D system on December 10, 2018, whereas the last participant started to use the system on April 16, 2019. The first participant stopped using the system (ie, either finished or withdrew from the study) on March 4, 2019, whereas the last one stopped using the system on October 16, 2019.

The admission process duration here has a median value of 8.76 days (Q1, Q3: 3.94, 15.07).

A total of 11 participants withdrew from the study (10 from the intervention group): 8 of them stopped the messages via sending the STOP commands, whereas the remaining 3 called the study team to ask for withdrawal. A total of 7 Pause X commands were received, and they were from 4 different participants.

SMS Message Response Rate of Participants

The metric here is the response rate, which is calculated according to equations 1 and 3. The median response rate was 17% (Q1, Q3: 13%, 22%). It can be concluded that there is a much lower response rate for the participants in the feasibility study, as there was no weekly prompt message sent in the latter study.

Like Versus Dislike Among Message Responses

To compare which BCT message is mostly liked or disliked by participants in the feasibility study, the total number of Like or Dislike was counted for all the BCT messages, and the results are shown in Figure 5. In the same way as Figure 4, Figure 5 shows that the great majority of the BCT messages sent in the feasibility study receive a positive rating (ie, Like).

Figure 5. Like versus Dislike among the SMS message response ratings for participants in the feasibility study.
View this figure
Impact of Like or Dislike Reminder

Unlike the pilot study, there were no weekly prompt SMS messages (ie, encouraging participants to rate all the received SMS messages) sent in the feasibility study. To remind the participants in the intervention group to use the Like or Dislike commands, a reminder of how to use these commands was sent to them when they had been in the study for 4 weeks. To determine the impact of Like or Dislike reminders (ie, how long the reminders will have an effect on the participants), the following 2 numbers have been plotted: the total number of Like or Dislike commands received per week and the number of participants sending Like or Dislike commands per week. The results are shown in Figure 6. From the results, it can be seen that in week 5 (ie, just after the reminders were sent), these 2 numbers reached their peak values. From week 9 (ie, 4 weeks after the reminders were sent), the number of participants sending Like or Dislike commands back per week remained almost constant. This suggests that the impact of Like or Dislike reminders persists for approximately 4 weeks.

Figure 6. Impact of Like/Dislike reminder on participants in the feasibility study.
View this figure

In these pilot and feasibility studies for the SuMMiT-D brief messaging, mobile phone–based intervention, participant withdrawal rates were 4.2% and 5.3%, respectively. In addition, from the message responses, it was concluded that most of the participants gave positive feedback to the BCT messages they received (ie, useful and like). Furthermore, when offered the opportunity to continue for another 3 months in the pilot study, 37 out of 46 participants chose to stay. These promising results suggest that the SuMMiT-D system is robust, user-friendly, useful, and positive for most participants.

Comparing the length of the admission process (registration and sign up), it can be seen that there is a significant improvement in the feasibility study compared with the pilot. This may be explained by changing the recruitment method from regular mail (ie, manually sending and receiving all forms: registration, questionnaires, and consent) to online recruitment.

One common issue in both studies is that the response rate of participants to the messages sent to them was lower than expected. This might have been associated with the demographics of those recruited to the studies (with a median age of about 65 years) where participants might not have been familiar with the use of mobile phones. In the pilot study, a weekly prompt message was sent to participants to encourage them to rate received messages. In the feasibility study, a one-time Like or Dislike reminder describing how to use the Like and Dislike commands was sent to participants. Our findings suggest that (1) prompt messages can potentially improve the total number of responses received and (2) a reminder encouraging participants to Like or Dislike messages has an effect persisting for about 4 weeks before a participant returns to their normal response rate. Potentially higher response rates may be obtained from participants if they were sent the reminder every 4 weeks.

The SuMMiT-D system uses brief messages as interventions to provide education and support self-management for people with type 2 diabetes. The brief messages are delivered through mobile phone–based SMS text messages. These patients are often concerned about starting new diabetes medicines or face difficulties in taking their medicines regularly. To enhance the support for patients’ self-management, these brief messages were personalized and tailored for each participant individually, according to each patient’s own preference and their feedback to previously received SMS text messages.

The SuMMiT-D system offers a model for technology-based self-management support [30,31], as it shows the clinical efficacy and cost-effectiveness of a text messaging intervention, compared with the usual care, for people with type 2 diabetes. In addition, the SuMMiT-D system is not only an effective tool for type 2 diabetes, but it could also be extended to other long-term conditions (eg, hypertension).

All the information and feedback that we gathered in each of the studies was carried out iteratively to improve the functionalities of the system. From the pilot study, we learned the following: (1) a recruitment process via regular post was time consuming and imposed a high workload from the study team, and in the feasibility study, we introduced the Sentry system to allow a digital (online) recruitment process, and (2) according to the participants’ feedback, they would have liked to receive the general lifestyle messages more frequently, and we increased the ratio of the general lifestyle and BCT messages from 1 to 5 in the pilot study to 1 to 2 in the feasibility studies.

In the SuMMiT-D pilot and feasibility studies, we have demonstrated the successful delivery of interventions to more than 200 participants via SMS text messages. In the main SuMMiT-D study, to further enhance the support for patients’ self-management, their EHR data will be routinely shared and collected via a GP data provider. EHR data, such as medication possession ratio and glycated hemoglobin, will be used to tailor the intervention messages. In addition, more automation in the recruitment process will be introduced, such that the overall workload is reduced.

Acknowledgments

This project was funded by the National Institute for Health Research (NIHR; Programme Grants for Applied Research [project reference RP-PG-1214-20003]). LT and AF are supported by the NIHR Oxford Biomedical Research Centre. AF is an NIHR Senior Investigator. The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.

Conflicts of Interest

ER supports a team receiving funds from the NIHR Health Technology Assessment (HTA) Programme.

Multimedia Appendix 1

Algorithm 1 pseudocode for receiving “More”/“Like” or “Dislike” after sending a behavior change technique message to a participant.

DOCX File , 13 KB

Multimedia Appendix 2

Histogram plot of the keyword percentage keyword percentage among the SMS SMS message responses for participants in the pilot study.

PNG File , 70 KB

References

  1. Tamayo T, Rosenbauer J, Wild S, Spijkerman A, Baan C, Forouhi N, et al. Diabetes in Europe: an update. Diabetes Res Clin Pract 2014 Feb;103(2):206-217. [CrossRef] [Medline]
  2. Us, diabetes and a lot of facts and stats. Diabetes UK. 2019.   URL: https:/​/www.​diabetes.org.uk/​resources-s3/​2019-02/​1362B_Facts%20and%20stats%20Update%20Jan%202019_LOW%20RES_EXTERNAL.​pdf [accessed 2020-07-13]
  3. U.K. Prospective Diabetes Study Group. U.K. prospective diabetes study 16. Overview of 6 years' therapy of type II diabetes: a progressive disease. U.K. Prospective Diabetes Study Group. Diabetes 1995 Nov 1;44(11):1249-1258. [CrossRef] [PMC]
  4. Asche C, LaFleur J, Conner C. A review of diabetes treatment adherence and the association with clinical and economic outcomes. Clin Ther 2011 Jan;33(1):74-109. [CrossRef] [Medline]
  5. UK Prospective Diabetes Study Group. Efficacy of atenolol and captopril in reducing risk of macrovascular and microvascular complications in type 2 diabetes: UKPDS 39. Br Med J 1998 Sep 12;317(7160):713-720. [CrossRef]
  6. Norris SL, Lau J, Smith SJ, Schmid CH, Engelgau MM. Self-management education for adults with type 2 diabetes: a meta-analysis of the effect on glycemic control. Diabetes Care 2002 Jul 1;25(7):1159-1171. [CrossRef] [Medline]
  7. Colhoun HM, Betteridge DJ, Durrington PN, Hitman GA, W Neil HA, Livingstone SJ, et al. Primary prevention of cardiovascular disease with atorvastatin in type 2 diabetes in the Collaborative Atorvastatin Diabetes Study (CARDS): multicentre randomised placebo-controlled trial. Lancet 2004 Aug;364(9435):685-696. [CrossRef]
  8. Salas M, Hughes D, Zuluaga A, Vardeva K, Lebmeier M. Costs of medication nonadherence in patients with diabetes mellitus: a systematic review and critical analysis of the literature. Value Health 2009 Sep;12(6):915-922 [FREE Full text] [CrossRef] [Medline]
  9. Rothe U, Müller G, Schwarz PE, Seifert M, Kunath H, Koch R, et al. Evaluation of a diabetes management system based on practice guidelines, integrated care, and continuous quality management in a Federal State of Germany: a population-based approach to health care research. Diabetes Care 2008 May 10;31(5):863-868. [CrossRef] [Medline]
  10. Pimouguet C, Le Goff M, Thiébaut R, Dartigues JF, Helmer C. Effectiveness of disease-management programs for improving diabetes care: a meta-analysis. Cand Med Assoc J 2011 Feb 8;183(2):E115-E127 [FREE Full text] [CrossRef] [Medline]
  11. Arianne E, Inge DP, Cor S, Marieke S, Hubertus V. Toward tailored disease management for type 2 diabetes. Am J Managed Care 2012 Oct;18(10):619-630.
  12. Inzucchi SE, Bergenstal RM, Buse JB, Diamant M, Ferrannini E, Nauck M, American Diabetes Association (ADA), European Association for the Study of Diabetes (EASD). Management of hyperglycemia in type 2 diabetes: a patient-centered approach: position statement of the American Diabetes Association (ADA) and the European Association for the Study of Diabetes (EASD). Diabetes Care 2012 Jun 19;35(6):1364-1379 [FREE Full text] [CrossRef] [Medline]
  13. Relman AS. The Institute of Medicine Report on The Quality of Health Care: Crossing the Quality Chasm: A New Health System for the 21st Century. N Engl J Med 2001 Aug 30;345(9):702-703. [CrossRef]
  14. The American Geriatrics Society Expert Panel on Person-Centered Care. Person-centered care: a definition and essential elements. J Am Geriatr Soc 2016 Jan 2;64(1):15-18. [CrossRef] [Medline]
  15. Coulourides Kogan A, Wilber K, Mosqueda L. Moving toward implementation of person-centered care for older adults in community-based medical and social service settings: 'you only get things done when working in concert with clients'. J Am Geriatr Soc 2016 Jan 02;64(1):e8-14. [CrossRef] [Medline]
  16. Robby N, Nancy W, Tamara N, Nicholas H, Rebecca J, Arun K, et al. Interventions for enhancing medication adherence. Cochrane Database of Systematic Reviews 2014 Nov 20;2014(11). [CrossRef]
  17. Ricci-Cabello I, Bobrow K, Islam SM, Chow CK, Maddison R, Whittaker R, et al. Examining development processes for text messaging interventions to prevent cardiovascular disease: systematic literature review. JMIR Mhealth Uhealth 2019 Mar 29;7(3):e12191 [FREE Full text] [CrossRef] [Medline]
  18. Farmer AJ, McSharry J, Rowbotham S, McGowan L, Ricci-Cabello I, French DP. Effects of interventions promoting monitoring of medication use and brief messaging on medication adherence for people with Type 2 diabetes: a systematic review of randomized trials. Diabet Med 2016 May 17;33(5):565-579. [CrossRef] [Medline]
  19. Beratarrechea A, Lee AG, Willner JM, Jahangir E, Ciapponi A, Rubinstein A. The impact of mobile health interventions on chronic disease outcomes in developing countries: a systematic review. Telemed J E Health 2014 Jan;20(1):75-82 [FREE Full text] [CrossRef] [Medline]
  20. Chow CK, Redfern J, Hillis GS, Thakkar J, Santo K, Hackett ML, et al. Effect of lifestyle-focused text messaging on risk factor modification in patients with coronary heart disease: a randomized clinical trial. J Am Med Assoc 2015 Sep 22;314(12):1255-1263. [CrossRef] [Medline]
  21. Bobrow K, Farmer AJ, Springer D, Shanyinde M, Yu L, Brennan T, et al. Mobile phone text messages to support treatment adherence in adults with high blood pressure (star): a single-blind, randomized trial. Circulation 2016 Jan 14:CIRCULATIONAHA.115.017530. [CrossRef]
  22. Chow CK, Redfern J, Hillis GS, Thakkar J, Santo K, Hackett ML, et al. Effect of lifestyle-focused text messaging on risk factor modification in patients with coronary heart disease: a randomized clinical trial. J Am Med Assoc 2015 Sep 22;314(12):1255-1263. [CrossRef] [Medline]
  23. Kassavou A, Sutton S. Automated telecommunication interventions to promote adherence to cardio-metabolic medications: meta-analysis of effectiveness and meta-regression of behaviour change techniques. Health Psychol Rev 2018 Mar 12;12(1):25-42. [CrossRef] [Medline]
  24. Farmer A, Allen J, Bartlett K, Bower P, Chi Y, French D, SuMMiT-D Collaborative Group. Supporting people with type 2 diabetes in effective use of their medicine through mobile health technology integrated with clinical care (SuMMiT-D Feasibility): a randomised feasibility trial protocol. BMJ Open 2019 Dec 29;9(12):e033504 [FREE Full text] [CrossRef] [Medline]
  25. Esendex.   URL: https://www.esendex.co.uk [accessed 2020-07-13]
  26. Chan A, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. Br Med J 2013 Jan 8;346(jan08 15):e7586 [FREE Full text] [CrossRef] [Medline]
  27. CakePHP.   URL: https://cakephp.org [accessed 2020-07-13]
  28. MySQL.   URL: https://www.mysql.com [accessed 2020-07-13]
  29. World Wide Web.   URL: https://www.w3.org [accessed 2020-07-13]
  30. HTML.   URL: https://www.w3.org/html [accessed 2020-07-13]
  31. CSS.   URL: https://www.w3.org/Style/CSS [accessed 2020-07-13]
  32. Javascript.   URL: https://www.w3.org/standards/webdesign/script [accessed 2020-07-13]
  33. PHP.   URL: https://www.php.net [accessed 2020-07-13]
  34. Health and Social Care Network (HSCN).   URL: https://digital.nhs.uk/services/health-and-social-care-network [accessed 2020-07-13]
  35. The Transport Layer Security (TLS) Protocol.   URL: https://tools.ietf.org/html/rfc524636 [accessed 2020-07-13]
  36. Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med 2013 Aug 20;46(1):81-95. [CrossRef] [Medline]
  37. Lauren E C, Rachel N C, Marijn B, Alexander J R, Marie J, Michael P K, et al. Links between behavior change techniques and mechanisms of action: an expert consensus study. Annals of Behavioral Medicine 2019 Jul 17;53(8):708-720. [CrossRef]
  38. Long H, Bartlett YK, Farmer AJ, French DP. Identifying brief message content for interventions delivered via mobile devices to improve medication adherence in people with type 2 diabetes mellitus: a rapid systematic review. J Med Internet Res 2019 Jan 9;21(1):e10421. [CrossRef]
  39. Yvonne Kiera B, Andrew F, Rustam R, David P G. Use of brief messages based on behavior change techniques to encourage medication adherence in people with type 2 diabetes: developmental studies. J Med Internet Res 2020 May 13;22(5). [CrossRef]
  40. Sentry (Secure Entry).   URL: https://sentry.phc.ox.ac.uk/sentry [accessed 2020-07-13]
  41. REDCap.   URL: https://www.project-redcap.org [accessed 2020-07-13]
  42. Secure Your Site with HTTPS.   URL: https://support.google.com/webmasters/answer/6073543?hl=en [accessed 2020-07-13]


API: application program interface
BCT: behavior change technique
EHR: electronic health record
GP: general practitioner
NHS: National Health Service
NIHR: National Institute for Health Research
Q1, Q3: first quartile and third quartile
SuMMiT-D: Support through Mobile Messaging and Digital Health Technology for Diabetes
TLS: transport layer security


Edited by G Eysenbach; submitted 27.02.20; peer-reviewed by D Wake, J Buekers; comments to author 29.06.20; revised version received 20.08.20; accepted 17.12.20; published 26.03.21

Copyright

©Yuan Chi, Carmelo Velardo, Julie Allen, Stephanie Robinson, Evgenia Riga, David Judge, Lionel Tarassenko, Andrew J Farmer. Originally published in JMIR Formative Research (http://formative.jmir.org), 26.03.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license information must be included.