Background: The Veterans Health Administration Pain Coach mobile health app was developed to support veterans with chronic pain.
Objective: Our objective was to evaluate early user experiences with the Pain Coach app and preliminary impacts of app use on pain-related outcomes.
Methods: Following a sequential, explanatory, mixed methods design, we mailed surveys to veterans at 2 time points with an outreach program in between and conducted semistructured interviews with a subsample of survey respondents. We analyzed survey data using descriptive statistics among veterans who completed both surveys and examined differences in key outcomes using paired samples t tests. We analyzed semistructured interview data using thematic analysis.
Results: Of 1507 veterans invited and eligible to complete the baseline survey, we received responses from 393 (26.1%). These veterans received our outreach program; 236 (236/393, 60.1%) completed follow-up surveys. We conducted interviews with 10 app users and 10 nonusers. Among survey respondents, 10.2% (24/236) used Pain Coach, and 58% (14/24) reported it was easy to use, though interviews identified various app usability issues. Veterans who used Pain Coach reported greater pain self-efficacy (mean 23.1 vs mean 16.6; P=.01) and lower pain interference (mean 34.6 vs mean 31.8; P=.03) after (vs before) use. The most frequent reason veterans reported for not using the app was that their health care team had not discussed it with them (96/212, 45.3%).
Conclusions: Our findings suggest that future efforts to increase adoption of Pain Coach and other mobile apps among veterans should include health care team endorsement. Our findings regarding the impact of Pain Coach use on outcomes warrant further study.
Chronic pain is a leading cause of disability  and poses a significant, costly problem [ ] among US adults, including veterans of the US military. In fact, veterans suffer from pain at disproportionately high rates compared with the general population [ ]. Reports suggest that nearly two-thirds of veterans experience pain [ ], with even greater pain prevalence observed among certain veteran cohorts [ - ]. This pain experience leads to negative consequences including functional impairment [ ], increased stress and mental health concerns [ , ], substance abuse [ , ], risky opioid use [ ], impaired sleep [ ], decreased quality of life [ ], and increased health care utilization [ ]. Accordingly, safe and effective strategies are needed to help veterans manage pain.
In recent years, national guidelines for the management of chronic pain have recommended use of nonpharmacological pain management strategies over medications (ie, opioid analgesics) , including US Veterans Health Administration (VA) and Department of Defense Clinical Practice Guidelines that suggest frontline treatment options such as behavioral therapy, activity-based therapy, and some nonopioid medications [ ]. In line with these guidelines, the VA health care system is committed to identifying non-medication–based pain treatment options and self-management support for veterans who experience chronic pain.
Mobile health apps are a platform for providing accessible self-management tools to patients, and a number of apps are being developed to help individuals manage pain [- ], including by the VA’s Office of Connected Care (OCC). Recently, the OCC developed the VA Pain Coach mobile health app as a tool to support veterans in managing chronic pain and related care. Through the app, veterans can reference educational resources about pain, including pain management techniques and tools besides medications; track and monitor their pain through a daily pain diary and monthly check-in; and review tables and graphs of changes in their pain level over time [ ]. The data that veterans enter into the app are viewable by VA care team members through VA care team–facing apps. A screenshot of the VA Pain Coach app interface, which is a web-based (rather than native) app, is included in .
Mobile health apps like VA Pain Coach have great potential for helping veterans communicate with their care team members and self-manage their health. Recent literature suggests that veterans are interested in using mobile health tools [, ]; however, the adoption of such tools has been limited to date [ , ]. The objective of this evaluation was to evaluate early user experiences with the Pain Coach app and preliminary impacts of app use on pain-related outcomes.
Beginning in summer 2019, we completed a sequential, explanatory, mixed methods, multisite evaluation of the Pain Coach app, wherein we first collected and analyzed quantitative data and subsequently, qualitative data. Specifically, we administered mailed surveys at 2 time points, with an outreach program in between, and then conducted semistructured interviews with a purposive sample of veterans who responded to the surveys.
Recruitment and Data Collection
We recruited veterans with a chronic pain diagnosis, including veterans who either used or did not use opioid therapy, from 3 geographically dispersed VA Medical Centers located in the Western and Southeastern regions of the United States.
We mailed the baseline surveys, along with a cover letter and a postage‑paid reply envelope, to veterans beginning in August 2019. We placed follow-up phone calls to veterans who did not return a survey in the mail to maximize their opportunity to participate. A member of our team mailed veterans who completed the survey a US $10 gift card to thank them for their time.
Veterans who completed the baseline survey subsequently received the components of an outreach program about the Pain Coach app. The components included (1) a frequently asked questions (FAQ) sheet about the Pain Coach app that veterans received in the mail, (2) a phone call to veterans from the VA National Telehealth Technology Helpdesk (NTTHD) to offer additional support and troubleshooting with the app as needed, and (3) an informational email sent to the VA care team members of these veterans explaining the Pain Coach app.
Beginning in February 2020, we mailed follow-up surveys to veterans who completed a baseline survey and received the outreach program components. Following the methods described in the previous section, we mailed surveys along with a cover letter and postage-paid reply envelope. We again placed follow-up phone calls to veterans who did not return a survey in the mail and compensated respondents with another US $10 gift card.
In accordance with the principles of purposive sampling, we identified a set of criteria (age, gender, geographic location, Pain Coach app use, and use of opioid therapy) with which to target our interview recruitment. We conducted interviews by telephone between April 2020 and June 2020. The interviews lasted 20 minutes to 30 minutes on average and were audio-recorded and transcribed verbatim to facilitate analysis.
Data Collection Instruments and Measures
We collected information on veteran sociodemographics including age, gender, race, ethnicity, relationship status, highest level of education completed, living arrangement, and self-reported health status (1=excellent to 5=poor). We measured veteran socioeconomic status by asking veterans to report how hard it is for them (and their family) to pay for basic necessities like food and heating/cooling (1=very hard to 4=not very hard). We also asked veterans to describe the technology that they own (desktop or laptop computer, tablet, cell phone).
Experience with and management of pain variables included prior-week pain intensity measured using a 10-point validated rating scale, ranging from 0 (no pain at all) to 10 (pain as bad as you can imagine) . We measured pain self-efficacy using the Pain Self-Efficacy Questionnaire [ ], an established 10-item scale that measures the extent to which individuals are confident that they can perform a range of activities despite their pain. We assessed pain interference with the Patient-Reported Outcomes Measurement Information System 8-item pain interference short form [ ], which measures the extent to which individuals felt their pain interfered with their life and activities in the prior week. We also asked veterans to report how much pain had interfered with their sleep in the prior week (1=not at all to 5=very much). Finally, we gathered information on pain outcomes using the Pain Outcomes Questionnaire - Short Form [ ], a 19‑item validated scale that assesses pain-related outcomes.
The 1-page FAQ sheet about the Pain Coach app that we mailed to veterans included a summary of the purpose of the app, details about its features, and where veterans could find more information about the app. Veterans also received proactive technical support in the form of telephone calls from the VA’s NTTHD in case they were having difficulty with the Pain Coach app. Callers from the NTTHD followed a script developed by our evaluation team to walk veterans through any challenges they were encountering using the app and troubleshooting to overcome those challenges. The final component of our outreach program was an informational email that briefly summarized the purpose and features of the Pain Coach app and the evaluation we were conducting, which was sent to the VA care team members of the veterans who returned our baseline survey.
We asked veterans to report whether they recalled receiving our outreach program components. In addition, we asked veterans to report whether they had used the Pain Coach app. Among veterans who reported having used the app, we assessed self-reported patterns of use.
We also asked self-reported app users about their experiences with and perspectives on the Pain Coach app. Questions asked veterans to report their level of agreement (1=strongly agree to 5=strongly disagree) on factors related to app usability and usefulness and whether the app helped them communicate with their VA care team about their pain. Additionally, we asked veterans to report their level of agreement (1=strongly agree to 5=strongly disagree) on items assessing perceived impact of the app on outcomes (eg, VA Pain Coach helped me to be more engaged in my pain management). Further, we asked veterans to rate their satisfaction with the app (1=very satisfied to 5=not at all satisfied) and their likelihood of recommending the app to other veterans (1=definitely will to 5=definitely will not recommend). Among veterans who reported not having used the app, we asked them to indicate their reasons for nonuse.
We also repeated all questions from the baseline survey regarding veterans’ experiences with and management of pain (please see the Baseline Survey section).
Our semistructured interviews with app users and nonusers followed guides and, reflective of data integration through “building” in mixed methods, included questions intended to elaborate on responses to specific survey items . We asked veterans who reported having used the Pain Coach app about their perceptions of the usability and usefulness of the app, impact of using the app on outcomes and communication with their VA care team, and general perceptions of the app and suggestions for improvement. In our interviews with veterans who reported not having used the app, we asked them to elaborate on their reasons for not having used it.
We analyzed survey data using descriptive statistics (means, percentages) among veterans who completed both a first and second survey. We assessed differences in key demographics (eg, age, relationship status, gender) among veterans who reported having used the Pain Coach app versus those who did not use the app using independent samples t tests and chi‑square tests. We also examined differences in key outcomes (ie, pain self-efficacy, pain interference, pain intensity, sleep, pain outcomes) on the pre- and post-surveys for Pain Coach app users and nonusers with paired samples t tests. Statistical analyses were performed with STATA MP Version 14.2 software (StataCorp, College Station, TX).
We analyzed interview transcripts using thematic analysis [, ]. We first developed a code list based on topics addressed in the interview guide and added emergent codes to the list during the coding process. We also identified exemplary quotes representative of key themes. Two team members analyzed each transcript, first reviewing the transcript independently and then meeting to discuss codes. Any identified discrepancies were resolved during these meetings until complete consensus was reached for all codes. Responses to open-ended survey questions were analyzed using similar procedures.
This work was reviewed by the Institutional Review Boards at the Edward Hines Jr. VA Hospital in Hines, IL and the VA Bedford Healthcare System in Bedford, MA and designated as program evaluation for quality improvement purposes, exempting it from further oversight (VA Handbook 1058.05).
We invited 1668 veterans to participate in the baseline survey; we adjusted this denominator to 1507 to reflect 9 surveys that were returned as undeliverable, 3 veterans who were deceased, and 149 who declined to participate. We received responses from 393 veterans (393/1507, 26.1% response rate). These 393 veterans received our outreach program and were invited to participate in the follow-up survey; we received completed follow-up surveys from 236 veterans (236/393, 60.1% response rate). In the follow-up survey, we asked veterans if they would be willing to complete a semistructured interview to further share their perspectives on the app; 144 (15 app users and 129 nonusers) of the 236 survey respondents expressed willingness to participate in an interview, and we completed interviews with 10 app users and 10 nonusers. In the following sections, we integrate our quantitative and qualitative data, “weaving” and reporting it together according to topics .
Sample Description: Survey Respondents
Our sample was predominantly male (195/234, 83.3%), White (154/236, 65.3%), of non-Hispanic ethnicity (213/217, 98.2%), married or in a civil union (157/233, 67.4%), and about 64 years old, on average (). The majority had completed at least some college or vocational school or were college graduates (192/235, 81.7%) and reported being in fair (100/235, 42.6%) or poor (46/235, 19.6%) health.
|Agea (years), mean (SD)||63.5 (11.2)|
|Agea (years), range||28.0-87.0|
|Gendera: male, n (%)||195 (83.3)|
|Gendera: female, n (%)||39 (16.7)|
|Race: White, n (%)||154 (65.3)|
|Race: Black or African American, n (%)||69 (29.2)|
|Race: Asian, n (%)||2 (0.9)|
|Race: Native Hawaiian or other Pacific Islander, n (%)||0 (0)|
|Race: American Indian or Alaskan Native, n (%)||9 (3.8)|
|Race: Other, n (%)||4 (1.7)|
|Ethnicityb: Hispanic or Latino, n (%)||4 (1.8)|
|Ethnicityb: not Hispanic or Latino, n (%)||213 (98.2)|
|Relationship statusc: Married or in a civil union, n (%)||157 (67.4)|
|Relationship statusc: Not married or in a civil uniond, n (%)||76 (32.6)|
|Education statuse: high school graduate or less, n (%)||43 (18.3)|
|Education statuse: at least some college or vocational school (1-4 years), n (%)||175 (74.5)|
|Education statuse: Master’s/professional/doctoral degree, n (%)||17 (7.2)|
|Living arrangement: my own apartment or house, n (%)||211 (89.4)|
|Living arrangement: friend’s or relative’s apartment or house, n (%)||19 (8.1)|
|Living arrangement: otherf, n (%)||8 (3.4)|
|Financial difficultyc,g: not very hard, n (%)||110 (47.2)|
|Financial difficultyc,g: somewhat hard/hard/very hard/do not know, n (%)||123 (52.8)|
|Self-reported health status (general)e: excellent, n (%)||1 (0.4)|
|Self-reported health status (general)e: very good, n (%)||17 (7.2)|
|Self-reported health status (general)e: good, n (%)||71 (30.2)|
|Self-reported health status (general)e: fair, n (%)||100 (42.6)|
|Self-reported health status (general)e: poor, n (%)||46 (19.6)|
|Technology ownership variables|
|Does own a desktop or laptop computere, n (%)||184 (78.3)|
|Does not own a desktop or laptop computere, n (%)||51 (21.7)|
|Does own a table computer (eg, iPad, Kindle Fire)h, n (%)||111 (48.1)|
|Does not own a table computer (eg, iPad, Kindle Fire)h, n (%)||120 (51.9)|
|Cell phone ownershipi,j: smartphonek, n (%)||181 (79.0)|
|Cell phone ownershipi,j: non-smartphone cell phone, n (%)||37 (16.2)|
|Cell phone ownershipi,j: none, n (%)||11 (4.8)|
dEngaged or in a relationship, single, separated, divorced, widowed.
fSchool or dormitory, hospital or detox center, nursing home or assisted living, car or street, jail/prison.
gHow hard is it for you (and your family) to pay for the very basics like food and heating/cooling?
jIf you have multiple cell phones, select the one you use most often.
kiPhone, Android, Blackberry, Windows Phone, Symbian, or some other type of smartphone.
The majority of respondents reported owning a smartphone (181/229, 79.0%) or non-smartphone cell phone (37/229, 16.2%) and desktop or laptop computer (184/235, 78.3%); just under one-half (111/231, 48.1%) reported owning a tablet computer.
Sample Description: Interview Participants
Veterans who participated in our semistructured interviews were predominantly male (15/20, 75%) and 56.0 years old, on average. Just under one-half (9/20, 45%) used opioid therapy to manage their pain, and one-half (10/20, 50%) reported having used the Pain Coach app.
Among respondents, 26.3% (62/236) recalled having received informational materials in the mail about the Pain Coach app, while 51.3% (121/236) did not recall having received these materials (53/236, 22.5% were unsure). About 7.6% (18/236) recalled having spoken with someone by phone from the NTTHD about the Pain Coach app, while 88.1% (208/236) reported that they did not recall speaking with anyone from the help desk (10/236, 4.2% were unsure). Finally, 11.4% (27/236) reported recalling having had communication about the Pain Coach app with their VA care team members in the past few months, while the majority did not recall such communication (193/236, 81.8%) or were unsure (16/236, 6.8%).
Veteran-Reported Use of the Pain Coach App
Among our respondents, 10.2% (24/236) reported having used the Pain Coach app while most (212/236, 89.8%) indicated not having used the app (). We compared sociodemographic characteristics among veterans who reported having used the app and those who reported not using it. We did not find significant differences in app adoption due to gender or relationship status among veterans. However, veterans who reported using the Pain Coach app were younger, on average, than those who reported not having used the app (57.6 years vs 64.2 years; P=.01).
Our follow-up semistructured interviews to the survey provided an opportunity to explore in more depth the motivations underlying app use. Some interviewees reported turning to the Pain Coach app as a means of tracking their pain or as part of seeking pain management strategies that could complement what they were already doing or could be alternatives to strategies that they had used previously. Others noted that their pain was chronic and ongoing and as such, were willing to try alternative approaches offered within the app to help them manage it. As one male veteran (52 years old) commented:
I definitely have a lot of years with a lot of pain in multiple sites of my body, and I wanted to figure out what, if anything, the app could do to help.
Some veterans also saw the app as a potential alternative to pain medications or taking pain medications throughout the day. As one female veteran (56 years old) explained:
I suffer from migraines. I suffer from back pain, lower back pain, and other pain. I just refuse to take medication because the medication was masking the symptoms. So, I decided if I use Pain Coach, I could have something to kind of sort of guide me along the way. It was a little more user friendly than taking a pill every time you ate.
Use of the Pain Coach app was also associated with changes that veterans were experiencing. One veteran described using the app to make sense of a type of pain he started feeling, while noting that the app might be useful for individuals who are new to managing pain in general. A male veteran (38 years old) noted:
I think that [the VA Pain Coach app] may be beneficial for someone that is dealing with...new pain or has not heard those things before...
Lastly, changes in an individual’s pain management strategy (eg, changes in their medications) also prompted use of the app.
|Pain Coach app use variables||Sample proportion, n (%)|
|Self-reported use of the VA Pain Coach app|
|Which of the following best describes your use of the VA Pain Coach app?a,b|
|I used it once but am no longer using it.||9 (40.9)|
|I used it more than once but am no longer using it.||7 (31.8)|
|I am still using it.||6 (27.3)|
|To the best you can recall, for about how long have you been using the VA Pain Coach app? Or, if you are no longer using the VA Pain Coach app, about how long did you use it for?a,c|
|3 months or longer||6 (26.1)|
|Between 1 and 3 months||4 (17.4)|
|Less than 1 month||7 (30.4)|
|1 week or less||6 (26.1)|
|Utilization of VA Pain Coach app toolsa|
|Deep breathing||14 (58.3)|
|Muscle relaxation||13 (54.2)|
|Sleep tips||12 (50.0)|
|Manage thoughts||11 (45.8)|
|Activity pacing||8 (33.3)|
|Plan a pleasant activity||7 (29.2)|
aAmong veterans who reported having used the app.
Patterns of Use and Usability of the Pain Coach App
Among veterans who reported having used the Pain Coach app, 41% (9/22) used the app once, 32% (7/22) used the app more than once but discontinued use, and 27% (6/22) reported they were still using the app on their follow-up survey (). One-half or more reported using the app to support deep breathing (14/24, 58%), muscle relaxation (13/24, 54%), and sleep tips (12/24, 50%).
The follow-up semistructured interviews were an opportunity for veterans to provide feedback on features of the app they were using, including the Pain Toolbox, the section of the app that describes different pain management strategies, and the informational resources section of the app. Veterans reported benefits from this content, noting that it helped alleviate the different physical and mental burdens that pain can present, including stress and rumination that can accompany persistent pain. Still other veterans described how the Toolbox and learning resources served as a reminder of pain management techniques that they were previously aware of but had since forgotten:
It reminded me of techniques that I had...used or I had totally forgotten about, so it was good in that aspect.
[male veteran, 52 years old]
The semistructured interviews also revealed aspects of the Toolbox and learning resources that veterans felt need to be improved. Some expressed concerns about the interface, while others noted that they did not think the content was useful or actionable beyond other health information resources available on the internet. For instance, one male veteran (54 years old) wanted to understand the rationale behind the pain management tools that were included in the Toolbox and why they might be effective at alleviating pain. In his opinion, it was a shortcoming merely to include the instructions or the how-to of the pain management tool without accompanying evidence:
I don’t know the purpose of doing them [the pain management techniques] if someone said, well this is why we want you to count to 10, because we want you to focus on “this,” because obviously there is more to it than just counting to 10.
Related to this point, other veterans felt that the presentation of general pain management strategies in the Toolbox was less effective than if they could provide a description of the pain they were experiencing so that the Toolbox could suggest strategies based on their specific type of pain. As described by one female veteran (60 years old):
Then they got this little tool thing, like, oh, you go to the pain Toolbox. Then, it says activity pacing, deep breathing, manage thoughts, muscle relaxation...But the thing is it doesn’t ask what pain, what was the pain. You can’t use all this stuff.
More than one-half of veterans who used the Pain Coach app reported that it was easy to use (14/24, 58.3%). One-half (12/24, 50.0%) indicated that they would have liked to have received more information on how to use the app.
Negative feedback about the app’s usability shared during the semistructured interviews stemmed from its web-based platform (ie, the Pain Coach app is not a native app). Log-in issues emerged as a key usability challenge. The lengthy process to go from VA’s Launchpad (the website through which veterans log into the Pain Coach app) to eventually logging into the app itself was cumbersome for some users. One male veteran (aged 54 years) commented:
There are so many steps to even get to where you can log in...even when the system is working, it’s a lot of steps just to log in to get to where you can log something in...You have to launch an app just to get to the app.
Beyond log-in issues, veterans commented on the input-driven nature of the Pain Coach app, including the effort it takes to self-report information about one’s experience of pain. Veterans wanted it to be faster and more efficient to document their pain levels. A male veteran (52 years of age) explained:
I think as far as documenting and everything...it just could be easier, more user friendly, something quick.
Usefulness, Communication, and Outcomes Associated with Pain Coach App Use
Many veterans found that being able to track their pain levels in the app was useful (11/24, 45.8%). About one-third (8/24, 33.3%) found that being able to set reminders to track their pain in the app was useful and that the app helped them track how their pain was impacting other facets of their life, such as mood and sleep over time. One-quarter (6/24, 25%) of veterans who used the app reported having a better understanding of how to manage their pain because of the Pain Coach app, and 29% (7/24) stated that the app introduced them to new pain management strategies. Just under one-half of veterans (11/24, 46%) found the educational information about how they can manage pain provided in the Pain Coach app to be useful. Approximately one-third (8/24, 33%) stated that they intend to continue using the app in the future, and just under one-half (11/24, 46%) reported being satisfied with the app. More than one-half of veterans reported that they would recommend the app to other veterans (13/24, 54%).
As part of the semistructured interviews, some veterans described feeling that the app and its features fit their needs and noted that Pain Coach can be a resource to facilitate recall of pain management strategies and to help veterans who want to learn more about how to manage their pain. Other veterans, however, felt that the amount of information to manage in the app was too much and at times, was not straightforward. A 38-year-old male veteran explained his experience this way:
I just think it’s a lot of information...too much information. ...you get agitated by trying to find different information and clicking all search feeds.
Other veterans offered specific suggestions to improve the usefulness of the Pain Coach app’s tracking features. Two key themes emerged. First, veterans suggested that it would be helpful to be able to document pain levels multiple times a day as levels fluctuate. One male veteran (54 years old) described using a different health app on his phone, which also did not allow tracking at multiple time points per day and expressed similar concerns about the Pain Coach app:
If I can get into it numerous times a day and track what’s going on, what happened, you know, it would help. But you know, for someone like myself, it’s kind of a pain. Aggravating. Because at different times, I mean all day long, it’s different for me.
Second, other veterans described how being able to track what they were doing while they experienced different levels of pain would be valuable to them. This might include a data entry field where a veteran could note their activities next to their pain ratings. A female veteran aged 60 years described it this way:
First when you...put in the pain [rating]. If they had something right there where you also have to put what activity...what type of activity you [were] doing, like a note.
Veterans also suggested that receiving reminders to use the app and other motivational messages from the app would elevate it beyond being what could feel at times like a data entry tool. As one male veteran (aged 76 years) said:
[If] some kind of little alarm or alert would pop up on the phone, “Hey, take your pain pills.” I mean, not your pain pill, but “Take your, do your app.”
Among Pain Coach app users, few (2/24, 8% strongly agree; 1/24, 4% agree) reported that the app helped them communicate with their VA care team about their pain. Our semistructured interviews with app users corroborated our survey findings about Pain Coach’s influence on veteran communication with their VA care team members. Our analysis revealed 2 main reasons why veterans felt the app did not help with that communication. First, several felt that their VA care team members simply lacked knowledge of the app or did not know that they were using the app. As one male veteran, aged 54 years, questioned:
Do they [VA care team] see the app? I mean, if they see the app, they didn’t tell me because I've seen them since I've logged into the app.
Beyond care team member awareness of the app and its use, other veterans described that the limited impact of the app on communication with their VA care team members was attributable to the quality of communication that already existed. Although some veterans reported good-quality communication with their care team members, others reported difficulties that they did not think the app would address. Veterans recognized that one of the main values of the Pain Coach app was to share information with their VA care team members; however, they questioned the point of using the app if that sharing did not happen. As one male veteran (54 years old) questioned:
Is this something that my primary care physician is going to see and say “Well, we see your pain was up this day.” You know what I mean? Yeah, so—it is nice for me to sit here and log it in, but is it just for me? ‘Cause if it was just for me, I have a pen and a pad and a paper right next to my chair here, so I could just keep track of it myself if I wanted to.
Approximately one-fifth (5/24, 21%) of veterans reported that using the Pain Coach app helped them to be more engaged with their pain management and managing their pain was less frustrating for them because of the app (5/24, 21%); 25% (6/24) found that they were able to manage their pain more effectively because of the app.
We compared pre- and post-survey scores on veterans’ self-reported experience of pain (pain self-efficacy, pain interference, pain intensity, sleep, pain outcomes) for Pain Coach app users and nonusers using paired samples t tests (see). Veterans who used the Pain Coach app reported greater pain self-efficacy after using the app (after, mean 23.1 vs before, mean 16.6; P=.01); conversely, among app nonusers, self-efficacy scores did not change from the pre (mean 22.9) to post (mean 23.0) periods (P=.46). Moreover, while both veterans who used the Pain Coach app (mean 34.6 vs mean 31.8; P=.03) and those who did not (mean 30.5 vs mean 28.7; P=.001) reported lower pain interference on the post survey as compared with the pre survey, reported pain interference scores decreased more among app users (2.8-point decrease) than nonusers (1.8-point decrease). Of note, the literature indicates that a 2- to 3-point change indicates a minimally important difference on this pain interference measure [ ]. In addition, veterans who used the Pain Coach app reported lower (though nonsignificant) pain intensity after using the app (after, mean 7.0 vs before, mean 7.5; P=.09); conversely, among app nonusers, pain intensity did not change from the pre (mean 6.6) to post (mean 6.7) periods (P=.52). Finally, both veterans who used the Pain Coach app (mean 119.5 vs mean 105.2; P=.04) and those who did not use the app (mean 101.3 vs mean 93.3; P<.001) reported improvements on the Pain Outcomes Questionnaire - Short Form, although scores among app users improved more than among nonusers.
|Outcome variables||App users (n=24)||App nonusers (n=212)|
|Baseline survey, mean||Follow-up survey, mean||P value||Baseline survey, mean||Follow-up survey, mean||P value|
|Pain self-efficacy ||16.6||23.1||.01||22.9||23.0||.46|
|Pain interferencea ||34.6||31.8||.03||30.5||28.7||.001|
|Pain intensityb ||7.5||7.0||.09||6.6||6.7||.52|
|Pain outcomes (overall score)a ||119.5||105.2||.04||101.3||93.3||<.001|
Reasons for Nonuse
In addition to understanding the experiences of those veterans who used the app during our evaluation period, we also gathered data from veterans who did not use the app to understand reasons for nonuse. The most frequent reason reported by veterans for not using the Pain Coach app was that their health care team had not talked with them about it (96/212, 45.3%;). This was followed by 22.2% (47/212) of veterans reporting that they did not use the app because they were not aware of it, 16.5% (35/212) did not think it would help them manage their pain, and 16.0% (34/212) reported that they are not comfortable using apps.
|Reasons for not using the app||Sample proportion, n (%)|
|My health care team has not talked with me about using the app||96 (45.3)|
|Not aware of the app||47 (22.2)|
|I do not think it would help me manage my pain on my own||35 (16.5)|
|I am not comfortable using apps||34 (16.0)|
|I do not see why it would benefit me||28 (13.2)|
|I do not have a reliable device on which to use the app||26 (12.3)|
|I have concerns about entering my information in the app||21 (9.9)|
|I had difficulty accessing the app (eg, downloading, forgot login/password)||19 (9.0)|
|Not enough information about the app||14 (6.6)|
|I do not think it would help my health care team to manage my pain.||13 (6.1)|
|I do not have internet access or stable internet connection||13 (6.1)|
|Haven\'t had the time||12 (5.7)|
|I already have too many apps or receive too much information as it is||11 (5.2)|
|I do not want to share information about my pain||9 (4.3)|
|Limited interest in using the app||8 (3.8)|
|It costs too much to use the app||5 (2.4)|
|Needs support to use the app||4 (1.9)|
|Using other pain management strategies||4 (1.9)|
|I am already using another app to help manage my pain||3 (1.4)|
Our semistructured interview data corroborated several of the key findings regarding reasons for nonuse. Limited awareness and having limited information were further described as key reasons for not using the Pain Coach app. In addition, other veterans described having limited interest in and knowledge of how to use the app, citing time pressures, other commitments, and in some cases, a sense of being overwhelmed by other electronic media. As one male veteran (72 years old) commented:
...I deal with too much computer stuff and emails and internet things, and this [is] just one more thing that, you know, I don’t need on my plate and that’s why, you know, I would never really use it.
For this same veteran, like others who reported not using the app, his lack of interest in Pain Coach also tied to his belief that its features and content could not help him manage his pain in lieu of other strategies he already tried:
I’ve been through everything from physical therapy to disc fusion at this point, and so, you know, I’ve done all the different options. If that’s what your coach thing does, it’s kind of like walking through the various options to managing your pain and, you know, tracking it, then, you know, at this point I’ve done everything. [laugh] So, what else is there?
Finally, several veterans who did not use the Pain Coach app explained that having members of their health care team talk with them about the app might have facilitated their willingness to try it.
I: Okay. Great. What else would have been helpful to you in getting started to use VA Pain Coach? P: Probably, my pain management team talking to me more about it.
[male veteran, 50 years old]
If I was able to talk to somebody, I guess, while I was already at the doctor. If, you know, that time is already set aside, somebody explain to me how it works or what the benefits of it were, it might help me be more willing to use it.
[female veteran, 28 years old]
Increasingly, patient-facing technologies such as mobile health apps are being used to offer pain management strategies to patients [, ]; however, information about the adoption and impact of such apps among veterans with chronic pain is limited. In this evaluation, we used mixed quantitative and qualitative methods to examine veteran perceptions of and experiences with Pain Coach, a VA mobile health app designed to facilitate pain self-management in the veteran population. Our participants reflect the diverse experiences of veterans managing chronic pain, including those from different geographic regions and those using different pain management approaches (eg, veterans using and not using opioid therapy). Our findings revealed that a fairly “light touch” outreach program yielded limited adoption of this app among veterans and that future mobile health app implementation efforts might benefit from the integration of health care team member encouragement to maximize patient engagement. Still, despite limited use of Pain Coach among the veterans in our sample, we identified important insights regarding usability issues, as well as encouraging, albeit preliminary, impacts of the app on increasing pain self-efficacy and reducing pain interference.
In order to facilitate app adoption among patients, awareness levels of apps must be increased. Our data revealed that traditional outreach approaches including mailed materials and phone calls appear to have limited impact among veterans in this regard. Although the FAQ sheet that we mailed to veterans about the Pain Coach app seemed to reach the greatest number of veterans in our sample, even this component was seemingly limited in its impact. Despite multiple call attempts, the technical support outreach efforts completed by the VA NTTHD did not seem to reach many veterans. Furthermore, although we provided information to VA health care team members about Pain Coach and its functionality via email, veterans reported little communication with their health care team about the app.
Our findings regarding VA care team member and veteran communication about the VA Pain Coach app are particularly important. The predominant reason veterans reported not having used the app was because their care team members had not talked with them about it. This was further corroborated in our interviews with veterans, during which some participants told us that they may have been more likely to try using the app if their VA care team members had recommended it to them. The implications of these findings are significant and underscore the important role that health care team member endorsement and recommendation play in the implementation, adoption, and use of mobile health apps like Pain Coach . These findings further expand on the evidence that endorsement of mobile health apps by health care providers is key to bolstering patient app adoption, including among veterans [ , ].
It is important to recognize that recommending an app to all patients may not resolve all implementation challenges either. As has been discussed in the literature, various considerations need to be made when recommending mobile apps to patients, including a patient’s technical competence, potential privacy issues, and other potential online harms [- ]. Our data further suggest other characteristics for providers to consider when determining whether a veteran might be a good candidate to use an app like Pain Coach. These include whether the individual is new to the experience and management of pain, whether they experienced some change in their pain or pain management strategies, and whether they might need a reminder of pain management strategies available to them. Similarly, those who have long-standing, chronic pain may be open to additional support to facilitate coping, thus also making them good candidates for app use.
Our evaluation further revealed insights about veteran perceptions of the usability and usefulness of the Pain Coach app. Veterans noted negative perceptions of the web-based platform of the app and in turn, the cumbersome process required to log in (a by-product of the Pain Coach not being a native app). In addition, while some veterans found the informational resources and tracking features in the Pain Coach app to be useful, participants offered suggestions for how these features could be improved, which may be integrated into future versions of Pain Coach or other pain-focused mobile health apps. These suggestions included noting evidence to support the effectiveness of the pain management strategies detailed in the app, facilitating faster and more intuitive pain tracking, allowing users to track their pain throughout the day as opposed to once per day to account for fluctuations in pain, and including a notes field where users could indicate what they were doing when they experienced that level of pain. Veterans also indicated that they would like for the app to send them reminders and motivational messages.
Previous research focused on mobile health apps for pain management also suggests that users value being able to track their pain in the app and want to share this information with their care providers . Similarly, the veterans in our evaluation recognized that one of the main values of the Pain Coach app was to share information with their VA care team members; however, they questioned the point of using the app if that sharing did not happen, and many did not feel the app facilitated communication. These perceptions could potentially be improved if veterans knew that their care team members (1) are aware of the app, (2) know that the veteran is using the app, and (3) know that the veteran is entering data into the app that are available to inform care. These findings are aligned with previous literature noting that patients want to share and discuss the data they enter into mobile health apps with their providers [ , ] and further highlight the importance of provider engagement in patient app adoption and use and subsequent impacts. In order to ensure that mobile health apps such as Pain Coach are able to effectively facilitate patient-care team communication, care team members must be aware that their patients are using such apps.
Finally, and although preliminary, our results suggested that the Pain Coach app may be effective in supporting veteran pain self-management, with users of the app in our sample reporting improved outcomes (ie, increased pain self-efficacy, reduced pain interference) after using the app. These findings are aligned with previous research suggesting that mobile health apps can effectively help individuals manage chronic pain  and the use of mobile health apps for pain can facilitate improved pain outcomes [ , ]. This is encouraging and further supports efforts to improve implementation of mobile health apps for pain management.
Our work was limited by lack of randomization and reliance on self-report data, including data on app usage, because reliable app activity log data were not available at the time this evaluation was conducted. In addition, the small number of veterans who reported using the app limits the comparisons that can be made with nonusers and generalizability beyond our sample and underscores the importance of interpreting the results, particularly those regarding impact of the app on veteran outcomes, with caution. In addition, VA Pain Coach is not a native app but rather a mobile web app accessed through an online browser; veterans may have experienced less usability concerns if they had been able to download VA Pain Coach to their personal device(s) rather than having to access a web portal in order to log in. Future work should leverage larger samples of Pain Coach app users, as well as examine the experiences of veterans who tried the app once but did not return, to understand why and how to better support sustained use. Beyond the kind of pre-post design we utilized here, more rigorous evaluations of the effectiveness of the Pain Coach app would require prospective designs with randomization and less reliance on self-report data.
The results of this evaluation have revealed that traditional outreach approaches (ie, mailed materials, phone calls) had only a marginal influence on awareness of a pain management mobile app among a sample of veterans with chronic pain. Future mobile health app marketing and outreach efforts should be combined with health care team member endorsement, as care team members appear to have a critical role to play in promoting app adoption. Further, addressing the usability issues inherent in apps like VA’s Pain Coach will only serve to advance implementation. Importantly, even though adoption of the Pain Coach app among veterans in this evaluation was quite limited, our results indicate that the app may be useful for self-management of pain (eg, increasing self-efficacy and reducing pain interference), signaling that more rigorous randomized evaluations may be warranted.
This work was supported by the US Department of Veterans Affairs, OCC, and Office of Research and Development, Health Services Research and Development Service, Quality Enhancement Research Initiative Program (PEC 15-470; PI: TPH). The authors also wish to thank Mr. Lincoln Clarke, MSA, for his invaluable contributions to data processing and review of data collection instruments for readability and appropriateness.
The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States Government.
All evaluation procedures described in this manuscript were completed in accordance with the ethical standards of the responsible committee on human experimentation (institutional or regional) and with the Helsinki Declaration.
Conflicts of Interest
- Kroenke K, Cheville A. Management of chronic pain in the aftermath of the opioid backlash. JAMA 2017 Jun 20;317(23):2365-2366. [CrossRef] [Medline]
- Phillips CJ. Economic burden of chronic pain. Expert Rev Pharmacoecon Outcomes Res 2006 Oct;6(5):591-601. [CrossRef] [Medline]
- Nahin RL. Severe pain in veterans: the effect of age and sex, and comparisons with the general population. J Pain 2017 Mar;18(3):247-254 [FREE Full text] [CrossRef] [Medline]
- Higgins DM, Buta E, Dorflinger L, Masheb RM, Ruser CB, Goulet JL, et al. Prevalence and correlates of painful conditions and multimorbidity in national sample of overweight/obese Veterans. J Rehabil Res Dev 2016;53(1):71-82 [FREE Full text] [CrossRef] [Medline]
- Phillips KM, Clark ME, Gironda RJ, McGarity S, Kerns RW, Elnitsky CA, et al. Pain and psychiatric comorbidities among two groups of Iraq and Afghanistan era veterans. J Rehabil Res Dev 2016;53(4):413-432 [FREE Full text] [CrossRef] [Medline]
- Travaglini LE, Cosgrave J, Klingaman EA. Pain and sleep problems predict quality of life for veterans with serious mental illness. Psychiatr Rehabil J 2019 Sep;42(3):229-237. [CrossRef] [Medline]
- Naylor JC, Wagner HR, Johnston C, Elbogen EE, Brancu M, Marx CE, VA Mid-Atlantic MIRECC Work Group, VA Mid-Atlantic MIRECC Women Veterans Work Group, et al. Pain intensity and pain interference in male and female Iraq/Afghanistan-era veterans. Womens Health Issues 2019 Jun 25;29 Suppl 1:S24-S31. [CrossRef] [Medline]
- Blakey SM, Wagner HR, Naylor J, Brancu M, Lane I, Sallee M, VA Mid-Atlantic MIRECC Workgroup, et al. Chronic pain, TBI, and PTSD in military veterans: a link to suicidal ideation and violent impulses? J Pain 2018 Jul;19(7):797-806 [FREE Full text] [CrossRef] [Medline]
- Powell MA, Corbo V, Fonda JR, Otis JD, Milberg WP, McGlinchey RE. Sleep quality and reexperiencing symptoms of PTSD are associated with current pain in U.S. OEF/OIF/OND veterans with and without mTBIs. J Trauma Stress 2015 Aug;28(4):322-329. [CrossRef] [Medline]
- Bennett AS, Watford JA, Elliott L, Wolfson-Stofko B, Guarino H. Military veterans' overdose risk behavior: Demographic and biopsychosocial influences. Addict Behav 2019 Dec;99:106036 [FREE Full text] [CrossRef] [Medline]
- Forman-Hoffman VL, Peloso PM, Black DW, Woolson RF, Letuchy EM, Doebbeling BN. Chronic widespread pain in veterans of the first Gulf War: impact of deployment status and associated health effects. J Pain 2007 Dec;8(12):954-961. [CrossRef] [Medline]
- Kay C, Wozniak E, Ching A, Bernstein J. Health care utilization by veterans prescribed chronic opioids. J Pain Res 2018;11:1779-1787 [FREE Full text] [CrossRef] [Medline]
- Dowell D, Haegerich TM, Chou R. CDC guideline for prescribing opioids for chronic pain--United States, 2016. JAMA 2016 Apr 19;315(15):1624-1645 [FREE Full text] [CrossRef] [Medline]
- VA/DoD Clinical Practice Guideline for Opioid Therapy for Chronic Pain. Veterans Health Administration/Department of Defense. 2017. URL: https://www.healthquality.va.gov/guidelines/Pain/cot/VADoDOTCPGProviderSummary022817.pdf [accessed 2021-12-30]
- Jaén I, Suso-Ribera C, Castilla D, Zaragoza I, García-Palacios A, Gómez Palones JL. Improving chronic pain management with eHealth and mHealth: study protocol for a randomised controlled trial. BMJ Open 2019 Dec 29;9(12):e033586 [FREE Full text] [CrossRef] [Medline]
- Ledel Solem IK, Varsi C, Eide H, Kristjansdottir OB, Børøsund E, Schreurs KMG, et al. A user-centered approach to an evidence-based electronic health pain management intervention for people with chronic pain: design and development of EPIO. J Med Internet Res 2020 Jan 21;22(1):e15889 [FREE Full text] [CrossRef] [Medline]
- Jamison RN, Jurcik DC, Edwards RR, Huang C, Ross EL. A pilot comparison of a smartphone app with or without 2-way messaging among chronic pain patients: who benefits from a pain app? Clin J Pain 2017 Aug;33(8):676-686 [FREE Full text] [CrossRef] [Medline]
- Lo WLA, Lei D, Li L, Huang DF, Tong K. The perceived benefits of an artificial intelligence-embedded mobile app implementing evidence-based guidelines for the self-management of chronic neck and back pain: observational study. JMIR Mhealth Uhealth 2018 Nov 26;6(11):e198 [FREE Full text] [CrossRef] [Medline]
- Kravitz RL, Schmid CH, Marois M, Wilsey B, Ward D, Hays RD, et al. Effect of mobile device-supported single-patient multi-crossover trials on treatment of chronic musculoskeletal pain: a randomized clinical trial. JAMA Intern Med 2018 Oct 01;178(10):1368-1377 [FREE Full text] [CrossRef] [Medline]
- Pain Coach for Veterans: User Manual. U.S. Department of Veterans Affairs. 2018. URL: https://mobile.va.gov/sites/default/files/user-manual-pain-coach-veterans.pdf [accessed 2021-12-30]
- Miller CJ, McInnes DK, Stolzmann K, Bauer MS. Interest in use of technology for healthcare among veterans receiving treatment for mental health. Telemed J E Health 2016 Oct;22(10):847-854. [CrossRef] [Medline]
- Lipschitz J, Miller CJ, Hogan TP, Burdick KE, Lippin-Foster R, Simon SR, et al. Adoption of mobile apps for depression and anxiety: cross-sectional survey study on patient interest and barriers to engagement. JMIR Ment Health 2019 Jan 25;6(1):e11334 [FREE Full text] [CrossRef] [Medline]
- Erbes CR, Stinson R, Kuhn E, Polusny M, Urban J, Hoffman J, et al. Access, utilization, and interest in mHealth applications among veterans receiving outpatient care for PTSD. Mil Med 2014 Nov;179(11):1218-1222. [CrossRef] [Medline]
- Krebs EE, Lorenz KA, Bair MJ, Damush TM, Wu J, Sutherland JM, et al. Development and initial validation of the PEG, a three-item scale assessing pain intensity and interference. J Gen Intern Med 2009 Jun;24(6):733-738 [FREE Full text] [CrossRef] [Medline]
- Nicholas MK. The pain self-efficacy questionnaire: Taking pain into account. Eur J Pain 2007 Mar;11(2):153-163. [CrossRef] [Medline]
- Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, PROMIS Cooperative Group. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol 2010 Nov;63(11):1179-1194 [FREE Full text] [CrossRef] [Medline]
- Clark ME, Gironda RJ, Young RW. Development and validation of the Pain Outcomes Questionnaire-VA. J Rehabil Res Dev 2003;40(5):381-395 [FREE Full text] [CrossRef] [Medline]
- Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and practices. Health Serv Res 2013 Dec;48(6 Pt 2):2134-2156 [FREE Full text] [CrossRef] [Medline]
- Strauss A, Corbin J. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, 2nd edition. Thousand Oaks, CA: Sage Publications; 1998.
- Lindlof R. Qualitative Communication Research Methods. Thousand Oaks, CA: Sage Publications; 1995.
- Chen CX, Kroenke K, Stump TE, Kean J, Carpenter JS, Krebs EE, et al. Estimating minimally important differences for the PROMIS pain interference scales: results from 3 randomized clinical trials. Pain 2018 Apr;159(4):775-782 [FREE Full text] [CrossRef] [Medline]
- Johnson SS, Levesque DA, Broderick LE, Bailey DG, Kerns RD. Pain self-management for veterans: development and pilot test of a stage-based mobile-optimized intervention. JMIR Med Inform 2017 Oct 17;5(4):e40 [FREE Full text] [CrossRef] [Medline]
- Machado GC, Pinheiro MB, Lee H, Ahmed OH, Hendrick P, Williams C, et al. Smartphone apps for the self-management of low back pain: A systematic review. Best Pract Res Clin Rheumatol 2016 Dec;30(6):1098-1109. [CrossRef] [Medline]
- Bhattarai P, Newton-John T, Phillips JL. Apps for older people's pain self-management: perspectives of primary care and allied health clinicians. Pain Med 2020 Apr 01;21(4):686-694. [CrossRef] [Medline]
- Jeffrey B, Bagala M, Creighton A, Leavey T, Nicholls S, Wood C, et al. Mobile phone applications and their use in the self-management of type 2 diabetes mellitus: a qualitative study among app users and non-app users. Diabetol Metab Syndr 2019;11:84 [FREE Full text] [CrossRef] [Medline]
- Bauer M, Glenn T, Monteith S, Bauer R, Whybrow PC, Geddes J. Ethical perspectives on recommending digital technology for patients with mental illness. Int J Bipolar Disord 2017 Dec;5(1):6 [FREE Full text] [CrossRef] [Medline]
- Torous J, Roberts LW. The ethical use of mobile health technology in clinical psychiatry. J Nerv Ment Dis 2017 Jan;205(1):4-8. [CrossRef] [Medline]
- Gangadharan SP. The downside of digital inclusion: expectations and experiences of privacy and surveillance among marginal internet users. New Media & Society 2015 Nov 09;19(4):597-615. [CrossRef]
- Minen MT, Jalloh A, Ortega E, Powers SW, Sevick MA, Lipton RB. User design and experience preferences in a novel smartphone application for migraine management: a think aloud study of the RELAXaHEAD application. Pain Med 2019 Feb 01;20(2):369-377 [FREE Full text] [CrossRef] [Medline]
- Jacomet C, Ologeanu-Taddei R, Prouteau J, Lambert C, Linard F, Bastiani P, et al. E-health. Patterns of use and perceived benefits and barriers among people living with HIV and their physicians. Part 2: Health apps and smart devices. Med Mal Infect 2020 Oct;50(7):582-589. [CrossRef] [Medline]
- Lordon RJ, Mikles SP, Kneale L, Evans HL, Munson SA, Backonja U, et al. How patient-generated health data and patient-reported outcomes affect patient-clinician relationships: A systematic review. Health Informatics J 2020 Dec;26(4):2689-2706 [FREE Full text] [CrossRef] [Medline]
- Irvine AB, Russell H, Manocchia M, Mino DE, Cox Glassen T, Morgan R, et al. Mobile-Web app to self-manage low back pain: randomized controlled trial. J Med Internet Res 2015 Jan 02;17(1):e1 [FREE Full text] [CrossRef] [Medline]
- Yang J, Weng L, Chen Z, Cai H, Lin X, Hu Z, et al. Development and testing of a mobile app for pain management among cancer patients discharged from hospital treatment: randomized controlled trial. JMIR Mhealth Uhealth 2019 May 29;7(5):e12542 [FREE Full text] [CrossRef] [Medline]
- Jibb LA, Stevens BJ, Nathan PC, Seto E, Cafazzo JA, Johnston DL, et al. Implementation and preliminary effectiveness of a real-time pain management smartphone app for adolescents with cancer: A multicenter pilot clinical study. Pediatr Blood Cancer 2017 Oct;64(10):1. [CrossRef] [Medline]
|FAQ: frequently asked questions|
|NTTHD: National Telehealth Technology Helpdesk|
|OCC: Office of Connected Care|
|VA: Veterans Health Administration|
Edited by G Eysenbach; submitted 20.09.21; peer-reviewed by B Chaudhry, E Sadeghi-Demneh; comments to author 12.10.21; revised version received 17.10.21; accepted 22.11.21; published 20.01.22Copyright
©Timothy P Hogan, Bella Etingen, Nicholas McMahon, Felicia R Bixler, Linda Am, Rachel E Wacks, Stephanie L Shimada, Erin D Reilly, Kathleen L Frisbee, Bridget M Smith. Originally published in JMIR Formative Research (https://formative.jmir.org), 20.01.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.