This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
Feedback for participants’ self-monitoring is a crucial and costly component of technology-based weight loss interventions. Detailed examination of interventionist time when reviewing and providing feedback for online self-monitoring data is lacking.
The aim of this study was to longitudinally examine the time counselors spent providing feedback on participant self-monitoring data (ie, diet, physical activity, weight) in a 12-month technology-based weight loss intervention. We hypothesized that counselors would compose feedback for participants more quickly over time.
The time the lay counselors (N=10) spent reviewing self-monitoring records and providing feedback to participants via email was longitudinally examined for all counselors across the three years of study implementation. Descriptives were observed for counselor feedback duration across counselors by 12 annual quarters (ie, 3-month periods). Differences in overall duration times by each consecutive annual quarter were analyzed using Wilcoxon-Mann-Whitney tests.
There was a decrease in counselor feedback duration from the first to second quarter (mean 53 to 46 minutes;
Counselors used increasingly less time to review online self-monitoring data and compose feedback after the initial 9 months of study implementation. Results inform counselor costs for future technology-based behavioral weight loss interventions. For example, regardless of increasing counselor efficiency, 25-30 minutes per feedback message is a high cost for interventions. One possibility for reducing costs would be generating computer-automated feedback.
ClinicalTrials.gov NCT02063178; https://clinicaltrials.gov/ct2/show/NCT02063178
Consistent weight and dietary self-monitoring are key elements for successful weight loss in both in-person [
In studies that have evaluated the cost-effectiveness of behavioral weight loss interventions [
One study examined costs of providing email feedback for online self-monitoring data, based on retrospective self-reported estimates of counselor time in a “typical week” and “after substantial implementation experience,” but bundled all intervention costs together (ie, session preparation, conducting the group session, review of self-monitoring journals, periodic contact of participants who might miss sessions or who have questions, posting on the bulletin board, record keeping, technical work by the webmaster) [
Thus, the purpose of this study was to longitudinally examine the time required to review self-monitoring data and compose feedback among newly trained counselors for participants engaged in a 12-month behavioral weight loss intervention. These data will be important since a detailed examination of interventionist time when reviewing and providing feedback for online self-monitoring data can inform the cost-effectiveness of technology-based programs (including the time for new counselors to “peak” in efficiency) and serve as a baseline for comparison if strategies are implemented for increasing efficiency. This study examined time spent on counselor self-monitoring feedback across the three years that the weight loss intervention was implemented. We hypothesize that counselors will deliver feedback on participant diet, physical activity, and weight self-monitoring more quickly over time.
Individuals receiving self-monitoring feedback in the behavioral weight loss intervention were active duty military personnel stationed at Lackland Air Force Base in San Antonio, Texas, enrolled in the Fit Blue weight loss study (2014-2017) [
Participants were asked to self-monitor food intake, physical activity, and weight daily. To record food intake and physical activity, participants used the Lose It! app or website and permitted their counselor to access this information. To monitor weight, participants used the BodyTrace e-scale provided to them at baseline, which uploaded to a secure personalized website. In the counselor-initiated condition, counselors provided feedback on dietary, physical activity, and weight self-monitoring at the same frequency as telephone sessions (ie, weekly for 4 months, then biweekly for 4 months, then monthly for 4 months) via email (28 total). The self-paced condition was provided feedback via email when requested (up to 28), although in practice this feedback was rarely requested in the self-paced condition [
Change in feedback durations across counselors.
Quarters from first review | Feedback duration across counselors (minutes) | ||||
|
Number of feedback emails | Quartile 1 | Median | Mean (SD) | Quartile 3 |
1 | 335 | 30 | 30 | 53 (54) | 60 |
2 | 380 | 20 | 30 | 46 (47) | 45 |
3 | 324 | 20 | 30 | 30 (25) | 30 |
4 | 286 | 15 | 25 | 26 (19) | 30 |
5 | 317 | 20 | 30 | 26 (12) | 30 |
6 | 309 | 17 | 25 | 27 (15) | 30 |
7 | 249 | 20 | 30 | 28 (14) | 30 |
8 | 231 | 25 | 30 | 27 (7) | 30 |
≥9 | 239 | 25 | 30 | 28 (12) | 30 |
Counselors (N=10) held bachelor’s or master’s degrees (ie, social work, counseling/psychology, child and family development, nursing, justice administration); however, they were considered lay interventionists since no prior counseling or research experience was required. Counselors were hired based on their interest in providing behavioral interventions and in research, and they were either retired from the military or familiar with military culture. All counselors were new to providing self-monitoring feedback in a weight loss intervention. Counselors attended a week-long training on the study protocol, behavioral weight management principles, feedback, and motivational interviewing. Counselors were taught to construct emails as “feedback sandwiches,” with reinforcement of behaviors sandwiched around identification of potential areas of behavior change, consistent with guidance from other studies [
Counselors contemporaneously logged the time it took them to review online self-monitoring data and construct each personalized email in the electronic study database.
All feedback duration times were included in analyses regardless of condition. Descriptives (ie, median, mean, SD, first quartile, third quartile, quartile range) were observed for feedback duration across counselors by 12 annual quarters (ie, 3-month periods). For counselors who joined the team later (N=2), the first quarter they provided feedback was compiled with the first quarter of feedback from the original counselors. Differences in overall duration times by each consecutive annual quarter were analyzed using the Wilcoxon-Mann-Whitney test.
Across all counselors, there was a significant decrease in overall duration to review self-monitoring data and compose feedback messages from the first to second quarter (
Feedback duration across intervention quarters. Counselor efficiency increases over time.
Counselors needed increasingly less time to review online self-monitoring data and compose personalized feedback to participants over the first 9 months the behavioral weight loss intervention was implemented, with a nonsignificant trend suggesting increased efficiency for the next 3 months as well. Thus, after 12 months, the mean amount of time spent in reviewing self-monitoring data and composing feedback decreased from 53 to 26 minutes. When examining median times, there was less variation across all quarters (ie, 25-30 minutes); however, a decrease in IQR in the first three quarters was notable. This narrowing time range indicated that counselors composed these feedback messages more consistently near the median (ie, 30 minutes) over time.
Although the standard deviation decreased, it remained high, likely due to variability across individual counselors and participant characteristics. Some participants might have logged similar data to previous weeks, or only logged one day, requiring shorter feedback messages. In response to these self-monitoring situations, counselors might write, “You continued to meet your calorie and fat goals and continue to make regular choices of fruits, vegetables, whole grains, and low- or no-calorie beverages” or “You met your calorie and fat goals on the one day that you were able to log. What were the barriers for logging on the other days?” Additionally, perhaps some counselors were quicker at crafting feedback than others.
Mean time was 32 minutes per feedback message, which is higher than the 10-15 minutes found by Hunter et al [
The decrease in time for feedback messages might be influenced by multiple factors. Perhaps counselors became more efficient at reviewing self-monitoring data and constructing feedback with experience and additional training. Another possibility is that counselors provided less detailed feedback and became “sloppier” over time. However, this is less likely given that the overall intervention results indicated participants experienced significant weight loss at 4-month and 12-month outcomes [
Regardless of increasing counselor efficiency, 25-30 minutes per feedback message is a high cost for interventions, even cost-effective interventions such as this one [
There are several limitations to consider. Counselors self-reported time spent reviewing self-monitoring data and constructing feedback, which may be less accurate compared to objective measurements. However, this level of detail is much greater than previous studies. In addition, findings are based on lay counselors and may differ from other weight loss professionals (eg, dietitians). Finally, future studies might examine a consortium of behavioral weight management studies in order to have a larger sample size of interventionists in analyses that examine these questions.
Current findings suggest that counselors, across the initial 9 months of a behavioral weight loss intervention, become quicker at reviewing participant self-monitoring data and composing individualized feedback. Although there was individual variability, findings indicate that after 9-12 months of experience, counselors composed self-monitoring feedback more consistently in about 30 minutes. Despite indications of increased counselor efficiency, time per feedback message was only reduced to 25-30 minutes. Weight loss programs might consider testing computer-automated feedback with human tailoring to reduce counselor time.
The study was funded by the National Institute of Diabetes and Digestive and Kidney Diseases (RO1 DK097158) of the National Institutes of Health, with the title of “Dissemination of the Look Ahead Weight Management Treatment in the Military,” and RCK and RAK as Principal Investigators
The research represents a Collaborative Research and Development Agreement with the United States Air Force (CRADA #13-168-SG-C13001). The trial is registered on ClinicalTrials.gov (NCT 02063178). The opinions expressed in this document are solely those of the authors and do not represent an endorsement by or the views of the United States Air Force, the Department of Defense, or the United States Government. Finally, we would like to thank the participants and the research team for their dedication to the research.
MCF wrote the first draft of the manuscript and conducted a literature review for the study. RCK was involved in study conceptualization, design, and implementation. MK ran statistical analyses for the study. LAG was involved in study conceptualization, design, and implementation. GWT was involved in study conceptualization, design, and implementation. RAK was involved in study conceptualization, study design, study implementation, and mentorship of student author. All authors reviewed and revised the manuscript draft.
None declared.