Background: Feedback for participants’ self-monitoring is a crucial and costly component of technology-based weight loss interventions. Detailed examination of interventionist time when reviewing and providing feedback for online self-monitoring data is lacking.
Objective: The aim of this study was to longitudinally examine the time counselors spent providing feedback on participant self-monitoring data (ie, diet, physical activity, weight) in a 12-month technology-based weight loss intervention. We hypothesized that counselors would compose feedback for participants more quickly over time.
Methods: The time the lay counselors (N=10) spent reviewing self-monitoring records and providing feedback to participants via email was longitudinally examined for all counselors across the three years of study implementation. Descriptives were observed for counselor feedback duration across counselors by 12 annual quarters (ie, 3-month periods). Differences in overall duration times by each consecutive annual quarter were analyzed using Wilcoxon-Mann-Whitney tests.
Results: There was a decrease in counselor feedback duration from the first to second quarter (mean 53 to 46 minutes; P<.001), and from the second to third (mean 46 to 30 minutes; P<.001). A trend suggested a decrease from the third to fourth quarter (mean 30 to 26 minutes; P=.053), but no changes were found in subsequent quarters. Consistent with the hypothesis, counselors may be increasing their efficiency in providing feedback; across 12 months, counselors spent less time reviewing participant self-monitoring and composing feedback (decreasing from mean 53 to 26 minutes).
Conclusions: Counselors used increasingly less time to review online self-monitoring data and compose feedback after the initial 9 months of study implementation. Results inform counselor costs for future technology-based behavioral weight loss interventions. For example, regardless of increasing counselor efficiency, 25-30 minutes per feedback message is a high cost for interventions. One possibility for reducing costs would be generating computer-automated feedback.
Trial Registration: ClinicalTrials.gov NCT02063178; https://clinicaltrials.gov/ct2/show/NCT02063178
Consistent weight and dietary self-monitoring are key elements for successful weight loss in both in-person  and technology-based programs [ ], and using technology for self-monitoring (eg, apps, smart scales) can increase self-monitoring adherence [ - ]. Personalized self-monitoring feedback on the frequency of weight, dietary, and exercise monitoring; reinforcing comments about weight loss behaviors; and presentation of behavior change possibilities are core elements of behavioral weight loss interventions [ - ]. Feedback on self-monitoring data appears to be a crucial component of these interventions since it is associated with greater self-monitoring engagement as well as greater weight loss in interventions [ - ]. In recent years, technology-based communication (eg, email) has been increasingly used for counselor’s feedback [ ], particularly since participants are now able to self-monitor food intake, weight, and physical activity online using either researcher-developed or commercial websites/apps [ ] rather than using paper and pencil diaries.
In studies that have evaluated the cost-effectiveness of behavioral weight loss interventions [- ], interventionist compensation emerges in the available studies as one of the costliest components [ , , ], including time for conducting the sessions and for providing feedback (ie, review of self-monitoring data, composing feedback). Interventionist costs, however, are often bundled in these analyses [ , , ]; that is, combining time required for the sessions together with time required for providing feedback as well as other intervention tasks. The amount of time for sessions is often quite rigid (eg, 60-90 minutes for group sessions; 20-30 minutes for individual sessions), with standard outlines of material to cover, but little is known about the time associated with providing self-monitoring feedback.
One study examined costs of providing email feedback for online self-monitoring data, based on retrospective self-reported estimates of counselor time in a “typical week” and “after substantial implementation experience,” but bundled all intervention costs together (ie, session preparation, conducting the group session, review of self-monitoring journals, periodic contact of participants who might miss sessions or who have questions, posting on the bulletin board, record keeping, technical work by the webmaster) [, ]. To our knowledge, only one behavioral intervention specifically reported isolated counselor feedback time, using an average weekly estimate for the entire intervention, but it is not clear whether this information was collected contemporaneously or retrospectively [ ]. Thus, previous information is limited to averages, and it is not clear that real-time data collection of each feedback message composed has been examined. Further, because counselor efficiency might increase over time, it will be important to examine the potential impact of implementation experience.
Thus, the purpose of this study was to longitudinally examine the time required to review self-monitoring data and compose feedback among newly trained counselors for participants engaged in a 12-month behavioral weight loss intervention. These data will be important since a detailed examination of interventionist time when reviewing and providing feedback for online self-monitoring data can inform the cost-effectiveness of technology-based programs (including the time for new counselors to “peak” in efficiency) and serve as a baseline for comparison if strategies are implemented for increasing efficiency. This study examined time spent on counselor self-monitoring feedback across the three years that the weight loss intervention was implemented. We hypothesize that counselors will deliver feedback on participant diet, physical activity, and weight self-monitoring more quickly over time.
Individuals receiving self-monitoring feedback in the behavioral weight loss intervention were active duty military personnel stationed at Lackland Air Force Base in San Antonio, Texas, enrolled in the Fit Blue weight loss study (2014-2017) [, ]. Recruitment used posters and bulletins, on-base presentations, advertisements, and word of mouth. Those interested were phone screened by study staff to assess eligibility (ie, >1 year left on base, >18 years of age, BMI >25.0 kg/m2, health care provider clearance, computer/email access). At baseline, 248 participants were randomized to either a counselor-initiated or self-paced 12-month intervention condition. Conditions varied in intensity and self-initiation required but were similar in intervention goals.
Participants were asked to self-monitor food intake, physical activity, and weight daily. To record food intake and physical activity, participants used the Lose It! app or website and permitted their counselor to access this information. To monitor weight, participants used the BodyTrace e-scale provided to them at baseline, which uploaded to a secure personalized website. In the counselor-initiated condition, counselors provided feedback on dietary, physical activity, and weight self-monitoring at the same frequency as telephone sessions (ie, weekly for 4 months, then biweekly for 4 months, then monthly for 4 months) via email (28 total). The self-paced condition was provided feedback via email when requested (up to 28), although in practice this feedback was rarely requested in the self-paced condition . The time estimates presented in the analyses are based on 2670 emails ( ). The protocol was approved by the Institutional Review Board of the Wilford Hall Ambulatory Surgical Center in San Antonio, Texas, and acknowledged by the Institutional Review Board at the University of Tennessee Health Science Center.
|Quarters from first review||Feedback duration across counselors (minutes)|
|Number of feedback emails||Quartile 1||Median||Mean (SD)||Quartile 3|
Counselor Characteristics and Training
Counselors (N=10) held bachelor’s or master’s degrees (ie, social work, counseling/psychology, child and family development, nursing, justice administration); however, they were considered lay interventionists since no prior counseling or research experience was required. Counselors were hired based on their interest in providing behavioral interventions and in research, and they were either retired from the military or familiar with military culture. All counselors were new to providing self-monitoring feedback in a weight loss intervention. Counselors attended a week-long training on the study protocol, behavioral weight management principles, feedback, and motivational interviewing. Counselors were taught to construct emails as “feedback sandwiches,” with reinforcement of behaviors sandwiched around identification of potential areas of behavior change, consistent with guidance from other studies . Counselors submitted practice emails that were discussed among counselors led by the principal investigator (RAK). Two counselors joined the team midway through the study timeline and were similarly trained. All counselors participated in ongoing biweekly 1-hour supervision and 1-hour motivational interviewing training and received roughly quarterly retraining on providing self-monitoring feedback to maintain and improve this skill.
Documentation of Self-monitoring Feedback Duration
Counselors contemporaneously logged the time it took them to review online self-monitoring data and construct each personalized email in the electronic study database.
All feedback duration times were included in analyses regardless of condition. Descriptives (ie, median, mean, SD, first quartile, third quartile, quartile range) were observed for feedback duration across counselors by 12 annual quarters (ie, 3-month periods). For counselors who joined the team later (N=2), the first quarter they provided feedback was compiled with the first quarter of feedback from the original counselors. Differences in overall duration times by each consecutive annual quarter were analyzed using the Wilcoxon-Mann-Whitney test.
Across all counselors, there was a significant decrease in overall duration to review self-monitoring data and compose feedback messages from the first to second quarter (P<.001; IQR 30-60 minutes versus 20-45 minutes; mean 53, SD 56 versus mean 46, SD 47;). There was a significant decrease in overall duration from the second to third quarter (P<.001; IQR 20-45 minutes versus 20-30 minutes; mean 46, SD 47 versus mean 30, SD 25). A nonsignificant trend suggested a decrease in duration from the third to fourth quarter (P=.053; IQR 20-30 minutes versus 15-30 minutes; mean 30, SD 25 versus mean 26, SD 19). There was no significant change in duration between later quarters ( ). Median time ranged from 25 to 30 minutes ( ). Median and mean feedback durations across all quarters are presented in .
Counselors needed increasingly less time to review online self-monitoring data and compose personalized feedback to participants over the first 9 months the behavioral weight loss intervention was implemented, with a nonsignificant trend suggesting increased efficiency for the next 3 months as well. Thus, after 12 months, the mean amount of time spent in reviewing self-monitoring data and composing feedback decreased from 53 to 26 minutes. When examining median times, there was less variation across all quarters (ie, 25-30 minutes); however, a decrease in IQR in the first three quarters was notable. This narrowing time range indicated that counselors composed these feedback messages more consistently near the median (ie, 30 minutes) over time.
Although the standard deviation decreased, it remained high, likely due to variability across individual counselors and participant characteristics. Some participants might have logged similar data to previous weeks, or only logged one day, requiring shorter feedback messages. In response to these self-monitoring situations, counselors might write, “You continued to meet your calorie and fat goals and continue to make regular choices of fruits, vegetables, whole grains, and low- or no-calorie beverages” or “You met your calorie and fat goals on the one day that you were able to log. What were the barriers for logging on the other days?” Additionally, perhaps some counselors were quicker at crafting feedback than others.
Mean time was 32 minutes per feedback message, which is higher than the 10-15 minutes found by Hunter et al , during which counselors similarly provided feedback on food intake, exercise, and weight. However, only time spent providing feedback was reported, which—unlike the current study—did not include time reviewing self-monitoring data [ ]. Further, it is unclear if the counselors in this previous research logged feedback duration contemporaneously or retrospectively [ ].
The decrease in time for feedback messages might be influenced by multiple factors. Perhaps counselors became more efficient at reviewing self-monitoring data and constructing feedback with experience and additional training. Another possibility is that counselors provided less detailed feedback and became “sloppier” over time. However, this is less likely given that the overall intervention results indicated participants experienced significant weight loss at 4-month and 12-month outcomes , and periodic retraining on self-monitoring feedback was conducted to increase the likelihood of maintaining good quality feedback. Further, participants might have needed increasingly less feedback about their behaviors over the 12 months that they participated in the intervention. However, our study compared time periods specific to counselor experience, which included the three years that the 12-month intervention was implemented. Thus, individual participants cycled throughout the three years that our results were analyzed and overlapped with participants at other stages of intervention. Nonetheless, future research should rate the quality of feedback over time alongside the time required to compose it.
Regardless of increasing counselor efficiency, 25-30 minutes per feedback message is a high cost for interventions, even cost-effective interventions such as this one . In order to improve dissemination of behavioral weight management programs to all who are eligible and interested, particularly in settings (eg, primary care) without individuals who may have the time, training, and supervising experience to provide self-monitoring feedback, it may be beneficial to develop strategies for decreasing the amount of time required for crafting each message. Some possibilities for reducing the costs of individualized counselor feedback would be generating computer-automated feedback or facilitating peer-group interaction to help promote self-monitoring behaviors [ ]. Although computer-automated feedback was comparable to counselor feedback in the short term in previous research, automated feedback was less effective for long-term weight loss [ ]. Given limited research on computer-generated feedback [ , , ] and the common use of counselor-generated feedback [ - ], it is clearly still important to understand in detail the time costs associated with counselor feedback. These details can inform future program budgets, especially since counselor compensation is one of the largest costs [ , , ]. However, since computer-tailored feedback might be more regularly incorporated into future interventions, a better understanding of the efficiency of counselors in crafting feedback may be beneficial to compare these different modalities.
There are several limitations to consider. Counselors self-reported time spent reviewing self-monitoring data and constructing feedback, which may be less accurate compared to objective measurements. However, this level of detail is much greater than previous studies. In addition, findings are based on lay counselors and may differ from other weight loss professionals (eg, dietitians). Finally, future studies might examine a consortium of behavioral weight management studies in order to have a larger sample size of interventionists in analyses that examine these questions.
Current findings suggest that counselors, across the initial 9 months of a behavioral weight loss intervention, become quicker at reviewing participant self-monitoring data and composing individualized feedback. Although there was individual variability, findings indicate that after 9-12 months of experience, counselors composed self-monitoring feedback more consistently in about 30 minutes. Despite indications of increased counselor efficiency, time per feedback message was only reduced to 25-30 minutes. Weight loss programs might consider testing computer-automated feedback with human tailoring to reduce counselor time.
The study was funded by the National Institute of Diabetes and Digestive and Kidney Diseases (RO1 DK097158) of the National Institutes of Health, with the title of “Dissemination of the Look Ahead Weight Management Treatment in the Military,” and RCK and RAK as Principal Investigators.
The research represents a Collaborative Research and Development Agreement with the United States Air Force (CRADA #13-168-SG-C13001). The trial is registered on ClinicalTrials.gov (NCT 02063178). The opinions expressed in this document are solely those of the authors and do not represent an endorsement by or the views of the United States Air Force, the Department of Defense, or the United States Government. Finally, we would like to thank the participants and the research team for their dedication to the research.
MCF wrote the first draft of the manuscript and conducted a literature review for the study. RCK was involved in study conceptualization, design, and implementation. MK ran statistical analyses for the study. LAG was involved in study conceptualization, design, and implementation. GWT was involved in study conceptualization, design, and implementation. RAK was involved in study conceptualization, study design, study implementation, and mentorship of student author. All authors reviewed and revised the manuscript draft.
Conflicts of Interest
- Boutelle K, Kirschenbaum D. Further support for consistent self-monitoring as a vital component of successful weight control. Obes Res 1998 May;6(3):219-224 [FREE Full text] [CrossRef] [Medline]
- Burke LE, Conroy MB, Sereika SM, Elci OU, Styn MA, Acharya SD, et al. The effect of electronic self-monitoring on weight loss and dietary intake: a randomized behavioral weight loss trial. Obesity (Silver Spring) 2011 Feb;19(2):338-344 [FREE Full text] [CrossRef] [Medline]
- Allen JK, Stephens J, Dennison Himmelfarb CR, Stewart KJ, Hauck S. Randomized controlled pilot study testing use of smartphone technology for obesity treatment. J Obes 2013;2013:151597 [FREE Full text] [CrossRef] [Medline]
- West JH, Belvedere LM, Andreasen R, Frandsen C, Hall PC, Crookston BT. Controlling Your "App"etite: How Diet and Nutrition-Related Mobile Apps Lead to Behavior Change. JMIR Mhealth Uhealth 2017 Jul 10;5(7):e95 [FREE Full text] [CrossRef] [Medline]
- Laing BY, Mangione CM, Tseng C, Leng M, Vaisberg E, Mahida M, et al. Effectiveness of a smartphone application for weight loss compared with usual care in overweight primary care patients: a randomized, controlled trial. Ann Intern Med 2014 Nov 18;161(10 Suppl):S5-12 [FREE Full text] [CrossRef] [Medline]
- Steinberg DM, Levine EL, Askew S, Foley P, Bennett GG. Daily text messaging for weight control among racial and ethnic minority women: randomized controlled pilot study. J Med Internet Res 2013 Nov 18;15(11):e244 [FREE Full text] [CrossRef] [Medline]
- Diabetes Prevention Program (DPP) Research Group. The Diabetes Prevention Program (DPP): description of lifestyle intervention. Diabetes Care 2002 Dec 01;25(12):2165-2171 [FREE Full text] [CrossRef] [Medline]
- Look AHEAD Research Group, Wadden TA, West DS, Delahanty L, Jakicic J, Rejeski J, et al. The Look AHEAD study: a description of the lifestyle intervention and the evidence supporting it. Obesity (Silver Spring) 2006 May;14(5):737-752 [FREE Full text] [CrossRef] [Medline]
- Khaylis A, Yiaslas T, Bergstrom J, Gore-Felton C. A review of efficacious technology-based weight-loss interventions: five key components. Telemed J E Health 2010 Nov;16(9):931-938 [FREE Full text] [CrossRef] [Medline]
- Krukowski RA, Harvey-Berino J, Ashikaga T, Thomas CS, Micco N. Internet-based weight control: the relationship between web features and weight loss. Telemed J E Health 2008 Oct;14(8):775-782 [FREE Full text] [CrossRef] [Medline]
- Tate DF, Wing RR, Winett RA. Using Internet technology to deliver a behavioral weight loss program. JAMA 2001 Mar 07;285(9):1172-1177. [CrossRef] [Medline]
- Tate DF, Jackvony EH, Wing RR. A randomized trial comparing human e-mail counseling, computer-automated tailored counseling, and no counseling in an Internet weight loss program. Arch Intern Med 2006 Aug 14;166(15):1620-1625. [CrossRef] [Medline]
- Spring B, Duncan JM, Janke EA, Kozak AT, McFadden HG, DeMott A, et al. Integrating technology into standard weight loss treatment: a randomized controlled trial. JAMA Intern Med 2013 Jan 28;173(2):105-111 [FREE Full text] [CrossRef] [Medline]
- Smith KJ, Kuo S, Zgibor JC, McTigue KM, Hess R, Bhargava T, et al. Cost effectiveness of an internet-delivered lifestyle intervention in primary care patients with high cardiovascular risk. Prev Med 2016 Jun;87:103-109. [CrossRef] [Medline]
- Smith KJ, Hsu HE, Roberts MS, Kramer MK, Orchard TJ, Piatt GA, et al. Cost-effectiveness analysis of efforts to reduce risk of type 2 diabetes and cardiovascular disease in southwestern Pennsylvania, 2005-2007. Prev Chronic Dis 2010 Sep;7(5):A109 [FREE Full text] [Medline]
- Hollenbeak CS, Weinstock RS, Cibula D, Delahanty LM, Trief PM. Cost-effectiveness of SHINE: A Telephone Translation of the Diabetes Prevention Program. Health Serv Insights 2016;9:21-28 [FREE Full text] [CrossRef] [Medline]
- Herman WH, Hoerger TJ, Brandle M, Hicks K, Sorensen S, Zhang P, Diabetes Prevention Program Research Group. The cost-effectiveness of lifestyle modification or metformin in preventing type 2 diabetes in adults with impaired glucose tolerance. Ann Intern Med 2005 Mar 01;142(5):323-332 [FREE Full text] [CrossRef] [Medline]
- Diabetes Prevention Program Research Group. Within-trial cost-effectiveness of lifestyle intervention or metformin for the primary prevention of type 2 diabetes. Diabetes Care 2003 Sep 22;26(9):2518-2523 [FREE Full text] [CrossRef] [Medline]
- Rushing J, Wing R, Wadden TA, Knowler WC, Lawlor M, Evans M, Look AHEAD Research Group. Cost of intervention delivery in a lifestyle weight loss trial in type 2 diabetes: results from the Look AHEAD clinical trial. Obes Sci Pract 2017 Mar 24;3(1):15-24 [FREE Full text] [CrossRef] [Medline]
- Krukowski RA, Pope RA, Love S, Lensing S, Felix HC, Prewitt TE, et al. Examination of costs for a lay health educator-delivered translation of the Diabetes Prevention Program in senior centers. Prev Med 2013 Oct;57(4):400-402 [FREE Full text] [CrossRef] [Medline]
- Krukowski RA, Tilford JM, Harvey-Berino J, West DS. Comparing behavioral weight loss modalities: incremental cost-effectiveness of an internet-based versus an in-person condition. Obesity (Silver Spring) 2011 Aug;19(8):1629-1635 [FREE Full text] [CrossRef] [Medline]
- McGloin AF, Eslami S. Digital and social media opportunities for dietary behaviour change. Proc Nutr Soc 2014 Oct 16;74(2):139-148. [CrossRef]
- Okorodudu DE, Bosworth HB, Corsino L. Innovative interventions to promote behavioral change in overweight or obese individuals: A review of the literature. Ann Med 2015 May 10;47(3):179-185 [FREE Full text] [CrossRef] [Medline]
- Manz KC, Waters TM, Clifton HE, Kocak M, Klesges RC, Talcott GW, et al. Cost-Effectiveness of a Weight Loss Intervention: An Adaptation of the Look AHEAD Lifestyle Intervention in the US Military. Obesity (Silver Spring) 2020 Jan 26;28(1):89-96 [FREE Full text] [CrossRef] [Medline]
- Ritzwoller DP, Glasgow RE, Sukhanova AY, Bennett GG, Warner ET, Greaney ML, Be Fit Be Well study investigators. Economic analyses of the Be Fit Be Well program: a weight loss program for community health centers. J Gen Intern Med 2013 Dec 4;28(12):1581-1588 [FREE Full text] [CrossRef] [Medline]
- Hunter CM, Peterson AL, Alvarez LM, Poston WC, Brundige AR, Haddock CK, et al. Weight management using the internet a randomized controlled trial. Am J Prev Med 2008 Feb;34(2):119-126. [CrossRef] [Medline]
- Krukowski RA, Hare ME, Talcott GW, Johnson KC, Richey PA, Kocak M, et al. Dissemination of the Look AHEAD intensive lifestyle intervention in the United States Air Force: study rationale, design and methods. Contemp Clin Trials 2015 Jan;40:232-239 [FREE Full text] [CrossRef] [Medline]
- Krukowski RA, Hare ME, Talcott GW, Gladney LA, Johnson KC, Richey PA, et al. Dissemination of the Look AHEAD Intensive Lifestyle Intervention in the United States Military: A Randomized Controlled Trial. Obesity (Silver Spring) 2018 Oct 11;26(10):1558-1565 [FREE Full text] [CrossRef] [Medline]
- Semler L. Coaching tips for offering feedback on participant self-monitoring records. Diabetes Prevention Support Center. 2016 Feb 16. URL: https://www.diabetesprevention.pitt.edu/coaching-tips-for-offering-feedback-on-participant-self-monitoring-records/ [accessed 2020-12-19]
- Venditti EM, Wylie-Rosett J, Delahanty LM, Mele L, Hoskin MA, Edelstein SL, Diabetes Prevention Program Research Group. Short and long-term lifestyle coaching approaches used to address diverse participant barriers to weight loss and physical activity adherence. Int J Behav Nutr Phys Act 2014 Feb 12;11(1):16 [FREE Full text] [CrossRef] [Medline]
- West D, Krukowski R, Larsen C. Treatment of obesity in community settings. In: Wadden TA, Bray GA, editors. Handbook of Obesity Treatment (Second Edition). New York, NY: The Guilford Press; 2018.
- Appel LJ, Clark JM, Yeh H, Wang N, Coughlin JW, Daumit G, et al. Comparative Effectiveness of Weight-Loss Interventions in Clinical Practice. N Engl J Med 2011 Nov 24;365(21):1959-1968. [CrossRef]
Edited by G Eysenbach; submitted 05.09.20; peer-reviewed by J Hong, HL Tam; comments to author 18.11.20; revised version received 02.02.21; accepted 13.04.21; published 05.05.21Copyright
©Margaret C Fahey, Robert C Klesges, Mehmet Kocak, Leslie A Gladney, Gerald W Talcott, Rebecca A Krukowski. Originally published in JMIR Formative Research (https://formative.jmir.org), 05.05.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.