This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2226) Mobile Health (mhealth) (2777) Research Instruments, Questionnaires, and Tools (545) Innovations and Technology for Healthy Eating Education (400)

Published on in Vol 6, No 10 (2022): October

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/40316, first published .
A Digital Smartphone-Based Self-administered Tool (R+ Dietitian) for Nutritional Risk Screening and Dietary Assessment in Hospitalized Patients With Cancer: Evaluation and Diagnostic Accuracy Study

A Digital Smartphone-Based Self-administered Tool (R+ Dietitian) for Nutritional Risk Screening and Dietary Assessment in Hospitalized Patients With Cancer: Evaluation and Diagnostic Accuracy Study

A Digital Smartphone-Based Self-administered Tool (R+ Dietitian) for Nutritional Risk Screening and Dietary Assessment in Hospitalized Patients With Cancer: Evaluation and Diagnostic Accuracy Study

Authors of this article:

Zhiwen Long1 Author Orcid Image ;   Shan Huang2, 3 Author Orcid Image ;   Jie Zhang2, 3 Author Orcid Image ;   Deng Zhang2 Author Orcid Image ;   Jun Yin1 Author Orcid Image ;   Chengyuan He1 Author Orcid Image ;   Qinqiu Zhang1, 4 Author Orcid Image ;   Huilin Xu1 Author Orcid Image ;   Huimin He1 Author Orcid Image ;   Ho Ching Sun1 Author Orcid Image ;   Ke Xie2, 3 Author Orcid Image
Article Authors Cited by (3) Tweetations (3) Metrics

    Original Paper

    1Recovery Plus Clinic, Chengdu, China

    2Department of Oncology, Sichuan Academy of Medical Sciences and Sichuan Provincial People's Hospital, Chengdu, China

    3School of Medicine, University of Electronic Science and Technology of China, Chengdu, China

    4College of Food Science, Sichuan Agricultural University, Ya'an, China

    *these authors contributed equally

    Corresponding Author:

    Ke Xie, MD

    Department of Oncology, Sichuan Academy of Medical Sciences and Sichuan Provincial People's Hospital

    Number 32, West 2nd Section, First Ring Road

    Qingyang District

    Chengdu, 610072

    China

    Phone: 86 18981838382

    Email: xieke@med.uestc.edu.cn


    Background: Malnutrition is a common and severe problem in patients with cancer that directly increases the incidence of complications and significantly deteriorates quality of life. Nutritional risk screening and dietary assessment are critical because they are the basis for providing personalized nutritional support. No digital smartphone-based self-administered tool for nutritional risk screening and dietary assessment among hospitalized patients with cancer has been developed and evaluated.

    Objective: This study aims to develop a digital smartphone-based self-administered mini program for nutritional risk screening and dietary assessment for hospitalized patients with cancer and to evaluate the validity of the mini program.

    Methods: We have developed the R+ Dietitian mini program, which consists of 3 parts: (1) collection of basic information of patients, (2) nutritional risk screening, and (3) dietary energy and protein assessment. The face-to-face paper-based Nutritional Risk Screening (NRS-2002), the Patient-Generated Subjective Global Assessment Short Form (PG-SGA-SF), and 3 days of 24-hour dietary recall (3d-24HRs) questionnaires were administered according to standard procedure by 2 trained dietitians as the reference methods. Sensitivity, specificity, positive predictive value, negative predictive value, κ value, and correlation coefficients (CCs) of nutritional risk screened in R+ Dietitian against the reference methods, as well as the difference and CCs of estimated dietary energy and protein intakes between R+ Dietitian and 3d-24HRs were calculated to evaluate the validity of R+ Dietitian.

    Results: A total of 244 hospitalized patients with cancer were recruited to evaluate the validity of R+ Dietitian. The NRS-2002 and PG-SGA-SF tools in R+ Dietitian showed high accuracy, sensitivity, and specificity (77.5%, 81.0%, and 76.7% and 69.3%, 84.5%, and 64.5%, respectively), and fair agreement (κ=0.42 and 0.37, respectively; CC 0.62 and 0.56, respectively) with the NRS-2002 and PG-SGA-SF tools administered by dietitians. The estimated intakes of dietary energy and protein were significantly higher (P<.001 for both) in R+ Dietitian (mean difference of energy intake: 144.2 kcal, SD 454.8; median difference of protein intake: 10.7 g, IQR 9.5-39.8), and showed fair agreement (CC 0.59 and 0.47, respectively), compared with 3d-24HRs performed by dietitians.

    Conclusions: The identified nutritional risk and assessment of dietary intakes of energy and protein in R+ Dietitian displayed a fair agreement with the screening and assessment conducted by dietitians. R+ Dietitian has the potential to be a tool for nutritional risk screening and dietary intake assessment among hospitalized patients with cancer.

    Trial Registration: Chinese Clinical Trial Registry ChiCTR1900026324; https://www.chictr.org.cn/showprojen.aspx?proj=41528

    JMIR Form Res 2022;6(10):e40316

    doi:10.2196/40316

    Keywords

    digital tool; nutritional risk screening; dietary assessment; validity; cancer patients 


    Cancer has become a primary public health problem in China over the past several decades [1]. Malnutrition is a common and severe problem in patients with cancer that directly increases the incidence of complications and significantly deteriorates their quality of life [2,3]. Hospitalized malnourished patients with cancer comprise 30%-80% of all patients with cancer [4]. Nausea, vomiting, and diarrhea induced by chemotherapy and radiotherapy can aggravate the nutritional status of patients with cancer [5,6], but nutritional support can improve their clinical outcomes [7-9]. Nutritional risk screening and dietary assessment are critical because they are the basis for providing personalized nutritional support [10].

    Guidelines from the Chinese Society for Parenteral and Enteral Nutrition (CSPEN) [11], the American Society for Parenteral and Enteral Nutrition (ASPEN) [12], and the European Society for Clinical Nutrition and Metabolism (ESPEN) [13] recommended that hospitalized patients with cancer should be screened for nutritional risk at admission to enable timely recognition and treatment of nutritional derangements. For those who are classified as having nutritional risk, further objective and quantitative assessment of nutritional intake should be undertaken, especially their energy and protein intakes, which are 2 of the most important nutrients for patients with cancer [14-16].

    However, it is difficult to screen all hospitalized patients for nutritional risk in hospitals in China, not to mention assessment of nutritional intake, which is more complex. On the one hand, there is a shortage of nutritional specialists in hospitals in China. Even some tertiary hospitals, which usually employ most medical specialists and provide full component of medical services, do not have departments of nutrition. On the other hand, clinicians and nurses are often under pressure to perform a variety of tasks. Nutritional risk screening and dietary assessment can significantly increase the burden of clinicians and nurses and so they do not perform these tasks routinely. In the coming years, there will still be a shortage of nutritional specialists in hospitals in China, meaning clinicians and nurses will continue to work in a busy clinical environment, but importantly patients’ need for nutritional care should not be neglected. Thus, there is an extremely urgent need for more effective, less time-consuming, and less people-demanding tools.

    Modern advancements in digital technologies provide a feasible solution for this problem. For example, a computer-based electronic version of the Malnutrition Universal Screening Tool (MUST), which is used for outpatient self-screening, displayed high validity for nutritional risk self-screening. A dietary assessment app (MyFood) showed good ability to estimate dietary intake [17] and another electronic system improved the documentation of nutritional intake, treatment, and nutritional care plans in hospitalized patients [18]. Several other digital tools based on computers or smartphones were also proved to enhance the efficacy in identifying patients at nutritional risk and assessing dietary intake [19-24]. However, to the best of our knowledge, no study has been conducted on developing a digital smartphone-based self-administered tool for nutritional risk screening and dietary assessment for hospitalized patients with cancer.

    We developed a mini program, R+ Dietitian, for smartphones as a support system for nutritional risk screening and dietary assessment for hospitalized patients with cancer. It is a self-administered tool and requires patients to input data about their disease, weight, dietary intake, among other variables. Further nutritional assessment and individualized nutritional care plan would be customized based on the data entered. Therefore, this mini program should be validated to ensure clinicians can provide appropriate nutritional care for patients.

    The aims of this study were (1) to develop a digital smartphone-based self-administered mini program for nutritional risk screening and dietary assessment of hospitalized patients with cancer, (2) to evaluate the validity of the mini program for patients’ self-screening nutritional risk compared with dietitians’ professional screening, and (3) to evaluate the validity of the mini program for estimating dietary energy and protein intakes compared with dietitians’ professional estimation using 3 days of 24-hour dietary recalls (3d-24HRs).


    Development of the R+ Dietitian Mini Program

    Overview

    R+ Dietitian was developed by dietitians, developers, and interaction designers at Recovery Plus Inc. (R+). Clinicians and nurses at the Oncology Department of Sichuan People’s Hospital were involved in the literary design process.

    The initial paper draft, including content and algorithm, was designed by dietitians at R+. Clinicians and nurses at the Oncology Department of Sichuan People’s Hospital reviewed the draft and then gave feedback to dietitians at R+. The content and language of the mini program were then modified before starting the technical development. Next, the beta version of R+ Dietitian was developed and then tested by 4 dietitians, 2 nurses, and 10 patients with cancer. Their feedback was used for further modification prior to commencing this study.

    Given the popularity of WeChat, which is China’s most popular messaging app with a monthly user base of more than 1 billion people [25], we developed R+ Dietitian as a subapp within the WeChat ecosystem. This was intended to save users’ time, as this avoids the need to additionally download a new app on their cell phones. Instead, users just need to open their WeChat profile to run R+ Dietitian quickly and smoothly.

    All contents displayed in R+ Dietitian were in Chinese. R+ Dietitian consists of the following 3 parts.

    Part 1: Collection of Basic Information of Patients

    In the first 2 interfaces of R+ Dietitian (Figure 1), patients recorded their basic information, including name, inpatient admission number, bed number, age (in years), gender, height (in centimeters), and weight (in kilograms).

    ‎
    Figure 1. Collection of basic information of patients.
    View this figure
    Part 2: Nutritional Risk Screening
    Design

    This part was designed based on the Nutritional Risk Screening (NRS-2002) tool and the Patient-Generated Subjective Global Assessment Short Form (PG-SGA-SF) tool. The NRS-2002 tool is recommended by the CSPEN to screen for nutritional risk in the hospital settings [11], whereas the PG-SGA-SF tool is usually used to screen for nutritional risk in patients with cancer [26-28]. Therefore, the estimated NRS-2002 and PG-SGA-SF scores would be given to participants automatically and separately, while their nutritional risk was separately assessed based on these 2 scores.

    Design and Nutritional Risk Score Estimation of NRS-2002 in R+ Dietitian

    The NRS-2002 tool includes 2 components, the initial screening and the final screening. The initial screening has 4 questions. If the answer is “Yes” to any question on the initial screening, the final screening is performed [29]. In China, the initial screening is skipped and only the final screening is performed according to the standard practice [30]. Therefore, the NRS-2002 in R+ Dietitian was aligned with the final screening of the NRS-2002, which consists of 3 sections: impaired nutritional status (scores 0-3), severity of the disease (scores 0-3), and age (scores 0-1), with a total score of 7. Age was already recorded by patients in part 1.

    Severity of the disease was evaluated by the question “Please choose the disease you have (multiple choice)” with a disease list covering all types of cancer, common chronic disease, and other diseases that are displayed in the NRS-2002.

    Impaired nutritional status involved 3 indicators: degree of weight loss, BMI, and degree of food intake reduction. BMI of patients was calculated as patients’ weight in kilogram (kg) divided by the square of height in meters (m). Degree of weight loss was evaluated by the question (Figure 2) “Has your weight changed over the past three months?” with 3 options, “slight weight gain or substantially unchanged (weight loss less than 5%),” “slight weight loss (weight loss of 5%-15%),” and “severe weight loss (weight loss exceeding 15%).” The exact values of 5% and 15% weight loss would be calculated automatically through the algorithm set in the back end of R+ Dietitian based on the weight recorded in part 1. The calculated weight loss will be displayed within parentheses next to each option so that patients could clearly choose the right option. For example, if a patient recorded the weight as 58 kg, then options such as “slight weight gain or substantially unchanged (weight loss less than 5 kg),” “slight weight loss (weight loss of 5-17 kg),” and “severe weight loss (weight loss exceeding 17 kg).” would be presented. If patients chose 1 of the latter 2 options, an additional question was presented, namely, “Weight loss in” with 3 options, “3 months,” “2 months,” and “1 month.” Degree of food intake reduction was evaluated by the question “Has there been a reduction in your food intake recently?” with 4 options, “substantially unchanged,” “slight reduction (reduction of about 25%-50%),” “moderate reduction (reduction of about 50%-75%),” “severe reduction (reduction of about 75%-100%).”

    Based on age, severity of the disease, and impaired nutritional status reported by patients, the estimated NRS-2002 score was automatically calculated through the algorithm set in the back end of R+ Dietitian and presented in the results interface. The algorithm was consistent with the scoring rules of the NRS-2002 [29].

    ‎
    Figure 2. Evaluation for degree of weight loss.
    View this figure
    Design and Nutritional Risk Score Estimation of the PG-SGA-SF Tool in R+ Dietitian

    The PG-SGA-SF tool comprises the first 4 options of the Patient-Generated Subjective Global Assessment (PG-SGA), including weight history, food intake, symptoms, and activities and function, and is designed to be used by patients independently [31]. Studies have shown that inclusion of the first 4 options did not add further value as a screening tool compared with inclusion of the first 3 options [32]. Therefore, the PG-SGS-SF tool in R+ Dietitian only evaluates weight history, food intake, and symptoms.

    Weight history was already evaluated by the question “Has your weight changed over the past three months?,” which was also used to evaluate the weight loss in the NRS-2002 tool. Food intake in the PG-SGS-SF tool was evaluated by the question “Has there been a reduction in your food intake recently?,” which was also used to evaluate the degree of food intake reduction in the NRS-2002 tool. Food intake was additionally evaluated by the question “What does your dietary look like recently?,” with the following 4 options: “fasting,” “liquid diet,” “soft food,” and “normal food.” Besides, a picture showing what the food looks like was presented next to the latter 3 options to help patients make choice. Symptoms were evaluated by the question “Have you had a bad appetite lately?” with 3 options, “never,” “occasionally,” and “frequently.”

    Based on weight history, food intake, and symptoms reported by patients, the estimated PG-SGA-SF score was automatically calculated through the algorithm set in the back end of R+ Dietitian and presented in the results interface. The algorithm was consistent with the scoring rules of the PG-SGS-SF tool.

    Part 3: Dietary Energy and Protein Assessment

    The 24-hour dietary recall (24HR) is a traditional method for assessing dietary nutrient intake and is widely used in nutrition research [33-35]. However, it requires patients to recall any food, beverages, and water that they consumed, as well as extensive training of the interviewer. These make it burdensome and time-consuming, limiting its utility in clinical practice. For describing the mean usual intake of target nutrients, a “short dietary screener” is feasible. Blalock and colleagues [36] developed a short instrument for assessing dietary intakes of calcium and vitamin D, which only includes 22 foods and beverages that are rich in these 2 nutrients. Similar short tools for assessing dietary intake of cholesterol [37] and saturated fat are also found in the literature [38]. In this mini program, we only wanted to estimate the dietary energy and protein intakes in patients, so, as for a quick assessment, patients’ recalling all the food they consumed is not necessary because some foods contribute less to the energy and protein intakes. Therefore, we shortened the 24HR method to make it suitable for self-reporting and easier to complete, thereby allowing the daily energy and protein intakes of patients to be estimated quickly.

    The China Health and Nutrition Surveys (CHNS) revealed that the main sources of dietary energy and protein among Chinese residents were cereals and animal foods [39-41]. Rice is the primary cereal type in China and milk is an important source of protein. Therefore, we designed 3 questions to assess dietary energy and protein intakes, “How much rice have you recently consumed on a daily basis?,” “How much meat have you recently consumed on a daily basis?,” and “How much milk have you recently consumed on a daily basis?.” An adjustable ruler (unit: gram or milliliter) presented under the 3 questions allowed the patients to report the amount of rice/meat/milk they consumed.

    Before this study began, we conducted a pilot nutrition survey using 3 days of 24HR data (3d-24HRs) among hospitalized patients. Data on foods and nutrients derived from the survey were used to create the algorithm for calculating energy and protein intakes in R+ Dietitian. Based on the rice/meat/milk consumption reported, the estimated dietary energy and protein intakes were automatically calculated through the algorithm set in the back end of R+ Dietitian. Besides, the patients’ requirements for energy and protein were automatically estimated by the algorithm, which was based on the guideline from the CSPEN [11]. The difference between the estimation of dietary intakes of energy and protein and patients’ requirements was calculated automatedly and was presented on the results screen.

    Validity Evaluation of R+ Dietitian

    Study Design, Setting, and Participants

    This was a prospective diagnostic accuracy study conducted at the Oncology Department of Sichuan People’s Hospital, Chengdu, China, from March 2021 to April 2021. Eligible patients were adults aged 18-80 years with pathologically confirmed tumors who were able to communicate normally. We excluded patients with mental or psychological disorders, those with incomplete data, and those who were unwilling or unable to provide written informed consent. All patients registering to the Oncology Department in March 2021 and April 2021 were evaluated for study eligibility by a researcher, and consequently, this was a convenience sample of oncology patients.

    Ethics Approval

    This study was performed according to the Declaration of Helsinki and was approved by the Medical Ethics Committee of Sichuan Provincial People’s Hospital (2019/243). Written informed consent was obtained from all participants.

    Test Methods
    Index Test

    Two dietitians (HX and QZ) were trained to use the R+ Dietitian program before the study started so that they can assist participants in using R+ Dietitian when needed. A clinician in the Oncology Department identified patients that met the inclusion and exclusion criteria and then informed 1 of the 2 dietitians about their eligibility.

    On the day of hospital admission, eligible participants were asked by a dietitian (HX or QZ) to use R+ Dietitian for nutritional risk screening and dietary assessment of energy and protein. Participants or their family members used smartphones to open the WeChat app and then scanned a QR code to run the R+ Dietitian mini program. Registering in R+ Dietitian was not mandatory, so participants can use the mini program directly. The 3 parts of R+ Dietitian were completed by participants one by one. Estimations of the NRS-2002 score, the PG-SGA-SF score, and dietary energy and protein intakes were immediately presented on the results interface of R+ Dietitian once participants completed the program.

    Reference Test

    We used face-to-face interviews as the reference method. On the day of hospital admission, after patients self-evaluated R+ Dietitian, paper-based NRS-2002, PG-SGA-SF, and 3d-24HRs tools were administered according to the standard procedure by 2 trained dietitians (HX and QZ). 3d-24HRs collected data on participants’ food intake in the 3 days before the day of hospital admission. Intake of all foods consumed by participants in the 3 days and corresponding cooking methods, including stir-fry, braising, stew, etc., were recorded. If the food was a composite dish, its composition and the corresponding proportion would be further asked and recorded. To ensure the precision of intake recalled by participants, dietitians showed pictures of standard cutlery to the participants to help them assess the intake when 3d-24HRs were performed.

    Data Collection

    The baseline demographic and medical characteristics of the participants, including age, sex, means of paying medical costs, occupation, marital status, residence, education level, chronic diseases, cancer type, and family history of cancer, were obtained from the electronic medical records by a researcher (JZ).

    Estimations of the NRS-2002 score, the PG-SGA-SF score, and dietary energy and protein intakes in R+ Dietitian were retrieved from the back end of R+ Dietitian by another researcher (CH). The NRS-2002 and PG-SGA-SF scores of the reference method were calculated according to the scoring rules of the 2 questionnaires and then entered into EpiData (EpiData Association) by the 2 dietitians (HX and QZ).

    Dietary data obtained from 3d-24HRs were managed by the 2 dietitians. First, the individual food and the corresponding specific amount consumed by the participants each day were entered into MS Excel (Microsoft Corporation). Next, the raw weight of every individual food was calculated based on the raw-to-cook ratio of the food. Then, if the dish was cooked with oil, the approximate amount of the oil consumed was estimated based on the intake of the dish of the participants. Finally, the energy and protein intakes from each individual food were calculated based on China Food Composition Tables. The total energy and protein intakes each day were then calculated. The final estimated daily dietary energy and protein intakes was the mean of 3-day intake and was entered into EpiData.

    Data Assessment and Statistical Analysis

    The NRS-2002 scores of 3 or above identified patients at nutritional risk [29], whereas the PG-SGA-SF scores of 4 or above identified patients at nutritional risk [27,42]. The difference in dietary energy and protein intakes estimated through R+ Dietitian from those measured by dietitians was calculated.

    All statistical analyses were performed using SAS 9.4 software (SAS Institute Inc.). First, continuous variables were analyzed for normality using the Kolmogorov–Smirnov test. Data were then described as mean (SD) or median (IQR). The Student t test or the rank sum test was applied accordingly. Categorical variables were described as frequencies and percentages, and then the χ2 test was applied. Accuracy, sensitivity, specificity, positive predictive value, negative predictive value (NPV), and κ values of the NRS-2002 and PG-SGA-SF tools in R+ Dietitian were calculated using the McNemar test for correlated proportions, respectively. The consistency of R+ Dietitian for dietary assessment with 3d-24HRs was tested using the Pearson or Spearman correlation test. All tests were 2-sided, and a significant level of 5% (P˂.05) was applied.

    The validity of R+ Dietitian was evaluated using the cutoffs recommended by van Bokhorst–de van der Schueren et al [43], which were based on sensitivity and specificity. “Good” indicated both sensitivity and specificity exceeding 80%; “fair” indicated sensitivity or specificity exceeding 50% but lower than 80%; and “poor” indicated sensitivity or specificity lower than 50%. For other validation indices, the cutoffs of the correlation coefficients (CCs) followed Guilford’s [44] description as follows: “good,” CC ≥0.75; “fair,” CC ≥0.4 and <0.75; and “poor,” CC <0.4. The cutoffs for good, fair, and poor κ values [45] were ≥0.6, ≥0.4 and <0.6, and <0.4, respectively.


    Participants

    Figure 3 illustrates the flow of participants through the study. From March 2021 to April 2021, 263 patients were assessed for eligibility and were invited to participate in this study. Patients who were excluded or who withdrew from this study and the corresponding reasons were noted. Overall, 244 patients were recruited to evaluate the validity of R+ Dietitian for nutritional risk screening, and 214 patients were included to evaluate the validity of R+ Dietitian for dietary intake assessment.

    ‎
    Figure 3. Flow of participants.
    View this figure

    Characteristics of Participants

    The baseline characteristics of the participants included in this study are presented in Table 1. The median (IQR) age of participants was 59 (51–68) years and the mean (SD) weight and BMI were 58.6 (8.9) kg and 22.2 (2.9) kg/m2, respectively. Most participants were male (156/244, 63.9%). The most common diagnosis was gastrointestinal tumor (107/244, 43.9%) and the least common diagnosis was head and neck cancer (17/244, 6.9%).

    Table 1. Baseline characteristics of participants (N=244).
    VariablesValues
    Age (years), median (IQR)59 (51-68)
    Weight (kg), mean (SD)58.6 (8.9)
    BMI (kg/m2), mean (SD)22.2 (2.9)
    Gender, n (%)

    		Male156 (63.9)

    		Female88 (36.1)
    Means of paying medical costs, n (%)

    		Social security230 (94.3)

    		Self-paid14 (5.7)
    Occupation, n (%)

    		Famer/worker27 (11.1)

    		Retirement45 (18.4)

    		Other172 (70.5)
    Marital status, n (%)

    		Married241 (98.8)

    		Unmarried3 (1.2)
    Residence, n (%)

    		Rural130 (53.3)

    		City114 (46.7)
    Education, n (%)

    		Primary28 (11.5)

    		Secondary197 (80.7)

    		Senior19 (7.8)
    Hypertension, n (%)

    		Yes40 (16.4)

    		No204 (83.6)
    Diabetes mellitus, n (%)

    		Yes22 (9)

    		No222 (91)
    Cancer family history, n (%)

    		Yes16 (6.6)

    		No228 (93.4)
    Cancer type, n (%)

    		Head and neck tumor17 (6.9)

    		Gastrointestinal tumor107 (43.9)

    		Respiratory tumor76 (31.1)

    		Other44 (18)

    Agreement Between the NRS-2002 Tool in R+ Dietitian and the NRS-2002 Tool Administered by Dietitians

    Table 2 presents the screening results of the NRS-2002 tool in R+ Dietitian in comparison to dietitians’ screening. According to the NRS-2002 tool in R+ Dietitian, 33.2% (81/244) of patients were at nutritional risk, which was significantly higher than in the screening by dietitians (42/244, 17.2%; P<.001; McNemar test). The NRS-2002 tool in R+ Dietitian was in fair agreement with the NRS-2002 tool administered by dietitians (Table 3; both sensitivity and specificity >50%, κ>0.42, and CC >0.4). In addition, self-screening by patients using the NRS-2002 tool in R+ Dietitian had a high NPV (95.1%), which indicates that self-screening by patients using the NRS-2002 in R+ Dietitian can strongly predict those that are not at nutritional risk.

    Table 2. Cross tabulation of nutritional risk according to the NRS-2002a tool in R+ Dietitian and the NRS-2002 tool administered by dietitians.
    R+ Dietitian (patients’ self-screening)Dietitians’ screeningTotal (N=244)
    Positive (n=42)Negative (n=202)
    Positive, n (%)34 (13.9)47 (19.3)81 (33.2)
    Negative, n (%)8 (3.3)155 (63.5) 163 (66.8)
    Total (n=244)42 (17.2)202 (82.8)244 (100)

    aNutritional Risk Screening.

    Table 3. Sensitivity, specificity, positive predictive value, negative predictive value, κ value, and correlation coefficient of patients’ self-screening using the NRS-2002 tool in R+ Dietitian.
    IndexNRS-2002a in R+ Dietitian (patients’ self-screening), % (95% CI)P value
    Accuracy77.5 (71.7-82.5) N/Ab
    Sensitivity81.0 (65.4-90.9) N/A
    Specificity76.7 (70.2-82.2) N/A
    Positive predictive value42.0 (31.3-53.5) N/A
    Negative predictive value95.1 (90.2-97.7) N/A
    κ value0.42 (0.30-0.54) N/A
    Correlation coefficient0.62 (0.54-0.70)<.001

    aNutritional Risk Screening.

    bNot applicable.

    Agreement Between the PG-SGA-SF Tools in R+ Dietitian and the PG-SGA-SF Tool Administered by Dietitians

    Table 4 shows the screening results of the PG-SGS-SF tool in R+ Dietitian in comparison to dietitians’ screening. According to the PG-SGS-SF tool in R+ Dietitian, 47.1% (115/244) of patients were at nutritional risk, which was significantly higher than in the screening by dietitians (58/244, 23.8%; P<.001) according to the McNemar test. Further, the PG-SGS-SF tool in R+ Dietitian was fairly in agreement with the NRS-2002 tool administered by dietitians (Table 5; both sensitivity and specificity >50% and CC >0.4). Besides, as with the NRS-2002 tool in R+ Dietitian, patients’ self-screening using the PG-SGS-SF tool in R+ Dietitian had a high NPV (95.1%).

    Table 4. Cross tabulation of nutritional risk according to the PG-SGA-SFa tool in R+ Dietitian and the PG-SGS-SF tool administered by dietitians.
    R+ Dietitian (patients’ self-screening)Dietitians’ screening, n (%)
    Positive (n=58)Negative (n=186)Total (N=244)
    Positive, n (%)49 (20.1)66 (27.0)115 (47.1)
    Negative, n (%)9 (3.7)120 (49.2)129 (52.9)
    Total (n=244)58 (23.8)186 (76.2)244 (100)

    aPatient-Generated Subjective Global Assessment Short Form.

    Table 5. Sensitivity, specificity, positive predictive value, negative predictive value, κ value, and correlation coefficient of patients’ self-screening using the PG-SGA-SFa tool in R+ Dietitian.
    IndexPG-SGA-SF in R+ Dietitian (patients’ self-screening), % (95% CI)P value
    Accuracy69.3 (63.1-75.0) N/Ab
    Sensitivity84.5 (72.1-92.2) N/A
    Specificity64.5 (57.1-71.3) N/A
    Positive predictive value42.6 (33.5-52.2) N/A
    Negative predictive value93.0 (86.8-96.6) N/A
    κ value0.37 (0.26-0.47) N/A
    Correlation coefficient0.56 (0.47-0.64)<.001

    aPatient-Generated Subjective Global Assessment Short Form.

    bNot applicable.

    Agreement Between Dietary Energy and Protein Intake Assessment in R+ Dietitian and Assessment Performed by Dietitians

    The dietary energy and protein intakes estimated by R+ Dietitian and dietitians are presented in Table 6. Both the estimated energy and protein intakes were significantly higher (P<.001 for both) in R+ Dietitian, compared with the 3d-24HRs. Estimations of energy and protein intakes in R+ Dietitian were 10.7% and 29.0% higher than those in 3d-24HRs, respectively (Table 7). Nevertheless, R+ Dietitian was still in moderate agreement with 3d-24HRs for both energy and protein intake assessment (Table 7; both CC >0.4).

    Table 6. Estimation of energy and protein intakes in R+ Dietitian compared with the 3d-24HRs administered by dietitians.
    Dietary intakeR+ Dietitian3d-24HRsaP value
    Energy (kcal), mean (SD)1578.3 (468.4)1434.1 (528.8)<.001
    Protein (g), median (IQR)79.0 (62.7-95.3)61.7 (43.0-82.8)<.001

    aThree days of 24-hour dietary recall.

    Table 7. Correlation coefficient and absolute and relative differences of dietary energy and protein intake estimation in R+ Dietitian against 3d-24HRsa.
    IndexEnergyP valueProteinP value
    Correlation coefficient (95% CI)0.59 (0.49-0.67)<.0010.47 (0.36-0.57)<.001
    Absolute difference (kcal or g), mean (SD)144.2 (454.8)<.00114.7 (29.2)<.001
    Relative difference (%), median (IQR)10.7 (9.5-39.8)<.00129 (3.1-68.1) <.001

    aThree days of 24-hour dietary recall.


    Principal Findings

    The R+ Dietitian mini program is developed for use among hospitalized patients with cancer. This prospective study demonstrates the validity of R+ Dietitian for nutritional risk screening and dietary energy and protein intake assessment in oncology department. Overall, the NRS-2002, PG-SGA-SF, and dietary energy and protein intake assessments in R+ Dietitian were in fair agreement with those performed by trained dietitians, although the number of patients identified at nutritional risk was more and the estimated dietary energy and protein intakes were higher in R+ Dietitian, compared with the reference methods. Both the NRS-2002 and the PG-SGA-SF tools in R+ Dietitian showed an excellent ability to predict those who were not at nutritional risk.

    To the best of our knowledge, no similar mini program has been developed and this is the first prospective study to examine the validity of a digital smartphone-based self-administered tool for nutritional risk screening and dietary assessment in the oncology department.

    The Validity of R+ Dietitian’s Nutritional Risk Screening

    For nutritional risk screening in hospitals, the NRS-2002 is recommended by the CSPEN [11] and the PG-SGA-SF is widely used among patients with cancer [26-28]. Therefore, we developed nutritional risk screening in R+ Dietitian based on both the NRS-2002 and the PG-SGA-SF. Our study shows that nutritional risk evaluated by the NRS-2002 and PG-SGA-SF tools in R+ Dietitian had fair validity.

    No similar smartphone-based app for nutritional risk self-screening has been developed, although a computer-based self-screening app is available [21,46]. This app is the electronic version of the MUST and was developed for hospital outpatients. It showed high agreement with the MUST administered by health care professionals (κ=0.74-1.00). These values are higher than the NRS-2002 and PG-SGA-SF tools in R+ Dietitian. The different degrees of agreement between the electronic version of the MUST and R+ Dietitian can be explained as follows: First, in the study of the electronic version of the MUST, the weight and height were measured using a weighing scale and a portable height measure, respectively, in the outpatients’ waiting room by patients. Consequently, the weight and height reported by patients are more likely to be objective than those in R+ Dietitian. Both dietitians in our study and health care professionals in the study of the electronic version of the MUST used weight and height scales to measure the weight and height, respectively, of participants. However, there was a higher deviation in the weight and height reported by patients and professionals in our study, compared with those in the study of the electronic version of the MUST. BMI calculated based on weight and height is a factor for nutritional risk scoring in both R+ Dietitian and the electronic version of the MUST. Weight was also used to calculate the degree of weight loss in the past 3-6 months, which is also a criterion for evaluating nutritional risk. Hence, the different methods applied to measure weight and height by participants may be one of the reasons for the difference in agreement between these 2 studies. However, bias induced from recall is common in surveys [47-51] and can hardly be avoided. Second, the NRS-2002 tool in R+ Dietitian additionally evaluates the degree of food intake reduction using the question, “Has there been a reduction in your food intake recently?” with 3 options, “slight reduction (reduction of about 25%-50%),” “moderate reduction (reduction of about 50%-75%),” and “severe reduction (reduction of about 75%-100%).” For this question, some patients may compare their recent food intake with food intake several weeks ago rather than their normal requirement of food intake, whereas dietitians compared patients’ recent food intake with their normal requirement. Third, the PG-SGS-SF tool in R+ Dietitian did not evaluate the symptoms affecting patients’ nutritional intake, but dietitians did. The electronic version of the MUST was developed for hospital outpatients and requires patients to use the weight scale and the portable height measure tool, which is less feasible for inpatients, as they are busy with completing their admission procedures while being admitted to the hospital. In addition, the electronic version of the MUST was computer based, which makes it less flexible compared with R+ Dietitian, which is a smartphone-based app for self-screening.

    Nutritional risk screening in R+ Dietitian showed a fair agreement with the dietitians’ screening. As the first smartphone-based tool for nutritional risk self-screening, it displayed potential ability to be used in clinical practice for nutritional risk screening.

    The Validity of R+ Dietitian’s Dietary Energy and Protein Intake Assessment

    The energy intake was overestimated by 144 kcal/day in R+ Dietitian compared with the estimation from 3d-24HRs performed by dietitians. Our finding is different from other self-administered dietary assessment apps. Compared with 24HRs or the Food Frequency Questionnaire (FFQ), most of these apps underestimated energy intake from –8 to –466 kcal/day [24,52-60], and 1 app overestimated energy intake by 55 kcal/day [17]. The absolute mean difference of estimated energy intake between R+ Dietitian and 3d-24HRs was higher in our study compared with the absolute difference in 5 studies (from 8.1 to 101 kcal/day) [24,55-57,60] but was lower compared with the absolute difference reported in 6 studies (from 145.1 to 466.0 kcal/day) [22,52-54,58,59]. The following might have contributed to the difference between R+ Dietitian and other apps. First, all the other validity studies required participants to use the experimental app to record their food intake for several days [17,23,24,53,54,56,57,59,60] or even few months [52,58]. Based on the data inputted, the daily energy intake and the mean energy intake over the study period would be calculated automatically by the app. By contrast, in our study, patients self-reported their mean food intake but the mean energy intake was calculated automatically by R+ Dietitian. The different methods of collecting data on food intake between R+ Dietitian and other tools may result in differences in the energy intake estimated. Second, other apps recorded all the foods consumed by participants, whereas only 3 food groups were reported by patients in R+ Dietitian. R+ Dietitian was developed to be a digital tool that can quickly assess energy and protein intakes of patients, thereby allowing clinicians, who generally have limited time to analyze these, to have an approximate overview of the intake level of the 2 nutrients (ie, energy and protein). Further and more complex and detailed dietary assessment is needed if patients are at nutritional risk. Third, patients in most validity studies for self-administered dietary assessment apps were young. For example, their sample population was recruited in high school [52] or in a university setting [24,53,55-57,59]. The median age of participants in our study was 59 years, which is much higher than that in these studies. Young and old people may have different tendencies in self-reporting studies. For the 2 studies that recruited old people or hospitalized patients [17,60], the absolute mean difference in estimated energy intake between the experimental app and the reference method was lower (55 kcal/day and 101 kcal/day, respectively), but in another study [58] this was higher (408.8 kcal/day) than that in our study (144 kcal/day).

    The estimation of protein intake in R+ Dietitian was higher than that from 3d-24HRs by 14.7 g/day. Mescoloto and colleagues [61] and Bucher Della Torre and coworkers [60] also reported overestimation of protein intake in 2 digital dietary recording apps by 2.1 and 2.0 g/day, respectively, compared with measured energy expenditure and 2 unannounced 24-h phone dietary recalls on overlapping days. In contrast to our study and these 2 studies [60,61], 7 studies found an underestimation of protein intake in phone-based dietary recording apps from –2.6 to –28 g/day [17,24,52,53,55,56,58], compared with 24HRs or FFQs or paper-based food records. The absolute difference between the experimental app and the reference method in our study was higher than that in 4 studies (from –2.6 to –10.5 g/day) [17,24,55,56], but was lower than that in 3 other studies (from –21.5 to 28.0 g/day) [52,53,58]. For the estimation of protein intake, just like the estimation of energy intake, other studies asked participants to use the dietary recording apps to record all the foods they consumed during the study period in real time, whereas we asked participants in our study to recall their intake of 3 recent food groups for a quick estimation during their hospital admission. In general, real-time recording may be more precise than recall in dietary assessment [62]. But still, the difference in the estimation of protein intake between R+ Dietitian and 3d-24HRs was lower than that between 3 dietary recording apps and 24HRs or paper-based food records [52,53,55].

    Nowadays, similar to traditional dietary intake assessment tools, including 24HRs, FFQs, and weighed or nonweighed food records, all smartphone-based digital dietary intake assessing apps require users to report on all the foods or food groups they consume [17,22-24,52-60,63]. However, this approach may not be suitable for all hospitalized patients. On the one hand, these apps require users to record their food intake for several days, or even few months, which cannot help clinicians to understand their patients’ nutritional intake within the short period available for initial assessment. The only 1 study that recruited hospitalized patients [17] asked them to use the experimental app to record food intake for 2 days; however, MyFood was developed for inpatients with nutritional risk rather than all hospitalized patients, which may limit its usage in other patient groups. In this case, the nutritional intake of other nonnutritional-risk patients can be missed by the clinicians. On the other hand, recording all foods or food groups is also time-consuming and burdensome for patients, so some may forget to record their food intake at times. R+ Dietitian was used on the day of hospital admission and only includes 3 food groups, reducing patients’ burden and helping clinicians to have a quick and approximate overview of the patients’ energy and protein intakes. If the patients are diagnosed to have nutritional risk, further comprehensive dietary assessments are needed.

    Strengths and Limitations

    To our knowledge, R+ Dietitian is the first digital smartphone-based self-administered tool for both nutritional risk screening and dietary assessment for hospitalized patients with cancer. This is a strong strength of this study. Another strength is that R+ Dietitian was developed based on the NRS-2002 and PG-SGA-SF tools. The NRS-2002 is the only tool validated by a retrospective analysis of 128 randomized controlled trials [29] and is recommended by several professional communities [11,64]. The PG-SGS-SF is specific for patients with cancer and has been proved to be an effective tool to screen nutritional risk for patients with cancer [28,31,42,65,66]. Experienced dietitians, nurses, and patients were involved in the development of R+ Dietitian, which is also an important strength. R+ Dietitian was developed as a subapp within the WeChat ecosystem, which is China’s most popular messaging app with a monthly user base of more than 1 billion people [25]. This makes it commonly available and easily accessible. All participants in our study had installed WeChat on their smartphones and thus the need to download a new app was not required, thereby saving their time.

    A limitation of this study is that we only recruited hospitalized patients with cancer in 1 hospital, which may limit the generalizability of our findings. However, R+ Dietitian was developed based on the NRS-2002 tool, which has been used among various types of patients. Hence, we propose that R+ Dietitian can also be used for other types of patients, but this needs further research. Besides, usability is one of the important factors determining the tool’s actual usefulness in practical settings [67]. The usability of R+ Dietitian was not evaluated in this study. The 2 dietitians responsible for asking the patients to use R+ Dietitian reported that none of the patients reported difficulties while using the tool.

    R+ Dietitian’s Potential for Nutritional Risk Screening and Dietary Intake Assessment Among Hospitalized Patients With Cancer

    Based on the findings in this study, we propose that R+ Dietitian has a large potential to be a tool for nutritional risk screening and dietary intake assessment among hospitalized patients with cancer. R+ Dietitian may provide support for nurses and clinicians to perform nutritional risk screening and dietary assessment among hospitalized patients with cancer, enhancing the rate and efficiency of nutritional risk screening and dietary assessment among this patient group. In addition, R+ Dietitian is a WeChat-based tool, which makes it commonly available and may potentially increase its use among hospitalized patients. Further validity of this study for other types of patients may be helpful to expand the underlying use of R+ Dietitian in hospital settings.

    Conclusions

    We have developed a digital smartphone-based self-administered instrument for nutritional risk screening and dietary assessment among hospitalized patients with canner. The instrument enables the evaluation of estimated dietary intake of energy and protein against individual’s requirements. The identified nutritional risk and assessment of dietary energy and protein intakes in R+ Dietitian displayed a fair agreement with the screening and assessment conducted by dietitians. R+ Dietitian has the potential to be a tool for nutritional risk screening and dietary intake assessment among hospitalized patients with cancer.

    Acknowledgments

    We acknowledge all study participants for supporting this study. The authors thank Enago for the English language review. This study was funded by Recovery Plus Inc., Chengdu, China.

    Authors' Contributions

    ZL contributed to the study design, data collection and analysis, and wrote the manuscript. SH contributed to the study design and patient recruitment. JZ contributed to patient recruitment and data collection. JY contributed to data collection and read and edited the manuscript. DZ contributed to data collection. HX and QZ contributed to calculation of dietary nutrients. HCS and HH contributed to data collection. CH and KX contributed to the study design, data analysis, and manuscript preparation and review.

    Conflicts of Interest

    This study was funded by Recovery Plus Inc., Chengdu, China, and the R+ Dietitian mini program used in this study was provided by this company. ZL, JY, HX, QZ, HCS, and HH are employees of Recovery Plus Inc.

    1. Cao W, Chen H, Yu Y, Li N, Chen W. Changing profiles of cancer burden worldwide and in China: a secondary analysis of the global cancer statistics 2020. Chin Med J (Engl) 2021 Mar 17;134(7):783-791 [FREE Full text] [CrossRef] [Medline]
    2. Clinton SK, Giovannucci EL, Hursting SD. The World Cancer Research Fund/American Institute for Cancer Research Third Expert Report on Diet, Nutrition, Physical Activity, and Cancer: Impact and Future Directions. J Nutr 2020 Apr 01;150(4):663-671 [FREE Full text] [CrossRef] [Medline]
    3. Rhondali W, Chisholm GB, Daneshmand M, Allo J, Kang D, Filbet M, et al. Association between body image dissatisfaction and weight loss among patients with advanced cancer and their caregivers: a preliminary report. J Pain Symptom Manage 2013 Jun;45(6):1039-1049 [FREE Full text] [CrossRef] [Medline]
    4. Zhang X, Pang L, Sharma SV, Li R, Nyitray AG, Edwards BJ. Malnutrition and overall survival in older patients with cancer. Clin Nutr 2021 Mar;40(3):966-977. [CrossRef] [Medline]
    5. Seo SH, Kim S, Kang Y, Ryoo B, Ryu M, Jeong JH, et al. Association of nutritional status-related indices and chemotherapy-induced adverse events in gastric cancer patients. BMC Cancer 2016 Nov 18;16(1):900 [FREE Full text] [CrossRef] [Medline]
    6. Sánchez-Lara K, Ugalde-Morales E, Motola-Kuba D, Green D. Gastrointestinal symptoms and weight loss in cancer patients receiving chemotherapy. Br J Nutr 2013 Mar 14;109(5):894-897. [CrossRef] [Medline]
    7. Tan S, Meng Q, Jiang Y, Zhuang Q, Xi Q, Xu J, et al. Impact of oral nutritional supplements in post-discharge patients at nutritional risk following colorectal cancer surgery: A randomised clinical trial. Clin Nutr 2021 Jan;40(1):47-53. [CrossRef] [Medline]
    8. Ukovic B, Porter J. Nutrition interventions to improve the appetite of adults undergoing cancer treatment: a systematic review. Support Care Cancer 2020 Oct;28(10):4575-4583. [CrossRef] [Medline]
    9. Capra S, Ferguson M, Ried K. Cancer: impact of nutrition intervention outcome--nutrition issues for patients. Nutrition 2001 Sep;17(9):769-772. [CrossRef] [Medline]
    10. Reber E, Schönenberger KA, Vasiloglou MF, Stanga Z. Nutritional Risk Screening in Cancer Patients: The First Step Toward Better Clinical Outcome. Front Nutr 2021;8:603936 [FREE Full text] [CrossRef] [Medline]
    11. Chinese Society for ParenteralEnteral Nutrition. [Guidelines on nutritional support in patients with tumor]. Zhonghua Wai Ke Za Zhi 2017 Nov 01;55(11):801-829. [CrossRef] [Medline]
    12. Mueller C, Compher C, Ellen DM, American Society for ParenteralEnteral Nutrition (A.S.P.E.N.) Board of Directors. A.S.P.E.N. clinical guidelines: Nutrition screening, assessment, and intervention in adults. JPEN J Parenter Enteral Nutr 2011 Jan;35(1):16-24. [CrossRef] [Medline]
    13. Arends J, Bachmann P, Baracos V, Barthelemy N, Bertz H, Bozzetti F, et al. ESPEN guidelines on nutrition in cancer patients. Clin Nutr 2017 Feb;36(1):11-48. [CrossRef] [Medline]
    14. Arends J. Energy and protein intake may have an impact on survival in patients with advanced cancer. Clin Nutr 2022 Jan;41(1):266-267. [CrossRef] [Medline]
    15. Pimentel GD, Pichard C, Laviano A, Fernandes RC. High protein diet improves the overall survival in older adults with advanced gastrointestinal cancer. Clin Nutr 2021 Mar;40(3):1376-1380. [CrossRef] [Medline]
    16. Stobäus N, Müller MJ, Küpferling S, Schulzke J, Norman K. Low Recent Protein Intake Predicts Cancer-Related Fatigue and Increased Mortality in Patients with Advanced Tumor Disease Undergoing Chemotherapy. Nutr Cancer 2015;67(5):818-824. [CrossRef] [Medline]
    17. Paulsen MM, Hagen MLL, Frøyen MH, Foss-Pedersen RJ, Bergsager D, Tangvik RJ, et al. A Dietary Assessment App for Hospitalized Patients at Nutritional Risk: Development and Evaluation of the MyFood App. JMIR Mhealth Uhealth 2018 Sep 07;6(9):e175 [FREE Full text] [CrossRef] [Medline]
    18. Paulsen MM, Paur I, Gjestland J, Henriksen C, Varsi C, Tangvik RJ, et al. Effects of using the MyFood decision support system on hospitalized patients' nutritional status and treatment: A randomized controlled trial. Clin Nutr 2020 Dec;39(12):3607-3617 [FREE Full text] [CrossRef] [Medline]
    19. Krznarić, Bender DV, Laviano A, Cuerda C, Landi F, Monteiro R, et al. A simple remote nutritional screening tool and practical guidance for nutritional care in primary practice during the COVID-19 pandemic. Clin Nutr 2020 Jul;39(7):1983-1987 [FREE Full text] [CrossRef] [Medline]
    20. Karagiozoglou-Lampoudi T, Daskalou E, Lampoudis D, Apostolou A, Agakidis C. Computer-based malnutrition risk calculation may enhance the ability to identify pediatric patients at malnutrition-related risk for unfavorable outcome. JPEN J Parenter Enteral Nutr 2015 May;39(4):418-425. [CrossRef] [Medline]
    21. McGurk P, Jackson JM, Elia M. Rapid and reliable self-screening for nutritional risk in hospital outpatients using an electronic system. Nutrition 2013 Apr;29(4):693-696. [CrossRef] [Medline]
    22. Lozano-Lozano M, Galiano-Castillo N, Martín-Martín L, Pace-Bedetti N, Fernández-Lao C, Arroyo-Morales M, et al. Monitoring Energy Balance in Breast Cancer Survivors Using a Mobile App: Reliability Study. JMIR Mhealth Uhealth 2018 Mar 27;6(3):e67 [FREE Full text] [CrossRef] [Medline]
    23. Svensson, Larsson C. A Mobile Phone App for Dietary Intake Assessment in Adolescents: An Evaluation Study. JMIR Mhealth Uhealth 2015 Nov 03;3(4):e93 [FREE Full text] [CrossRef] [Medline]
    24. Rangan AM, O'Connor S, Giannelli V, Yap ML, Tang LM, Roy R, et al. Electronic Dietary Intake Assessment (e-DIA): Comparison of a Mobile Phone Digital Entry App for Dietary Data Collection With 24-Hour Dietary Recalls. JMIR Mhealth Uhealth 2015 Oct 27;3(4):e98 [FREE Full text] [CrossRef] [Medline]
    25. Science on WeChat. Nat Methods 2020 Sep;17(9):863. [CrossRef] [Medline]
    26. Zhang Q, Li X, Zhang X, Ding J, Liu T, Qian L, Investigation on Nutrition Statusits Clinical Outcome of Common Cancers (INSCOC) Group. PG-SGA SF in nutrition assessment and survival prediction for elderly patients with cancer. BMC Geriatr 2021 Dec 10;21(1):687 [FREE Full text] [CrossRef] [Medline]
    27. Dewansingh P, Euwes M, Krijnen WP, Strijbos JH, van der Schans CP, Jager-Wittenaar H. Patient-Generated Subjective Global Assessment Short Form better predicts length of stay than Short Nutritional Assessment Questionnaire. Nutrition 2021;91-92:111366 [FREE Full text] [CrossRef] [Medline]
    28. Carriço M, Guerreiro CS, Parreira A. The validity of the Patient-Generated Subjective Global Assessment Short-form© in cancer patients undergoing chemotherapy. Clin Nutr ESPEN 2021 Jun;43:296-301. [CrossRef] [Medline]
    29. Kondrup J, Rasmussen HH, Hamberg O, Stanga Z, Ad Hoc ESPEN Working Group. Nutritional risk screening (NRS 2002): a new method based on an analysis of controlled clinical trials. Clin Nutr 2003 Jun;22(3):321-336. [CrossRef] [Medline]
    30. National Health Commission of the People's Republic of China. Hygiene Standard: Nutritional risk screening. National Health Commission of the People's Republic of China. 2013.   URL: http://www.nhc.gov.cn/wjw/yingyang/201308/9c70beb3461e4676aa4afaeed8cdaab5.shtml [accessed 2022-10-04]
    31. Gomes-Neto AW, van Vliet IMY, Osté MCJ, de Jong MFC, Bakker SJL, Jager-Wittenaar H, et al. Malnutrition Universal Screening Tool and Patient-Generated Subjective Global Assessment Short Form and their predictive validity in hospitalized patients. Clin Nutr ESPEN 2021 Oct;45:252-261 [FREE Full text] [CrossRef] [Medline]
    32. Abbott J, Teleni L, McKavanagh D, Watson J, McCarthy AL, Isenring E. Patient-Generated Subjective Global Assessment Short Form (PG-SGA SF) is a valid screening tool in chemotherapy outpatients. Support Care Cancer 2016 Sep;24(9):3883-3887. [CrossRef] [Medline]
    33. Shan Z, Guo Y, Hu FB, Liu L, Qi Q. Association of Low-Carbohydrate and Low-Fat Diets With Mortality Among US Adults. JAMA Intern Med 2020 Apr 01;180(4):513-523 [FREE Full text] [CrossRef] [Medline]
    34. Huang K, Yu D, Guo Q, Yang Y, Wei X, Zhao L, et al. Validation of the MSM and NCI Method for Estimating the Usual Intake of Nutrients and Food According to Four Seasons of Seven Consecutive Daily 24 Hour Dietary Recalls in Chinese Adults. Nutrients 2022 Jan 19;14(3):445 [FREE Full text] [CrossRef] [Medline]
    35. Htet MK, Fahmida U, Do TT, Dibley MJ, Ferguson E. The Use of Tablet-Based Multiple-Pass 24-Hour Dietary Recall Application (MP24Diet) to Collect Dietary Intake of Children under Two Years Old in the Prospective Cohort Study in Indonesia. Nutrients 2019 Nov 27;11(12):2889 [FREE Full text] [CrossRef] [Medline]
    36. Blalock SJ, Norton LL, Patel RA, Cabral K, Thomas CL. Development and assessment of a short instrument for assessing dietary intakes of calcium and vitamin D. J Am Pharm Assoc (2003) 2003;43(6):685-693. [CrossRef] [Medline]
    37. Stuhldreher WL, Orchard TJ, Donahue RP, Kuller LH, Gloninger MF, Drash AL. Cholesterol screening in childhood: sixteen-year Beaver County Lipid Study experience. J Pediatr 1991 Oct;119(4):551-556. [CrossRef] [Medline]
    38. Kris-Etherton P, Eissenstat B, Jaax S, Srinath U, Scott L, Rader J, et al. Validation for MEDFICTS, a dietary assessment instrument for evaluating adherence to total and saturated fat recommendations of the National Cholesterol Education Program Step 1 and Step 2 diets. J Am Diet Assoc 2001 Jan;101(1):81-86. [CrossRef] [Medline]
    39. Ju L, Yu D, Fang H, Guo Q, Xu X, Li S, et al. [Trends and food sources composition of energy, protein and fat in Chinese residents, 1992-2012]. Wei Sheng Yan Jiu 2018 Sep;47(5):689-704. [Medline]
    40. Jiang H, Zhang J, Du W, Su C, Zhang B, Wang H. Energy intake and energy contributions of macronutrients and major food sources among Chinese adults: CHNS 2015 and CNTCS 2015. Eur J Clin Nutr 2021 Feb;75(2):314-324. [CrossRef] [Medline]
    41. Zhou C, Liu C, Zhang Z, Liu M, Zhang Y, Li H, et al. Variety and quantity of dietary protein intake from different sources and risk of new-onset diabetes: a Nationwide Cohort Study in China. BMC Med 2022 Jan 13;20(1):6 [FREE Full text] [CrossRef] [Medline]
    42. Jager-Wittenaar H, de Bats HF, Welink-Lamberts BJ, Gort-van Dijk D, van der Laan BFAM, Ottery FD, et al. Self-Completion of the Patient-Generated Subjective Global Assessment Short Form Is Feasible and Is Associated With Increased Awareness on Malnutrition Risk in Patients With Head and Neck Cancer. Nutr Clin Pract 2020 Apr;35(2):353-362 [FREE Full text] [CrossRef] [Medline]
    43. van Bokhorst-de van der Schueren MAE, Guaitoli PR, Jansma EP, de Vet HCW. A systematic review of malnutrition screening tools for the nursing home setting. J Am Med Dir Assoc 2014 Mar;15(3):171-184. [CrossRef] [Medline]
    44. Guilford JP. Fundamental Statistics in psychology and Education. Vol. 565. New York, NY: McGraw-Hill Book Company; 2018:1-565.
    45. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977 Mar;33(1):159-174. [Medline]
    46. Cawood AL, Walters ER, Sharp SKE, Elia M, Stratton RJ. 'Self-screening' for malnutrition with an electronic version of the Malnutrition Universal Screening Tool ('MUST') in hospital outpatients: concurrent validity, preference and ease of use. Br J Nutr 2018 Sep;120(5):528-536. [CrossRef] [Medline]
    47. Ryu H, Choi Y, Kim E, Park J, Lee S, Yoon J, et al. Misclassification and characterization of exposure to humidifier disinfectants using a questionnaire. BMC Public Health 2021 Jul 27;21(1):1458 [FREE Full text] [CrossRef] [Medline]
    48. Gotlin MJ, Kingery MT, Baron SL, McCafferty J, Jazrawi LM, Meislin RJ. Recall Bias in Retrospective Assessment of Preoperative Patient-Reported American Shoulder and Elbow Surgeons Scores in Arthroscopic Rotator Cuff Repair Surgery. Am J Sports Med 2020 May;48(6):1471-1475. [CrossRef] [Medline]
    49. Lizama N, Heyworth J, Thomson A, Slevin T, Fritschi L. Self-reported shift work, recall bias, and belief about disease causation in a case-control study of breast cancer. Cancer Epidemiol 2017 Oct;50(Pt A):9-15. [CrossRef] [Medline]
    50. Sinnett S, Bengle R, Brown A, Glass AP, Johnson MA, Lee JS. The validity of Nutrition Screening Initiative DETERMINE Checklist responses in older Georgians. J Nutr Elder 2010 Oct;29(4):393-409. [CrossRef] [Medline]
    51. Keller HH, Goy R, Kane S. Validity and reliability of SCREEN II (Seniors in the community: risk evaluation for eating and nutrition, Version II). Eur J Clin Nutr 2005 Oct;59(10):1149-1157. [CrossRef] [Medline]
    52. Lee J, Song S, Ahn JS, Kim Y, Lee JE. Use of a Mobile Application for Self-Monitoring Dietary Intake: Feasibility Test and an Intervention Study. Nutrients 2017 Jul 13;9(7):748 [FREE Full text] [CrossRef] [Medline]
    53. Chen J, Berkman W, Bardouh M, Ng CYK, Allman-Farinelli M. The use of a food logging app in the naturalistic setting fails to provide accurate measurements of nutrients and poses usability challenges. Nutrition 2019 Jan;57:208-216. [CrossRef] [Medline]
    54. Wellard-Cole L, Chen J, Davies A, Wong A, Huynh S, Rangan A, et al. Relative Validity of the Eat and Track (EaT) Smartphone App for Collection of Dietary Intake Data in 18-to-30-Year Olds. Nutrients 2019 Mar 14;11(3):621 [FREE Full text] [CrossRef] [Medline]
    55. Teixeira V, Voci SM, Mendes-Netto RS, da Silva DG. The relative validity of a food record using the smartphone application MyFitnessPal. Nutr Diet 2018 Apr;75(2):219-225. [CrossRef] [Medline]
    56. Carter MC, Burley VJ, Nykjaer C, Cade JE. 'My Meal Mate' (MMM): validation of the diet measures captured on a smartphone application to facilitate weight loss. Br J Nutr 2013 Feb 14;109(3):539-546. [CrossRef] [Medline]
    57. Ambrosini GL, Hurworth M, Giglia R, Trapp G, Strauss P. Feasibility of a commercial smartphone application for dietary assessment in epidemiological research and comparison with 24-h dietary recalls. Nutr J 2018 Jan 09;17(1):5 [FREE Full text] [CrossRef] [Medline]
    58. Recio-Rodriguez JI, Rodriguez-Martin C, Gonzalez-Sanchez J, Rodriguez-Sanchez E, Martin-Borras C, Martínez-Vizcaino V, EVIDENT Investigators. EVIDENT Smartphone App, a New Method for the Dietary Record: Comparison With a Food Frequency Questionnaire. JMIR Mhealth Uhealth 2019 Feb 08;7(2):e11463 [FREE Full text] [CrossRef] [Medline]
    59. Pendergast FJ, Ridgers ND, Worsley A, McNaughton SA. Evaluation of a smartphone food diary application using objectively measured energy expenditure. Int J Behav Nutr Phys Act 2017 Mar 14;14(1):30 [FREE Full text] [CrossRef] [Medline]
    60. Bucher Della Torre S, Carrard I, Farina E, Danuser B, Kruseman M. Development and Evaluation of e-CA, an Electronic Mobile-Based Food Record. Nutrients 2017 Jan 18;9(1):76. [CrossRef] [Medline]
    61. Mescoloto SB, Caivano S, Domene S. Avaliação de um aplicativo móvel para a estimativa de ingestão alimentar [Evaluation of a mobile application for estimation of food intake]. Revista de Nutrição 2017 Feb;30:91-98 [FREE Full text] [CrossRef]
    62. Shim J, Oh K, Kim HC. Dietary assessment methods in epidemiologic studies. Epidemiol Health 2014;36:e2014009 [FREE Full text] [CrossRef] [Medline]
    63. Rangan AM, Tieleman L, Louie JCY, Tang LM, Hebden L, Roy R, et al. Electronic Dietary Intake Assessment (e-DIA): relative validity of a mobile phone application to measure intake of food groups. Br J Nutr 2016 Jun;115(12):2219-2226. [CrossRef] [Medline]
    64. Kondrup J, Allison SP, Elia M, Vellas B, Plauth M, EducationalClinical Practice Committee‚ European Society of ParenteralEnteral Nutrition (ESPEN). ESPEN guidelines for nutrition screening 2002. Clin Nutr 2003 Aug;22(4):415-421. [CrossRef] [Medline]
    65. Ma C, Cruz C, Eslamparast T, Taylor L, Farhat K, Shommu N, et al. The patient generated subjective global assessment short form is a useful screening tool to detect risk for malnutrition in patients with cirrhosis. Clin Nutr ESPEN 2022 Aug;50:330-333. [CrossRef] [Medline]
    66. De Groot LM, Lee G, Ackerie A, van der Meij BS. Malnutrition Screening and Assessment in the Cancer Care Ambulatory Setting: Mortality Predictability and Validity of the Patient-Generated Subjective Global Assessment Short form (PG-SGA SF) and the GLIM Criteria. Nutrients 2020 Jul 30;12(8):2287. [CrossRef] [Medline]
    67. Trtovac D, Lee J. The Use of Technology in Identifying Hospital Malnutrition: Scoping Review. JMIR Med Inform 2018 Jan 19;6(1):e4 [FREE Full text] [CrossRef] [Medline]

    ‎
    3d-24HRs: 3 days of 24-hour dietary recall
    24HR: 24-hour dietary recall
    ASPEN: American Society for Parenteral and Enteral Nutrition
    CC: correlation coefficient
    CHNS: China Health and Nutrition Surveys
    CSPEN: Chinese Society for Parenteral and Enteral Nutrition
    ESPEN: European Society for Clinical Nutrition and Metabolism
    FFQ: Food Frequency Questionnaire
    MUST: Malnutrition Universal Screening Tool
    NPV: negative predictive value
    NRS-2002: Nutritional Risk Screening
    PG-SGA-SF: Patient-Generated Subjective Global Assessment Short Form

    Edited by A Mavragani; submitted 15.06.22; peer-reviewed by M Paulsen; comments to author 21.07.22; revised version received 22.09.22; accepted 23.09.22; published 26.10.22

    Copyright

    ©Zhiwen Long, Shan Huang, Jie Zhang, Deng Zhang, Jun Yin, Chengyuan He, Qinqiu Zhang, Huilin Xu, Huimin He, Ho Ching Sun, Ke Xie. Originally published in JMIR Formative Research (https://formative.jmir.org), 26.10.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.