This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2226) Web-based and Mobile Health Interventions (3128) Human Factors and Usability Case Studies (748)

Published on in Vol 6, No 4 (2022): April

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/33886, first published .
Message Frame–Tailoring in Digital Health Communication: Intervention Redesign and Usability Testing

Message Frame–Tailoring in Digital Health Communication: Intervention Redesign and Usability Testing

Message Frame–Tailoring in Digital Health Communication: Intervention Redesign and Usability Testing

Authors of this article:

Inge S van Strien-Knippenberg1 Author Orcid Image ;   Maria B Altendorf1 Author Orcid Image ;   Ciska Hoving2 Author Orcid Image ;   Julia C M van Weert1 Author Orcid Image ;   Eline S Smit1 Author Orcid Image
Article Authors Cited by (1) Tweetations (3) Metrics

    Original Paper

    1Department of Communication Science, Amsterdam School of Communication Research, University of Amsterdam, Amsterdam, Netherlands

    2Department of Health Promotion, School for Public Health and Primary Care, Maastricht University, Maastricht, Netherlands

    Corresponding Author:

    Inge S van Strien-Knippenberg, MSc

    Department of Communication Science

    Amsterdam School of Communication Research

    University of Amsterdam

    Nieuwe Achtergracht 166

    Amsterdam, 1018 WV

    Netherlands

    Phone: 31 645139863

    Email: i.vanstrien@amsterdamumc.nl


    Background: Message frame–tailoring based on the need for autonomy is a promising strategy to improve the effectiveness of digital health communication interventions. An example of a digital health communication intervention is Personal Advice in Stopping smoking (PAS), a web-based content-tailored smoking cessation program. PAS was effective in improving cessation success rates, but its effect sizes were small and disappeared after 6 months. Therefore, investigating whether message frame–tailoring based on the individual’s need for autonomy might improve effect rates is worthwhile. However, to our knowledge, this has not been studied previously.

    Objective: To investigate whether adding message frame–tailoring based on the need for autonomy increases the effectiveness of content-tailored interventions, the PAS program was redesigned to incorporate message frame–tailoring also. This paper described the process of redesigning the PAS program to include message frame–tailoring, providing smokers with autonomy-supportive or controlling message frames—depending on their individual need for autonomy. Therefore, we aimed to extend framing theory, tailoring theory, and self-determination theory.

    Methods: Extension of the framing theory, tailoring theory, and self-determination theory by redesigning the PAS program to include message frame–tailoring was conducted in close collaboration with scientific and nonscientific smoking cessation experts (n=10), smokers (n=816), and communication science students (n=19). Various methods were used to redesign the PAS program to include message frame–tailoring with optimal usability: usability testing, think-aloud methodology, heuristic evaluations, and a web-based experiment.

    Results: The most autonomy-supportive and controlling message frames were identified, the cutoff point for the need for autonomy to distinguish between people with high and those with low need for autonomy was determined, and the usability was optimized.

    Conclusions: This resulted in a redesigned digital health communication intervention that included message frame–tailoring and had optimal usability. A detailed description of the redesigning process of the PAS program is provided.

    Trial Registration: Netherlands Trial Register NL6512 (NRT6700); https://www.trialregister.nl/trial/6512

    JMIR Form Res 2022;6(4):e33886

    doi:10.2196/33886

    Keywords

    digital health communication; web-based computer tailoring; smoking cessation intervention; message framing; usability testing 


    Background

    In 2018, a total of 22.4% of the adult Dutch population indicated to be occasional smokers [1]. In 2017, an attempt to quit smoking at least once was undertaken by 35.7% of Dutch smokers; however, most smokers did not succeed in stopping smoking permanently [2]. Therefore, it is important to provide effective interventions to support smokers to stop smoking and prevent the relapse of smoking. For instance, computer-tailored interventions appear to be a promising and cost-effective solution [3-6].

    Computer tailoring refers to an automated communication strategy intended to reach every individual with tailored messages by adapting these messages to the individual’s unique characteristics and his or her behavioral and motivational state [3,7,8]. Generally, computer tailoring focuses on tailoring the content of health communication interventions [9-11]. An example of a web-based computer-tailored intervention is Personal Advice in Stopping smoking (PAS) [12]. This is a computer-tailored smoking cessation program, which provides tailored feedback based on respondents’ answers to web-based questionnaires based on the Integrated-Change (I-Change) Model [13]. The I-Change Model combines insights from several behavior change theories, such as the Transtheoretical Model [14], Theory of Planned Behavior [15], Social Cognitive Theory [16], and the Health Belief Model [17]. As such, the I-Change Model assumes a person’s behavioral intention to be the most proximal predictor of behavior. In turn, this behavioral intention is proposed to be predicted by three motivational constructs: attitude (represented by perceived advantages and disadvantages of the behavior), perceived social influence (composed of perceived social norms, social modeling, and social pressure), and self-efficacy (a person’s perceived ability to perform the behavior). Furthermore, the I-Change Model includes several premotivational factors: predisposing (eg, past behavioral experiences), awareness (eg, knowledge), and information (eg, message source) factors. Finally, the I-Change Model suggests that although a positive behavioral intention is needed for behavior change to occur, several postmotivational factors play a role in bridging the gap between intention and behavior [18], with perceived barriers to change widening this gap and ability factors (eg, skills and the formation of action plans) narrowing this gap.

    Given its theoretical grounding, the PAS program starts with a baseline assessment consisting of questions on sociodemographic characteristics, smoking behavior, and addiction level, but, then, continues to ask questions about the respondent’s intention to quit smoking, attitude, social influence, self-efficacy beliefs, and action and coping planning on how to stop smoking and prevent the relapse of smoking [12]. Then, the PAS participants receive tailored feedback messages based on their answers [12]. Questions and feedback messages alternate, meaning that, participants answer a question—or a set of related questions—and immediately receive the feedback message associated with their answer and, then, receive the next question. At the end of the program, participants receive an overview of their tailored feedback messages, which can be printed and sent to them through email if desired. A detailed description is provided elsewhere [12,19].

    An investigation of the effects of PAS among Dutch smokers indicated a significant effect on smoking abstinence, reported 6 weeks after the initiation of participation in the program. However, 6 months after participation initiation, significant intervention effects could no longer be found [19]. In other web-based computer-tailored interventions to promote health behavior, intervention effects also declined after intervention completion [20]. Therefore, and because overall effect sizes remain small [3,20], it is worth exploring additional strategies that might enhance intervention effects. A proposed strategy to improve the effectiveness in addition to tailoring what information is presented (ie, content tailoring), is tailoring how this information is presented, which is known as message frame–tailoring [10].

    Message frame–tailoring refers to an integration of the message frame theory [21] and theory around tailoring of health information [22]. According to the message frame theory, the manner in which information is presented is referred to as framing of information, which contains 2 aspects: selection and salience [21]. Selection refers to choosing several aspects to communicate, and salience is defined as “making a piece of information more noticeable, meaningful, or memorable” [21]. Selecting elements to communicate and making them more (or less) salient to the audience will capture the audience’s attention and, as a result, make the information more likely to be read and processed. This is where tailoring theory comes in, as tailoring of information is also done to increase the likelihood that information is read, processed, remembered, and acted upon. Tailoring is “any combination of information or change strategies intended to reach one specific person and increase the personal relevance of the information for this particular person, based on characteristics that are unique to that person, related to the outcome of interest, and derived from an individual assessment” [8]. Relevance of the presented information for an individual is a key concept in this definition, as personally relevant information may lead to more in-depth information processing [22]. Combining message frame theory with theory around tailoring leads us to the definition of the main concept described in this paper, that is, message frame–tailoring, which is “adjusting the perspective when formulating a message based on people’s individual needs” [10]. In this study, message frames were tailored to respondents’ need for autonomy, a theoretical concept derived from self-determination theory (SDT) [23]. Tailoring health messages to respondents’ need for autonomy is important because health messages should not only be read and considered as personally relevant but, above all, should encourage behavior change by increasing the recipients’ autonomous motivation for change. According to SDT, autonomous motivation plays an important role in enhancing the initiation and maintenance of health-related behavior [24]. To be autonomously motivated, people need to experience a sense of volition and freedom to make their own choices. On the other hand, people are likely to not feel autonomously motivated when they feel controlled and experience pressure to think, feel, or behave in a manner that has been imposed on them [25]. In SDT, this prerequisite for autonomous motivation is called fulfilling the need for autonomy [23]. However, according to SDT, individual differences in the need for autonomy exist regarding health and health-related decision-making. For example, some people prefer to choose their own path toward lifestyle improvement, whereas others prefer to be guided by clear-cut expert advice [26,27]. To meet these individual differences in the need for autonomy, the manner of information provision in health messages can be tailored based on this need, to accommodate an individual’s associated processing and communication style preferences and further increase the personal relevance of the message [10].

    In our operationalization, message frame–tailoring based on the need for autonomy consists of 2 components: language (ie, autonomy-supportive or controlling) and choice (ie, provision of choice or no choice) [28,29]. Autonomy-supportive language, or noncontrolling language, is defined as “a vocalization that would allow choice or support self-initiation” [30] and can be used for individuals with high need for autonomy. Autonomy-supportive language encourages individuals to take responsibility for their own behavior and make decisions based on their own values [28,30]. This involves minimizing pressure [28] and using words such as could, might, and would. Controlling language is defined as “a vocalization to pressure a person to behave (or think or feel) a particular way” [30] and can be used for individuals with low need for autonomy. Controlling language is characterized by the use of commands, orders, imperatives, and suggestive questions [30]. Examples are words such as must, should, and ought. Choice is operationalized as offering participants the option to receive additional information about a certain topic [31]. Tailoring message frames to a person’s need for autonomy may enhance the motivation to change and maintain health-related behavior. As such, message frame–tailoring based on the need for autonomy has been suggested as a promising avenue to explore further [10,26,27,32]. However, to date, to the best of our knowledge, no health-related computer-tailored intervention has been reported that incorporates message frame–tailoring based on the need for autonomy, except the study by Smit et al [10].

    Objective

    To investigate whether adding message frame–tailoring based on the need for autonomy increases the effectiveness of content-tailored interventions, the PAS program [12] was redesigned to incorporate message frame–tailoring. This paper describes the process of extending framing theory, tailoring theory, and SDT by redesigning the PAS program to include message frame–tailoring. The redesigning process was conducted in close collaboration with communication science students, experts, and the actual end users (ie, smokers).


    Overview

    The process of redesigning the original PAS program to incorporate message frame–tailoring consisted of four steps: (1) the feedback messages were rewritten in an autonomy-supportive and a controlling manner and a first pilot test was conducted, (2) the message frames were integrated into the program and a second pilot test was conducted, (3) a web-based experiment with a 2×2 between-participants design with a control condition was conducted, and (4) message frame–tailoring was integrated into the program and a usability test was conducted. An overview of the steps is presented in Figure 1, and the 4 steps are described in detail in subsequent sections. OverNite Software Europe provided technical support and usability advice during the redesign process.

    The study was registered in the Netherlands Trial Register (NL6512/NRT-6700).

    ‎
    Figure 1. Overview of the redesign process. HCOS: Health Causality Orientations Scale; PAS: Personal Advice in Stopping smoking.
    View this figure

    Rewrite the Feedback Messages and Conduct a First Pilot Test

    To tailor message frames based on the need for autonomy, the message frames needed to be developed. The feedback messages of the original PAS program were rewritten using autonomy-supportive and controlling language strategies. To test whether the language strategies were perceived as such, a pilot test was conducted among communication science students. After consenting to participate, students were randomly assigned to either evaluate the autonomy-supportive or controlling messages. The students received a questionnaire with 5 messages to be reviewed using a 5-point response scale, ranging from very autonomy-supportive to very controlling [33]. Students were also able to write comments on each message in a textbox. A total of 19 students participated (mean age 23.2, SD 1.44 years; 2/19, 11% were men; 15/19, 79% were nonsmokers). Of the 19 students, 10 (53%) students received the autonomy-supportive messages and 8 (42%) students received the controlling messages. Quantitative results were analyzed using an independent sample t test (2-tailed) using SPSS (IBM Corp), and the qualitative comments were inspected to identify relevant comments regarding language use.

    As the 5-point scale ranged from very autonomy-supportive (rating=1) to very controlling (rating=5), we expected lower scores for the autonomy-supportive messages than the controlling messages. Results showed significantly lower scores for the autonomy-supportive messages (mean 2.46, SD 0.62; P<.001) than the controlling messages (mean 3.68, SD 0.32), which means that participants detected the language manipulation as intended. However, based on participants’ comments and inspection of the differences between messages that had high or low scores and those that had average scores, we further emphasized communication style (autonomy-supportive vs controlling). The main change consisted of emphasizing with signal words on the type of language style—autonomy-supportive or controlling—used at the beginning of each message (eg, starting the message with “you must...” in a controlling message).

    Subsequently, the second message frame element, choice, was added. The following question was included 6 times throughout the program: “Do you want to receive these tips?” which could be answered with “yes” or “no.” In addition, participants were asked if they wanted to choose a quit date themselves. Furthermore, participants could choose whether and for which potential difficult situations they wanted to formulate coping plans to refrain from smoking in these situations.

    Integrate Message Frames in the Program and Conduct a Second Pilot Test

    Adding message frames (autonomy-supportive vs controlling language and provision of choice vs no provision of choice) to the feedback messages of the original PAS program resulted in four versions of the program: (1) autonomy-supportive language and no choice, (2) autonomy-supportive language and choice, (3) controlling language and no choice, and (4) controlling language and choice. Besides the 4 message frames, 1 control condition was added to the program, consisting of generic smoking cessation advice with a neutral message frame (ie, the message frame was not manipulated and was the same as in the original smoking cessation intervention) and no content tailoring (ie, the messages were not tailored based on participants’ answers). Following is an example of a control message:

    When smokers are stressed, tensed, or dreary, they often find it difficult not to smoke. To help you with this, we give you a few tips. If you find yourself feeling emotional and wanting a cigarette, it’s good to do something else. You can take a walk: the outside air may do you good. Or you can exercise. This helps you to change your mind and reduces the desire for a cigarette.

    Examples of feedback messages for the 4 message frames are presented in Table 1. For more examples, refer to the study by Altendorf et al [31].

    The next step in the redesign process was conducting a second pilot test to investigate the intervention from experts’ and smokers’ perspectives. Totally, 5 experts (4/5, 80% men) and 11 smokers (8/11, 73% men) participated in the usability test and the subsequent interviews. The experts were working as professionals or researchers in the fields of health communication, public health, eHealth intervention development, and smoking cessation support. First, participants received a link to the website and were instructed to complete the program, while paying attention to the comprehensibility of the program and the feedback messages. Subsequently, a researcher interviewed the participants on the following topics: time taken to conduct the program; comprehensibility of instructions, questions, and answer categories; and awareness of the message frame used. Participants were rewarded with a shopping voucher worth €10 (US $11). During the usability tests and the interviews, the researcher took notes. Results were discussed within the research team to determine whether changes to the program were required to resolve issues mentioned in the feedback.

    The participants took between 20-60 minutes (average 35 minutes) to complete the program. As the introduction was perceived as very long, it was shortened. In addition, some questions seemed to be unclear; therefore, a brief instruction was added. Some questions needed to be reformulated (eg, using more familiar words and not using double negatives). In addition, participants wished to receive the summary of feedback messages by email; thus, this option was added. The message frames were generally perceived as intended; therefore, no modifications were made regarding this aspect. After incorporating these changes, the PAS program with message frames was ready for use in the next step.

    Table 1. Examples of feedback messages for the different message frames.
    Condition and processAutonomy-supportive languageControlling language
    No choice

    		MessageYou answered that you will succeed in not smoking if you are stressed, tensed, or dreary. You doubt whether you will succeed in not smoking when you are angry. We would like to offer you some tips. If you notice that you are emotional and would like to have a cigarette, you can try to do something else. For example, you could take a walk: the outside air might do you good. Or you could exercise. This might help you to change your mind and it reduces the desire for a cigarette.You think that you will succeed in not smoking if you are stressed, tensed, or dreary. You doubt whether you will succeed in not smoking when you are angry. You need to do something else when you are emotional and want a cigarette. Take a walk: the outside air often works well. Or go exercise: this should help to change your mind and reduces the desire for a cigarette.
    Choice

    		Message 1You answered that you will succeed in not smoking if you are stressed, tensed, or dreary. You doubt whether you will succeed in not smoking when you are angry. Below, you can choose whether you would like to receive some tips on what you can do when you are emotional and would like to have a cigarette.You think that you will succeed in not smoking if you are stressed, tensed, or dreary. You doubt whether you will succeed in not smoking when you are angry. Below, you can choose whether you would like to receive some tips on what you can do when you are emotional and would like to have a cigarette.

    		Choice questionaDo you want to receive these tips?Do you want to receive these tips?

    		AnswerYesYes

    		Message 2If you notice that you are emotional and would like to have a cigarette, you can try to do something else. For example, you could take a walk: the outside air might do you good. Or you could exercise. This might help you to change your mind and it reduces the desire for a cigarette.You need to do something else when you are emotional and want a cigarette. Take a walk: the outside air often works well. Or go exercise: this should help to change your mind and reduces the desire for a cigarette.

    aIf respondents answered “no” to the choice question, the intervention continued without the provision of the tips.

    Conduct a 2×2 Web-Based Experiment With a Control Condition

    To tailor message frames based on individuals’ need for autonomy, we had to identify (1) the most autonomy-supportive and the most controlling message frame based on participants’ perceived autonomy support and (2) the cutoff point to distinguish between people with high need for autonomy and those with low need for autonomy. Therefore, a web-based experiment with a 2×2 (language: autonomy-supportive vs controlling; choice vs no choice) between-participants design with a control condition was conducted. An extensive description of the method and results of this experiment is provided elsewhere [31]. However, we summarize the most important outcomes below.

    Participants’ need for autonomy was measured using the Health Causality Orientations Scale (HCOS), consisting of 4 vignettes, each of which contained 3 items that could be answered on a 5-point Likert scale (1=low need for autonomy; 5=high need for autonomy) [34]. The primary outcome measure was participants’ perceived autonomy support, which was measured with the Virtual Climate Care Questionnaire, consisting of 15 items that can be answered on a 7-point Likert scale (1=low perceived autonomy support; 7=high perceived autonomy support) [35]. Therefore, the HCOS measured participants’ general need for autonomy and the Virtual Climate Care Questionnaire measured how the participants perceived the messages in the PAS program; that is, whether they perceived them as autonomy-supporting.

    Results showed that participants’ perceived autonomy support was generally high (mean 3.46, SD 0.78). On the basis of the results, it was not possible to identify the most autonomy-supportive and the most controlling message frames. Therefore, we decided to follow SDT principles, according to which the provision of choice and the use of autonomy-supportive language enhances an individual’s autonomy [23]. Additional inspection of the data supported this decision, as it appeared from the evaluative comments that especially participants with high need for autonomy mentioned that the word must was salient in the advice and not appreciated. Furthermore, especially participants with high need for autonomy chose to receive more information when offered choice and wanted to choose a quit date themselves more often than participants with low need for autonomy.

    Regarding participants’ need for autonomy, the results also showed a high overall score (mean 3.87, SD 0.76). The Johnson-Neyman procedure was followed to identify the need for autonomy score at which the three-way interaction of the need for autonomy, language use, and provision of choice on perceived autonomy support changed from insignificant to significant, resulting in a range of 3.8 to 4.4. The qualitative data showed that participants with need for autonomy ≥3.8 more often made unappreciative comments about not being able to set a quit date themselves and about the word must used in their feedback messages. On the basis of these results, 3.8 was chosen as the cutoff point of the HCOS to distinguish between individuals with high need for autonomy and those with low need for autonomy.

    On the basis of the evaluative comments from participants, some changes were implemented. First, the participants perceived a period of 2 weeks to set a quit date as very soon. In the original PAS program, the quit date was set 4 weeks after participation initiation, but based on results that showed the importance of immediate action when a quit attempt was considered [36,37], it was decided to shorten the period to 2 weeks. To clarify this to the participants, a brief explanation about the benefits of setting a quit date within a short period was added. Participants in the no choice conditions were still given a quit date within 2 weeks, whereas participants in the choice conditions were offered an additional question of whether they wanted to choose a quit date within 2 weeks. On the basis of their answer (“yes,” “no,” or “I am not sure yet”), the participants would see a different calendar to set a quit date (ie, a calendar with dates within a time span of 2 weeks, a calendar with dates with a time span of >2 weeks, or a commenting section to enter a date). Second, participants wished to comment on their former quit attempts. Therefore, a question was added about the number of previous quit attempts, followed by a feedback message in which the participants were asked to note why former quit attempts did not succeed and what they have learned from this. Third, participants wanted to receive more information about smoking cessation support tools (eg, medication or nicotine patches), the costs of smoking, and the use of replacing smoking with other oral stimuli (eg, chewing sunflower seeds). New feedback messages regarding these topics were added. Fourth, it was noted that the program was not suitable for a tablet or mobile phone; therefore, the program was made compatible with different devices. This included automatic adjustments to the screen size, alignment of the buttons (eg, forward and print) and pictures, and centering error messages. Finally, as the participants seemed to have difficulties in answering questions that were presented in matrices, all questions were presented as separate questions.

    Integrate Message Frame–Tailoring in the Program and Conduct a Usability Test

    On the basis of the results of the web-based experiment as described in the previous step, the PAS program was developed into a program with message frame–tailoring. The message frame for participants with high need for autonomy (HCOS score ≥3.8) consisted of feedback messages with autonomy-supportive language and choice, and the message frame for participants with low need for autonomy (HCOS score <3.8) consisted of feedback messages with controlling language and no choice. The flow of the redesigned PAS program with content tailoring and message frame–tailoring is presented in Figure 2.

    To optimize the PAS program with the tailored message frames, a usability test among experts and smokers was conducted. Regarding user experience, it is argued that, besides the aspect of ease of use, perceived usefulness is also important, which refers to the extent to which the content of the program satisfies the information needs of the participant [38]. Therefore, we aimed to not only identify problems with the use of the program but also to test the content of the program from experts’ and smokers’ perspectives. Experts were working as professionals or researchers in the fields of health communication, public health, eHealth intervention development, and smoking cessation support.

    Both smokers and experts were asked to evaluate the program using the think-aloud method, based on which they were asked to verbalize their thoughts while using the program [39]. As verbalizing one’s thoughts while conducting a task is not common, the participants first received a brief task to practice. After completing the usability testing, a semistructured interview was conducted with the smokers to evaluate the program and further explore the problems mentioned. Examples of questions are, “What do you think of the layout of the program?” and “Which are the two most important elements that could be improved?” To identify 80% to 90% of the user issues, 5 to 9 end users were needed [40,41]. The semistructured interviews with experts were based on the heuristic evaluation method [39,42]. The heuristic evaluation method consists of ten heuristic principles, such as simple and natural dialogue, speak the user’s language, consistency, and good error messages [42]. To identify most problems by using the heuristic evaluation method, 3-5 experts were needed [39].

    As the program will be used on different devices, smokers and experts were instructed to use the program on a laptop, tablet, or mobile phone. The entire procedure took approximately 1 hour to complete, and the participants were rewarded with a shopping voucher worth €25 (US $27.5). Experts and smokers were recruited through the network of the research team. A total of 5 experts (2/5, 40% were men; 4/5, 80% were nonsmokers) and 7 smokers (mean age 31.86, SD 14.15 years; range 20-62 years; 4/7, 57% were men) participated in the usability test. All sessions were video-recorded, which were selectively transcribed and coded using ATLAS.ti 8.0 [43]. The data were analyzed according to the content analysis approach [44]. Deductive content analysis allows us to go beyond general findings and validate theories and models that guided the research. Inductive content analysis allows for inclusion of codes and categories that are derived from the data in an iterative process [45]. First, codes were derived using the heuristic evaluation method [39,42]. Then, the transcripts were thoroughly read and coded. In the next step, categories were identified and ordered into themes that represented the answers to the research questions. When new themes emerged from the collected data, they were added to the analysis [45]. Initially, 2 transcripts were coded independently by 2 coders (ISvSK and MBA). Disagreements were resolved through discussion, and after reaching consensus, 1 coder continued coding the remaining transcripts (ISvSK).

    ‎
    Figure 2. Flowchart of the redesigned Personal Advice in Stopping smoking program that incorporates both content tailoring and message frame–tailoring based on the need for autonomy. HCOS: Health Causality Orientations Scale.
    View this figure

    As we aimed to test the content of the program and identify problems with its use, user experience was divided into 2 dimensions: the content of the program and the usability of the program. Modifications that were made regarding the content of the program included the following: replacing difficult words, shortening the introduction and instructions, reformulating questions, changing the labels of answer categories, and adding the quit date to the summary. Modifications that were made regarding the usability of the program included the following: adding a sidebar to show the different parts of the program, modernizing the logo and the colors, displaying only the core messages in bold font, aligning the illustrations on a mobile phone, adding a red frame signaling an unanswered question on the mobile phone, and adding the option to directly add the quit date to a personal agenda. An overview of the codes, subcodes, definitions, examples, and modifications is presented in Table 2.

    It is worth noting that participants also wished to receive feedback messages with more illustrations and that some participants wished for shorter feedback messages with more bullet points. However, as the original feedback messages were effective in supporting smokers to quit smoking [19] and making large modifications to this content would mean retesting the effectiveness of the feedback messages, it was decided to stay as close to the original feedback message format as possible.

    Table 2. Overview of the codes, subcodes, definitions, examples, and modifications of the usability test.
    Code and subcodeDefinitionExampleModifications
    1–Language use

    		Level and styleWords, concepts, or sentence structures that are unclear or ambiguous. In addition, the use of formal language.“I don’t even know what this is.” [smoker]Difficult words were replaced or explained. Spelling mistakes were corrected.

    		Amount of informationThe amount of information provided in the program, especially in the instruction, introduction, and informed consent.“Uhm...well...it’s a lot of text, I know that a lot of low educated people really hate a lot of text.” [expert]The introduction and instruction were shortened.
    2–Questions and answers

    		Number and content of questionsThe number, relevance, understandability, and topics of the questions in the program.“I have to read this sentence four times [...] I find it hard to follow. So, I don’t know what I have to answer.” [expert]Questions were reformulated.

    		Answer categoriesThe answer possibilities, way of answering, and unclarity and inconsistencies regarding scale questions.“Maybe you also need some kind of range here, don’t you? Never, uhm, a few times, very often or I try to do it regularly.” [smoker]The labels of some answer categories were changed. Some answer options were changed from a textbox to a dropdown menu.
    3–Feedback messages

    		Content and numberThe number, length, topics, and content of the feedback messages.“This is good, good arguments here, especially at the end. With time and money, that is good.” [smoker]A short explanation of the summary of the feedback messages was added. In addition, the quit date was added to the summary.

    		Credibility and relevanceThe concreteness, reliability, and credibility of the feedback messages and whether the messages are perceived as true and suitable to support in quitting smoking.“They are all things that I have already read and heard, but it is all true.” [smoker]No modifications were made regarding this aspect.

    		IllustrationsThe support of illustrations to the text and the number of illustrations.“I do indeed see here a picture of someone who gives some kind of support, so that does support the text.” [expert]No modifications were made regarding this aspect.
    4–Tailoring

    		Content tailoringThe relevancy of the content for a particular participant.“I think it’s good that for each answer category a story is told what really applies.” [smoker]No modifications were made regarding this aspect.

    		Message framingThe use of autonomy-supportive or controlling language and the provision of choice.“You must, there is a lot ‘must.’ I don’t feel like quitting smoking anymore.” [expert]No modifications were made regarding this aspect.
    5–Program structure

    		StructureThe order and structure of the different parts of the program and whether the order is clear. In addition, the feedback that a participant receives via the progress bar and whether it is possible to return to the program.“Step 3. [...] I don’t remember so well that I saw step 2, but those steps are a bit out of the blue. Maybe I should be taken a little more by the hand.” [expert]To make the structure of the program clearer, a sidebar was added, which continuously shows the different parts of the program throughout the completion process, including which part a participant is working on. In addition, the titles of the different parts were made larger.

    		InstructionsThe unclarity of content of the instructions, especially concerning how to complete the program and how to answer the questions.“How to complete the questionnaire. [...] Well, all basic things.” [smoker]No modifications were made regarding this aspect.

    		DurationThe duration of the program and time required to complete the program.“It is long, so maybe people will get bored at some point.” [expert]No modifications were made regarding this aspect.
    6–Layout

    		DesignThe font size and style, colors, logos, buttons, and illustrations.“It is very straightforward, so well...there is little...it will be made more beautiful? It is not very inviting.” [smoker]A modern logo was developed and modern colors were used. In addition, the buttons were changed into modern icons. As the print button seemed to appear too often, this was resolved; some inconsistencies in the font size were also resolved.

    		ReadabilityThe readability of the text, with regard to the layout of the pages, the feedback messages, the summary of the feedback messages, and sentence alignment.“Shorter sentences, only graphic now in terms of location, you read more easily. This sentence is easier to read than, for example, this sentence, whereby your eye has to go all the way from left to right.” [smoker]The alignment of the sentences was modified. Furthermore, it seemed unclear which information was displayed in bold font; thus, it was decided to display only the core message in bold font.

    		Layout on various devicesThe layout of the program on the various devices (ie, laptop, mobile phone, and tablet).Someone with a mobile phone: “There is also a picture next to it, that makes the text a bit narrower.” [expert]Changes were made to the alignment of the illustrations on mobile phones.
    7–Error messages

    		Time of appearanceWhether the error messages appear at the right time.“I have to give an answer, so that works.” [expert]A red frame signaling an unanswered question was added on mobile phones.

    		ContentThe content of the error messages and whether they are perceived as helpful.“It was clear that it was about the age question. It was not mentioned that it was about the age question, but I think that’s also not possible in an error message.” [expert]No modifications were made regarding this aspect.
    8–Features

    		CalendarThe appearance and ease of use of the calendar.“You can also place a link here, or something like that so that it will be added to your agenda automatically.” [smoker]The option to directly add the quit date to a personal agenda was added.

    		Print or emailThe option to print or email the summary of the feedback messages and whether this works. In addition, the layout of the printed summary.“I like that I can print it because it’s a lot to read at once.” [expert]No modifications were made regarding this aspect.

    After incorporating the modifications in response to the usability test, the redesign process was completed. The redesigned PAS program—including message frame–tailoring—is shown in Figures 3 and 4. For a comparison with the original interface, refer to the study by Smit et al [19]. This redesigned PAS program has been evaluated in a real-life setting for effectiveness and cost-effectiveness; results of the effectiveness evaluation were recently published elsewhere [46], and the results of the cost-effectiveness evaluation are currently being prepared for publication.

    ‎
    Figure 3. Screenshot that shows the colors, logo, and sidebar of the redesigned Personal Advice in Stopping smoking program.
    View this figure
    ‎
    Figure 4. Screenshot that shows the print button and enlarged titles of the redesigned Personal Advice in Stopping smoking program.
    View this figure

    Ethics Approval

    This study was approved by the institutional review board of the University of Amsterdam (reference number 2017-PC-7599).


    Principal Findings

    This study aimed to extend the message framing theory, tailoring theory, and SDT by including message frame–tailoring in a digital health communication intervention. In addition to tailoring what information is presented (ie, content tailoring), we wanted to improve the effectiveness of a digital health communication intervention by tailoring how this information is presented (ie, message frame–tailoring). This paper offers a detailed description of the incorporation of message frame–tailoring in a web-based computer-tailored smoking cessation program, PAS. The PAS is taken as an example to explore the effects of—in addition to the known effects of content tailoring—the promising strategy of message frame–tailoring [10]. The process of incorporating message frame–tailoring was conducted in close collaboration with experts and smokers, and a usability test was conducted to optimize the redesigned program. An extensive description of the various methods through which experts’ and smokers’ opinions were included throughout the redesigning process is provided. If the redesigned PAS program is more effective in supporting smokers in their quit attempt than the original PAS program, adding message frame–tailoring to other digital health communication interventions also might be of value.

    Potential Strengths

    This study has several potential strengths. First, to the best of our knowledge, this study is the first to explore message frame–tailoring in web-based computer-tailored health communication. The existing literature on message framing mostly includes studies on gain and loss framing, focusing on positive (health) outcomes and costs in terms of health loss, respectively. Regarding tailoring, so far, scholars have mainly tailored the content of web-based interventions; that is, adjusting what health information is provided based on individuals’ current health behavior or their self-reported scores on known predictors of the desired health behavior (change) [9]. To build on tailoring theory, advance the strategy of web-based computer tailoring, and further increase its effectiveness, it has been argued that tailoring how information is provided, in addition to tailoring what information is given, appears to be a promising strategy [10,32]. To our knowledge, message frame–tailoring based on the need for autonomy and operationalized as differences in the provision of choice and the use of autonomy-supportive or controlling language has not been studied previously. By developing a redesigned PAS program including message frame–tailoring, we were able to test whether message frame–tailoring increases the effectiveness of the PAS program. The results of the effect evaluation were published elsewhere [46]. These results indicated that message frame–tailoring based on the need for autonomy can be an effective additional strategy to include in the computer-tailored interventions in addition to content tailoring, but only for people with high need for autonomy and not for people with low need for autonomy. Regarding cost-effectiveness, the combination of message frame–tailoring and content tailoring in web-based smoking cessation programs seems to have high potential for both cost-effectiveness (ie, considering smoking abstinence as the outcome of interest) and cost-utility (ie, considering quality of life as the outcome), thereby providing good value for money. However, when the willingness to pay for each abstinent smoker becomes high (ie, ≥€5000 [≥US $5500]), the addition of message frame–tailoring might not be worth the effort and only content tailoring is preferred. A manuscript reporting on the results of this economic evaluation is currently in the preparation phase. Given these results, it is plausible that adding message frame–tailoring to other web-based computer-tailored or digital health communication interventions will also increase their effectiveness and cost-effectiveness, at least for people with high need for autonomy. This is important because the overall effect sizes of such interventions are generally positive but small [3], limited evidence suggests that such interventions may be cost-effective [4-6], and the internet is highly accessible [47,48]. This paper provides detailed insight into how to integrate this novel tailoring strategy with more traditional tailoring efforts. Furthermore, the application of message frame–tailoring can support the offering of person-centered care by providing feedback in a manner that matches the individual’s preferences and needs, which is one of the key characteristics of person-centered care [49,50]. Moving beyond the tailoring of what information is provided to additionally tailoring how this information is provided takes into account individual differences in communication style preferences. As such, message frame–tailoring can be considered as the next step in the field of tailored and person-centered health communication. Self-management support interventions, such as the PAS program, have shown to be the most frequent category of interventions with the potential to result in positive health impact for patients with chronic diseases [51]. Therefore, continuation of (research) efforts aimed at the development of improved versions of such programs is warranted.

    Another strength concerns the approach taken in developing the redesigned PAS program: scientific and nonscientific experts, smokers, and communication science students were actively involved throughout the process. Involving different types of stakeholders, including end users, in research removes barriers regarding the implementation of research findings and interventions, such as the redesigned PAS program, in practice [52,53]. By describing the process of involving participants in redesigning the PAS program and the valuable insights we have gained as a result, we aim to encourage researchers to involve participants in the development and redesign process of digital health communication interventions.

    Finally, we chose to use an existing, already effective intervention [19]. Using an existing intervention as a starting point and adding an additional tailoring strategy to it is a cost-effective and efficient approach because the intervention does not have to be developed from scratch and the time and money that has already been invested are optimally used.

    Potential Limitations

    This study also has few limitations. First, in the 2×2 web-based experiment, most participants’ need for autonomy was moderately high. This may not be representative of the need for autonomy in the general population; therefore, the cutoff point of the need for autonomy, based on which the message frame is tailored, might be very high, resulting in suboptimal message frame–tailoring. In addition, research has shown that the need for autonomy might be different for different groups of people [46]. More research is needed to assess the optimum cutoff point for the need for autonomy and to test whether this point differs between subgroups.

    Second, we mainly focused on the communication of information in an autonomy-supportive or controlling manner, especially pertaining to the type of words used; for example, could in the autonomy-supportive condition and must in the controlling condition. Moreover, in the case of the quit date, we also tried to design the way of responding in an autonomy-supportive versus controlling manner; that is, the smokers in the choice conditions received different types of calendars to enter their quit date based on their individual choices in this regard; for example, a calendar to choose a date from or a commenting section in which they could enter the date themselves, whereas smokers in the no-choice conditions were given a prespecified date by the intervention. This method of designing could be further explored as a way to make the interface more compatible with the individual’s need for autonomy. For instance, more questions can be designed such that individuals with high need for autonomy would receive open entry fields, whereas individuals with low need for autonomy would receive limited multiple-choice answers. In addition, how the program is run can be adapted to an individual’s need for autonomy; for example, a fixed order for those with low need for autonomy and a self-determined order for those with high need for autonomy. More attention to the design of the intervention as tailored to an individual’s need for autonomy could possibly further increase the effects of this and similar interventions—therefore, such possibilities warrant more attention in future research.

    Third, although using an existing, already effective intervention is a cost-effective and efficient approach, it also has limitations. For example, some participants requested shorter feedback messages with more bullet points or more illustrations. However, as we wanted to be able to make a comparison between the effectiveness of the PAS program with and that without message frame–tailoring, it was decided to make no adaptations regarding these aspects. By changing the content of the program, it would not be possible to attribute any differences in effectiveness between the original PAS program and the redesigned PAS program to the effect of message frame–tailoring. When choosing between developing a new intervention from scratch and redesigning an existing effective intervention, we recommend considering both the cost-effectiveness and efficiency of the approaches and the potential limitations of the approaches, such as (not) being able to meet all participants’ needs regarding the usability of the program.

    Acknowledgments

    The authors would like to thank OverNite Software Europe for the technical support and usability advice during the redesign of the Personal Advice in Stopping smoking program. The authors would also like to thank Fairouz Kasri for transcribing the video recordings. They also thank all the smokers, students, and experts who participated in the redesign process. This study was supported by the Dutch Cancer Society (KWF Kankerbestrijding; grant number UVA 2015-7913). The Dutch Cancer Society was not involved in the study design, data collection, data analysis, or report writing.

    Conflicts of Interest

    None declared.

    1. Roken | De Staat van Volksgezondheid en Zorg. Centraal Bureau voor de Statistiek and Rijksinstituut voor Volksgezondheid en Milieu. 2018.   URL: https://www.staatvenz.nl/kerncijfers/roken [accessed 2019-03-19]
    2. Springvloet L, Bommelé J, Willemsen M, van Laar M. Kerncijfers roken 2017. National Expertisecentrum Tabaksonmoediging. 2018.   URL: https://www.trimbos.nl/docs/cdade14e-20dc-462a-9f63-1b71ee995a84.pdf [accessed 2019-03-20]
    3. Lustria ML, Noar SM, Cortese J, Van Stee SK, Glueckauf RL, Lee J. A meta-analysis of web-delivered tailored health behavior change interventions. J Health Commun 2013;18(9):1039-1069. [CrossRef] [Medline]
    4. Schulz DN, Smit ES, Stanczyk NE, Kremers SP, de Vries H, Evers SM. Economic evaluation of a web-based tailored lifestyle intervention for adults: findings regarding cost-effectiveness and cost-utility from a randomized controlled trial. J Med Internet Res 2014 Mar 20;16(3):e91 [FREE Full text] [CrossRef] [Medline]
    5. Smit ES, Evers SM, de Vries H, Hoving C. Cost-effectiveness and cost-utility of Internet-based computer tailoring for smoking cessation. J Med Internet Res 2013 Mar 14;15(3):e57 [FREE Full text] [CrossRef] [Medline]
    6. Stanczyk NE, Smit ES, Schulz DN, de Vries H, Bolman C, Muris JW, et al. An economic evaluation of a video- and text-based computer-tailored intervention for smoking cessation: a cost-effectiveness and cost-utility analysis of a randomized controlled trial. PLoS One 2014;9(10):e110117 [FREE Full text] [CrossRef] [Medline]
    7. de Vries H, Brug J. Computer-tailored interventions motivating people to adopt health promoting behaviours: introduction to a new approach. Patient Educ Couns 1999 Feb;36(2):99-105. [CrossRef] [Medline]
    8. Kreuter MW, Skinner CS. Tailoring: what's in a name? Health Educ Res 2000 Feb;15(1):1-4. [CrossRef] [Medline]
    9. Rimer BK, Kreuter MW. Advancing tailored health communication: a persuasion and message effects perspective. J Commun 2006 Aug;56(s1):S184-S201. [CrossRef]
    10. Smit ES, Linn AJ, van Weert JC. Taking online computer-tailoring forward. Eur Heal Psychol 2015;17(1):25-31.
    11. Williams-Piehota P, Schneider TR, Pizarro J, Mowad L, Salovey P. Matching health messages to information-processing styles: need for cognition and mammography utilization. Health Commun 2003;15(4):375-392. [CrossRef] [Medline]
    12. Smit ES, de Vries H, Hoving C. The PAS study: a randomized controlled trial evaluating the effectiveness of a web-based multiple tailored smoking cessation programme and tailored counselling by practice nurses. Contemp Clin Trials 2010 May;31(3):251-258. [CrossRef] [Medline]
    13. de Vries H, Mudde A, Kremers S, Wetzels J, Uiters E, Ariza C, et al. The European Smoking Prevention Framework Approach (ESFA): short-term effects. Health Educ Res 2003 Dec;18(6):649-677. [CrossRef] [Medline]
    14. Prochaska JO, DiClemente CC. Stages and processes of self-change of smoking: toward an integrative model of change. J Consult Clin Psychol 1983;51(3):390-395. [CrossRef]
    15. Ajzen I. From intentions to actions: a theory of planned behavior. In: Kuhl J, Beckmann J, editors. Action control: from cognition to behavior. Berlin, Heidelberg: Springer; 1985:11-39.
    16. Bandura A. Social foundations of thought and action: a social cognitive theory. Hoboken, NJ, USA: Prentice-Hall; 1986.
    17. Janz NK, Champion VL, Strecher VJ. The health belief model. In: Glanz K, Rimer BK, Lewis FM, editors. Health behavior and health education: theory, research, and practice. San Francisco, CA, USA: Jossey-Bass; 2002:45-65.
    18. Armitage CJ, Conner M. Efficacy of the Theory of Planned Behaviour: a meta-analytic review. Br J Soc Psychol 2001 Dec;40(Pt 4):471-499. [CrossRef] [Medline]
    19. Smit ES, de Vries H, Hoving C. Effectiveness of a web-based multiple tailored smoking cessation program: a randomized controlled trial among Dutch adult smokers. J Med Internet Res 2012 Jun 11;14(3):e82 [FREE Full text] [CrossRef] [Medline]
    20. Krebs P, Prochaska JO, Rossi JS. A meta-analysis of computer-tailored interventions for health behavior change. Prev Med 2010;51(3-4):214-221 [FREE Full text] [CrossRef] [Medline]
    21. Entman RM. Framing: toward clarification of a fractured paradigm. J Commun 1993 Dec;43(4):51-58 [FREE Full text] [CrossRef]
    22. Kreuter MW, Farrell DW, Olevitch LR, Brennan LK. Tailoring health messages: customizing communication with computer technology. Mahwah, NJ, USA: Lawrence Erlbaum Associates; 1999.
    23. Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 2000 Jan;55(1):68-78. [CrossRef] [Medline]
    24. Ng JY, Ntoumanis N, Thøgersen-Ntoumani C, Deci EL, Ryan RM, Duda JL, et al. Self-determination theory applied to health contexts: a meta-analysis. Perspect Psychol Sci 2012 Jul;7(4):325-340. [CrossRef] [Medline]
    25. Williams GC, Niemiec CP, Patrick H, Ryan RM, Deci EL. The importance of supporting autonomy and perceived competence in facilitating long-term tobacco abstinence. Ann Behav Med 2009 Jun;37(3):315-324 [FREE Full text] [CrossRef] [Medline]
    26. Resnicow K, Zhou Y, Hawley S, Jimbo M, Ruffin MT, Davis RE, et al. Communication preference moderates the effect of a tailored intervention to increase colorectal cancer screening among African Americans. Patient Educ Couns 2014 Dec;97(3):370-375 [FREE Full text] [CrossRef] [Medline]
    27. Resnicow K, Davis RE, Zhang G, Konkel J, Strecher VJ, Shaikh AR, et al. Tailoring a fruit and vegetable intervention on novel motivational constructs: results of a randomized study. Ann Behav Med 2008 Apr;35(2):159-169. [CrossRef] [Medline]
    28. Williams GC, Cox EM, Kouides R, Deci EL. Presenting the facts about smoking to adolescents: effects of an autonomy-supportive style. Arch Pediatr Adolesc Med 1999 Sep;153(9):959-964. [CrossRef] [Medline]
    29. Deci EL, Eghrari H, Patrick BC, Leone DR. Facilitating internalization: the self-determination theory perspective. J Pers 1994 Mar;62(1):119-142. [CrossRef] [Medline]
    30. Deci EL, Driver RE, Hotchkiss L, Robbins RJ, Wilson IM. The relation of mothers' controlling vocalizations to children's intrinsic motivation. J Exp Child Psychol 1993 Apr;55(2):151-162. [CrossRef] [Medline]
    31. Altendorf MB, Smit ES, Azrout R, Hoving C, van Weert JC. A smoker's choice? Identifying the most autonomy-supportive message frame in an online computer-tailored smoking cessation intervention. Psychol Health 2021 May;36(5):549-574. [CrossRef] [Medline]
    32. Altendorf MB, van Weert JC, Hoving C, Smit ES. Should or could? Testing the use of autonomy-supportive language and the provision of choice in online computer-tailored alcohol reduction communication. Digit Health 2019;5:2055207619832767 [FREE Full text] [CrossRef] [Medline]
    33. Miller CH, Lane LT, Deatrick LM, Young AM, Potts KA. Psychological reactance and promotional health messages: the effects of controlling language, lexical concreteness, and the restoration of freedom. Human Comm Res 2007 Apr;33(2):219-240 [FREE Full text] [CrossRef]
    34. Smit ES, Bol N. From self-reliers to expert-dependents: identifying classes based on health-related need for autonomy and need for external control among mobile users. Media Psychol 2019 Apr 30;23(3):391-414. [CrossRef]
    35. Smit ES, Dima AL, Immerzeel SA, van den Putte B, Williams GC. The Virtual Care Climate Questionnaire: development and validation of a questionnaire measuring perceived support for autonomy in a virtual care setting. J Med Internet Res 2017 May 08;19(5):e155 [FREE Full text] [CrossRef] [Medline]
    36. Ferguson SG, Shiffman S, Gitchell JG, Sembower MA, West R. Unplanned quit attempts--results from a U.S. sample of smokers and ex-smokers. Nicotine Tob Res 2009 Jul;11(7):827-832. [CrossRef] [Medline]
    37. West R, Sohal T. "Catastrophic" pathways to smoking cessation: findings from national survey. BMJ 2006 Feb 25;332(7539):458-460 [FREE Full text] [CrossRef] [Medline]
    38. Bolle S, Romijn G, Smets EM, Loos EF, Kunneman M, van Weert JC. Older cancer patients' user experiences with web-based health information tools: a think-aloud study. J Med Internet Res 2016 Jul 25;18(7):e208 [FREE Full text] [CrossRef] [Medline]
    39. Jaspers MW. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inform 2009 May;78(5):340-353. [CrossRef] [Medline]
    40. Molich R, Nielsen J. Improving a human-computer dialogue. Commun ACM 1990 Mar;33(3):338-348. [CrossRef]
    41. Nielsen J. Estimating the number of subjects needed for a thinking aloud test. Int J Hum Comput Stud 1994 Sep;41(3):385-397. [CrossRef]
    42. Nielsen J, Molich R. Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1990 Mar Presented at: CHI '90; April 1-5, 1990; Seattle, WA, USA p. 249-256. [CrossRef]
    43. ATLAS.ti.   URL: http://atlasti.com/ [accessed 2022-03-24]
    44. Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs 2008 Apr;62(1):107-115. [CrossRef] [Medline]
    45. Guest G, MacQueen KM, Namey EE. Introduction to applied thematic analysis. In: Guest G, MacQueen KM, Namey EE, editors. Applied Thematic Analysis. Thousand Oaks, CA, USA: Sage Publications; 2012.
    46. Altendorf M, Hoving C, Van Weert JC, Smit ES. Effectiveness of message frame-tailoring in a web-based smoking cessation program: randomized controlled trial. J Med Internet Res 2020 Apr 03;22(4):e17251 [FREE Full text] [CrossRef] [Medline]
    47. Internet; toegang, gebruik en faciliteiten; 2012-2019. Centraal Bureau voor de Statistiek. 2018.   URL: https://opendata.cbs.nl/statline/#/CBS/nl/dataset/83429NED/table?fromstatweb [accessed 2019-06-20]
    48. Internet usage statistics - the internet big picture. Internet World Stats. 2019.   URL: https://www.internetworldstats.com/stats.htm [accessed 2019-06-20]
    49. Jayadevappa R, Chhatre S. Patient centered care - a conceptual model and review of the state of the art. Open Health Serv Policy J 2011 Apr 27;4(1):15-25. [CrossRef]
    50. Epstein RM, Street Jr RL. The values and value of patient-centered care. Ann Fam Med 2011;9(2):100-103 [FREE Full text] [CrossRef] [Medline]
    51. Poitras ME, Maltais ME, Bestard-Denommé L, Stewart M, Fortin M. What are the effective elements in patient-centered and multimorbidity care? A scoping review. BMC Health Serv Res 2018 Jun 14;18(1):446 [FREE Full text] [CrossRef] [Medline]
    52. O'Brien L, Marzano M, White RM. 'Participatory interdisciplinarity': towards the integration of disciplinary diversity with stakeholder engagement for new models of knowledge production. Science and Public Policy 2013 Jan 07;40(1):51-61. [CrossRef]
    53. Talwar S, Wiek A, Robinson J. User engagement in sustainability research. Sci Pub Pol 2011 Jun 01;38(5):379-390. [CrossRef]

    ‎
    HCOS: Health Causality Orientations Scale
    I-Change: Integrated-Change
    PAS: Personal Advice in Stopping smoking
    SDT: self-determination theory

    Edited by A Mavragani; submitted 28.09.21; peer-reviewed by M Nitsch, E Mansfield; comments to author 21.12.21; revised version received 07.02.22; accepted 11.03.22; published 21.04.22

    Copyright

    ©Inge S van Strien-Knippenberg, Maria B Altendorf, Ciska Hoving, Julia C M van Weert, Eline S Smit. Originally published in JMIR Formative Research (https://formative.jmir.org), 21.04.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.