Published on 08.07.19 in Vol 3, No 3 (2019): Jul-Sep
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/13417, first published Jan 16, 2019.
Development and Usability of ADappt: Web-Based Tool to Support Clinicians, Patients, and Caregivers in the Diagnosis of Mild Cognitive Impairment and Alzheimer Disease
Background: As a result of advances in diagnostic testing in the field of Alzheimer disease (AD), patients are diagnosed in earlier stages of the disease, for example, in the stage of mild cognitive impairment (MCI). This poses novel challenges for a clinician during the diagnostic workup with regard to diagnostic testing itself, namely, which tests are to be performed, but also on how to engage patients in this decision and how to communicate test results. As a result, tools to support decision making and improve risk communication could be valuable for clinicians and patients.
Objective: The aim of this study was to present the design, development, and testing of a Web-based tool for clinicians in a memory clinic setting and to ascertain whether this tool can (1) facilitate the interpretation of biomarker results in individual patients with MCI regarding their risk of progression to dementia, (2) support clinicians in communicating biomarker test results and risks to MCI patients and their caregivers, and (3) support clinicians in a process of shared decision making regarding the diagnostic workup of AD.
Methods: A multiphase mixed-methods approach was used. Phase 1 consisted of a qualitative needs assessment among professionals, patients, and caregivers; phase 2, consisted of an iterative process of development and the design of the tool (ADappt); and phase 3 consisted of a quantitative and qualitative assessment of usability and acceptability of ADappt. Across these phases, co-creation was realized via a user-centered qualitative approach with clinicians, patients, and caregivers.
Results: In phase 1, clinicians indicated the need for risk calculation tools and visual aids to communicate test results to patients. Patients and caregivers expressed their needs for more specific information on their risk for developing AD and related consequences. In phase 2, we developed the content and graphical design of ADappt encompassing 3 modules: a risk calculation tool, a risk communication tool including a summary sheet for patients and caregivers, and a conversation starter to support shared decision making regarding the diagnostic workup. In phase 3, ADappt was considered to be clear and user-friendly.
Conclusions: Clinicians in a memory clinic setting can use ADappt, a Web-based tool, developed using multiphase design and co-creation, for support that includes an individually tailored interpretation of biomarker test results, communication of test results and risks to patients and their caregivers, and shared decision making on diagnostic testing.
JMIR Form Res 2019;3(3):e13417
Dementia is a syndrome diagnosis that is used to describe decline in cognitive functioning that is severe enough to result in a loss of independence in performing everyday activities . A growing proportion of individuals presenting at memory clinics do not (yet) fulfill the criteria for dementia. These individuals are labeled as having mild cognitive impairment (MCI) if cognitive impairment can be objectified [ ]. In the course of 3 years, roughly half of the MCI patients develop dementia, whereas the other half remain stable or improve [ ]. Therefore, the label of MCI entails a prognosis, rather than a diagnosis.
The prognosis of MCI patients depends on the etiology of the symptoms. The most common underlying cause is Alzheimer disease (AD), a neurodegenerative disease that develops gradually, with dementia as the final stage . AD biomarkers, assessed through, for example, magnetic resonance imaging (MRI) or cerebrospinal fluid (CSF), reflect AD related pathological processes and can therefore provide information on the underlying cause of cognitive impairment [ ]. In early stages of the disease, AD biomarkers are particularly valuable, as this information allows a more precise estimate of the patient’s risk of developing dementia [ ]. However, interpreting and communicating biomarker results is complex, as the understanding of probabilistic information is known to be difficult for clinicians and patients and caregivers [ ]. Moreover, test results may be unclear or conflicting and thus may not always offer the certainty that clinicians, patients, and caregivers are looking for [ ].
As a result, the clinician has to face a growing number of challenges during the diagnostic workup, for example, which and how many diagnostic (biomarker) tests should be performed, how much and what type of information does the patient actually want, what can we expect from a specific diagnostic test, does a patient prefer to be provided with information on the likely course of their symptoms, and how can this information best be conveyed. Given that patients may weigh potential benefits and harms of AD biomarker testing differently, considering the patients’ preferences and needs are essential when making decisions for or against biomarker testing and the disclosure of results. In this situation, where an increasing number of reasonable options emerge to address the patients’ situation, clinicians and patients should engage in a process of shared decision making to ensure that the decisions made best fit the individual . However, in a memory clinic setting, involving patients in decision making is often limited to providing information only [ , ], and explicit risk communication about the development of dementia is barely observed in individuals with MCI. This demonstrates that there is room to support clinician-patient communication [ ].
In the context of the Alzheimer Biomarkers in Daily Practice (ABIDE) project, we aimed to develop a Web-based tool for clinicians in a memory clinic setting that (1) provides personalized risk estimates of progression to dementia to aid the clinician in interpreting test results, (2) supports clinicians in communicating biomarker test results and risks to MCI patients and their caregivers, and (3) supports clinicians to engage patients in decision making regarding the diagnostic workup of AD. Following a user-centered design, this paper describes the process of development, usability, and acceptability of the Web-based diagnostic support tool named ADappt.
This study was conducted as part of the ABIDE project, which has been funded in the context of the Dutch Deltaplan Dementia . ABIDE has been designed to improve and facilitate the use and interpretation of AD biomarkers in clinical practice, taking into account patients’ preferences toward diagnostic testing and communication of test results. Here, we describe the development and testing of the usability and acceptability of ADappt, a Web-based tool to support clinicians in the diagnostic process of AD in patients with MCI. The tool consists of 3 modules: (1) risk calculation: this module provides personalized risk estimates of progression to dementia to aid the clinician in interpreting test results (MRI and CSF); (2) risk communication: this module provides a summary of the biomarker results, including a graphic representation of the risk to facilitate communication of test results; (3) shared decision making: this module entails a conversation starter to support clinicians to engage patients in shared decision making regarding diagnostic testing.
The overall study adopted a user-centered mixed-method approach with emphasis on co-creation and was conducted in the 3 phases described in. The board of the medical ethics review committee of the Vrije Universiteit (VU) medical center reviewed and approved this study. For each module, input and feedback of end users (professionals, patients, and caregivers) was obtained. Modules were developed in independent yet overlapping trajectories such that module 1 was developed first, closely followed by module 2. Module 3 was developed last, based on a set of earlier studies conducted in the context of the ABIDE project [ - , ].
Thematic content analysis was used to analyze the qualitative data collected during focus groups, panels, and one-on-one interviews . In total, 2 coders (IvM and LV), with a background in psychology, used an inductive approach to independently formulate lists of themes that appeared from summarized data. The themes were discussed to reach consensus.
Module 1: Risk Calculation
Phase 0: Defining the Context
In 2016, the Geneva Task Force on the roadmap of Alzheimer biomarkers published a strategic roadmap to foster the clinical validation of AD biomarkers . This roadmap adopted a 5-phase approach from cancer research and dedicated 1 phase to the diagnostic and prognostic performance of biomarkers in MCI patients. Both for MRI and CSF biomarkers, evidence still appeared incomplete [ , ]. In particular, limited evidence was available on the combination of biomarkers and how they perform to predict prognosis in individual MCI patients. In a previous study, we therefore constructed individualized prediction models that provide personalized risk estimates for MCI patients, on the basis of MRI, CSF, or the combination of these 2 biomarkers based on data from the Amsterdam Dementia Cohort [ , , ]. Probabilities (with CIs) of progression to AD dementia for 1 and 3 years of follow-up can be calculated using these models (for a detailed description see van Maurik et al [ ]). These models serve as input for the risk calculation module of ADappt.
Phase 1: Needs Assessment
Focus Group Clinicians
We organized a 2-hour focus group (n=8, 7 neurologists and 1 geriatrician) led by experienced focus group leaders (RP and MK). During this focus group, we discussed the interpretation of biomarker test results and communication of results in terms of risks to patients based on 2 clinical cases that were shortly introduced. Used examples of risk visualizations were based on a commonly used format in cardiovascular risk management  and breast cancer therapy risks (Adjuvant! Online) [ ].
Phase 2: Design and Development
Test Functional Design
A prototype of the risk calculation module (module 1) was developed from the results of phase 1. Functionality of the initial design was tested by end users in 2 rounds. In both rounds, end users were asked, via a questionnaire, if they considered the module to be logical, clear, and useful. In the first round, a neurologist, a geriatrician, and 2 researchers provided input on a preliminary design and layout (risk calculation functionality was not yet available). From their input the prototype was further developed. Next, 9 professionals (5 neurologists, 2 geriatricians, and 2 health care professionals) provided input on the prototype with a fully functioning risk calculation.
Phase 3: Usability and Acceptability
Questionnaire Feedback Prototype
During the annual Dutch Dementia Conference (Dementia Update) in 2017, the risk calculation module was presented and interested attendees were given the opportunity to try out the prototype. A total of 24 attendees (4 geriatricians, 10 neurologists, 5 with another profession [internist or nursing home physician specialist], and 5 did not indicate their profession) provided written feedback by filling out a brief questionnaire.
Module 2: Risk Communication
Phase 0: Defining the Context
In a previous survey study , clinicians indicated that they find it difficult to convey MCI as a diagnosis because of the uncertainty of the diagnosis of MCI and the lack of treatment. Nonetheless, the majority of clinicians indicated to always disclose the risk of developing dementia [ ]. This however does not sufficiently fulfill the information need of patients and caregivers, as patients expressed a wish for more information on the prognosis of the disease [ ].
In risk communication, numeric formats are generally preferred, relative to other formats, to increase patients’ understanding [, , ]. For example, verbal communication of risk (unlikely, possible, and rare) is vulnerable to a high degree of variability in interpretation and therefore not considered a best practice [ ]. Among numeric formats, natural frequencies (20 out of 100 people similar to you) are favored over other formats (probabilities, odds, or classical probabilities), as a reference class is included that reduces misinterpretation [ , ]. A numeric format should ideally be complemented with a visual representation of risk and include a specific time frame in which an event may occur [ ].
Phase 1: Needs Assessment
Focus Group Clinicians
Input from clinicians with regard to risk communication was obtained in the same focus group as described in phase 1 for module 1 (see the section above for details).
Focus Group Patients and Caregivers
A total of 4 1-hour focus groups were organized for MCI patients and were led by an experienced focus group leader (RP). A total of n=13 patients participated (3 to 4 participants per session, mean age 66 years (SD 8); 31% (4/13) female) and 63% (8/13) were accompanied by a caregiver (all partners, n=4 (50%) female). The topics that were discussed included how patients and caregivers looked back on receiving the diagnosis and test results and whether they had any recommendations for the disclosure of test results. In a second part, participants brainstormed hands-on in a paper, pencil, and scissors session on useful (graphical) summaries of test results.
Phase 2: Design and Development
During the annual Dutch Dementia Symposium (Dementia Update) in 2017, clinicians were invited to attend a short panel discussion on their preferences about tools to support communication of test results, in terms of graphics, written summaries, and explanations of risks and uncertainty. A total of 5 clinicians attended this panel discussion and provided feedback.
Individual Interviews With Patients and Caregivers
To evaluate the first prototypes of the risk communication module, we conducted 5 interviews, each with 1 MCI patient (mean age 65 years [SD 9], n=2 (40%) female) and his or her caregiver (all partners, n=3 (60%) female). Interviews took 15 to 30 min and were conducted by the software developer to ensure that the feedback of patients was optimally used to improve this module (co-creation). Patients were presented with an example of the module, in which the diagnostic test results of a fictive case of a man or woman of 60 years were displayed, and patients were asked for their input on the storyline, graphics, style, and the possibility to print these results.
Module 3: Shared Decision-Making Module
Phase 0/1: Defining the Context/Needs Assessment
In shared decision making, clinicians and patients work together to decide which care plan best fits with individual preferences and needs when there is more than 1 reasonable option [, ]. In the diagnostic routine for AD, there is typically more than 1 reasonable option to choose, making this a situation where shared decision making is the preferred approach. However, in a previous study, we found that shared decision making in the memory clinic is often limited to information giving [ ], and shared decision making involves the following 4 steps: (1) create choice awareness, (2) provide information, (3) explore preferences, and (4) decide together [ , ].
Phase 2: Design and Development
For each step in shared decision making, we constructed example phrases in co-creation with communication experts (LV and ES). These phrases might function as a conversation starter for clinicians to engage patients in shared decision making. To inform a patient on the possible pros and cons, we also developed a list with the example language on (dis)advantages for the following commonly used diagnostic tests: neuropsychological investigation, imaging (MRI or computer tomography [CT]), lumbar puncture, amyloid Positron Emission Tomography (PET), consultation with other specialist (neurologist, geriatrician, and psychiatrist), clinical geneticist, and awaiting policy. This list was based on the literature and expert opinions (FB and WF).
All Modules: Pilot Study in 4 Local Memory Clinics
To test the ADappt tool comprising all 3 modules, we organized a multicenter usability pilot study. Both clinicians and patients were asked for feedback in 4 local memory clinics (Reinier de Graaf Ziekenhuis, Elisabeth Tweesteden Ziekenhuis, Jeroen Bosch Ziekenhuis, and Spaarne Gasthuis) and 1 academic memory clinic (Amsterdam University Medical Center (UMC)). Participating clinicians (4 neurologists, 1 geriatrician, and 2 not specified) were asked to use the tool for a minimum of 2 weeks and a maximum of 7 weeks. One of the authors (MvB) was present in the memory clinics to assist clinicians in the first week of the pilot. Clinicians were asked to complete the System Usability Scale (SUS) after using the tool . The maximum score on the SUS is 5, and a higher score indicates better usability. Technical details of the tool are summarized in .
Module 1: Risk Calculation Module
Here, we describe the results of the co-creation steps leading to the final layout of the risk calculation module shown in.
Phase 1: Needs Assessment
Focus Group Clinicians
On reviewing the discussion on the preferences with regard to risk calculation, 3 main findings emerged (the findings and adaptations are presented in). First, clinicians preferred the reporting of percentages over a risk table or bar chart, as in the latter, the information was perceived not to be clear at one glance. Second, regarding reliability and validity; clinicians whished information about CIs and information on how the models were constructed and how they perform. Therefore, we included a link to the publication, explaining how the models were constructed. Finally, clinicians emphasized user-friendliness of the module, for example, every professional should be able to use the module.
|Group||Aim||Themes||Changes made based on the results|
|Clinicians||To explore the need for risk calculation and communication and preferences with regard to visualization||Module 1: Risk calculation. Reporting percentages is considered pleasant.; Reliability and validity; CIs should be added and it should be possible for clinicians to look up how the risk models were constructed and how they perform; The module should be user-friendly such that every professional is able to use it.; Module 2: A graphical representation of risk alone could be difficult for patients to understand and therefore would not be sufficient; Clinicians often show brain images to support the communication of test results; Clinicians provide patients with written information.||We included a link to the original paper that describes how the risk models were constructed; We accompanied the graphical presentation of risk with a textual explanation; The module allows summary sheet for patients to be printed|
|Patients and caregivers||To explore communication needs and to explore what kind of information should be communicated when disclosing (risk related) test results and how it should be visualized||Module 2: Risk Communication. Patients would like to know their exact progression risk. The explanation of risk should be simple. To compare their results with what is normal for their age. How should I discuss the results with family or friends?; Visualization: Patients like to see their brain images (MRIa or CTb) at home. Patients like to know which results have led to the presented risk and why. The preference for visualization of risk differs per person. How should I deal with the diagnosis? Are there things that I can do myself (lifestyle changes, tips and tricks)? Which steps do we need to take (case manager etc)?||We included an a priori risk as reference group for comparison purposes. The result page includes information on each diagnostic test and whether this result was abnormal. A summary is given at the end with the opportunity to include personal notes. We included 2 ways to visualize risk: a bar chart and an icon array. Users can switch between these 2|
|Clinicians||To explore the preferences with regard to visualization of test results for patients||Clinicians like to communicate as dichotomous’ as possible. However, biomarker results always have a gray area. This could be visualized by a traffic light; Clinicians would like to visualize risk in a line chart with time on the horizontal axes; The brain images (MRI or CT) could be visualized with a figure of the brain: normal versus atrophy; Clinicians emphasize that the module should be an aid, and that there should remain room for personal variation||We added a traffic light visualization for all diagnostic tests; An open text field provides the ability to personalize the result page based on the needs of clinicians and patients|
|Interviews||To explore whether the storyline, visualization, and style of the printout page are clear||The printout contains too much and too complex information; The presentation of the a priori risk is confusing and sometimes daunting; Patients prefer to see their test results first followed by the risk; The graphical presentation of risk in 2 ways works well; The font is too small; The print of the results page is considered as valuable.||A total of 2 communication experts reviewed the text on understandable language; We removed the a priori risk from this module; We changed the flow of the printout: test results are presented first followed by risk; Colors and font were optimized|
aMRI: magnetic resonance imaging.
bCT: computer tomography.
Phase 2: Design and Development
Test Functional Design
End users agreed on the proposed order and name of the tool. In general, the information in the module was considered to be clear. Suggestions for improvement included (1) a clear disclaimer that the module calculates risk of AD dementia only, (2) to use a different term instead of a priori and a posteriori risk, (3) adding value ranges, and (4) the module should also work when only CSF or MRI values were entered.
Phase 3: Usability and Acceptability
Questionnaire Feedback Prototype
The responses of the n=24 clinicians are summarized in. The applicability and user-friendliness of the module was rated highest. More than half of the respondents indicated that they would use the final version of the module in the future. The perceived reliability of the module was rated relatively low.
|Is it clear where ADappt could be used for (rate 1-10)?||8.3|
|How user-friendly would you rate this tool (rate 1-10)?||8.3|
|How reliable would you rate ADappt (rate 1-10)?||6.4|
|Would you use the final version of ADappt in your daily clinical routine (% yes)?||58%|
Module 2: Risk Communication Module
To facilitate explanation of the test results, we developed in co-creation (steps described below) a risk communication sheet. The final layout is shown in. The final design of the summary sheet (module 2). Here we use the same example as in . First, a general description of MCI is given, followed by the personalized results of this person. For each test, we give a summary of what kind of information is retrieved from the test, followed by the interpretation of the test result and how it effects the chance of AD dementia. Finally, we summarize the findings and give a verbal explanation (1 out of 100) of risk and visualization of this risk, either with an array grid or bar chart (not shown). Of note, the MRI is a stock image, not personalized.
Phase 1: Needs Assessment
Focus Group Clinicians
With regard to risk communication, clinicians mentioned that they often show brain images to patients to support communication of test results and that they provide patients with written information (findings and adaptations are presented in). In addition, clinicians confirmed that a graphical representation of risk only is not sufficient to explain a risk to patients and caregivers. Therefore, we accompanied the visual presentation of risk with a textual explanation.
Focus Group Patients and Caregivers
An important theme during the focus group with patients and caregivers was that patients would like to know their exact progression risk. Other themes with regard to risk were that explanations should be simple and that they would like to compare their results with a normal reference category. In addition, patients indicated to struggle with how they should communicate their diagnosis and test results with family and friends (). In terms of visualization, patients preferred to see their own brain images. However, in view of ever-more complicated privacy regulations, we decided not to include this in the summary sheet. Patients would also like to know how the test results contributed to the presented risk. Individual differences emerged with regard to the preference for the visualization of risk (bar chart or icon arrays). Therefore, we decided to incorporate both options in the module, to be used depending on patients’ preference. Finally, patients and caregivers would like to be provided with information on how to handle the diagnosis, what they can do in terms of lifestyle changes and which steps they should take next (for example a case manager). As this is likely to be different for each patient, we included an open field at the end of the summary sheet that can be used by the clinician to include such information.
Phase 2: Design and Development
One of the themes emerging from the panel discussion with clinicians is that they prefer to communicate test results in a dichotomous way (normal or abnormal,). However, biomarker results are originally continuous in nature, and in terms of interpretation, there is always a gray area. A possible way to visualize this is by using a traffic light. Therefore, we included a traffic light for all biomarkers on the summary sheet, with red indicating abnormal, green indicating normal, and gray indicating borderline results. As an alternative to including the MRI or CT images of patients, clinicians suggested to implement an example MRI image on the written summary for patients. For risk visualization, clinicians suggested to use a line chart with time on the horizontal axis. However, as the risk calculation module only provides valid predictions up to 3 years, a line graph might incorrectly imply that this line could be extrapolated to longer follow-up periods. For this reason, we decided not to make use of this suggestion. Instead, we included 2 options for risk visualization: a bar chart and an icon array. Finally, clinicians emphasized that the communication module should be an aid, allowing for tailoring of the communication to individual patients, rather than a strict protocol. This whish is met by providing an open text field in the summary sheet, where the clinician can add information.
During the individual interviews with patients and caregivers, the storyline of the printout was considered as too long and too complex. Therefore, 2 communication experts reviewed, optimized, and abbreviated the text (LV and ES). Moreover, we removed the a priori risk (ie, risk without biomarkers, see example in) from the summary sheet for patients as this was experienced as confusing and sometimes daunting. Patients and caregivers only valued the probability based on the diagnostic tests. Patients would like to see their individual test results first, followed by their overall risk as a result of these individual test results. The graphical presentation of risk as either a bar chart or icon array worked well and the printout page was considered valuable. Themes and adaptations are summarized in .
Module 3: Shared Decision-Making Module
Phase 2: Design and Development
presents the conversation starter with example phrases for a clinician to engage patients and caregivers in each of the 4 steps of shared decision making (create choice awareness, provide information, explore preferences, and decide together [ , ]). In , we show example language that can be used to inform a patient on pros and cons with regard to imaging. Phrases for other diagnostic tests can be found on our website [ ].
|Creating choice awareness||To make the patient aware that there is more than 1 way forward, that a decision needs to be made on whether and which diagnostic test to use, and that the best way forward depends on what matters most to the patient||“The aim is that we decide (together) which diagnostic test we will perform.”|
|Provide information||To assess patients’ preferences||“There is much to tell about the different diagnostic tests. Some patients would like to hear as much as possible, while other patients do not want to know too much. What do you prefer?”|
|To inform on the different diagnostic test possibilities||“What would you like to know about the possibilities for diagnostic testing?”; or “There are 3 possibilities, namely…”|
|To inform on the pros and cons in general or for a specific test||“Would you like to know more about the possible pros and cons?”|
|To inform which results can be expected||“I will tell you something about the kind of results that you can expect when we perform a specific test”|
|Explore preferences||To explore preferences and considerations from the patient||“I have told you about the possibilities, what are your considerations?”|
|To estimate the decision preferences of the patient. Does the patient want to be involved in the decision or does the patient want the clinician to decide||“Some patients would like to decide with the clinician which test to perform, while others would like the clinician to decide. What do you prefer?”|
|Decide||To make a shared decision; a balanced decision that both parties support, and communicate this||“We decide to do [X], because you indicated that [preferences patient].”|
|To formulate an advice, in which the preferences of the patient are taken into account, and communicate this||“I propose [X], based on [preferences patient, guidelines, experience, preferences clinician]. Do you agree?”|
|Reason why MRI or CT scan is suggested||Pros||Cons|
|“A scan of the brain is a common part of dementia diagnostics. With MRIa or CTb scan we visualize the brain. We can exclude treatable diseases. Moreover, we can view characteristics of dementia, such as shrinkage of the brain or wearing of the blood vessels.”|
aMRI: magnetic resonance imaging.
bCT: computer tomography.
Pilot Study in 4 Local Memory Clinics
In total, 7 clinicians from 5 memory clinics commented on the ADappt tool and 5 of them completed the survey. The usability of the tool was rated with a score of SUS=4.4 (SD 0.5, max score=5; see). Moreover, clinicians rated the tool as well integrated, easy, and consistent. A total of 3 clinicians used the tool in their daily practice, of which 1 used it on a regular basis during the pilot. summarizes the response frequencies of each question in the survey:
[…] it was very useful, because this patient had somewhat conflicting biomarkers, which did not point in one direction, so when using the app, it resulted in a much lower risk than that these people expected, and actually they went home very satisfied.
[…] I did use the website, but mainly the result page to provide the patient with more insight in what it could mean when it results in MCI and what the risk on dementia eventually is and that you can give them a printout to take home, that is very positively received also by the patient.
Clinicians experienced some technical problems while using the tool. For example, the site was blocked by the firewall in 1 of the hospitals, leading to the need to use it on a personal device. Some clinicians commented that they already feel time pressure during a consult and do not find the time to actually open the tool in the consult. A total of 4 patients (n=3 (75%) female, mean age 72 years [SD 2]) completed a short questionnaire after the consult in which the tool was used. The result page was valued as clear and useful by all. The result pages helped them to better understand the diagnosis and helped them explain the results to relatives and/or friends. These patients recommended the result page to other patients.
This study reports the development and initial usability testing of ADappt—a Web-based tool to facilitate risk calculation and support clinician-patient risk communication and shared decision making in a memory clinic setting. We worked closely with end users, clinicians, patients, and caregivers, to identify initial requirements for the tool and make necessary changes to enhance the functionalities, usability, and information provided by the tool. Needs assessment (phase 1) uncovered the need of patients and caregivers for written summaries and information on prognosis. Clinicians expressed their need for visual aids to communicate results and prognosis. In the development phase (phase 2), we developed a risk calculation tool for clinicians and a risk communication tool (summary sheet) for patients and caregivers. In addition, a conversation starter was developed to support clinicians and patients in decision making around diagnostic testing.
Already in an early stage of usability testing (phase 3), the tool was considered clear and user-friendly, and a majority of memory clinic professionals indicated they would use such a tool in the future. However, clinicians appeared to have concerns about the reliability of the risk prediction tool. At the moment of testing, the algorithms were not yet published, and therefore a reference to a detailed description of the models was not included in the tool. To date, the module has been updated with the published algorithms . For the tool to be used in real clinical practice, generalizability of prediction models in the risk calculation module is essential. Therefore, we are generating updated risk models based on data from a range of multi and monocenter cohorts and in alignment with the recently published National Institute on Aging and Alzheimer’s Association (NIA-AA) criteria [ ]. These updated models will be applicable to CSF biomarkers and volumetric MRI markers measured with different assays and platforms. In addition, we will add models based on amyloid PET. Another short-term update will be the translation of the tool into English.
Our small pilot study demonstrated that the tool, especially the printout page (risk communication module), was considered valuable by patients and caregivers. Although the usability of the tool was rated to be high by clinicians, only few clinicians actually used the tool in clinical practice. This was mainly due to practical reasons; clinicians experienced technical hurdles for the use of such a tool in clinical practice (eg, computers in consulting room firewalled). Despite these hurdles and that relying on computer algorithms may be counterintuitive for clinicians, experience from different disease areas shows that computer algorithms are welcome in clinical practice and are seen as the future of medicine. For example, in oncology clinicians use Adjuvant Online to calculate the risk of recurrence with and without adjuvant therapy for breast cancer patients . Recently U-Prevent was introduced for clinicians to calculate cardiovascular risk and therapy benefit [ , ]. The major challenge for ADappt is its large-scale implementation. In an ideal situation, ADappt would be embedded in electronic patient file systems. Alternatively, ADappt can be used as a simple add-on help for both the clinician and patient to be quickly consulted on their mobile phone. In the short term, an e-learning could be useful to show how the clinicians can integrate the tool in their routine. Moreover, the amount of text in the tool was still experienced as a barrier by clinicians and could be further improved by using graphics instead of plain text.
The 3 modules of ADappt could aid clinicians and patients in a number of ways. In the diagnostic process for AD, it is currently not common practice to actively engage patients in diagnostic decision making [, ]. ADappt provides a conversation starter and an overview of pros and cons per diagnostic test and could therefore help clinicians to engage patients and caregivers in diagnostic decision making [ ]. In an observational study of ABIDE, we currently examine communication and decision-making processes during pre- and postdiagnostic clinician-patient encounters. This study will provide input to further improve the shared decision-making module, as soon as the results of this study become available [ ].
Second, ADappt supports clinicians in the communication of test results to patients and contributes to personalized diagnostic care and harmonization of clinical practice. After diagnostic testing, the interpretation and communication of test results, especially when patients are not yet demented (ie, MCI) is not straightforward. The ADappt tool clearly shows whether a patient is unlikely to develop AD dementia in the following years and therefore could be reassured, or whether a patient is indeed likely to convert to AD dementia and should therefore be followed up, arranged care, or referred for participation in clinical trials. The importance of an appropriate diagnosis and prognosis of MCI is acknowledged in current guidelines. These guidelines state that an appropriate diagnosis of MCI is important for patients and caregivers to understand the cause of their complaints and to arrange care planning based on communicated prognostic probabilities . Moreover, clear communication of the test results and the corresponding prognosis has been given high priority by patients and caregivers. Tools similar to ADappt can help in this regard. One of the strengths of this study is that we designed and co-created ADappt with the end users, that is, clinicians, patients, and caregivers in memory clinic settings, to optimize future use of the tool. Most suggestions and whishes were met in the design and development of the tool. Only 2 suggestions did not find their way into the tool immediately. First, a recurring theme was the use of CT or MRI images to support the communication of diagnosis and test results. Both clinicians and patients indicated that they would like to see a patients’ CT or MRI image to be included on the summary sheet. However, this would require the summary sheet to make a link to, for example, the electronic patient file that is challenging, both from a technical and privacy perspective. As an alternative, we suggested that the clinician shows the patient their own MRI on screen in the consulting room, and we included an example MRI on the summary sheet. Second, in early phases of the development of the summary sheet, patients indicated that they would like to see their results in comparison with a reference class (ie, what is normal for my age). In the first design of the summary sheet, this reference class was presented as separate probabilities and patients could compare this with their personal risk. However, this led to confusion and patients suggested that this should be removed from the summary sheet. Another strength of the tool is its compatibility with the most recent versions of the most commonly used Web browsers and could be used on all commonly used devices (computer and mobile phone). Moreover, the tool is designed as a flexible platform with the idea that extra modules can easily be added in the future. This would also enable adding modules on lifestyle advice [ ], extension to other biomarkers or settings, such as the general practitioner, or to advanced care planning.
Among the limitations is that other factors, such as cardiovascular disease on MRI, cardiovascular risk factors, and amyloid PET biomarkers are currently not accounted for in the risk prediction. Clinicians do consider these factors to be important for prognosis in risk prediction and should therefore be accounted for in the models. However, the algorithms incorporated in the tool had a different starting point—namely, how to extract maximum information from MRI and CSF biomarker values for an individual patient, given that the clinician decided to order these tests. For a user-centered development process, the number of participants in this project was adequate. However, to achieve more generalizable results with regard to implementation or cost-effectiveness, future validation should include larger samples and a different study design.
In conclusion, this study presents the first tool to support clinicians and patients in memory clinic settings with decisions on diagnostic testing, individual tailored interpretation of diagnostic test results, and communication of test results. At the moment, the tool is available for academic use only. The tool is developed in a multiphase design, where co-creation with end users was an important feature. Owing to its flexibility, it is possible to add extra modules, guidelines, or new prediction models in the future. Moreover, as the tool currently focuses on clinicians, we envision that in the future a similar platform would be valuable for patients and caregivers to facilitate them to engage in shared decision making and to aid them in managing their own health care trajectory.
Research of the Alzheimer center, Amsterdam, is part of the neurodegeneration research program of Amsterdam Neuroscience. The Alzheimer Center, Amsterdam, is supported by Stichting Alzheimer Nederland and Stichting VUmc fonds. Wiesje van der Flier holds the Pasman chair. The authors would like to thank Stichting Buytentwist for their valuable contribution. This study is funded by ZonMW-Memorabel (ABIDE; project no 733050201), a project in the context of the Dutch Deltaplan Dementie.
ABIDE study group: Amsterdam, the Netherlands (Alzheimer Center and Department of Neurology, Amsterdam Neuroscience, VU University Medical Center): Wiesje M van der Flier, PhD; Philip Scheltens, MD, PhD; Femke H Bouwman, MD, PhD; Marissa D Zwan, PhD; Ingrid S van Maurik, MSc; Arno de Wilde, MD; Wiesje Pelkmans, MSc; Rosha Babapour Mofrad, MSc; Silvia Ingala, MSc; Colin Groot, MSc; Ellen Dicks, MSc; Els Dekkers (Department of Radiology and Nuclear Medicine, Amsterdam Neuroscience, VU University Medical Center); Bart NM van Berckel, MD, PhD; Frederik Barkhof, MD, PhD; Mike P Wattjes, MD, PhD (Neurochemistry laboratory, Department of Clinical Chemistry, Amsterdam Neuroscience, VU University Medical Center); Charlotte E Teunissen, PhD; Eline A Willemse, MSc (Department of Medical Psychology, University of Amsterdam, Academic Medical Center); Ellen M Smets, PhD; Leonie NC Visser, PhD; Marleen Kunneman, PhD; Sophie Pelt; Sanne Schepers, MSc; Laxini Murugesu, MSc; Bahar Azizi, MSc; Anneke Hellinga, MSc (BV Cyclotron); E van Lier, MSc; Haarlem, the Netherlands (Spaarne Gasthuis) Niki M Schoonenboom, MD, PhD; Utrecht, the Netherlands (Department of Neurology and Neurosurgery, Brain Center Rudolf Magnus, University Medical Center Utrecht); Geert Jan Biessels, MD, PhD; Jurre H Verwer, MSc (Department of Geriatrics, University Medical Center Utrecht); Dieneke H Koek, MD, PhD (Department of Radiology and Nuclear Medicine); Monique G Hobbelink, MD (Vilans, Center of Expertise in long term care); Mirella M Minkman, PhD; Cynthia S Hofman, PhD; Ruth Pel, MSc; Meppel, the Netherlands (Espria) Esther Kuiper, MSc; Berlin, Germany (Piramal Imaging GmbH) Andrew Stephens, MD, PhD; Rotrkreuz, Switzerland (Roche Diagnostics International Ltd) Richard Bartra-Utermann, MD.
Memory clinic panel: The members of the memory clinic panel are Niki M Schoonenboom, MD, PhD (Spaarne Gasthuis, Haarlem); Barbera van Harten, MD, PhD; Niek Verwey, MD, PhD; Peter van Walderveen, MD (Medisch Centrum Leeuwarden, Leeuwarden); Ester Korf, MD, PhD (Admiraal de Ruyter Ziekenhuis, Vlissingen); Gerwin Roks, MD, PhD (Sint Elisabeth Ziekenhuis, Tilburg); Bertjan Kerklaan, MD, PhD (Onze Lieve Vrouwe Gasthuis, Amsterdam); Leo Boelaarts, MD (Medisch Centrum Alkmaar, Alkmaar); Annelies WE Weverling, MD (Diaconessenhuis, Leiden); Rob J van Marum, MD, PhD (Jeroen Bosch Ziekenhuis, ‘s-Hertogenbosch); Jules J Claus, MD, PhD (Tergooi Ziekenhuis, Hilversum); Koos Keizer, MD, PhD (Catherina Ziekenhuis, Eindhoven).
Conflicts of Interest
PS has acquired grant support (for the institution) from GE Healthcare, Danone Research, Piramal, and Merck. In the past 2 years, he has received consultancy/speaker fees (paid to the institution) from Lilly, GE Healthcare, Novartis, Sanofi, Nutricia, Probiodrug, Biogen, Roche, Avraham, and EIP Pharma. WMvdF performs contract research for Biogen MA Inc. Research programs of WMvdF have been funded by ZonMW, Health Holland, Pasman stichting, NWO, EU-FP7, EU-JPND, Alzheimer Nederland, Cardiovasculair Onderzoek Nederland, stichting Dioraphte, Gieskes-Strijbis fonds, Boehringer Ingelheim, Piramal Neuroimaging, Roche BV, Janssen Stellar, and Combinostics. All funding is paid to her institution. All other coauthors report no conflicts of interest.
- Citron M. Alzheimer's disease: strategies for disease modification. Nat Rev Drug Discov 2010 May;9(5):387-398. [CrossRef] [Medline]
- Albert MS, DeKosky ST, Dickson D, Dubois B, Feldman HH, Fox NC, et al. The diagnosis of mild cognitive impairment due to Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease. Alzheimers Dement 2011 May;7(3):270-279 [FREE Full text] [CrossRef] [Medline]
- Visser PJ, Verhey F, Knol DL, Scheltens P, Wahlund L, Freund-Levi Y, et al. Prevalence and prognostic value of CSF markers of Alzheimer's disease pathology in patients with subjective cognitive impairment or mild cognitive impairment in the DESCRIPA study: a prospective cohort study. Lancet Neurol 2009 Jul;8(7):619-627. [CrossRef] [Medline]
- Scheltens P, Blennow K, Breteler MM, de Strooper B, Frisoni GB, Salloway S, et al. Alzheimer's disease. Lancet 2016 Jul 30;388(10043):505-517. [CrossRef] [Medline]
- Jack CR, Bennett DA, Blennow K, Carrillo MC, Dunn B, Haeberlein SB, et al. NIA-AA Research Framework: toward a biological definition of Alzheimer's disease. Alzheimers Dement 2018 Apr;14(4):535-562 [FREE Full text] [CrossRef] [Medline]
- van Maurik IS, Zwan MD, Tijms BM, Bouwman FH, Teunissen CE, Scheltens P, Alzheimer’s Disease Neuroimaging Initiative. Interpreting biomarker results in individual patients with mild cognitive impairment in the Alzheimer's biomarkers in daily practice (ABIDE) project. JAMA Neurol 2017 Dec 1;74(12):1481-1491 [FREE Full text] [CrossRef] [Medline]
- Lipkus IM. Numeric, verbal, and visual formats of conveying health risks: suggested best practices and future recommendations. Med Decis Making 2007;27(5):696-713. [CrossRef] [Medline]
- Banningh LJ, Vernooij-Dassen M, Rikkert MO, Teunisse J. Mild cognitive impairment: coping with an uncertain label. Int J Geriat Psychiatry 2008 Feb;23(2):148-154. [CrossRef]
- van der Flier WM, Kunneman M, Bouwman FH, Petersen RC, Smets EM. Diagnostic dilemmas in Alzheimer's disease: room for shared decision making. Alzheimers Dement (N Y) 2017 Sep;3(3):301-304 [FREE Full text] [CrossRef] [Medline]
- Kunneman M, Smets EMA, Bouwman FH, Schoonenboom NS, Zwan MD, Pel-Littel R, et al. Clinicians' views on conversations and shared decision making in diagnostic testing for Alzheimer's disease: the ABIDE project. Alzheimers Dement (N Y) 2017 Sep;3(3):305-313 [FREE Full text] [CrossRef] [Medline]
- de Wilde A, van Maurik IS, Kunneman M, Bouwman F, Zwan M, Willemse EA, et al. Alzheimer's biomarkers in daily practice (ABIDE) project: rationale and design. Alzheimers Dement (Amst) 2017;6:143-151 [FREE Full text] [CrossRef] [Medline]
- Kunneman M, Pel-Littel R, Bouwman FH, Gillissen F, Schoonenboom NS, Claus JJ, et al. Patients' and caregivers' views on conversations and shared decision making in diagnostic testing for Alzheimer's disease: the ABIDE project. Alzheimers Dement (N Y) 2017 Sep;3(3):314-322 [FREE Full text] [CrossRef] [Medline]
- Vaismoradi M, Turunen H, Bondas T. Content analysis and thematic analysis: implications for conducting a qualitative descriptive study. Nurs Health Sci 2013 Sep;15(3):398-405. [CrossRef] [Medline]
- Frisoni GB, Boccardi M, Barkhof F, Blennow K, Cappa S, Chiotis K, et al. Strategic roadmap for an early diagnosis of Alzheimer's disease based on biomarkers. Lancet Neurol 2017 Dec;16(8):661-676. [CrossRef] [Medline]
- Mattsson N, Lönneborg A, Boccardi M, Blennow K, Hansson O, Geneva Task Force for the Roadmap of Alzheimer's Biomarkers. Clinical validity of cerebrospinal fluid Aβ42, tau, and phospho-tau as biomarkers for Alzheimer's disease in the context of a structured 5-phase development framework. Neurobiol Aging 2017 Dec;52:196-213. [CrossRef] [Medline]
- Ten Kate M, Barkhof F, Boccardi M, Visser PJ, Jack CR, Lovblad K, Geneva Task Force for the Roadmap of Alzheimer's Biomarkers. Clinical validity of medial temporal atrophy as a biomarker for Alzheimer's disease in the context of a structured 5-phase development framework. Neurobiol Aging 2017 Apr;52:167-82.e1. [CrossRef] [Medline]
- van der Flier WM, Pijnenburg YAL, Prins N, Lemstra AW, Bouwman FH, Teunissen CE, et al. Optimizing patient care and research: the Amsterdam Dementia cohort. J Alzheimers Dis 2014;41(1):313-327. [CrossRef] [Medline]
- van der Flier WM, Scheltens P. Amsterdam dementia cohort: performing research to optimize care. J Alzheimers Dis 2018;62(3):1091-1111 [FREE Full text] [CrossRef] [Medline]
- Cooney MT, Dudina A, D'Agostino R, Graham IM. Cardiovascular risk-estimation systems in primary prevention: do they differ? Do they make a difference? Can we see the future? Circulation 2010 Jul 20;122(3):300-310. [CrossRef] [Medline]
- Paridaens RJ, Gelber S, Cole BF, Gelber RD, Thürlimann B, Price KN, et al. Adjuvant! Online estimation of chemotherapy effectiveness when added to ovarian function suppression plus tamoxifen for premenopausal women with estrogen-receptor-positive breast cancer. Breast Cancer Res Treat 2010 Aug;123(1):303-310 [FREE Full text] [CrossRef] [Medline]
- Gurmankin AD, Baron J, Armstrong K. The effect of numerical statements of risk on trust and comfort with hypothetical physician risk communication. Med Decis Making 2004;24(3):265-271. [CrossRef] [Medline]
- Wallsten TS, Budescu DV, Zwick R, Kemp SM. Preferences and reasons for communicating probabilistic information in verbal or numerical terms. Bull Psychon Soc 2013 Nov 5;31(2):135-138. [CrossRef]
- Hoffrage U, Gigerenzer G, Krauss S, Martignon L. Representation facilitates reasoning: what natural frequencies are and what they are not. Cognition 2002 Jul;84(3):343-352. [Medline]
- Zipkin DA, Umscheid CA, Keating NL, Allen E, Aung K, Beyth R, et al. Evidence-based risk communication: a systematic review. Ann Intern Med 2014 Aug 19;161(4):270-280. [CrossRef] [Medline]
- Stiggelbout AM, Van der Weijden T, De Wit MP, Frosch D, Légaré F, Montori VM, et al. Shared decision making: really putting patients at the centre of healthcare. Br Med J 2012;344:e256 [FREE Full text] [Medline]
- Hargraves I, LeBlanc A, Shah ND, Montori VM. Shared decision making: the need for patient-clinician conversation, not just information. Health Aff (Millwood) 2016 Apr;35(4):627-629. [CrossRef] [Medline]
- Stiggelbout AM, Pieterse AH, de Haes JC. Shared decision making: concepts, evidence, and practice. Patient Educ Couns 2015 Oct;98(10):1172-1179. [CrossRef] [Medline]
- Kunneman M, Montori VM, Castaneda-Guarderas A, Hess EP. What is shared decision making? (and what it is not). Acad Emerg Med 2016 Dec;23(12):1320-1324 [FREE Full text] [CrossRef] [Medline]
- Brooke J. SUS: a quick and dirty usability scale. In: Jordan PW, Thomas B, MCClelland AL, editors. Usability Evaluation in Industry. London: Taylor and Francis; 1996:-94.
- van Maurik IS. Alzheimercentrum. ADappt overview 28-12-2018 URL: https://www.alzheimercentrum.nl/professionals/adappt/ [WebCite Cache]
- van Maurik IS. ADappt. URL: https://adappt.health/ [WebCite Cache]
- van Maurik IS. Alzheimercentrum Amsterdam. ADappt diagnostic tests URL: https://www.alzheimercentrum.nl/professionals/adappt-diagnostische-testen/ [WebCite Cache]
- Jaspers NE, Ridker PM, Dorresteijn JA, Visseren FL. The prediction of therapy-benefit for individual cardiovascular disease prevention: rationale, implications, and implementation. Curr Opin Lipidol 2018 Dec;29(6):436-444. [CrossRef] [Medline]
- Jaspers NE, Visseren FL, Numans ME, Smulders YM, van Loenen Martinet FA, van der Graaf Y, et al. Variation in minimum desired cardiovascular disease-free longevity benefit from statin and antihypertensive medications: a cross-sectional study of patient and primary care physician perspectives. BMJ Open 2018 May 26;8(5):e021309 [FREE Full text] [CrossRef] [Medline]
- Petersen RC, Lopez O, Armstrong MJ, Getchius TSD, Ganguli M, Gloss D, et al. Practice guideline update summary: mild cognitive impairment: report of the Guideline Development, Dissemination, and Implementation Subcommittee of the American Academy of Neurology. Neurology 2018 Jan 16;90(3):126-135. [CrossRef] [Medline]
- Ngandu T, Lehtisalo J, Solomon A, Levälahti E, Ahtiluoto S, Antikainen R, et al. A 2 year multidomain intervention of diet, exercise, cognitive training, and vascular risk monitoring versus control to prevent cognitive decline in at-risk elderly people (FINGER): a randomised controlled trial. Lancet 2015 Jun 6;385(9984):2255-2263. [CrossRef] [Medline]
|ABIDE: Alzheimer biomarkers in daily practice|
|AD: Alzheimer disease|
|CSF: cerebrospinal fluid|
|CSS: cascading style sheets|
|CT: computer tomography|
|MCI: mild cognitive impairment|
|MRI: magnetic resonance imaging|
|PET: positron emission tomography|
|SUS: system usability scale|
Edited by G Eysenbach; submitted 16.01.19; peer-reviewed by E Zafeiridi, S Chokshi; comments to author 28.03.19; revised version received 30.04.19; accepted 30.04.19; published 08.07.19
©Ingrid S van Maurik, Leonie NC Visser, Ruth E Pel-Littel, Marieke M van Buchem, Marissa D Zwan, Marleen Kunneman, Wiesje Pelkmans, Femke H Bouwman, Mirella Minkman, Niki Schoonenboom, Philip Scheltens, Ellen MA Smets, Wiesje M van der Flier. Originally published in JMIR Formative Research (http://formative.jmir.org), 08.07.2019.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license information must be included.