Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 08.07.19 in Vol 3, No 3 (2019): Jul-Sep

Preprints (earlier versions) of this paper are available at, first published Jan 16, 2019.

This paper is in the following e-collection/theme issue:

    Original Paper

    Development and Usability of ADappt: Web-Based Tool to Support Clinicians, Patients, and Caregivers in the Diagnosis of Mild Cognitive Impairment and Alzheimer Disease

    1Alzheimer Center Amsterdam, Department of Neurology, Amsterdam Neuroscience, Vrije Universiteit Amsterdam, Amsterdam UMC, Amsterdam, Netherlands

    2Department of Epidemiology and Biostatistics, Vrije Universiteit Amsterdam, Amsterdam UMC, Amsterdam, Netherlands

    3Department of Medical Psychology, Amsterdam Public Health Research Insitute, University of Amsterdam, Amsterdam UMC, Amsterdam, Netherlands

    4Vilans Center of Expertise for Long Term Care, Utrecht, Netherlands

    5Medical Decision Making, Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, Netherlands

    6Tilburg University, TIAS School for Business and Society, Tilburg, Netherlands

    7Department of Neurology, Spaarne Gasthuis, Haarlem, Netherlands

    Corresponding Author:

    Ingrid S van Maurik, MSc

    Alzheimer Center Amsterdam, Department of Neurology

    Amsterdam Neuroscience

    Vrije Universiteit Amsterdam, Amsterdam UMC

    De Boelelaan 1118

    Amsterdam, 1081 HZ


    Phone: 31 204440816

    Fax:31 204448529



    Background: As a result of advances in diagnostic testing in the field of Alzheimer disease (AD), patients are diagnosed in earlier stages of the disease, for example, in the stage of mild cognitive impairment (MCI). This poses novel challenges for a clinician during the diagnostic workup with regard to diagnostic testing itself, namely, which tests are to be performed, but also on how to engage patients in this decision and how to communicate test results. As a result, tools to support decision making and improve risk communication could be valuable for clinicians and patients.

    Objective: The aim of this study was to present the design, development, and testing of a Web-based tool for clinicians in a memory clinic setting and to ascertain whether this tool can (1) facilitate the interpretation of biomarker results in individual patients with MCI regarding their risk of progression to dementia, (2) support clinicians in communicating biomarker test results and risks to MCI patients and their caregivers, and (3) support clinicians in a process of shared decision making regarding the diagnostic workup of AD.

    Methods: A multiphase mixed-methods approach was used. Phase 1 consisted of a qualitative needs assessment among professionals, patients, and caregivers; phase 2, consisted of an iterative process of development and the design of the tool (ADappt); and phase 3 consisted of a quantitative and qualitative assessment of usability and acceptability of ADappt. Across these phases, co-creation was realized via a user-centered qualitative approach with clinicians, patients, and caregivers.

    Results: In phase 1, clinicians indicated the need for risk calculation tools and visual aids to communicate test results to patients. Patients and caregivers expressed their needs for more specific information on their risk for developing AD and related consequences. In phase 2, we developed the content and graphical design of ADappt encompassing 3 modules: a risk calculation tool, a risk communication tool including a summary sheet for patients and caregivers, and a conversation starter to support shared decision making regarding the diagnostic workup. In phase 3, ADappt was considered to be clear and user-friendly.

    Conclusions: Clinicians in a memory clinic setting can use ADappt, a Web-based tool, developed using multiphase design and co-creation, for support that includes an individually tailored interpretation of biomarker test results, communication of test results and risks to patients and their caregivers, and shared decision making on diagnostic testing.

    JMIR Form Res 2019;3(3):e13417




    Dementia is a syndrome diagnosis that is used to describe decline in cognitive functioning that is severe enough to result in a loss of independence in performing everyday activities [1]. A growing proportion of individuals presenting at memory clinics do not (yet) fulfill the criteria for dementia. These individuals are labeled as having mild cognitive impairment (MCI) if cognitive impairment can be objectified [2]. In the course of 3 years, roughly half of the MCI patients develop dementia, whereas the other half remain stable or improve [3]. Therefore, the label of MCI entails a prognosis, rather than a diagnosis.

    The prognosis of MCI patients depends on the etiology of the symptoms. The most common underlying cause is Alzheimer disease (AD), a neurodegenerative disease that develops gradually, with dementia as the final stage [4]. AD biomarkers, assessed through, for example, magnetic resonance imaging (MRI) or cerebrospinal fluid (CSF), reflect AD related pathological processes and can therefore provide information on the underlying cause of cognitive impairment [5]. In early stages of the disease, AD biomarkers are particularly valuable, as this information allows a more precise estimate of the patient’s risk of developing dementia [6]. However, interpreting and communicating biomarker results is complex, as the understanding of probabilistic information is known to be difficult for clinicians and patients and caregivers [7]. Moreover, test results may be unclear or conflicting and thus may not always offer the certainty that clinicians, patients, and caregivers are looking for [8].

    As a result, the clinician has to face a growing number of challenges during the diagnostic workup, for example, which and how many diagnostic (biomarker) tests should be performed, how much and what type of information does the patient actually want, what can we expect from a specific diagnostic test, does a patient prefer to be provided with information on the likely course of their symptoms, and how can this information best be conveyed. Given that patients may weigh potential benefits and harms of AD biomarker testing differently, considering the patients’ preferences and needs are essential when making decisions for or against biomarker testing and the disclosure of results. In this situation, where an increasing number of reasonable options emerge to address the patients’ situation, clinicians and patients should engage in a process of shared decision making to ensure that the decisions made best fit the individual [9]. However, in a memory clinic setting, involving patients in decision making is often limited to providing information only [9,10], and explicit risk communication about the development of dementia is barely observed in individuals with MCI. This demonstrates that there is room to support clinician-patient communication [9].

    In the context of the Alzheimer Biomarkers in Daily Practice (ABIDE) project, we aimed to develop a Web-based tool for clinicians in a memory clinic setting that (1) provides personalized risk estimates of progression to dementia to aid the clinician in interpreting test results, (2) supports clinicians in communicating biomarker test results and risks to MCI patients and their caregivers, and (3) supports clinicians to engage patients in decision making regarding the diagnostic workup of AD. Following a user-centered design, this paper describes the process of development, usability, and acceptability of the Web-based diagnostic support tool named ADappt.


    Study Design

    This study was conducted as part of the ABIDE project, which has been funded in the context of the Dutch Deltaplan Dementia [11]. ABIDE has been designed to improve and facilitate the use and interpretation of AD biomarkers in clinical practice, taking into account patients’ preferences toward diagnostic testing and communication of test results. Here, we describe the development and testing of the usability and acceptability of ADappt, a Web-based tool to support clinicians in the diagnostic process of AD in patients with MCI. The tool consists of 3 modules: (1) risk calculation: this module provides personalized risk estimates of progression to dementia to aid the clinician in interpreting test results (MRI and CSF); (2) risk communication: this module provides a summary of the biomarker results, including a graphic representation of the risk to facilitate communication of test results; (3) shared decision making: this module entails a conversation starter to support clinicians to engage patients in shared decision making regarding diagnostic testing.

    Figure 1. Study overview. AD: Alzheimer disease; SDM: shared decision making.
    View this figure

    The overall study adopted a user-centered mixed-method approach with emphasis on co-creation and was conducted in the 3 phases described in Figure 1. The board of the medical ethics review committee of the Vrije Universiteit (VU) medical center reviewed and approved this study. For each module, input and feedback of end users (professionals, patients, and caregivers) was obtained. Modules were developed in independent yet overlapping trajectories such that module 1 was developed first, closely followed by module 2. Module 3 was developed last, based on a set of earlier studies conducted in the context of the ABIDE project [9-10,12].

    Thematic content analysis was used to analyze the qualitative data collected during focus groups, panels, and one-on-one interviews [13]. In total, 2 coders (IvM and LV), with a background in psychology, used an inductive approach to independently formulate lists of themes that appeared from summarized data. The themes were discussed to reach consensus.

    Module 1: Risk Calculation

    Phase 0: Defining the Context

    In 2016, the Geneva Task Force on the roadmap of Alzheimer biomarkers published a strategic roadmap to foster the clinical validation of AD biomarkers [14]. This roadmap adopted a 5-phase approach from cancer research and dedicated 1 phase to the diagnostic and prognostic performance of biomarkers in MCI patients. Both for MRI and CSF biomarkers, evidence still appeared incomplete [15,16]. In particular, limited evidence was available on the combination of biomarkers and how they perform to predict prognosis in individual MCI patients. In a previous study, we therefore constructed individualized prediction models that provide personalized risk estimates for MCI patients, on the basis of MRI, CSF, or the combination of these 2 biomarkers based on data from the Amsterdam Dementia Cohort [6,17,18]. Probabilities (with CIs) of progression to AD dementia for 1 and 3 years of follow-up can be calculated using these models (for a detailed description see van Maurik et al [6]). These models serve as input for the risk calculation module of ADappt.

    Phase 1: Needs Assessment
    Focus Group Clinicians

    We organized a 2-hour focus group (n=8, 7 neurologists and 1 geriatrician) led by experienced focus group leaders (RP and MK). During this focus group, we discussed the interpretation of biomarker test results and communication of results in terms of risks to patients based on 2 clinical cases that were shortly introduced. Used examples of risk visualizations were based on a commonly used format in cardiovascular risk management [19] and breast cancer therapy risks (Adjuvant! Online) [20].

    Phase 2: Design and Development
    Test Functional Design

    A prototype of the risk calculation module (module 1) was developed from the results of phase 1. Functionality of the initial design was tested by end users in 2 rounds. In both rounds, end users were asked, via a questionnaire, if they considered the module to be logical, clear, and useful. In the first round, a neurologist, a geriatrician, and 2 researchers provided input on a preliminary design and layout (risk calculation functionality was not yet available). From their input the prototype was further developed. Next, 9 professionals (5 neurologists, 2 geriatricians, and 2 health care professionals) provided input on the prototype with a fully functioning risk calculation.

    Phase 3: Usability and Acceptability
    Questionnaire Feedback Prototype

    During the annual Dutch Dementia Conference (Dementia Update) in 2017, the risk calculation module was presented and interested attendees were given the opportunity to try out the prototype. A total of 24 attendees (4 geriatricians, 10 neurologists, 5 with another profession [internist or nursing home physician specialist], and 5 did not indicate their profession) provided written feedback by filling out a brief questionnaire.

    Module 2: Risk Communication

    Phase 0: Defining the Context

    In a previous survey study [10], clinicians indicated that they find it difficult to convey MCI as a diagnosis because of the uncertainty of the diagnosis of MCI and the lack of treatment. Nonetheless, the majority of clinicians indicated to always disclose the risk of developing dementia [10]. This however does not sufficiently fulfill the information need of patients and caregivers, as patients expressed a wish for more information on the prognosis of the disease [12].

    In risk communication, numeric formats are generally preferred, relative to other formats, to increase patients’ understanding [7,21,22]. For example, verbal communication of risk (unlikely, possible, and rare) is vulnerable to a high degree of variability in interpretation and therefore not considered a best practice [7]. Among numeric formats, natural frequencies (20 out of 100 people similar to you) are favored over other formats (probabilities, odds, or classical probabilities), as a reference class is included that reduces misinterpretation [7,23]. A numeric format should ideally be complemented with a visual representation of risk and include a specific time frame in which an event may occur [24].

    Phase 1: Needs Assessment
    Focus Group Clinicians

    Input from clinicians with regard to risk communication was obtained in the same focus group as described in phase 1 for module 1 (see the section above for details).

    Focus Group Patients and Caregivers

    A total of 4 1-hour focus groups were organized for MCI patients and were led by an experienced focus group leader (RP). A total of n=13 patients participated (3 to 4 participants per session, mean age 66 years (SD 8); 31% (4/13) female) and 63% (8/13) were accompanied by a caregiver (all partners, n=4 (50%) female). The topics that were discussed included how patients and caregivers looked back on receiving the diagnosis and test results and whether they had any recommendations for the disclosure of test results. In a second part, participants brainstormed hands-on in a paper, pencil, and scissors session on useful (graphical) summaries of test results.

    Phase 2: Design and Development
    Panel Clinicians

    During the annual Dutch Dementia Symposium (Dementia Update) in 2017, clinicians were invited to attend a short panel discussion on their preferences about tools to support communication of test results, in terms of graphics, written summaries, and explanations of risks and uncertainty. A total of 5 clinicians attended this panel discussion and provided feedback.

    Individual Interviews With Patients and Caregivers

    To evaluate the first prototypes of the risk communication module, we conducted 5 interviews, each with 1 MCI patient (mean age 65 years [SD 9], n=2 (40%) female) and his or her caregiver (all partners, n=3 (60%) female). Interviews took 15 to 30 min and were conducted by the software developer to ensure that the feedback of patients was optimally used to improve this module (co-creation). Patients were presented with an example of the module, in which the diagnostic test results of a fictive case of a man or woman of 60 years were displayed, and patients were asked for their input on the storyline, graphics, style, and the possibility to print these results.

    Module 3: Shared Decision-Making Module

    Phase 0/1: Defining the Context/Needs Assessment

    In shared decision making, clinicians and patients work together to decide which care plan best fits with individual preferences and needs when there is more than 1 reasonable option [25,26]. In the diagnostic routine for AD, there is typically more than 1 reasonable option to choose, making this a situation where shared decision making is the preferred approach. However, in a previous study, we found that shared decision making in the memory clinic is often limited to information giving [10], and shared decision making involves the following 4 steps: (1) create choice awareness, (2) provide information, (3) explore preferences, and (4) decide together [27,28].

    Phase 2: Design and Development

    For each step in shared decision making, we constructed example phrases in co-creation with communication experts (LV and ES). These phrases might function as a conversation starter for clinicians to engage patients in shared decision making. To inform a patient on the possible pros and cons, we also developed a list with the example language on (dis)advantages for the following commonly used diagnostic tests: neuropsychological investigation, imaging (MRI or computer tomography [CT]), lumbar puncture, amyloid Positron Emission Tomography (PET), consultation with other specialist (neurologist, geriatrician, and psychiatrist), clinical geneticist, and awaiting policy. This list was based on the literature and expert opinions (FB and WF).

    All Modules: Pilot Study in 4 Local Memory Clinics

    To test the ADappt tool comprising all 3 modules, we organized a multicenter usability pilot study. Both clinicians and patients were asked for feedback in 4 local memory clinics (Reinier de Graaf Ziekenhuis, Elisabeth Tweesteden Ziekenhuis, Jeroen Bosch Ziekenhuis, and Spaarne Gasthuis) and 1 academic memory clinic (Amsterdam University Medical Center (UMC)). Participating clinicians (4 neurologists, 1 geriatrician, and 2 not specified) were asked to use the tool for a minimum of 2 weeks and a maximum of 7 weeks. One of the authors (MvB) was present in the memory clinics to assist clinicians in the first week of the pilot. Clinicians were asked to complete the System Usability Scale (SUS) after using the tool [29]. The maximum score on the SUS is 5, and a higher score indicates better usability. Technical details of the tool are summarized in Textbox 1.

    Textbox 1. Technical development of the tool.
    View this box


    Module 1: Risk Calculation Module

    Here, we describe the results of the co-creation steps leading to the final layout of the risk calculation module shown in Figure 2.

    Phase 1: Needs Assessment
    Focus Group Clinicians

    On reviewing the discussion on the preferences with regard to risk calculation, 3 main findings emerged (the findings and adaptations are presented in Table 1). First, clinicians preferred the reporting of percentages over a risk table or bar chart, as in the latter, the information was perceived not to be clear at one glance. Second, regarding reliability and validity; clinicians whished information about CIs and information on how the models were constructed and how they perform. Therefore, we included a link to the publication, explaining how the models were constructed. Finally, clinicians emphasized user-friendliness of the module, for example, every professional should be able to use the module.

    Figure 2. Final design of the risk calculation module (module 1).Left: start page with disclaimer. Middle: risk calculation module based on demographic information only (probability of AD dementia without biomarkers). As an example, we entered data of a fictional female MCI patient, age 67 years and mini mental state examination (MMSE) score of 28, resulting in a 1 year probability of progression of 13% and 3 year probability of 47%. Right: adding biomarkers for the same patient; Hippocampal atrophy (visually rated with Medial temporal lobe (MTA) atrophy scale), global cortical atrophy (GCA), Abeta, and and total tau. In this case, the normal biomarkers (the tool calculates with continuous data) resulted in a strong decrease of progression probabilities. To appreciate the change in probabilities, the tool repeats the initial risk based on demographic data at the lower part of the module.
    View this figure
    Table 1. Overview of themes for each focus group.
    View this table
    Phase 2: Design and Development
    Test Functional Design

    End users agreed on the proposed order and name of the tool. In general, the information in the module was considered to be clear. Suggestions for improvement included (1) a clear disclaimer that the module calculates risk of AD dementia only, (2) to use a different term instead of a priori and a posteriori risk, (3) adding value ranges, and (4) the module should also work when only CSF or MRI values were entered.

    Phase 3: Usability and Acceptability
    Questionnaire Feedback Prototype

    The responses of the n=24 clinicians are summarized in Table 2. The applicability and user-friendliness of the module was rated highest. More than half of the respondents indicated that they would use the final version of the module in the future. The perceived reliability of the module was rated relatively low.

    Table 2. Overview of first round of usability rating.
    View this table

    Module 2: Risk Communication Module

    To facilitate explanation of the test results, we developed in co-creation (steps described below) a risk communication sheet. The final layout is shown in Figure 3. The final design of the summary sheet (module 2). Here we use the same example as in Figure 2. First, a general description of MCI is given, followed by the personalized results of this person. For each test, we give a summary of what kind of information is retrieved from the test, followed by the interpretation of the test result and how it effects the chance of AD dementia. Finally, we summarize the findings and give a verbal explanation (1 out of 100) of risk and visualization of this risk, either with an array grid or bar chart (not shown). Of note, the MRI is a stock image, not personalized.

    Phase 1: Needs Assessment
    Focus Group Clinicians

    With regard to risk communication, clinicians mentioned that they often show brain images to patients to support communication of test results and that they provide patients with written information (findings and adaptations are presented in Table 1). In addition, clinicians confirmed that a graphical representation of risk only is not sufficient to explain a risk to patients and caregivers. Therefore, we accompanied the visual presentation of risk with a textual explanation.

    Focus Group Patients and Caregivers

    An important theme during the focus group with patients and caregivers was that patients would like to know their exact progression risk. Other themes with regard to risk were that explanations should be simple and that they would like to compare their results with a normal reference category. In addition, patients indicated to struggle with how they should communicate their diagnosis and test results with family and friends (Table 1). In terms of visualization, patients preferred to see their own brain images. However, in view of ever-more complicated privacy regulations, we decided not to include this in the summary sheet. Patients would also like to know how the test results contributed to the presented risk. Individual differences emerged with regard to the preference for the visualization of risk (bar chart or icon arrays). Therefore, we decided to incorporate both options in the module, to be used depending on patients’ preference. Finally, patients and caregivers would like to be provided with information on how to handle the diagnosis, what they can do in terms of lifestyle changes and which steps they should take next (for example a case manager). As this is likely to be different for each patient, we included an open field at the end of the summary sheet that can be used by the clinician to include such information.

    Phase 2: Design and Development
    Panel Clinicians

    One of the themes emerging from the panel discussion with clinicians is that they prefer to communicate test results in a dichotomous way (normal or abnormal, Table 1). However, biomarker results are originally continuous in nature, and in terms of interpretation, there is always a gray area. A possible way to visualize this is by using a traffic light. Therefore, we included a traffic light for all biomarkers on the summary sheet, with red indicating abnormal, green indicating normal, and gray indicating borderline results. As an alternative to including the MRI or CT images of patients, clinicians suggested to implement an example MRI image on the written summary for patients. For risk visualization, clinicians suggested to use a line chart with time on the horizontal axis. However, as the risk calculation module only provides valid predictions up to 3 years, a line graph might incorrectly imply that this line could be extrapolated to longer follow-up periods. For this reason, we decided not to make use of this suggestion. Instead, we included 2 options for risk visualization: a bar chart and an icon array. Finally, clinicians emphasized that the communication module should be an aid, allowing for tailoring of the communication to individual patients, rather than a strict protocol. This whish is met by providing an open text field in the summary sheet, where the clinician can add information.

    Individual Interviews

    During the individual interviews with patients and caregivers, the storyline of the printout was considered as too long and too complex. Therefore, 2 communication experts reviewed, optimized, and abbreviated the text (LV and ES). Moreover, we removed the a priori risk (ie, risk without biomarkers, see example in Figure 2) from the summary sheet for patients as this was experienced as confusing and sometimes daunting. Patients and caregivers only valued the probability based on the diagnostic tests. Patients would like to see their individual test results first, followed by their overall risk as a result of these individual test results. The graphical presentation of risk as either a bar chart or icon array worked well and the printout page was considered valuable. Themes and adaptations are summarized in Table 1.

    Figure 3. Final design of the risk communication module (module 2). Here we use the same example as in Figure 2. First, a general description of MCI is given, followed by the personalized results of this person. For each test, we give a summary of what kind of information is retrieved from the test, followed by the interpretation of the test result and how it effects the chance of AD dementia. Finally, we summarize the findings and give a verbal explanation (1 out of 100) of risk and visualization of this risk, either with an array grid or bar chart (not shown). Of note, the MRI is a stock image, not personalized.
    View this figure

    Module 3: Shared Decision-Making Module

    Phase 2: Design and Development

    Table 3 presents the conversation starter with example phrases for a clinician to engage patients and caregivers in each of the 4 steps of shared decision making (create choice awareness, provide information, explore preferences, and decide together [27,28]). In Table 4, we show example language that can be used to inform a patient on pros and cons with regard to imaging. Phrases for other diagnostic tests can be found on our website [32].

    Table 3. Example phrases to start shared decision making.
    View this table
    Table 4. Example phrases for magnetic resonance imaging (MRI) and computed tomography (CT).
    View this table

    Pilot Study in 4 Local Memory Clinics

    In total, 7 clinicians from 5 memory clinics commented on the ADappt tool and 5 of them completed the survey. The usability of the tool was rated with a score of SUS=4.4 (SD 0.5, max score=5; see Figure 4). Moreover, clinicians rated the tool as well integrated, easy, and consistent. A total of 3 clinicians used the tool in their daily practice, of which 1 used it on a regular basis during the pilot. Figure 1 summarizes the response frequencies of each question in the survey:

    […] it was very useful, because this patient had somewhat conflicting biomarkers, which did not point in one direction, so when using the app, it resulted in a much lower risk than that these people expected, and actually they went home very satisfied.
    […] I did use the website, but mainly the result page to provide the patient with more insight in what it could mean when it results in MCI and what the risk on dementia eventually is and that you can give them a printout to take home, that is very positively received also by the patient.
    Figure 4. Overview of pilot results: responses of clinicians to each question of the system usability scale.
    View this figure

    Clinicians experienced some technical problems while using the tool. For example, the site was blocked by the firewall in 1 of the hospitals, leading to the need to use it on a personal device. Some clinicians commented that they already feel time pressure during a consult and do not find the time to actually open the tool in the consult. A total of 4 patients (n=3 (75%) female, mean age 72 years [SD 2]) completed a short questionnaire after the consult in which the tool was used. The result page was valued as clear and useful by all. The result pages helped them to better understand the diagnosis and helped them explain the results to relatives and/or friends. These patients recommended the result page to other patients.


    This study reports the development and initial usability testing of ADappt—a Web-based tool to facilitate risk calculation and support clinician-patient risk communication and shared decision making in a memory clinic setting. We worked closely with end users, clinicians, patients, and caregivers, to identify initial requirements for the tool and make necessary changes to enhance the functionalities, usability, and information provided by the tool. Needs assessment (phase 1) uncovered the need of patients and caregivers for written summaries and information on prognosis. Clinicians expressed their need for visual aids to communicate results and prognosis. In the development phase (phase 2), we developed a risk calculation tool for clinicians and a risk communication tool (summary sheet) for patients and caregivers. In addition, a conversation starter was developed to support clinicians and patients in decision making around diagnostic testing.

    Already in an early stage of usability testing (phase 3), the tool was considered clear and user-friendly, and a majority of memory clinic professionals indicated they would use such a tool in the future. However, clinicians appeared to have concerns about the reliability of the risk prediction tool. At the moment of testing, the algorithms were not yet published, and therefore a reference to a detailed description of the models was not included in the tool. To date, the module has been updated with the published algorithms [6]. For the tool to be used in real clinical practice, generalizability of prediction models in the risk calculation module is essential. Therefore, we are generating updated risk models based on data from a range of multi and monocenter cohorts and in alignment with the recently published National Institute on Aging and Alzheimer’s Association (NIA-AA) criteria [5]. These updated models will be applicable to CSF biomarkers and volumetric MRI markers measured with different assays and platforms. In addition, we will add models based on amyloid PET. Another short-term update will be the translation of the tool into English.

    Our small pilot study demonstrated that the tool, especially the printout page (risk communication module), was considered valuable by patients and caregivers. Although the usability of the tool was rated to be high by clinicians, only few clinicians actually used the tool in clinical practice. This was mainly due to practical reasons; clinicians experienced technical hurdles for the use of such a tool in clinical practice (eg, computers in consulting room firewalled). Despite these hurdles and that relying on computer algorithms may be counterintuitive for clinicians, experience from different disease areas shows that computer algorithms are welcome in clinical practice and are seen as the future of medicine. For example, in oncology clinicians use Adjuvant Online to calculate the risk of recurrence with and without adjuvant therapy for breast cancer patients [20]. Recently U-Prevent was introduced for clinicians to calculate cardiovascular risk and therapy benefit [33,34]. The major challenge for ADappt is its large-scale implementation. In an ideal situation, ADappt would be embedded in electronic patient file systems. Alternatively, ADappt can be used as a simple add-on help for both the clinician and patient to be quickly consulted on their mobile phone. In the short term, an e-learning could be useful to show how the clinicians can integrate the tool in their routine. Moreover, the amount of text in the tool was still experienced as a barrier by clinicians and could be further improved by using graphics instead of plain text.

    The 3 modules of ADappt could aid clinicians and patients in a number of ways. In the diagnostic process for AD, it is currently not common practice to actively engage patients in diagnostic decision making [10,25]. ADappt provides a conversation starter and an overview of pros and cons per diagnostic test and could therefore help clinicians to engage patients and caregivers in diagnostic decision making [9]. In an observational study of ABIDE, we currently examine communication and decision-making processes during pre- and postdiagnostic clinician-patient encounters. This study will provide input to further improve the shared decision-making module, as soon as the results of this study become available [11].

    Second, ADappt supports clinicians in the communication of test results to patients and contributes to personalized diagnostic care and harmonization of clinical practice. After diagnostic testing, the interpretation and communication of test results, especially when patients are not yet demented (ie, MCI) is not straightforward. The ADappt tool clearly shows whether a patient is unlikely to develop AD dementia in the following years and therefore could be reassured, or whether a patient is indeed likely to convert to AD dementia and should therefore be followed up, arranged care, or referred for participation in clinical trials. The importance of an appropriate diagnosis and prognosis of MCI is acknowledged in current guidelines. These guidelines state that an appropriate diagnosis of MCI is important for patients and caregivers to understand the cause of their complaints and to arrange care planning based on communicated prognostic probabilities [35]. Moreover, clear communication of the test results and the corresponding prognosis has been given high priority by patients and caregivers. Tools similar to ADappt can help in this regard. One of the strengths of this study is that we designed and co-created ADappt with the end users, that is, clinicians, patients, and caregivers in memory clinic settings, to optimize future use of the tool. Most suggestions and whishes were met in the design and development of the tool. Only 2 suggestions did not find their way into the tool immediately. First, a recurring theme was the use of CT or MRI images to support the communication of diagnosis and test results. Both clinicians and patients indicated that they would like to see a patients’ CT or MRI image to be included on the summary sheet. However, this would require the summary sheet to make a link to, for example, the electronic patient file that is challenging, both from a technical and privacy perspective. As an alternative, we suggested that the clinician shows the patient their own MRI on screen in the consulting room, and we included an example MRI on the summary sheet. Second, in early phases of the development of the summary sheet, patients indicated that they would like to see their results in comparison with a reference class (ie, what is normal for my age). In the first design of the summary sheet, this reference class was presented as separate probabilities and patients could compare this with their personal risk. However, this led to confusion and patients suggested that this should be removed from the summary sheet. Another strength of the tool is its compatibility with the most recent versions of the most commonly used Web browsers and could be used on all commonly used devices (computer and mobile phone). Moreover, the tool is designed as a flexible platform with the idea that extra modules can easily be added in the future. This would also enable adding modules on lifestyle advice [36], extension to other biomarkers or settings, such as the general practitioner, or to advanced care planning.

    Among the limitations is that other factors, such as cardiovascular disease on MRI, cardiovascular risk factors, and amyloid PET biomarkers are currently not accounted for in the risk prediction. Clinicians do consider these factors to be important for prognosis in risk prediction and should therefore be accounted for in the models. However, the algorithms incorporated in the tool had a different starting point—namely, how to extract maximum information from MRI and CSF biomarker values for an individual patient, given that the clinician decided to order these tests. For a user-centered development process, the number of participants in this project was adequate. However, to achieve more generalizable results with regard to implementation or cost-effectiveness, future validation should include larger samples and a different study design.

    In conclusion, this study presents the first tool to support clinicians and patients in memory clinic settings with decisions on diagnostic testing, individual tailored interpretation of diagnostic test results, and communication of test results. At the moment, the tool is available for academic use only. The tool is developed in a multiphase design, where co-creation with end users was an important feature. Owing to its flexibility, it is possible to add extra modules, guidelines, or new prediction models in the future. Moreover, as the tool currently focuses on clinicians, we envision that in the future a similar platform would be valuable for patients and caregivers to facilitate them to engage in shared decision making and to aid them in managing their own health care trajectory.


    Research of the Alzheimer center, Amsterdam, is part of the neurodegeneration research program of Amsterdam Neuroscience. The Alzheimer Center, Amsterdam, is supported by Stichting Alzheimer Nederland and Stichting VUmc fonds. Wiesje van der Flier holds the Pasman chair. The authors would like to thank Stichting Buytentwist for their valuable contribution. This study is funded by ZonMW-Memorabel (ABIDE; project no 733050201), a project in the context of the Dutch Deltaplan Dementie.

    ABIDE study group: Amsterdam, the Netherlands (Alzheimer Center and Department of Neurology, Amsterdam Neuroscience, VU University Medical Center): Wiesje M van der Flier, PhD; Philip Scheltens, MD, PhD; Femke H Bouwman, MD, PhD; Marissa D Zwan, PhD; Ingrid S van Maurik, MSc; Arno de Wilde, MD; Wiesje Pelkmans, MSc; Rosha Babapour Mofrad, MSc; Silvia Ingala, MSc; Colin Groot, MSc; Ellen Dicks, MSc; Els Dekkers (Department of Radiology and Nuclear Medicine, Amsterdam Neuroscience, VU University Medical Center); Bart NM van Berckel, MD, PhD; Frederik Barkhof, MD, PhD; Mike P Wattjes, MD, PhD (Neurochemistry laboratory, Department of Clinical Chemistry, Amsterdam Neuroscience, VU University Medical Center); Charlotte E Teunissen, PhD; Eline A Willemse, MSc (Department of Medical Psychology, University of Amsterdam, Academic Medical Center); Ellen M Smets, PhD; Leonie NC Visser, PhD; Marleen Kunneman, PhD; Sophie Pelt; Sanne Schepers, MSc; Laxini Murugesu, MSc; Bahar Azizi, MSc; Anneke Hellinga, MSc (BV Cyclotron); E van Lier, MSc; Haarlem, the Netherlands (Spaarne Gasthuis) Niki M Schoonenboom, MD, PhD; Utrecht, the Netherlands (Department of Neurology and Neurosurgery, Brain Center Rudolf Magnus, University Medical Center Utrecht); Geert Jan Biessels, MD, PhD; Jurre H Verwer, MSc (Department of Geriatrics, University Medical Center Utrecht); Dieneke H Koek, MD, PhD (Department of Radiology and Nuclear Medicine); Monique G Hobbelink, MD (Vilans, Center of Expertise in long term care); Mirella M Minkman, PhD; Cynthia S Hofman, PhD; Ruth Pel, MSc; Meppel, the Netherlands (Espria) Esther Kuiper, MSc; Berlin, Germany (Piramal Imaging GmbH) Andrew Stephens, MD, PhD; Rotrkreuz, Switzerland (Roche Diagnostics International Ltd) Richard Bartra-Utermann, MD.

    Memory clinic panel: The members of the memory clinic panel are Niki M Schoonenboom, MD, PhD (Spaarne Gasthuis, Haarlem); Barbera van Harten, MD, PhD; Niek Verwey, MD, PhD; Peter van Walderveen, MD (Medisch Centrum Leeuwarden, Leeuwarden); Ester Korf, MD, PhD (Admiraal de Ruyter Ziekenhuis, Vlissingen); Gerwin Roks, MD, PhD (Sint Elisabeth Ziekenhuis, Tilburg); Bertjan Kerklaan, MD, PhD (Onze Lieve Vrouwe Gasthuis, Amsterdam); Leo Boelaarts, MD (Medisch Centrum Alkmaar, Alkmaar); Annelies WE Weverling, MD (Diaconessenhuis, Leiden); Rob J van Marum, MD, PhD (Jeroen Bosch Ziekenhuis, ‘s-Hertogenbosch); Jules J Claus, MD, PhD (Tergooi Ziekenhuis, Hilversum); Koos Keizer, MD, PhD (Catherina Ziekenhuis, Eindhoven).

    Conflicts of Interest

    PS has acquired grant support (for the institution) from GE Healthcare, Danone Research, Piramal, and Merck. In the past 2 years, he has received consultancy/speaker fees (paid to the institution) from Lilly, GE Healthcare, Novartis, Sanofi, Nutricia, Probiodrug, Biogen, Roche, Avraham, and EIP Pharma. WMvdF performs contract research for Biogen MA Inc. Research programs of WMvdF have been funded by ZonMW, Health Holland, Pasman stichting, NWO, EU-FP7, EU-JPND, Alzheimer Nederland, Cardiovasculair Onderzoek Nederland, stichting Dioraphte, Gieskes-Strijbis fonds, Boehringer Ingelheim, Piramal Neuroimaging, Roche BV, Janssen Stellar, and Combinostics. All funding is paid to her institution. All other coauthors report no conflicts of interest.


    1. Citron M. Alzheimer's disease: strategies for disease modification. Nat Rev Drug Discov 2010 May;9(5):387-398. [CrossRef] [Medline]
    2. Albert MS, DeKosky ST, Dickson D, Dubois B, Feldman HH, Fox NC, et al. The diagnosis of mild cognitive impairment due to Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease. Alzheimers Dement 2011 May;7(3):270-279 [FREE Full text] [CrossRef] [Medline]
    3. Visser PJ, Verhey F, Knol DL, Scheltens P, Wahlund L, Freund-Levi Y, et al. Prevalence and prognostic value of CSF markers of Alzheimer's disease pathology in patients with subjective cognitive impairment or mild cognitive impairment in the DESCRIPA study: a prospective cohort study. Lancet Neurol 2009 Jul;8(7):619-627. [CrossRef] [Medline]
    4. Scheltens P, Blennow K, Breteler MM, de Strooper B, Frisoni GB, Salloway S, et al. Alzheimer's disease. Lancet 2016 Jul 30;388(10043):505-517. [CrossRef] [Medline]
    5. Jack CR, Bennett DA, Blennow K, Carrillo MC, Dunn B, Haeberlein SB, et al. NIA-AA Research Framework: toward a biological definition of Alzheimer's disease. Alzheimers Dement 2018 Apr;14(4):535-562 [FREE Full text] [CrossRef] [Medline]
    6. van Maurik IS, Zwan MD, Tijms BM, Bouwman FH, Teunissen CE, Scheltens P, Alzheimer’s Disease Neuroimaging Initiative. Interpreting biomarker results in individual patients with mild cognitive impairment in the Alzheimer's biomarkers in daily practice (ABIDE) project. JAMA Neurol 2017 Dec 1;74(12):1481-1491 [FREE Full text] [CrossRef] [Medline]
    7. Lipkus IM. Numeric, verbal, and visual formats of conveying health risks: suggested best practices and future recommendations. Med Decis Making 2007;27(5):696-713. [CrossRef] [Medline]
    8. Banningh LJ, Vernooij-Dassen M, Rikkert MO, Teunisse J. Mild cognitive impairment: coping with an uncertain label. Int J Geriat Psychiatry 2008 Feb;23(2):148-154. [CrossRef]
    9. van der Flier WM, Kunneman M, Bouwman FH, Petersen RC, Smets EM. Diagnostic dilemmas in Alzheimer's disease: room for shared decision making. Alzheimers Dement (N Y) 2017 Sep;3(3):301-304 [FREE Full text] [CrossRef] [Medline]
    10. Kunneman M, Smets EMA, Bouwman FH, Schoonenboom NS, Zwan MD, Pel-Littel R, et al. Clinicians' views on conversations and shared decision making in diagnostic testing for Alzheimer's disease: the ABIDE project. Alzheimers Dement (N Y) 2017 Sep;3(3):305-313 [FREE Full text] [CrossRef] [Medline]
    11. de Wilde A, van Maurik IS, Kunneman M, Bouwman F, Zwan M, Willemse EA, et al. Alzheimer's biomarkers in daily practice (ABIDE) project: rationale and design. Alzheimers Dement (Amst) 2017;6:143-151 [FREE Full text] [CrossRef] [Medline]
    12. Kunneman M, Pel-Littel R, Bouwman FH, Gillissen F, Schoonenboom NS, Claus JJ, et al. Patients' and caregivers' views on conversations and shared decision making in diagnostic testing for Alzheimer's disease: the ABIDE project. Alzheimers Dement (N Y) 2017 Sep;3(3):314-322 [FREE Full text] [CrossRef] [Medline]
    13. Vaismoradi M, Turunen H, Bondas T. Content analysis and thematic analysis: implications for conducting a qualitative descriptive study. Nurs Health Sci 2013 Sep;15(3):398-405. [CrossRef] [Medline]
    14. Frisoni GB, Boccardi M, Barkhof F, Blennow K, Cappa S, Chiotis K, et al. Strategic roadmap for an early diagnosis of Alzheimer's disease based on biomarkers. Lancet Neurol 2017 Dec;16(8):661-676. [CrossRef] [Medline]
    15. Mattsson N, Lönneborg A, Boccardi M, Blennow K, Hansson O, Geneva Task Force for the Roadmap of Alzheimer's Biomarkers. Clinical validity of cerebrospinal fluid Aβ42, tau, and phospho-tau as biomarkers for Alzheimer's disease in the context of a structured 5-phase development framework. Neurobiol Aging 2017 Dec;52:196-213. [CrossRef] [Medline]
    16. Ten Kate M, Barkhof F, Boccardi M, Visser PJ, Jack CR, Lovblad K, Geneva Task Force for the Roadmap of Alzheimer's Biomarkers. Clinical validity of medial temporal atrophy as a biomarker for Alzheimer's disease in the context of a structured 5-phase development framework. Neurobiol Aging 2017 Apr;52:167-82.e1. [CrossRef] [Medline]
    17. van der Flier WM, Pijnenburg YAL, Prins N, Lemstra AW, Bouwman FH, Teunissen CE, et al. Optimizing patient care and research: the Amsterdam Dementia cohort. J Alzheimers Dis 2014;41(1):313-327. [CrossRef] [Medline]
    18. van der Flier WM, Scheltens P. Amsterdam dementia cohort: performing research to optimize care. J Alzheimers Dis 2018;62(3):1091-1111 [FREE Full text] [CrossRef] [Medline]
    19. Cooney MT, Dudina A, D'Agostino R, Graham IM. Cardiovascular risk-estimation systems in primary prevention: do they differ? Do they make a difference? Can we see the future? Circulation 2010 Jul 20;122(3):300-310. [CrossRef] [Medline]
    20. Paridaens RJ, Gelber S, Cole BF, Gelber RD, Thürlimann B, Price KN, et al. Adjuvant! Online estimation of chemotherapy effectiveness when added to ovarian function suppression plus tamoxifen for premenopausal women with estrogen-receptor-positive breast cancer. Breast Cancer Res Treat 2010 Aug;123(1):303-310 [FREE Full text] [CrossRef] [Medline]
    21. Gurmankin AD, Baron J, Armstrong K. The effect of numerical statements of risk on trust and comfort with hypothetical physician risk communication. Med Decis Making 2004;24(3):265-271. [CrossRef] [Medline]
    22. Wallsten TS, Budescu DV, Zwick R, Kemp SM. Preferences and reasons for communicating probabilistic information in verbal or numerical terms. Bull Psychon Soc 2013 Nov 5;31(2):135-138. [CrossRef]
    23. Hoffrage U, Gigerenzer G, Krauss S, Martignon L. Representation facilitates reasoning: what natural frequencies are and what they are not. Cognition 2002 Jul;84(3):343-352. [Medline]
    24. Zipkin DA, Umscheid CA, Keating NL, Allen E, Aung K, Beyth R, et al. Evidence-based risk communication: a systematic review. Ann Intern Med 2014 Aug 19;161(4):270-280. [CrossRef] [Medline]
    25. Stiggelbout AM, Van der Weijden T, De Wit MP, Frosch D, Légaré F, Montori VM, et al. Shared decision making: really putting patients at the centre of healthcare. Br Med J 2012;344:e256 [FREE Full text] [Medline]
    26. Hargraves I, LeBlanc A, Shah ND, Montori VM. Shared decision making: the need for patient-clinician conversation, not just information. Health Aff (Millwood) 2016 Apr;35(4):627-629. [CrossRef] [Medline]
    27. Stiggelbout AM, Pieterse AH, de Haes JC. Shared decision making: concepts, evidence, and practice. Patient Educ Couns 2015 Oct;98(10):1172-1179. [CrossRef] [Medline]
    28. Kunneman M, Montori VM, Castaneda-Guarderas A, Hess EP. What is shared decision making? (and what it is not). Acad Emerg Med 2016 Dec;23(12):1320-1324 [FREE Full text] [CrossRef] [Medline]
    29. Brooke J. SUS: a quick and dirty usability scale. In: Jordan PW, Thomas B, MCClelland AL, editors. Usability Evaluation in Industry. London: Taylor and Francis; 1996:-94.
    30. van Maurik IS. Alzheimercentrum. ADappt overview 28-12-2018   URL: [WebCite Cache]
    31. van Maurik IS. ADappt.   URL: [WebCite Cache]
    32. van Maurik IS. Alzheimercentrum Amsterdam. ADappt diagnostic tests   URL: [WebCite Cache]
    33. Jaspers NE, Ridker PM, Dorresteijn JA, Visseren FL. The prediction of therapy-benefit for individual cardiovascular disease prevention: rationale, implications, and implementation. Curr Opin Lipidol 2018 Dec;29(6):436-444. [CrossRef] [Medline]
    34. Jaspers NE, Visseren FL, Numans ME, Smulders YM, van Loenen Martinet FA, van der Graaf Y, et al. Variation in minimum desired cardiovascular disease-free longevity benefit from statin and antihypertensive medications: a cross-sectional study of patient and primary care physician perspectives. BMJ Open 2018 May 26;8(5):e021309 [FREE Full text] [CrossRef] [Medline]
    35. Petersen RC, Lopez O, Armstrong MJ, Getchius TSD, Ganguli M, Gloss D, et al. Practice guideline update summary: mild cognitive impairment: report of the Guideline Development, Dissemination, and Implementation Subcommittee of the American Academy of Neurology. Neurology 2018 Jan 16;90(3):126-135. [CrossRef] [Medline]
    36. Ngandu T, Lehtisalo J, Solomon A, Levälahti E, Ahtiluoto S, Antikainen R, et al. A 2 year multidomain intervention of diet, exercise, cognitive training, and vascular risk monitoring versus control to prevent cognitive decline in at-risk elderly people (FINGER): a randomised controlled trial. Lancet 2015 Jun 6;385(9984):2255-2263. [CrossRef] [Medline]


    ABIDE: Alzheimer biomarkers in daily practice
    AD: Alzheimer disease
    CSF: cerebrospinal fluid
    CSS: cascading style sheets
    CT: computer tomography
    MCI: mild cognitive impairment
    MRI: magnetic resonance imaging
    PET: positron emission tomography
    SUS: system usability scale

    Edited by G Eysenbach; submitted 16.01.19; peer-reviewed by E Zafeiridi, S Chokshi; comments to author 28.03.19; revised version received 30.04.19; accepted 30.04.19; published 08.07.19

    ©Ingrid S van Maurik, Leonie NC Visser, Ruth E Pel-Littel, Marieke M van Buchem, Marissa D Zwan, Marleen Kunneman, Wiesje Pelkmans, Femke H Bouwman, Mirella Minkman, Niki Schoonenboom, Philip Scheltens, Ellen MA Smets, Wiesje M van der Flier. Originally published in JMIR Formative Research (, 08.07.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.