Participants (n=10) were recruited from a bank of individuals at the Institut Universitaire de Gériatrie de Montréal who volunteered to be contacted to participate in research studies. Inclusion criteria consisted of a minimum age of 65 years, no history of cognitive impairment, and the ability to read and communicate in either French or English. After the Institut Universitaire de Gériatrie de Montréal provided our research team with the participant bank, individuals were contacted via phone. Those who were available and interested were then recruited as a convenience sample.

A telephone-use questionnaire was administered, which was developed by our research team to assess participant comfort with the phone modality as well as each participant’s technological circumstances (eg, device used; Multimedia Appendix 1). In light of the social and health context during the period of our study, we also administered a COVID-19 questionnaire, which is available in Multimedia Appendix 2. Subsequently, the MoCA-Blind was administered remotely, following published adaptations and guidance [5]. The MoCA-Blind omits the subtest items that require a functional vision for its completion, such as the clock-drawing task. Further, the tapping portion of the attention subtest was adapted, whereby participants had the option of tapping on their phones with either their hand or a pen. Testing sessions concluded with a semistructured debriefing interview (Multimedia Appendix 3), where participants shared their experiences during remote research. This interview guide was developed by the research team using a phenomenological approach [31].

This study was conducted in the summer of 2021 in the context of pandemic-related lockdown regulations in Quebec, Canada. Following recruitment, initial phone contact consisted of obtaining verbal consent and the administration of the telephone-use questionnaire. Subsequently, each participant received a checklist of questions and reminders by email to help troubleshoot any problems, such as their use of a hearing aid, how they planned to use the phone, and ensuring they had the necessary technical tools and a suitable environment for the protocol tasks (eg, no background noise and being alone). A second phone call was made within the subsequent 2 weeks, where checklist items were revisited with participants to ensure a smooth administration of the protocol. Following this, the MoCA-Blind and the semistructured debriefing interview were conducted over the phone. During the intervals between different segments of the phone call, verification questions were posed to ascertain the audibility and modality of telephone use. Phone calls were recorded using NoNotes (NoNotes Inc), a third-party phone call recording app. Recorded testing sessions, note-taking, and subsequent transcription facilitated scoring and qualitative coding of textual data. All interviews were transcribed verbatim and analyzed by 2 research assistants on Microsoft Word. Given the bilingual context of research in Quebec, Canada, some interviews were translated from Canadian French to English using the DeepL software (DeepL SE). The resulting translations were proofread by a fluently bilingual member of the team to maximize the preservation of the meaning of the original interviews. All participant information (eg, audio recordings, contact information, and test results) were identified by an ID number to preserve confidentiality.

Research assistants (n=4) were recruited from a laboratory at the Université de Montréal’s School of Optometry focused on sensory-cognitive aging. Given Quebec’s bilingual population, research assistants are required to be functionally fluent in French and English to conduct testing sessions in accordance with participants’ language preferences.

A Microsoft Excel sheet was created for each research assistant to report qualitative field notes after each completed testing session. This form was divided into 2 sections. The first section pertained to the observations made during the testing sessions, such as the length of the call, participant behavior, problems encountered, and troubleshooting solutions. The second part centered on personal experiences when testing participants over the phone, such as how comfortable research assistants were testing over the phone, their opinion of the phone modality, and test session facilitators. Additionally, research assistants individually reported information on their environmental setup during remote administration, including information about how they used the phone (eg, phone to ear, on speaker, and on headphones; Multimedia Appendix 4).

The research assistants provided written informed consent and completed a demographic questionnaire (Multimedia Appendix 5). After each completed test session, they reflected on the operational aspects of the study using the previously described Microsoft Excel sheet to detail qualitative insights. All testing sessions were conducted in French, over the phone, and were recorded using the NoNotes software.

The sample consisted of 8 women and 2 men, all older adults (mean_age 69.33, SD 12.20; range 67-76 years); information on participant gender was provided by the recruitment agent at the Institut de Gériatrie de Montréal. Information on participant education level was not collected. Results of the telephone use questionnaire indicated that half of our participants (5/10) felt technology to be very important, while others reported that it was important (3/10) or of little importance (2/10). Participants reported making phone calls almost every day (4/10), every day (3/10), or once a week (3/10). None of the participants reported requiring assistance when making these phone calls, but 2 reported encountering technological issues when using their devices. In total, 5 participants reported primary use of their mobile phone, 4 reported using their landline phone more often, while 1 used both equally. A total of 9 participants obtained scores equal to or above the adjusted cutoff on the MoCA-Blind, while 1 participant obtained a score of 16, which was outside the normal range using the adjusted threshold of 18/22 [5].

Broadly, participants viewed the process of remote research as acceptable, thereby suggesting telephone administration to be appropriate for use in research. When discussing the MoCA-Blind, participants considered the instrument suitable for remote administration because of its short straightforward test instructions and items.

Participants reported a demand for telephone-based research methods, citing its convenience as facilitative to participation:

In addition, several participants described the appeal of participating in research from the comfort of a personal environment (practicality), as it allowed for better time management, was convenient, and minimized distractions stemming from the laboratory environment (eg, formality of environment). Conducting the study’s tests in a familiar environment encouraged feelings of relaxation and physical comfort and increased concentration and stimulation:

Participants acknowledged several aspects of our research method as facilitative to implementation. First, they reported feeling efficiently prepared for the testing sessions because of the reminder phone calls and checklists (eg, questions pertaining to hearing aid use and adherence) used by our team. This preparedness allowed participants to plan their schedule around the study participation, troubleshoot technology, arrange their environmental setting, and prepare for the session both mentally and physically. Additionally, they acknowledged and valued our research team’s punctuality, respect, trustworthiness, professionalism, and organization.

Although participants deemed the telephone modality as appropriate to conduct research, there were several potential barriers to its implementation. First, there was concern about the length of remote research activities. Participants emphasized limiting the length of the research session, describing the consent process as one area that could be shortened. Despite the efforts of the research team to expedite the process, the emphasis placed on obtaining consent over the phone was identified as being long and tedious:

Furthermore, some portions of the MoCA-Blind were identified as challenging in a remote context. Concern was raised for the attentional tapping task, as the phone presented some distraction and apprehension.

One participant articulated a preference for in-laboratory research, citing a perceived absence of stimulation in her home setting. Similarly, other participants noted an absence of personal connection as well as confidentiality as issues with the telephone approach. The telephone also appeared to create some apprehension, as participants feared the occurrence of technological issues such as a bad phone connection or lack of audibility. Interestingly, one participant suggested the use of videoconference platforms as an alternative to telephone-based methods, with the aim of enhancing interpersonal interactions during the research process.

Finally, 2 participants indicated the potential for inaccurate data due to cheating due to a lack of supervision and the accessibility of materials such as writing tools and paper.

Four research assistants were recruited from a research laboratory from the Université de Montréal’s School of Optometry (mean_age 29.75, SD 12.20; range 23-48 years), comprised of 3 women (2 master graduates, including author SD, and 1 undergraduate) and 1 man (laboratory coordinator). All research assistants self-reported their gender. French was the first language for 3 of the research assistants, while Portuguese was the first language of the fourth; however, all were functionally fluent in both English and French.

Research assistants noted several aspects of the research process as important for facilitating successful implementation. First, they denoted the importance of administering tests and questionnaires with minimal response choices as well as clear and simple instructions to promote acceptability.

In addition, all research assistants noted that sessions were affected by the rapport between the research assistant and the participant. Specifically, sessions were perceived as more fluid when the participant appeared to be engaged and interested in the research. Further, one research assistant related participants’ skillfulness with modern technology (eg, smartphones) also facilitated the process through flexibility and convenience.

Recording the research session was also described as a desirable aspect of remotely conducted research. One research assistant expressed their intention to review the recorded sessions to facilitate increased proficiency in future testing sessions.

Research assistants further explained that remote research methods were practical and implementable, provided that appropriate preparatory measures are taken. For example, presession organization was identified as a key facilitator, including preparing necessary documents as well as charging and troubleshooting technology (eg, telephone and recording software) to prevent any potential issues.

Relatedly, research assistants were able to arrange their own environmental settings according to their preferences. This allowed them to frequently create a calm and distraction-free environment. Importantly, the use of hands-free technology was considered essential to enhance physical comfort and ease of manipulating materials during the research sessions (eg, “With all the information in front of you and the headset to free your hands, it was easy to navigate.” [Research assistant 2]). By preparing in advance, research assistants were able to better conduct the various research tasks, allowing them to be responsive to participants’ needs.

They also mentioned that presession contact with the participant streamlined the implementation of the remote interaction. For instance, providing potential participants with a copy of the consent form by email was perceived by research assistants as accelerating the process and providing participants with more time to consider and provide consent that is better informed.

Research assistants also reported on their feelings of comfort while engaging in remote research. The novelty of administering the different tests and questionnaires over the phone initially generated feelings of anxiety for some. After several administrations, they reported an increase in their confidence in conducting the study, suggesting that practice improved their sense of competency (eg, “This session was directly following the other participant’s session, so I felt confident.” [Research assistant 2]). However, it was crucial to prevent excessive scheduling within a limited time frame to prevent fatigue and potential compromising of research data.

Research assistants indicated that questionnaires with a wide range of response options were not well suited for remote research. Concerns about implementation included the repetitive nature of the response options and the participants’ difficulty in retaining multiple response options in their working memory.

Research assistants also commented on several telephone-related barriers regarding practicality. These included delays in responses, overlapping communication between the participant and the researcher resulting in the decayed provision and reception of information, as well as limited settings on the telephone (eg, lack of speakerphone option). Researchers experienced technical difficulties (eg, dropped calls and diminished speech perception), which may have been a result of weak signal connection or issues with the recording software.

Another important concern raised about the efficacy of the remote administration centered on language. One research assistant described apprehension about their accented speech interfering with participant comprehension because French was not the research assistant’s first language. An additional articulated barrier was communication in the case of participants living with hearing impairment, whereby research assistants were concerned that information could be misunderstood or missed. To compensate, they were careful to speak clearly at a slow, steady pace.

Perspectives of both participants and researchers converged on remote research using the MoCA-Blind as a feasible endeavor. Importantly, we found that implementing remote research was supported by a thorough evaluation of specific factors, including the technological infrastructure of researcher and participant [37], participant engagement and comfort [38], data security and privacy [38], ethical considerations [39], and the adaptability of research protocols. Careful attendance to these variables could act as a foundation to support and proliferate research on remotely delivered cognitive screening tools.

A key facilitator identified in this study was the implementation of preparatory measures by the research team, which included troubleshooting technical equipment, establishing a welcoming and cordial rapport with the participants, and ensuring transparency and clarity regarding the testing procedures. These preparatory steps played a crucial role in creating a comfortable environment for the participants, fostering a sense of trust and cooperation throughout the testing sessions. Cognitive testing is often stressful [40], and intentional attention to fostering rapport and comfort has been recommended for telephone cognitive screening [41]. The establishment of rapport with the participants likely fostered a positive atmosphere, reducing potential anxiety or discomfort associated with the research setting.

More broadly, remote methods facilitated data analysis and data interpretation. The ability to record test sessions and revisit them during the subsequent analysis phase facilitated accurate note-taking and comprehensive analysis. By preserving the exact content of the test sessions through recorded sessions, researchers were released from relying solely on their memory to score the various tasks. This possibility to revisit the recordings enhanced the quality of the data because it became possible to check its accuracy during the analysis process.

Our participants expressed valid concerns regarding the lengthy consent process in research. To address this concern, we streamlined the consent process by either providing participants with a copy of the consent form via email prior to the testing session or by summarizing key sections of the form while ensuring that important ethical considerations (eg, risks, inconvenience, confidentiality, and privacy) were emphasized. This procedure was conceptually similar to short-form consent [42]. However, we acknowledge that these concerns may not extend to other contexts where remotely delivered cognitive tools may be used (eg, clinical practice). Participants also emphasized being mindful of session length when considering study design. This concern has been articulated previously [43], with several studies arguing that remotely delivered research can be more effortful than face-to-face contact [44-46]. Participants also noted a lack of personal connection to the process, which might lessen their interest in participation. This lack of interpersonal connection could be due to the loss of nonverbal communication [47], which may restrict rapport [48]. Interestingly, one participant suggested videoconferencing as an avenue to increase interpersonal connection; this idea has been suggested as viable in the literature [49]. Additionally, confidentiality was a noted concern for participants, in keeping with other investigations [43,50], due to risks such as the presence of others in the room [51] or participants being unsure what will happen with the data [52]. Finally, some participants mentioned opportunities for dishonesty on the part of the participant in remote research, particularly as it pertains to cognitive test performance. To our knowledge, this is the first time that participant dishonesty has been articulated as a challenge for remote research. However, this notion is consistent with the described challenges of remote academic dishonesty during the COVID-19 pandemic [53,54].

From the research assistant’s perspective, one barrier was that some instruments may not be well suited to remote modalities. This is consistent with other research recommending against the use of complicated instruments when conducting telephone interviews [55]. If the use of such tools cannot be avoided, participants could bring a pen and paper to the telephone to track response options [56] or be prompted as reminders of response categories [57]. Research assistants also discussed the potential challenges of telephone-delivered research tools with people living with hearing impairment (ie, information could be misunderstood or missed). This challenge has been described in other investigations into remote research [51,58]. It is possible that people with hearing impairment may find face-to-face methods more beneficial because they can engage in lip- and speech reading and benefit from nonverbal communication to aid in their understanding [59]. Thus, it is important to find strategies to make remote research more accessible, and remote research with videoconferencing might be ideal for persons with hearing impairment due to the possibility of lip- and speech reading and closed captioning [2]. Interestingly, our research assistants brought forth possibilities that have been conveyed in extant literature such as paying particular attention to articulation, tone, and repetition when needed [60].

Notably, participant and research assistant responses converged on 2 barriers to this remote research interaction. The first was an awareness of technical problems that are disrupting the research process. Technological difficulties continue to plague remote research [61], with problems like signal disruption [60] or lower vocal clarity [62], making our findings regarding technological disruptions perhaps unsurprising. Second, both participants and research assistants articulated concerns about remote cognitive screening administration, albeit from differing perspectives. Participants displayed some apprehension about completing the MoCA-Blind tasks correctly. Some literature on remote administration of cognitive assessments supports the presence of participant apprehension or anxiety during administration [63], while in other cases, remote methods could be seen as less anxiety-provoking [64]. Relatedly, research assistants were concerned with communication breakdowns (eg, through accented speech or overlapping verbalizations) interfering with test administration. Communication breakdowns may be more likely to occur in remote research, due to a decrease in the ability to control communication, that is, for the researcher to lead and the participant to follow [65], and may require increased attention to verbal cueing [66]. In any case, more study is needed to disentangle the complexity of how remote cognitive screening using the telephone can be accomplished most effectively.