Evaluating the Use of a Note-Taking App by Japanese Resident Physicians: Nationwide Cross-Sectional Study

Introduction

Note-taking during learning has long been a mainstay of educational practice and data over the last 60 years, demonstrating its contribution to improved learning and test scores [1,2]. In 1995, a study on effective note-taking among students reported that free note-taking by learners was a particularly effective process [3]. A 2002 report noted that the most effective way for medical students to perform well was to take written notes on materials prepared in advance by the teachers [4]. Since 2010, the use of mobile technology in education has increased in medical and pharmacy schools and is correlated with high student satisfaction [5,6]. Note-taking is also useful to improve understanding during the class and to retrospectively reflect on learned material [7].

In recent years, medical students have been using digital devices extensively for learning and literature searches [8]. Moreover, in medical education, digital methods are often used mainly as a replacement for traditional methods [9]. These days, medical students have increasingly encouraged incorporating digital tools, such as the iPad, for note-taking [10]. It has been suggested that digital note-taking may be better than nondigital note-taking concerning readability and search capabilities during revision of the notes [11]. Furthermore, students tend to have a positive attitude toward learning the use of digital tools [12]. They are reportedly able to use mobile devices as online information resources, which leads to improved learning efficiency in the classroom [13]. Multiple mobile devices are also being used in clinical settings. They are used for telemedicine, and have been developed to support clinical decision-making, providing timely feedback to residents in addition to improving clinical skills [14,15].

However, there are also disadvantages to the use of digital tools such as apps. One of the disadvantages of using apps in clinical settings is the challenge of using them well in multiple electronic tools [8]. First, medical students are not used to digital technology, and a tendency exists for them to prefer paper-based materials to digital ones [5], which is supported by a study that reported the digital group performing worse than the group that took notes using paper and pen [16]. Multitasking may have been a factor in the poor performance [16]. Additionally, a study in Mexico has also raised the issue of the difficulty of using online content when the internet connection is poor, and whether or not the instructor is willing to allow the use of online content [17].

However, we are unaware of the type of individuals using a note-taking app and the background factors, such as the extent to which clinical skills differ between app users and nonapp users.

Therefore, we sought to understand the characteristics of note-taking residents using a note-taking app and determine whether there is a difference in the basic medical knowledge compared to that of nondigital note-taking residents.

Methods

Design and Participants

This was a nationwide analytical cross-sectional survey of resident physicians in Japan. Since 2004 in Japan, newly graduated resident physicians have undergone mandatory cross-disciplinary 2-year training (including rotations in internal medicine, emergency medicine, pediatrics, gynecology, psychiatry, surgery, and community medicine) after passing the national medical examination during their final year of medical school [18]. The departments for rotational training and duration and timing during the 2 years of training vary from hospital to hospital and are not set in stone. Resident physicians in their first (postgraduate year [PGY] 1) and second (PGY 2) years take the same General Medicine In-Training Examination (GM-ITE), and examination participants complete questionnaires that survey the actual status of resident physicians immediately after the examination. Participants were gathered by an announcement from the training supervisor at each teaching hospital, and resident physicians participated voluntarily. This study was conducted between January 17, 2023, and January 30, 2023, and followed STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines [19].

Variables

The GM-ITE assesses general clinical knowledge and its practical relevance by the medical training guidelines of the Ministry of Health, Labor, and Welfare (MHLW) of Japan. This exam provides feedback to individual resident physicians and evaluates training programs and facilities. The GM-ITE comprises 80 multiple-choice questions, with optional tests that are answered by approximately 50% of all resident physicians in Japan. The GM-ITE is a test organized by the Japan Association for Medical Education Program (JAMEP). JAMEP is a nonprofit organization established with the aim of contributing to the promotion of Japanese medicine through support for training and education to improve the quality of medical care, as a third-party organization that checks the quality of medical education in Japan. The authors received the data from the person in charge at JAMEP. In line with the MHLW’s goals for resident physicians, the 2022 GM-ITE consisted of a total of 80 structured questions, including (1) eight questions on medical history taking and professionalism, (2) eighteen questions on symptomatology and clinical reasoning, (3) eighteen questions on physical examination and clinical skills, and (4) thirty-six questions on basic clinical knowledge, including disease knowledge. The GM-ITE is answered on the computer, and after the test is finished, the participant decides whether or not to agree to participate in questionnaires on the computer. Thereafter, they answer the following questions. Baseline characteristics included resident physicians’ age, gender, year in training, and hospital location (urban or rural; the 20 cities designated by the Ministry of Internal Affairs and Communications and the 23 wards of Tokyo were defined as urban areas, and all other cities were defined as rural areas), general hospital type (community-based or university hospital), number of beds, whether rotation in a general medicine department was completed, number of emergency department shifts per month, average number of inpatients cared for daily, use of medical online resources, self-study time per day, and duty hours per week. The ages of the resident physicians were divided into 3 groups. Group 1 consisted of resident physicians aged 24 and 25 years, group 2 consisted of physicians aged 26 and 27 years, and group 3 consisted of physicians aged 28, 29, and 30 years and older. The main outcome question regarding app use was asked as follows. First, participants were asked whether they kept records of the cases they had experienced, and those who did not were excluded. They were then asked whether they used the application software to record the cases they had experienced, and they responded with either yes or no. The relationship between the total GM-ITE score and app use was also investigated. The GM-ITE is taken by residents under supervision at hospitals. However, due to the recent COVID-19 pandemic, some residents took the examination from their homes. The data of residents who took the exam at home were excluded from the analysis. Participants who did not collect cases for self-study, either on paper or through an app, were excluded. Additionally, those with missing data were excluded.

Ethical Considerations

This study was approved by the JAMEP Ethics Committee (No. 22‐31). The study was conducted in accordance with the principles of the Declaration of Helsinki [20]. All the methods were performed following the Ethical Guidelines for Medical and Health Research Involving Human Subjects.

All participants reviewed this study document detailing data anonymization, voluntariness, and the dissemination of research outcomes before involvement. Only participants who provided informed consent were included in this study. The data are being carefully managed by the JAMEP staff in a locked file. The data were anonymized at the time of passing them to the authors; thus, it is not possible to identify the participants. In particular, the questions in this study are limited to minor invasive questions. The participants answered the questions on a completely voluntary basis, and no compensation, including rewards, was provided.

Data Analysis

The results are presented as means (SD) for continuous variables or as prevalence (%) for categorical variables. During data analysis, the subgroup analysis (app users vs app nonusers) were performed to determine the factors associated with the app use. Comparisons between the 2 groups were performed using the chi-square test. Univariate or multivariate analyses with a mixed logistic regression model were performed using a note-taking app to collect cases as the objective variable.

Other fixed covariates were sex, age, PGY, hospital location, hospital type, bed number, general medicine department rotation, number of emergency department duties per month, average number of inpatients cared for daily, use of medical online resources, self-study time per day, and duty hours per week. The GM-ITE scores were divided into 4 quartiles from top to bottom. Group 1 had the lowest score groups and group 4 had the highest score.

The total GM-ITE score was used as the objective variable for the analysis of secondary outcomes. Univariate or multivariate analyses with a linear mixed model were conducted using the GM-ITE total score as the objective variable, with a random effect of the training hospital. The fixed covariate was the presence or absence of app use, in addition to the previously mentioned variables.

The 95% CIs were defined using the mixed logistic regression and linear mixed models. All calculations were performed using SAS (version 9.4; SAS Institute). P values <.05 were considered statistically significant. The sample size was not calculated for this exploratory study.

Results

A total of 9011 residents from 662 teaching hospitals participated in the 2022 GM-ITE. Participants who did not provide consent (n=2375), took the GM-ITE at home (n=573), did not record their cases for self-study purposes (n=1366), and had at least 1 missing data variable (n=864) were excluded. A total of 3833 participants were included in the final analysis (Figure 1).

Most (n=2591, 67.6%) of the participants were female, 32.4% (n=1242) were PGY-1 resident physicians, and 52.4% (n=2009) were of age group 2. Most (n=2628, 68.6%) of them were training in rural areas, and 84.4% (3236) practiced in community-based hospitals, while less than half (n=1772, 46.2%) had completed a general medicine rotation. Emergency department duty: most (n=2766, 72.2%) of the participants had 3‐5 duties per month in the emergency department. Almost half (n=1896, 49.5%) of the participants had less than 60 duty hours per week. Nearly half (n=1933, 50.4%) reported caring for 5‐9 inpatients daily. Furthermore, less than one-third of the participants (n=1095, 28.6%) were using online medical resources, while 41.4% (n=1586) and 41% (n=1570) spent 0‐30 minutes and 31‐60 minutes for self-study time per day, respectively (Table 1). The participants’ GM-ITE scores ranged from 12 to 71 points.

A substantial proportion (n=1750, 45.3%) of participants were using the note-taking app.

Among this study’s participants, PGY-2 residents used the app more frequently than PGY-1 residents (n=896, 48.6% vs n=843, 42.4%, respectively; adjusted odds ratio [aOR] 1.10, 95% CI 1.03 to 1.41; P=.02). Participants in community-based hospitals were more likely to use the app than those in university hospitals (n=1513, 46.8% vs n=226, 37.9%, aOR 1.42, 95% CI 1.04 to 1.94; P=.03). Those who worked in hospitals with a larger number of beds were also more likely to use the app (aOR 1.08, 95% CI 1.02 to 1.13; P=.004). Those who had rotated through the Department of General Medicine were more likely to use the app than those who had not (n=881, 49.7% vs n=858, 41.6%, aOR 1.19, 95% CI 1.02 to 1.39; P=.03). The number of inpatients cared for per day was most frequently 10‐14 among app users, which showed a significant difference (aOR 1.62, 95% CI 1.18 to 2.21; P=.003). Participants in the app user group also used online medical resources more frequently (n=1365, 49.9% vs n=374, 34.2%, aOR 1.52, 95% CI 1.3 to 1.79; P<.001; Table 2).

The GM-ITE total score was used for evaluation. The following variables showed significant differences in the GM-ITE total score in the multivariate analysis. Female scores tended to be lower than male scores (adjusted difference [aD] −1.18, 95% CI −1.69 to −0.68; P<.001). PGY-2 scores tended to be higher than PGY-1 scores (aD 2.45, 95% CI 1.92 to 2.99; P<.001). The age of the resident physician tended to be lower in both group 2 and group 3 than in group 1, with lower GM-ITE scores (group 2 [aD −2.55, 95% CI −3.17 to −1.92; P<.001] and group 3 [aD -5.88, 95%CI −6.62 to −5.13; P<.001]). Predictors of the high GM-ITE scores were working at community-based hospitals (aD 3.64, 95% CI 2.44 to 4.84; P<.001), hospitals with more beds (aD 0.36, 95% CI 0.17 to 0.55; P<.001), high number of emergency department duties per month (aD 1.98, 95% CI 0.15 to 3.80; P=.03), high number of inpatient (aD 1.40, 95% CI 0.32 to 2.47; P=.01), using medical online resources (aD 1.45, 95% CI 0.9 to 1.99; P<.001), and high number of self-study time (aD 1.41, 95% CI 0.13 to 2.69; P=.03). The GM-ITE score tended to be higher for the group working 60 to 79 hours per week than for the group working 0 to 59 hours per week (aD 0.97, 95% CI 0.45 to 1.59; P<.001). However, for the group working 80 hours or more per week, there was no difference in score from the group working 0 to 59 hours per week, suggesting that working 60 to 79 hours per week is optimal for improving the GM-ITE score. Moreover, app users had significantly higher scores (aD 0.74, 95% CI 0.25 to 1.22; P=.003; Table 3).

‎

Discussion

Principal Findings

To our knowledge, this is the first study to compare the characteristics of note-taking app users with those of nonusers among resident physicians actively engaged in clinical practice in Japan. The results showed that app users, compared to nonapp users, tend to be more likely to use online medical resources, have rotated in general medicine, in community-based hospitals, be more experienced trainees, and engage in more self-study. GM-ITE scores were also higher among those who used the app.

Resident physicians who used online resources tended to use the apps more frequently. The use of online resources and apps is considered to be associated with digital literacy. A previous report stated that one of the disadvantages of digital note-taking was a limited period during which students were not able to use the apps easily [21]. Students found apps to be difficult to use if they were using an app that was unfamiliar to them [21]. Another study reported that higher information and digital literacy were associated with satisfactory information retrieval among health care professionals [22]. In a study comparing the effectiveness of information literacy versus digital literacy among family physicians in the Middle East, those with higher information literacy were more effective in searching for information relevant to their practice [23]. Similar to previous reports, both information literacy and digital literacy were involved and affected the frequency of app use.

Participants who used apps tended to have rotated through the Department of General Medicine compared to nonapp users. Japan has an increasingly aging population, and Japanese general medicine doctors are often tasked with treating older adults and are required to deal with multimorbidity [24,25]. Additionally, there have been reports that general medicine doctors based in clinics often use apps to consult with patients during consultations; thus, general practice is highly compatible with app use [26]. Furthermore, as general medicine doctors are keen propagators of education, they could encourage their trainees to use the app, which could have contributed to the results of this study [27].

Our results also indicate that resident physicians in community-based hospitals work in an environment with more app use than in university hospitals. A study of Japanese resident physicians reported that they experienced more cases, spent more time on self-study, worked more hours, and tended to have higher GM-ITE scores in community-based hospitals than university hospitals [28]. Therefore, it is possible that residents in busy community-based hospitals need to compile a large amount of information for self-study, thereby using digital aides to collect and organize this information more efficiently and effectively.

Compared with PGY-1 residents, PGY-2 residents showed a higher rate of app use. There are 2 possible reasons for this. First, PGY-2 residents may have exchanged information with each other on the efficiency of using apps, such as those used for case collection, during their initial year of training and thus have been influenced to use the technology preferentially. In the Japanese training environment, PGY-1s often spend most of their time on mandatory rotations, while PGY-2 students are provided more opportunities for elective rotations. Almost all physicians in Japan proceed to a single specialty department by their PGY-3 year [29]. Therefore, it is possible that PGY-2 residents were motivated to study hard to prepare for their specialty training.

It may be unsurprising that total GM-ITE scores were higher among app users, given that those using digital note-taking have the advantage of being able to organize what they have learned more efficiently than paper note-taking. In the previous studies, high-achieving medical students were aware of the need to manage their time efficiently [30,31]. Increased efficiency is one of the critical concepts of app use. A 2014 report indicated that paper-based systems were more likely to be retained in short-term memory and contribute to higher conceptual awareness; however, a 2022 report showed that short-term memory retention was not significantly different between paper and digital note-taking [32,33]. No reports have compared the facts of paper and digital note-taking on long-term memory; however, digital note-taking may be superior to traditional paper note-taking per efficiency and the amount of information that can be recorded and retained without loss or degradation [11,33]. Thus, the results of this study also support the results of previous reports.

With respect to the GM-ITE score, a positive correlation was observed with being a community-based hospital, self-study time, working hours, the number of emergency shifts, the number of inpatients, and the year of residency, which was consistent with the results of previous reports [27,28]. This suggests that the GM-ITE is a test that is related to clinical practice. The fact that male doctors tended to score higher may have been influenced by the gender hierarchy in Japan [34].

Finally, the app user group spent more time in self-study. App users may have been more motivated, resulting in a more positive effect on clinical performance. Previous studies have also reported that being self-motivated was associated with higher rates of graduation from graduate school [35]. From the results of this study, we consider the possibility that being motivated may have led to an interest in various things, including app use and improved clinical performance.

Limitations

This study has several limitations. First, our methodology did not specify detailed note-taking procedures, leaving these potentially important specifics unknown. In particular, the survey did not examine the motivation of residents or their ideas about personal efficiency. In this questionnaire survey, the answer item is “30 years old or older”; hence, the exact average age could not be calculated. In addition, this study did not examine the specific product names of the apps, descriptions of data collection methods, frequency of use, satisfaction, or challenges; we wish to investigate these variables in future studies. Also, as this was a cross-sectional study, causal relationships remain unclear. Further, a sample size calculation was not performed because this was an exploratory study. This may have affected the interpretation and effectiveness of the research results.

Conclusions

This pioneering study on app-based note-taking among physicians revealed that app users are often efficient and motivated, offering valuable insights for medical education. Building on these findings, future research should further explore app-based note-taking and its potential integration into clinical practice.