This study complies with the Ethical Guidelines for Medical and Biological Research Involving Human Subjects published by the Ministry of Health, Labor, and Welfare of Japan, and all research plans were reviewed and approved by the Research Ethics Committee of the Keio University Faculty of Pharmacy (approval 211111-5). A web-based, unmarked questionnaire survey was used in this study. Informed consent was obtained from all participants by presenting them with an explanatory document and consent form prior to the survey administration. Only those who agreed to these documents were invited to participate in the survey. All procedures, including informed consent and the explanatory and consent documents presented to participants, were reviewed and approved by the Research Ethics Committee in compliance with ethical guidelines.

After the screening survey, 1000 individuals were invited to participate and 638 individuals completed the questionnaire. The demographic breakdown of the respondents was as follows: 68.8% (n=439) male and 31.2% (n=199) female. According to age group, 1.3% (n=8) were aged 20-29 years, 6.4% (n=41) aged 30-39 years, 13.2% (n=84) aged 40-49 years, 26.6% (n=170) aged 50-59 years, 27% (n=172) aged 60-69 years, 22.6% (n=144) aged 70-79 years, and 3% (n=19) aged 80-89 years or older (Table 1). The most prevalent diseases among respondents, accounting for more than 5% of the sample, were hypertension (n=169, 42.2%), hyperlipidemia (n=128, 20.1%), type 2 diabetes (n=84, 13.2%), constipation (n=55, 8.6%), psycho-nervous system disease (n=55, 8.6%), gastritis or gastroesophageal reflux disease (n=52, 8.2%), insomnia (n=38, 6%), and heart disease (n=32, 5%; Table 2). Regarding the duration of drug use, 2.7% (n=17) reported a period of 3 months to less than 6 months, 5% (n=32) for 6 months to less than 1 year, 16.3% (n=104) for 1 year to less than 3 years, and 76% (n=485) for 3 years or more (Table 3).

Results of the regularization model are presented in Table 4, whereas the results of the filter method model with feature selection using univariate analysis are shown in Table 5. Table S1 in Multimedia Appendix 4 presents the results of the univariate analysis. A total of 19 variables were selected in the regularized model, and 4 of them were found to be statistically significant: inflammatory bowel disease (IBD; P=.01), asthma or chronic obstructive pulmonary disease (COPD; P<.001), “Using the drug at approximately the same time each day” (P=.02), and “Taking meals at approximately the same time each day” (P=.02).

The process of constructing a logistic regression model with multicollinearity eliminated is detailed in Multimedia Appendix 4, where Table S1 presents the regularization model variable and Table S2 shows the process with the filter method variable. The VIF is available in Tables S3 and S4 in Multimedia Appendix 4. In the filter model, 24 variables were selected, and 4 of them were found to be statistically significant: type 1 diabetes (P=.03), “I would like to have my medication reduced” (P=.006), “Using the drug at approximately the same time each day” (P=.01), and “Taking meals at approximately the same time each day” (P=.002).

The results of the feature importance calculation using LightGBM are displayed in Figure 1. The top 5 variables with the highest feature importance scores were “Age,” “Using the drug at approximately the same time each day,” “Taking meals at approximately the same time each day,” “I would like to reduce my medication,” and “I think I want to take my medicine.”

In this study, we conducted a questionnaire-based survey of medication adherence factors and compliance in the general patient population in Japan and used multiple models to determine their associations. While numerous factors related to medication adherence have been suggested previously, the relative importance of these items has not been demonstrated. In this study, we presented the relative importance of medication adherence factors through results such as feature importance and Table 6. The respondents were divided into two categories: one consisted of individuals who were taking their medication correctly, while the other included those who were not. Subsequently, two logistic regression analyses were conducted. Two characteristics showed significant differences were “Using the drug at approximately the same time each day” and “Taking meals at approximately the same time each day.” In the LightGBM, these two items were the second and third most common, similar to the results of the logistic regression analysis.

This is the first study to use a machine learning model to calculate the importance of factors related to medication compliance, which is important in determining intervention priorities. In this study, age, acceptance or refusal of medication, and number of medications taken were identified as important characteristics of feature importance. By calculating feature importance, we can quantitatively demonstrate the relative impact of each factor on the model’s predictions. This enhances the interpretability of the analysis and is useful for clinical applications such as prioritizing interventions.

In addition, some features, such as age, were not significantly different in the logistic regression; however, it ranked high in feature importance in LightGBM. The linear predictors in the generalized linear model are fixed at first-order expressions. This can be challenging in handling cases with explanatory variables, which are represented by polynomials of second- or higher-order, or special functions. Therefore, these features are likely to exhibit nonlinear relationships. Previous studies have shown that medication adherence improves with increasing age for many diseases; however, it declines after the age of 70 years due to the effects of cognitive decline [24,55]. Negative effects have also been observed in some diseases; however, these demonstrate a nonlinear age-related relationship. These results indicate that age is one of the most important factors in medication compliance and that age-related interventions in clinical practice can be effective [56,57].

The rank order and value of feature importance are presented for the psychological factors of medication adherence, and the mean of feature importance was calculated for each group (Table 6). The items related to the awareness of medication (acceptance, refusal, and expectation), lifestyle-related items, and other items were approximately in the same rank order, and the feature importance values deviated from each other by a factor of more than 2. Although whether the bottom two variables in the lifestyle-related items were far apart from the top two variables is unclear, the mean feature importance indicated that the psychological factors of medication compliance and related adherence were (1) lifestyle-related items, (2) awareness of medication (acceptance, refusal, and expectation), (3) relationships with health care professionals, and (4) others, in order of importance. To the best of our knowledge, this is the first study to calculate and discuss the relative significance of each medication adherence factor in terms of feature importance in the general patient population [20].

Comparing the prediction accuracy of the models created in this study, the area under the curve, an evaluation index of prediction accuracy, improved from 0.69 for normal models to 0.76 for regularization models. In addition, a comparison of the calculated coefficients and 95% CIs for asthma and COPD suggests that the regularization terms were effective in suppressing the overestimation of the coefficient. In addition, because regularization also has the effect of suppressing overfitting, the appropriate variable selection is presumed to lead to an improvement in accuracy [58].

In regularization models, the underestimation of partial correlation coefficients within multicollinearity groups can cause missing variables that are relevant when significance is the criterion [59,60]. Multimedia Appendix 4 shows the results of creating a normal logistic regression model from the variables selected in the two models and further reducing the variables until multicollinearity was solved based on the VIF. When significant variables were removed, other variables became significant. Although the mathematical basis is unclear, the order of the variables in the LightGBM results was almost identical to that in the logistic regression analysis, suggesting that the important variables in the model were shifting. For instance, for traits such as “Anxious about taking medication” where the tendency of the trait was similar to that of the other traits, and the explanatory power in the model was lost to the other variables. However, by removing the other variables, the explanatory power that should have been carried by that variable was demonstrated.

The variables selected by the filter method are presented as results after they were removed based on the VIF. Unlike the regularization model, variables that became significant sometimes ceased to be significant in the process of resolving multicollinearity, indicating that the model was not stable. This is because the filter method selects noisy variables, and variable selection using the regularization method may be useful for selecting variables with complex relationships, such as confounders.

First, regarding external validity, the population surveyed in this study was recruited from patients registered on an internet panel, and any deviation from the actual demographics may have affected the results [61]. The results of this study may be influenced by Japan’s cultural and social background and health care system. Additionally, because the survey was conducted using the internet, older people and those who do not use the internet may have been underrepresented, potentially leading to sampling bias. The differences between our sample and the actual demographics in Japan are discussed below. Therefore, we compared the present population with statistical information published by the Japanese government [62]. No major differences were found in terms of gender and disease rates. However, in terms of age groups, the number of patients in the age group above the late 60s was smaller than the actual demographics. This may be due to a decrease in the number of participants due to barriers to internet access in the case of those older than 60 years. In addition, the random recruitment in this study limits us to collect sufficient data for the analysis of diseases with low incidence rates. In particular, two variables entered into the two logistic regression models in the current analysis—IBD and asthma or COPD—had statistically unwieldy values, with a count of zero in one group when the response variable was classified into two groups (Multimedia Appendix 3). The coefficients and CIs diverged for these two variables, which may have affected the other variables [63]. Therefore, we recreated a model without these two variables (Multimedia Appendix 5). In the regularization model, 15 of the 17 variables were equal and the statistically significant variables remained the same, except for eye drops (dosage forms used), which was newly significant. Most of the variables that ranked high in LightGBM feature importance remained consistent despite these changes, suggesting that the impact of IBD and asthma or COPD on the overall model is likely limited. However, a new variable selection “I think I want to go off my medicine” emerged as one of the top variables in LightGBM feature importance after removing the variables. The elimination of the two anomalous variables might have enabled the correct variable selection.

The most important factor influencing medication compliance was consistent with the timing of medication intake and meal consumption. The subsequent factors were the number of medications taken and feelings of acceptance or refusal of medication. Although these factors have been mentioned in previous studies, we were able to calculate their importance using a machine learning model. Few studies have mentioned adherence and lifestyles of patients, and further research could shed light on medication adherence in terms of daily behaviors of people.

In addition, when adherence factors are used as features, multicollinearity may be generated because of similarities in their respective characteristics. Therefore, caution should be exercised when discussing the relationship between response variables using generalized linear models. When multicollinearity is addressed, examining the relevance or considering alternative models, such as regularization or decision trees, that can effectively handle the issue of multicollinearity is important.