In total, 1168 events were reported by the 75 young adults: 729 non-drinking events (ie, days with no drinking event), 236 low-risk drinking events (ie, days on which 1 to 3 or 1 to 4 drinks per occasion were consumed by women or men, respectively), and 203 BDEs (ie, high-risk drinking days on which ≥4 or ≥5 drinks were consumed per occasion by women or men, respectively). We included the participants in our analysis if they reported at least 1 non-drinking at least 1 low-risk drinking or BDE and if there were at least 3 days with the minimum amount of phone sensor data needed for analysis. We excluded 414 events from the 75 participants (mean 5.52 events per person) because they did not provide GPS-based location data, and more than half of their key features were missing. Missing key phone sensor features might have occurred, for example, because the AWARE app allowed users to disable GPS collection. As such, we excluded days that were missing a few hours of sensor data (eg, owing to smartphone running out of battery), resulting in a final data set of 754 events from the 75 participants: BDEs (122/754, 16.2%), low-risk drinking events (143/754, 19%), and non-drinking events (489/756, 64.9%). For missing sensor values (ie, a few minutes, such as 5 or 10 minutes, of gap in data collection), we interpolated the average value between 2 adjacent sensor data points.

Our strategy for drinking prediction modeling is as follows: using behavioral features from sensor data collected over a given analysis window of w hours (where we varied w), we predicted whether the participant will have a BDE within d hours (with varying d to optimize the model; Figure 2). To build this predictive model, we first created a data set containing the sensor data across the participants of the study (a data set split by “rows”; 80/20 at the level of rows), calculating features in 15-minute windows (eg, the mean of the magnitude of acceleration per minute averaged every 15 minutes). For the 15-minute window, we computed a statistical values (eg, the mean of the raw sensor) for numerical data. We encoded the time of day as a number from 0 to 23 to represent all segments of the day.

We used 15-minute segment instances as our unit of analysis because this time frame captures known social and behavioral predictors of binge drinking. The day of the week was encoded as 0 to 6. If an individual reported that they did not drink the previous day, we labeled each of the day’s 15-minute windows as a non-drinking event. When the participant reported a drinking event (BDE or low-risk drinking event), we labeled the windows before and after the drinking event as non-drinking event. For a woman, windows during a drinking event were labeled as BDE if they consumed ≥4 standard drinks, and for a man, windows during a drinking event were labeled as BDE if they consumed ≥5 standard drinks. Otherwise, the windows were coded as low-risk drinking. More than 60% (75/122; the total number of reports=122 and the number of BDE reports=75) of the BDEs reported started between 4 PM and 10 PM, as shown in Table 1, and on a Friday or Saturday, as shown in Table 2. As suggested in earlier work [40], we assumed that behavioral data aggregated during the day by smartphone-based sensors, in our case, can be applied to predict same-day drinking events.

We explored different prediction windows using 1-, 3-, and 6-hour distances prior to drinking onset to see how far in advance predictions could be made about BDEs (Figure 2). Similar to research on physical activity [41], we hypothesized that shorter window distances (eg, 1 hour) would reflect more recent behavior and would be more strongly related to same-day alcohol consumption, whereas other types of sensor data might require cumulative behavior over a longer period (eg, 6 hours of sedentary activity) to be associated with increased likelihood of alcohol use later that day. Interventions could be delivered at any of these window distances and could be tailored to be context specific at the time of delivery using a JIT framework [32]. In addition, from a technical standpoint, window size, or how much data are needed to accurately predict drinking events, impacts how much data are required to be kept on the smartphone, impacting privacy and phone storage considerations. We used window sizes of 1, 3, 6, 9, and 12 hours in our analysis. For example, if we wanted to predict whether a BDE would occur at 6 PM, we tested the predictive ability 1 versus 3 hours in advance (ie, at 5 PM vs 3 PM). Similarly, if we wanted to predict whether a BDE would occur at 3 PM, we tested the predictive ability using sensor data collected between 2 PM and 3 PM versus between 12 PM and 3 PM (ie, sensor data from 1 vs 3 hours).

We split our data into 2 nonoverlapping data sets to represent the randomly selected data set: training (80%) and testing (20%) data sets, which were split by rows. The training set was used to train ML algorithms and for model optimization. The trained model was then tested on the “unseen” data set. We used 10-fold cross-validation to identify the best model with our training data set. We report our final results on the 20% holdout test data set. To balance our data set (consisting of mostly non-drinking events), we oversampled the instances of the underrepresented classes using the synthetic minority over-sampling technique (SMOTE) [42] in our training data set, balancing BDEs and low-risk drinking events. Although SMOTE may affect the importance scores of the features we used because we filtered the features before the data were processed with SMOTE. The test data set was kept unseen and SMOTE did not affect our overall analysis.

We built our prediction models by parsing the data for each person into 15-minute epochs. Each 15-minute epoch contained 70 sensor features and a drinking label (N: non-drinking event, D: low-risk drinking event, or BDE). We then identified all sensor features from all the 15-minute epochs in the 24 hours before the onset of a drinking event as our primary period for predictive modeling. For example, if a person reported starting drinking at 9 PM on a day they self-reported a BDE, then we would include all sensor data from the 8:45 PM–to–9 PM epoch on the previous day to the 8:45 PM–to–9 PM epoch on the current day. For days when there was no reported drinking, we randomly selected a time between 6 PM and midnight and marked this observation as the “start of the non-drinking event.” There were missing observations in the data set, so not all labeled days had ninety-six 15-minute epochs of sensor data.

As part of our modeling, we assessed the performance of the predictive models using a range of analysis window sizes (1-, 3-, 6-, 9-, and 12-hour windows) to compare the accuracies of the ML models. To test different analysis window sizes, the “prediction distance” window was held constant at “1 hour” prior to a drinking event. Note that the results indicated that a prediction window of 6 hours on weekdays and 3 hours on weekends prior to a drinking event had the best performance across analysis window sizes; therefore, the prediction windows of 6 hours on weekdays and 3 hours on weekends are reported here (refer to Multimedia Appendix 1 for results of testing analysis window sizes at other prediction distances).

We also estimated a baseline model that used only the day of the week to classify non-drinking events, low-risk drinking events, and BDEs, with the idea that a prediction model is needed to demonstrate better performance relative to baseline.

We performed hyperparameter tuning for the prediction models (separate models for weekday and weekend) for a 3-way classification task: non-drinking event (N) versus low-risk drinking event (D) versus BDE using Optuna [48]. We also trained a prediction model to predict BDE (vs N and D) for weekdays and weekends as a particularly risky form of drinking. We used popular ML classifiers and trained 5 algorithms, extreme gradient boosting (XGBoost), random forest, decision tree, support vector machine, and logistic regression, using all selected sensor features.

Hyperparameter tuning uses a search strategy to identify an optimal set of parameters that maximizes the performance of a model. To evaluate and hyperparameter tune our models, we implemented 10-fold cross-validation and then min-max scaling for feature normalization and used oversampling on the training set with SMOTE to handle class imbalance. We ran 100 iterations of Optuna to find the best hyperparameter by optimizing the F₁-score of the model. As part of hyperparameter optimization, we tried different values of XGBoost. Table 3 presents the setting of the XGBoost parameters.

We chose the hyperparameter combination that achieved the best performance on the validation set for the final model that was then evaluated on the test set. To try multiple combinations of parameters, cross-validate each, and determine which values result in the best performance, we tested a variety of parameters, such as the model type, distance from the drinking events (ie, prediction window), the amount of data used (ie, analysis window size), and hyperparameters. The analysis window size was particularly important, as it defined how much data were used to fit the model. For example, an analysis window size of 3 and a prediction distance (ie, prediction window) of 6 means that the hours [17,18,49] before the event are used to predict the class of the event (N, D, and BDE).

To evaluate the performance of the models, we computed the following metrics to measure the overall performance: accuracy, F₁-score, κ, precision, and recall. To measure the performance of the models across different classes, we computed precision, recall, and F₁-score. In addition, we built a baseline model using only the day of the week to test how much the phone sensor features improved model performance.

First, we considered the F₁-score to optimize the prediction model. The F₁-score is a balanced measure of precision and recall. Second, we used the κ value if the F₁-scores were the same. The κ value is a measure of the similarity between observations and predictions while correcting for agreement that happens by chance [50]. The κ value is also used to test performance for imbalanced classes and multiple classes.

To support JITAI development, understanding key contributors to the prediction of BDEs is needed to refine and adapt algorithm-based decisions. The SHAP feature importance plot in Figure 5 shows the 20 (default, but configurable) most important features according to their level of contributions (mean absolute SHAP values across all the instances) to the ML prediction model, only targeting BDE prediction. The results show that “time of day” contributed the most to predicting BDEs on both weekends and weekdays.

In Figure 5, we present the SHAP summary plots for our best models for both weekdays and weekends, which show the contribution of sensor features to the prediction of BDEs (left) and feature importance (right) on weekdays (top) and on weekends (bottom) using the test data set. SHAP summary plots show each sample in the test set as a data point and their impact in predicting BDEs based on the relative values of features. The color spectrum depicts the values of features. If the value of a feature is relatively high for a specific instance, it is represented in red. If the value is relatively low, it is represented in blue. The plots on the right-side show feature contribution in absolute SHAP values for the most influential 20 features.

Overall, the longitudinal and latitudinal coordinate statistics, radius of gyration, and movement features were among the most highly influential features for BDE prediction. SHAP results show that (1) latitude and longitude were the most important features, in addition to time of day contributing to the prediction of BDEs both on weekdays and weekends, and (2) radius of gyration is a relatively more impactful predictor of BDEs on weekends than on weekdays.

From the summary plots on the left, we can see that acceleration features tend to be negatively related to the BDE prediction on both the weekends and weekdays, meaning that higher acceleration values decrease the probability of BDE prediction. This greater activity or movement of the smartphone is associated with a lower likelihood of BDEs. In addition, radius of gyration and number of locations are positively related to the BDE prediction (ie, greater radius of gyration and number of locations increase the probability of BDE prediction).

We present the interaction effect between key features: time of day with GPS-derived travel pattern or location. As shown in Figure 6, the results demonstrate the interaction between the radius of gyration and time of day features and their impact on BDE prediction. The hours of the day are represented with colors, and the SHAP value represents the impact on the BDE prediction—positive SHAP value indicates a positive contribution to BDE prediction.

The radius of gyration followed similar patterns on both weekends and weekdays for values greater than approximately 1000 m; however, radius of gyration and day of week (weekday or weekend) had opposite patterns for values between 0 and 800 m contributing to BDEs on weekdays. When radius of gyration is between 0 and 800 m, SHAP values for weekdays follow an inverted u–curve peaking at around 500 m, whereas for weekends, the same radius of gyration range follows a u-curve (Figure 6 left [weekdays] vs right [weekends]).

In addition, we narrowed down the coordinates to solely represent the Pittsburgh area (ie, the place where data were collected) to localize the meaning of the values. Looking at the average latitude plots, we see that the patterns are inverse of each other for weekdays and weekends for the same latitudinal coordinates, which means that young adults visit certain areas before BDEs on the weekends but not on weekdays. This is highly likely owing to bars and other drinking spaces being in the vicinity of workplaces in the city.

PDPs show the marginal effect that 1 or 2 features have on the ML model [52]. In Figure 7, we visualize the impact of radius of gyration (which refers to the radius of the circle that includes all locations visited during a time window) on each prediction class.0 on the y-axis represents the expected (mean) probability value of each respective class. Positive y-axis values indicate positive contribution to the prediction of the class, whereas the negative y-axis values indicate negative contribution to the prediction of that class. x-axis values are radius of gyration values in meters, divided into 9 equal portions (ie, the number of samples between grids are equal).

From the plots, we can see that young adults are more likely to travel within a larger area (radius of gyration) before BDEs compared with non-drinking or low-risk drinking events on weekdays. By contrast, it is shown that a greater radius of gyration on weekends is negatively related to non-drinking positively related to low-risk drinking events and BDEs. However, specifically the range between 1223 and 10,952 m was positively related to BDEs. Furthermore, on weekends, the probability of not drinking alcohol declines as the radius of gyration increases, whereas on weekdays, it follows a more linear pattern with a slight increase in probability at higher radius of gyration.

In Figure 8, we present the PDP of BDE on weekdays (left) and weekends (right) and the interaction of 15-minute average longitudes and latitudes on a contour plot representing a section in the Pittsburgh metropolitan area (city of Pittsburgh and surrounding smaller towns). 9W6D refers to the model by taking 9-hour window and 6-hour analysis distance from the onset of BDE and 12W3D refers to the model by taking 12-hour window and 3-hour analysis distance from the onset of BDE.

This plot can be used as a map to display which visited areas increase the probability of BDEs on weekends versus weekdays. The horizontal axes represent the average latitude values, and the vertical axes represent the average longitude values. To preserve privacy and simplify the plot, rather than mapping out the exact locations of participants and the probability of BDEs at specific locations, we plotted their general locations (zip code level or neighborhood level) to give a bird’s eye view on their locations and calculate the probabilities in general areas rather than specific locations. The color spectrum represents the probability of the person starting a BDE within 6 to 15 hours later that day. From dark to bright colors, the probability of BDEs increases.

From the weekend plot, we found that in certain geographic locations (represented in yellow in the plot), the probability of BDEs was 24%, higher than the expected probability of 21.4% (relative increase of 11% and absolute increase of 2.6%).

As for the weekday plot, in the areas colored in yellow, the average probability of binge drinking behavior was 17%, higher than the expected probability of BDE of 11.9% in the entire data set, which indicates a 1.43× increase in BDE prevalence (weekday plot analyzes a greater area because of the differences in how the data are dispersed). Thus, it can be concluded that just by looking at the 15-minute average locations of young adults who reported hazardous alcohol use and were sampled from an ED in the Pittsburgh metropolitan area, we can make meaningful inferences about their probability of binge drinking later that day. With more fine-grained and personalized GPS data, it may be possible to increase the predictive power of location information.

In summary, we found that a combination of time of day and GPS-derived travel and location features provided interpretable patterns of young adults who are likely to report same-day BDEs. Our findings not only optimized the ML model with “windows of opportunity” but also identified “key location features” to trigger JITAIs to support the design of intervention strategies and messages for BDE prevention among young adults.

We attempted to predict imminent same-day BDEs in young adults with the use of smartphone-based sensors. We used ML coupled with XAI to investigate windows of opportunity to prevent BDEs. Our results confirm that smartphone sensors can be used to predict imminent BDE events before their onset using same-day behaviors and comparing them with the behaviors during non-drinking and low-risk drinking events on weekdays and weekends. The key contributing features we found are time of day, travel patterns and boundaries (eg, radius of gyration), and accelerations of body movements, which could be used for triggering JITAI before the onset of BDEs and thus preventing the potential risks of BDEs.

As our previous work confirmed that the detection of BDEs is feasible using phone sensor data [17,18], moving toward the prevention of BDEs, we focused on predicting imminent (same-day) BDEs before their onset. In this feasibility study, we demonstrated how to build an ML model using passively sensed smartphone sensor data, which predicted young adult drinking behaviors (whether they will engage in BDEs, low-risk drinking events, or non-drinking events), with 94.3% accuracy for weekday drinking behavior (WDXGBoost-W9D6) and 95% accuracy for weekend drinking behavior (WEXGBoost-W12D3). The prediction model using phone sensor data for both weekday and weekend drinking had higher accuracy relative to a baseline model using only day of the week to predict drinking behavior (60.6% accuracy). We also found that 9 hours (for weekdays) and 12 hours (for weekends) of phone sensor data collected at a 6-hour and 3-hour prediction distance (weekday and weekend, respectively) maximized the F₁-score (which balances precision and recall) for predicting BDEs. The analysis window size is important for estimating data storage needs and potential privacy risks of storing sensitive data on the phone for prediction. This ML prediction model advances our prior work using phone sensors to “detect” episodes of alcohol use that have already started [17,18] by allowing the delivery of JIT support messages 3 and 6 hours before a predicted drinking event, when alternative plans to support drinking limit goals can be considered and implemented.

There were similarities in the most important phone sensor features for the “detection” and “prediction” of drinking events. For example, time (eg, time of day and day of the week) was important for both the “detection” [17,18] and prediction of drinking events. However, there were important differences in key features used to “detect” relative to “predict” a drinking event. As an example, microlevel features, such as screen interaction duration and number of activity changes, were key features relevant to the “detection” of a drinking event, whereas the macrolevel features of travel and location (eg, radius of gyration and latitude and longitude) were key features relevant to the “prediction” of an imminent drinking event. XAI further revealed that the interaction of time (eg, time of day and day of the week) with travel and location features contributed to the prediction of drinking events in young adults.

In predicting drinking events, we found that young adults who traveled more (eg, larger radius of gyration) and spent longer duration in important locations were more likely to report BDEs (ie, compared with non-drinking events) and drinking events (ie, non-drinking and drinking events) later that day. Furthermore, the participants who interacted with their smartphones more (ie, higher acceleration values) and had longer call durations were less likely to drink alcohol later in the day. These results suggest that smartphone interaction activity and communication during the day might indicate work, school, and social obligations and activities that could help to regulate or constrain drinking behavior. By contrast, young adults who had fewer communication interactions using the phone had an increased likelihood of drinking later that day. The low level of communication activity involving the phone in the hours before a drinking event may reflect a sense of social isolation, which is a direction to be examined in future research. The most important smartphone sensor features contributing to drinking behavior prediction may serve as early warning signals, which have parallels in the mental health literature, where, for example, certain signs and symptoms signal increases in depressive symptom severity [56].

Further analyses using XAI indicated differences in the importance of features in predicting BDEs in young adults on weekdays relative to weekends. For example, the radius of gyration was a more important predictor of BDEs on weekends than on weekdays. The relevance of GPS-derived travel data in relation to episodes of alcohol use is consistent with prior smartphone sensor work with young adults [16]. Our use of XAI adds to this emerging literature by showing, for the first time, that the interaction of day of the week, time of day, and location information contributed to the prediction of BDEs in young adults. The enhanced explainability of the ML prediction model provides novel insight into factors that serve as early warning signs of an imminent and potentially preventable BDE in young adults.

The ability to predict upcoming drinking events 3 and 6 hours before their likely onset during weekends and weekdays, respectively, using only smartphone sensor data could support JIT intervention [7], potentially boosting digital intervention effects [57]. This time window prior to drinking onset permits the opportunity for a more proximally timed, proactive intervention in young adult drinkers to modify their drinking intentions, enhance their motivation to set drinking limit goals, and provide them with tips to successfully meet their health goals when such support may be most salient. In several studies, including Suffoletto et al [35], we found that proactively prompting goal commitment to limit drinking before drinking onset is a critical component of JIT interventions. As such, a prediction-based intervention approach avoids delivering messages or support during a drinking event when cognitive and motivational impairment due to acute alcohol use might limit message utility.

The drinking prediction models presented in this paper, testing a range of distances from the onset of drinking (eg, 1-, 3-, and 6-hour prediction distances), suggest temporal changes that are the most predictive of drinking on specific days (weekends vs weekdays), which are relevant to tailoring intervention content [7]—for example, farther from the onset of drinking, enhancing the motivation to limit drinking, or planning alternative healthy activities might be prioritized. In contrast, once drinking starts (ie, relevant to a drinking detection model), specific techniques to reduce the risk associated with a BDE could be provided (eg, alternate alcoholic and nonalcoholic drinks) [4]. We envision the future use of our predictive models to prevent the onset of BDEs and suggest some considerations to developers that need to be considered for designing JIT intervention systems. In this system, young adults willingly download an app on their smartphone, consent to the analysis of phone sensor data, and receive intervention messages to avoid BDEs and related negative consequences. To make the app relevant and useful to a young adult, JIT resources could be sent to promote rewarding, personalized nonalcohol events that shift attention and intention away from drinking to healthy alternative activities.

Our results suggest that an eventual BDE prediction and intervention app requires a certain amount of sensor data to be temporarily stored on one’s smartphone. The phone needs to store a window of 12 hours (WEXGBoost-W12D3) and 9 hours (WDXGBoost-W9D6) of data to run the prediction algorithm for BDEs on weekends and weekdays, respectively. Older data can be removed on an ongoing basis to maximize data privacy. Furthermore, although our current system runs on a server, the prediction model is efficient enough that it could be executed directly on a smartphone, with data never leaving the smartphone. Regarding privacy and ethical issues in the collection of phone sensor data, there is a need to carefully review the data collection process with the participants, including the collection of specific types of sensor data (eg, location data) and the granularity at which data will be collected (ie, the precision of location data to minimize identifiability), and provide them with examples of the variables that will be derived (eg, the radius of daily travel from location data) so that they can evaluate the potential risks to privacy and provide informed consent for data collection. The voluntariness of data collection and ability to opt out of data collection at any time need to be made clear and discussed with the participants when obtaining their informed consent. The project’s app used reminders (eg, flashing red banner or notification saying that the app was running) to keep the participants informed of phone sensor data collection. Secure methods of data transmission and storage (eg, on the phone and server) were used but are subject to potential breach, a risk which the participants need to understand.

It is challenging to balance privacy preservation and the accuracy of knowing an individual’s location and travel patterns. The collection and analysis of GPS data raise privacy issues because the participant’s location (eg, home address) might be identifiable. The deidentification and level of granularity of data must be considered in protecting the privacy and confidentiality of the participants. Interestingly, based on our supplementary analysis (details in Multimedia Appendix 1) using the rounded GPS latitude and longitude coordinates, we found a trade-off relationship between privacy and accuracy. The supplementary experiment showed that accuracy was 5 percentage points lower compared with our best performing model (94%) when the latitude and longitude coordinates were rounded (to 1 decimal place; ie, 0.1 refers to approximately 11.1 km). In this scenario (GPS data rounded to 1 decimal place), GPS data do not represent the specific address that the participants visit during the day but instead represent a relatively large region, ranging from 10 to 50 km. As such, it is important to consider the granularity of GPS data to make meaningful BDE predictions that balance privacy-preserving strategies and the performance of the prediction algorithms. Alternatively, after a full explanation of the risks and data to be collected, the participants may agree to the collection and analysis of granular location data for research and health care purposes. These location data, along with analysis of key behavioral features captured by smartphone-based sensors impacting BDEs, permit the use of XAI techniques, including feature importance and contribution (via SHAP and PDPs) to determine potential features contributing to the prediction of imminent BDEs. In addition, XAI models that explore counterfactuals (eg, non-BDEs) and contrastive examples (eg, low-risk drinking) [58,59] may be used to understand hypothetical alternative scenarios and generate possible feature combinations to obtain an expected outcome (eg, non-drinking). However, risks such as overtrust and reliance [60] on clinician decision support systems need to be carefully investigated.

More frequent reminder systems or a more effective incentive mechanism could improve compliance with sustainable data collection using phone sensors and phone surveys over a long period. The development of a “dashboard” could help a researcher to monitor participant compliance. Alternatively, newer transdermal alcohol biosensors might facilitate the collection of data on alcohol consumption with less stigma and participant burden for use in providing “ground truth” to train the ML model.

Regarding battery drain, future research on the use of smartphone sensors for predicting drinking events might identify an optimal set of sensors to use. This optimal set could use a lower overall sampling rate or optimize the sampling rate for the sensors based on the algorithm chosen to minimize battery drain. Although we used the default sampling rates specified in the AWARE data collection framework, in the future, we can explore different sampling rates to optimize both the performance of the predictive models and the battery life of the smartphones. Future studies should identify which phone sensor features are acceptable to participants for monitoring purposes to allow for more personalized support.