This study was designed as a retrospective feasibility study using workplace daily report data collected from 2 Japanese companies. The company A dataset was used to train the BERT-based regression model, whereas the company B dataset was used as an external test dataset to evaluate whether the trained model and a locally deployed Llama model could estimate within-person longitudinal changes in workday happiness from daily report texts.

Because this was a retrospective feasibility study based on existing workplace daily report datasets, the sample size was determined by the number of consenting employees who had available paired data consisting of daily report text and self-reported workday happiness scores during the data collection periods rather than by an a priori power calculation. The sample therefore reflected the available longitudinal workplace data from employees in the 2 participating Japanese companies. Participant and dataset characteristics are described in the Participants and Datasets section.

We collected 2 types of data: daily report text and self-reported workday happiness scores. Daily reports consisted of free-text descriptions in Japanese of the participants’ daily lives with no character limit. Participants were instructed as follows (translated from Japanese): “Daily report text (required): Please write your diary in about 3 lines (it may be unrelated to work). Free description with no character limit.” Examples of the daily reports are provided in Table 1.

Participants also rated their workday happiness on an 11-point scale ranging from 0 (extremely unhappy) to 10 (extremely happy). The item wording was as follows (translated from Japanese): “Were you happy today? 0 (very unhappy) to 10 (very happy).” This item was based on the ladder by Cantril [20].

Daily reports were collected using an original web browser–based application. Participants were instructed that entries should typically be submitted once or more per workday to reflect the events of that day at the end of the workday.

We used data from 2 companies. Company A is a major advertising and marketing company in Japan. In company A, 92 members submitted daily reports over 2 months from September 1, 2022, to October 31, 2022, yielding 1725 data points. The mean age of participants in company A was 39.0 (SD 9.0) years, and 22.8% (21/92) were female. Company B is a major Japanese electronics manufacturer. In company B, 11 employees submitted daily reports over 6 months from December 19, 2022, to May 18, 2023, yielding 652 data points. The mean age of participants in company B was 42.3 (SD 10.1) years, and 9.1% (1/11) were female. The average number of characters per entry was 58.8 in company A and 72.2 in company B (Table 2).

Participants were employees of 2 Japanese companies that implemented daily workplace report systems during the study periods. Participants were recruited through workplace-based study invitations and provided informed consent before data collection. Eligible participants were employees who submitted daily report text entries with corresponding self-reported workday happiness scores during the data collection period. Reports were included in the analysis when both the daily report text and the self-reported workday happiness score were available. Reports with missing text, missing happiness scores, or invalid date information were excluded from the corresponding analyses.

The purpose of this study was not primarily to estimate the absolute level of workday happiness but rather to assess whether changes in workday happiness could be captured over time. Therefore, we used the Pearson correlation coefficient as the main evaluation metric following previous work [15]. We interpreted the Pearson correlation coefficient (r) as an effect size index indicating the degree to which self-reported and estimated scores increased and decreased in the same direction over time, that is, the extent of agreement in temporal fluctuation. Following prior work, we used r=0.40 as a pragmatic reference point for this feasibility evaluation rather than as a threshold for real-world workplace implementation [19]. To contextualize this value, we also reviewed prior work in personnel and applied psychology. For example, Schmidt and Hunter [26] reported that several commonly used predictors of job performance had validity coefficients in this range. Bosco et al [27] argued that correlations should be interpreted using empirical benchmarks from the relevant research context rather than generic rules of thumb. However, a moderate correlation suggests only that the model may capture some within-person temporal signal.

Because the Pearson correlation coefficient does not quantify the absolute magnitude of prediction errors, we additionally used the mean absolute error (MAE) and root mean squared error (RMSE) as error-based metrics. The MAE represented the average absolute difference between self-reported and estimated workday happiness scores, whereas the RMSE gave greater weight to larger prediction errors. Both MAE and RMSE were calculated on the original workday happiness scale from 0 to 10.

Model performance was evaluated at the participant level in the external test dataset from company B. For each participant and each model, we calculated the Pearson correlation coefficient between self-reported and estimated workday happiness scores. Pearson correlation tests were conducted as 2-tailed tests with a significance level of α=.05. To quantify uncertainty in the participant-level correlation estimates, we calculated 95% CIs for Pearson correlation coefficients using the Fisher z transformation. CIs were not calculated when the Pearson r was undefined because the self-reported workday happiness score was constant. We also calculated MAE and RMSE for each participant and each model to quantify absolute prediction errors on the original workday happiness scale from 0 to 10.

Participant-level MAE and RMSE values were compared between the BERT-based model and the Llama model using 2-tailed paired t tests, with statistical significance set at α=.05. Effect sizes for paired t tests were reported as Cohen dz. Days without diary entries or self-reported workday happiness scores were treated as missing and excluded from the corresponding participant-level analyses. No imputation was performed.

We also conducted exploratory participant-level analyses to describe whether diary text characteristics varied according to model performance. Specifically, we conducted 4 exploratory participant-level Pearson correlation analyses: BERT performance vs average sentence length, Llama performance vs average sentence length, BERT performance vs Self-BLEU, and Llama performance vs Self-BLEU. Average sentence length was calculated for each participant across daily report texts. Lexical diversity was evaluated using Self-BLEU [28]. A higher Self-BLEU value indicated that a participant’s daily reports were more similar to one another and, therefore, less lexically diverse, whereas a lower Self-BLEU value indicated greater lexical diversity. Self-BLEU was calculated using SacreBLEU [29]. Participant U01 was excluded from these exploratory analyses because model performance based on the Pearson correlation coefficient could not be calculated for them. For visualization only, participants were divided into below-median and above-median performance groups for each model in the box plots. These groups were not used to calculate separate correlations.

This study was approved by Kyoto University, including Nara Institute of Science and Technology (review 26-P-16). The company A and company B datasets used in this study were collected as part of a previous field study of daily workplace reports [30]. Before beginning the questionnaire, participants read an online explanation of the study overview and provided electronic informed consent.

To protect participant privacy and confidentiality, all datasets used in the present analysis were deidentified prior to analysis. Participants were instructed not to include personal information or company-confidential information in their free-text diary entries. In the present manuscript, participants in company B are referred to using pseudonymous labels (U01-U11), and no directly identifying information is reported. Because free-text daily reports may contain potentially identifiable information, raw text data are not publicly shared. Instead, only aggregated or statistically processed results are reported. Participation was voluntary. Participants did not receive financial compensation or other incentives. The model outputs in this study were analyzed retrospectively for research purposes and were not used to make employment, personnel, or managerial decisions about individual participants.

Using r=0.40 as a pragmatic feasibility benchmark, 81.8% (9/11) of the users met or exceeded this level, indicating that the model achieved practically meaningful alignment for most users. The correlation coefficients of workday happiness estimation for each user in company B are shown in Table 3, and the time series are shown in Figure 2. As shown in Table 3, model performance varied across users. Hereafter, individual users are denoted as U01 to U11. Error-based metrics also showed variability across participants (Table S1 in Multimedia Appendix 1). For the BERT-based model, MAE ranged from 1.04 to 2.78, and RMSE ranged from 1.25 to 3.50 on the original workday happiness scale from 0 to 10. For the Llama model, MAE ranged from 0.76 to 4.39, and RMSE ranged from 1.06 to 4.64. The participant-level mean MAE and RMSE were 1.52 and 1.87 for the BERT-based model and 1.71 and 2.07 for the Llama model, respectively. As an exploratory comparison of absolute prediction error between models, paired t tests were conducted using participant-level MAE and RMSE values. No statistically significant difference was observed between the BERT-based model and the Llama model for MAE (t₁₀=–0.72; P=.49; Cohen dz=–0.22) or RMSE (t₁₀=–0.67; P=.52; Cohen dz=–0.20).

For U01, the correlation coefficient could not be calculated because the self-reported workday happiness score remained consistently at 8 (Figure 2 and Table 3). As shown in Figure 2, while users’ reported levels of happiness remained constant, the language model’s estimated levels of happiness showed some variation. Next, 9.1% (1/11) of the users showed a negative correlation coefficient, although it was not significant (BERT: P=.70; Llama: P=.72; Figure 2 and Table 3; U03). We examined the daily report text of this user and found that the user frequently used the expression “A nice day that...” As shown in Figure 2, only 1 user (U05) repeatedly self-reported a workday happiness score of 0 after April 2023. Although the user initially provided diverse texts, after April 2023, the entries were primarily zeros, and the content became uniform. In addition to this user, there were other users for whom the number of entries decreased since April 2023 (Figure 2; U2, U3, U5, U6, and U9).

In the comparison between models, Llama outperformed BERT for several users (U04, U05, U07, and U11), whereas BERT performed better for U09, and the 2 models showed similar performance for U06.

The 95% CIs indicated uncertainty in participant-level performance estimates. Although several point estimates met or exceeded the pragmatic feasibility reference point of r=0.40, some CI lower bounds were below this value.

This study examined whether daily report texts could be used to estimate within-person longitudinal changes in workday happiness among Japanese employees. A BERT-based regression model trained on company A data and a locally deployed Llama model were evaluated using an external test dataset from company B. Overall, the findings suggest preliminary feasibility for capturing broad within-person temporal trends in workday happiness from daily report text. However, MAE and RMSE showed participant-level variation in prediction error, indicating that the findings should be interpreted cautiously and not as definitive validation.

The findings suggest that daily report texts may contain information related to within-person changes in workday happiness. Open-ended responses may provide contextual information about work-related events or conditions that coincide with changes in self-reported workday happiness. For instance, if a decline in workday happiness is inferred and a respondent states, “I could not make any progress at work because the Wi-Fi did not connect,” this may indicate the need to investigate Wi-Fi connectivity issues. Similarly, a comment such as “The meeting was too long and I became fatigued” could serve as a catalyst for re-evaluating meeting durations. Thus, workday happiness estimation models may help identify recurring workplace issues that warrant further human review. However, the present findings should be interpreted as preliminary feasibility evidence rather than evidence that model outputs can directly identify the causes of changes in workday happiness or prescribe specific interventions.

Second, even when BERT-based models and LLMs are used as predictive models for workday happiness, their use in workplace settings requires careful consideration. Prediction performance alone is not sufficient to justify workplace deployment because the main issue is how estimated scores are interpreted and used in practice. If such estimates are treated as definitive judgments of employees or are used for personnel evaluation, they may undermine trust and discourage honest diary writing [31,32]. Therefore, predicted workday happiness scores should be used as supportive indicators for happiness interventions and workplace improvement rather than as direct evidence for employee assessment. In addition, because diary texts may contain sensitive personal information and model predictions may be biased or difficult to interpret, organizations need clear data handling policies.

Third, to collect daily report text over an extended period, it is necessary to mitigate participant dropout. Prior research also indicates that diary studies need to address dropout [33,34]. Several participants discontinued diary submission before the end of the observation period. This suggests that keeping a diary is burdensome, thereby posing a constant risk of attrition. Many participants stopped inputting their entries after April 2023, suggesting that environmental changes might have imposed stress that hindered their ability to continue diary writing (Figure 2). The potential solution involves using services that make it easier to write daily report texts. For example, a service could be provided to facilitate diary entry [35], or a service using LLMs to assist with writing could be used [36]. With these methods, the data collection rate may increase without requiring participants to be repeatedly prompted to submit diary entries.

Fourth, considering user-specific characteristics such as gender and age may improve the model’s performance in workday happiness estimation. Research on subjective descriptions of happiness collected through daily report text has demonstrated that a wide variety of descriptions are gathered [37]. What is considered workday happiness depends on attributes such as culture [38] and age [39]. In fact, taking into account age, gender, and personality has been shown to enhance model performance [40-43]. Therefore, collecting additional data through separate questionnaires may improve workday happiness estimation accuracy.

Finally, differences in model performance may reflect user-specific characteristics, including the length and content of daily reports, the explicitness of emotional expressions, and the types of work-related events described. A previous study comparing ChatGPT with fine-tuned BERT-style models reported that relative performance varied across natural language understanding tasks; ChatGPT performed worse on paraphrasing and similarity tasks but showed comparable or better performance on other tasks [44]. In the present study, model performance differences could also not be explained by text length alone. Other factors such as diary content, emotional clarity, individual writing style, and differences in the determinants of workday happiness may also have influenced model performance.

Previous studies have shown that happiness and subjective well-being can be estimated from text, including social media language, conversational text, free-text questionnaire responses, and LLM-based assessments [10-16]. For example, Kjell et al [14] reported relatively high performance when predicting questionnaire-based well-being scores from free-text responses using BERT-based representations. The present findings are consistent with the literature, suggesting that free-text language contains information related to self-reported happiness.

Unlike previous studies that estimated well-being from cross-sectional free-text questionnaire responses [14-16] or from social media language at the individual, population, or geographically aggregated level [10,13], the present study examined within-person longitudinal changes in workday happiness using routinely collected workplace daily reports. In addition, the BERT-based model was trained on data from one company and evaluated on data from another, yielding a more stringent preliminary evaluation than a random split within a single dataset.

This study has several limitations.

First, the relationship between specific events and workday happiness varies across individuals. For example, the statement “Today I have a meeting!!!” may evoke workday happiness for some individuals but not for others. Understanding the relationship between holidays and workday happiness requires not only sufficient textual data but also extensive information about the writers’ beliefs, values, and current circumstances.

Second, the generalizability of the model is limited. The model used in this study was based on data collected from experiments conducted by two companies: company A, an advertising and marketing firm, and company B, an electronics manufacturer. In addition, expressions of workday happiness may be influenced not only by industry but also by occupation, organizational culture, daily report writing practices, and cultural background. Therefore, the present findings may not directly generalize to nonoffice workers, smaller companies, or non-Japanese workplace settings.

Third, the daily report text collected may have been influenced by the workday happiness scores collected at the same time. Prior research on affect labeling [45] has shown that putting feelings into words can attenuate emotional experience and may function as a form of implicit emotion regulation. Future studies should adopt a design that separates the collection of free-form diary entries from self-reported happiness assessments.

Fourth, the small external test sample limits the statistical robustness and generalizability of the findings. Although repeated-measure datasets can be useful for initial feasibility analyses of within-person variability in related fields [46,47], the company B dataset included reports from only 11 participants, and the exploratory analyses of diary characteristics were based on 10 (90.9%) participants after excluding U01. Therefore, participant-level estimates and exploratory associations should be interpreted as preliminary and hypothesis generating rather than as definitive evidence.

Fifth, there are limitations inherent to the language models themselves when using BERT- or LLM-based approaches. Because these models are not fully transparent regarding which linguistic features drive their predictions, it is difficult to identify the causes of failure or ensure construct-level validity. In this study, we did not conduct a detailed evaluation of prompt design and limited the analysis to a simple prompt. Therefore, even when correlations with self-reports are observed, the extracted signals may partly reflect model-specific biases, and careful robustness testing is required.

Sixth, in the BERT-based evaluation, the company A training dataset and the company B external test dataset differed in both data collection timing and duration, which may have affected model performance. Because this study used existing workplace datasets, we could not separate the effects of company context from seasonal or temporal effects. In contrast, the Llama model was evaluated in a zero-shot setting and was not trained on the company A dataset. Future studies should collect data from multiple companies during the same or comparable periods and examine whether seasonality and differences in observation period affect model performance.

Finally, it should be emphasized that the target construct addressed in this study was work-related happiness derived from workplace diary texts (referred to as “workday happiness”) and that it should be distinguished from general happiness or broader subjective happiness.

Measuring workday happiness as a determinant of well-being is increasingly recognized as crucial. While previous studies have used free-text descriptions to estimate happiness on a broad scale, limited research has focused on tracking individual fluctuations in workday happiness over time owing to the challenges associated with longitudinal data collection. This study addresses this issue by obtaining longitudinal data from two workplaces over 2 and 6 months. Subsequently, the data were used to construct a workday happiness estimation model and assess individual workday happiness levels. Evaluation of model performance using correlation coefficients showed variability in the correlation values among individuals. Although the sample size of 11 participants limits the generalizability of the findings, this study provides preliminary evidence supporting the feasibility of the proposed approach. Our findings highlight the potential of longitudinal, text-based approaches to capture the dynamic nature of workday happiness and support the design of interventions that foster healthier and more sustainable working lives.