A quantitative cross-sectional study design was applied to assess the usability of the digital library of the SCFHS. The study participants included the trainee residents from all specialties with valid registration at the SCFHS. A list containing the registered names of 3455 professionals was obtained from the SCFHS, which constituted the sample frame. Assuming that 50% of the population conveys positive perceptions of digital library usability, with a 95% CI and a precision of 5%, the estimated sample size using the single proportion formula was calculated to be 384. This study was performed over a period of 2 months, in March and April of 2018. The study tool was constructed using the SurveyMonkey platform. The tool was sent online to all registered email IDs listed in the sample frame. Two reminder emails were sent after 2 weeks to increase the response rate.

This study adopted the validated questionnaire of the System Usability Scale (SUS) developed by John Brooke in 1996 [10]. The SUS system is a robust, reliable, and valid tool, which has been extensively used by researchers. It has also shown a high degree efficiency in testing library systems, repositories, and websites [11]. The first section comprises 10 questions measuring the system usability of various hardware, software, websites, and applications, with scaled responses on a 5-point Likert scale ranging from strongly disagree to strongly agree. The SUS is considered to be an easy-to-use tool with simple and straightforward statements arranged as odd numbers (1, 3, 5, 7, 9) that are positively expressed and statements with even numbers (2, 4, 6, 8, 10) that are negatively expressed.

The 10 items on the SUS are as follows: (1) I think that I would like to use this system frequently, (2) I found the system unnecessarily complex, (3) I thought the system was easy to use, (4) I think that I would need the support of a technical person to be able to use this system, (5) I found the various functions in this system to be well-integrated, (6) I thought there was too much inconsistency in this system, (7) I would imagine that most people would learn to use this system very quickly, (8) I found the system very cumbersome to use, (9) I felt very confident using the system, and (10) I needed to learn a lot of things before I could get going with this system.

The sequence of the statements was created so as to reduce concurrence and extreme bias. The second section comprised questions on demographic characteristics, accessibility, training and experiences in using databases, and suggestions to improve the digital library. A pilot study was performed with 12 participants to ensure reliability and validity of the questions in section 2, and these responses were not included in the main analysis.

The SUS scores were calculated with reference to the guidance outlined by Jeff Sauro [12]. The scores for each question were converted to a new number, summed, and then multiplied by 2.5 to convert the original scores ranging from 0 to 40 to a range of 0 to 100. The SUS scores were interpreted with a cutoff of 68. A score above 68 indicates average performance, whereas a score below 68 is considered to be below average. Furthermore, comparison and easy reference of individual items of findability, complexity, consistency, and confidence can be achieved using targeted benchmark scores that make the responses more meaningful [12].

The data were analyzed using the IBM software Statistical Package for Social Sciences version 22.0. Descriptive statistics (frequencies, percentages, mode, and mean and SD) were used to describe the categorical and quantitative variables. Testing of associations between categorical and continuous variables was performed by the Student t test and analysis of variance, respectively. A P value less than .05 was judged to indicate statistical significance.

The study was reviewed and approved by the Institutional Review Board Committee of College of Medicine, with reference number CMED 305-F8-2018-18. All participants were informed about the purpose of the study and electronic informed consent was obtained on the first page of the survey tool. The study ensured that the participants’ data will be confidential, private, and used for research purposes only.

This study focusing on the usability of the SCFHS digital library by registered trainee health practitioners is the first of its kind in Saudi Arabia. Some of the main findings indicate that digital library use in terms of perceived complexity, consistency, and confidence is below average among the trainees. Since the overall SUS score obtained in this study was 52.9, a grade D categorization may be given, denoting low usability [13]. Benchmarking of the scores of individual SUS items provides useful interpretation [14]. In this study, item 1 that reflects the digital library’s frequent use was the sole item that obtained a score above the benchmark score. This finding suggests the participants’ increased perceived usability toward using the digital library, whereas the rest of the SUS items obtained low scores, suggesting difficulty in all attributes of measures of complexity, consistency, and confidence.

Nevertheless, on a positive note, the SUS score showed a significant increase among those who attended a voluntary introductory training session on an ad hoc basis that was provided either by the individual databases or by their affiliated institutions through webinars and YouTube sessions. However, it must be emphasized that no formal training sessions were organized facilitating use of the digital library. These key findings may draw attention. Research studies have linked high computer literacy and social media to increased digital library usability among users. Umukoro et al [15] examined the factors associated with increased digital library use among university students by performing a mixed methods study in Nigeria, and established three major predictors among the users. They linked high system and service quality, computer skills, and level of satisfaction with electronic services to increased e-Library use, whereas the determinants of not using the digital library were primarily lack of awareness of services, inadequate computer skills, and absence of user-training facilities. These results are highly consistent with our findings, which could indicate the chief reasons for low usability. Similarly, Piccoli et al [16] built a conceptual framework of contributing factors for effective electronic learning (e-Learning) that depends on the interface between human and design factors. Human factors constitute the instructors’ and students’ roles such as motivation, learning, and training, whereas design factors include technology, course contents, and an interactive environment that have a direct effect on the level of use. A qualitative analysis demonstrated incomprehensible website design and content as some of the major themes for low usability of a digital library, which suggested that the search protocol should be established in accordance with the end users’ expectations to improve website usefulness [17]. Since some users may demonstrate computer literacy, the reasons for low usability remain ambiguous. In-depth qualitative research is required to comprehend the factors associated with low SUS scores. The poor system usability may reflect a combination of factors or attributes that regulate actual usability.

The other main finding of training-assisted improvement in digital library usability suggests that the respondents may enhance usability with essential training skills and operative assistance. These findings are in accordance with similar studies from Nigeria and Zimbabwe, where lack of training and awareness were identified as principal barriers for poor electronic resource utilization [4,5,18]. These findings are further supported by a study that demonstrated improved e-Learning behavior after implementation of a successful end-user internet literacy training program for university students [19]. Our study thereby suggests the definite need for training measures to be reinstated by the SCFHS for the effective use of digital library facilities. Future studies obtaining pre and post evaluation measures of efficacy of training programs are highly recommended.

Another issue of concern is the lack of awareness of digital library facilities. The finding that only 3.7% of the participants knew about the digital library facility through the SCFHS website is alarming. The content display on the website plays a constructive role in spreading awareness about the digital library. There is an immediate need to exemplify content guidance to improve awareness and accessibility of the digital library in the SCFHS website. Hinchliffe and Mummery [20] elucidated the significance of the involvement of intended audience and end users in designing websites through usability testing to improve and optimize user experiences. Likewise, we recommend performing similar research to obtain users’ suggestions for content directions of the SCFHS website highlighting the digital library facilities, which can result in increased usability.

Furthermore, the library usage patterns demonstrate that preparation for presentations and obtaining information on routine patient care are the two most common reasons for use. Although these results are reasonably expected from the trainees, the suboptimal usage for other relevant purposes warrants further clarification. The use of a digital library during examinations and for research and additional support materials was found to be low. However, the use of alternate sources such as institutional libraries and the Saudi Digital Library may account for the probable low use. These findings are strong indicators of the suboptimal awareness of available online databases. A similar study performed at a university library in the United States showed that 94.5% of residents accessed the resources for patient care, 92% of nursing professionals accessed the library for class preparations, and 76% accessed the library for research purposes [1]. There is great potential for research to further discover the reasons for low usage among the participants. Library orientation for residents and fellows, web helpdesk service, training on online database searching skills, and web-based notification systems are some of the suggested methods to improve usability [21].

This study found an inclined preponderance toward the use of UpToDate as the most popular online database. This is also reflected in the lower utilization of other multiple databases. Provision of training facilities ought to incorporate content description and specifications of other online databases such as Cochrane, Best evidence, and DynaMed to maximize benefits. These three evidence-based medicine databases containing full free texts and voluminous literature on clinical trials with a level of evidence and recommendations based on established guidelines serve as priceless assets for both research and clinical practice. The reported low use of these resources raises concerns. Moreover, this may lead to concealing a significant amount of variance in perceived usability scores. These findings point toward the need for further exploratory research to identify the determinants and reasons for low usage, which can inform exhaustive measures to be taken to improve awareness and usability.

This study has certain limitations. The relatively small sample size restricts the generalizability of the findings. Furthermore, the sample included only trainees, and other health care providers who could have served as potential resource participants were not included, which again may lead to limited generalizability. However, the strength of this study is that it is the first such investigation in the region of Saudi Arabia, and the multiple findings generated can be considered as a vital basis for future research. Another relevant point is the ambiguity in the usability of each of the multiple online databases. This study assessed the overall usability of the digital library; however, details of the usability of individual databases were not explored, which can be a subject of future research.

The concept of digital libraries has revolutionized the usability of libraries by providing instant and simplified access. The digital library of the SCFHS is a major source of referencing for health professionals in Saudi Arabia. This study is the first to evaluate the usability of the digital library of SCFHS.

Lower usability scores were obtained, demonstrating the below-average utilization of resources. The participants demonstrated low confidence in accessing the digital library, and needed technical support, guidance, and training programs for efficient access and use of the contents. The online database UpToDate was the most popular database accessed for clinical support decisions, whereas the remaining databases showed limited usability. However, training and orientation were associated with higher usability scores, suggesting a promising solution to improve usage. A well-structured training program facilitating simplified and efficient navigation of the digital library contents is the need of the hour. In addition, the SCFHS website should consider increasing awareness through innovative methods to increase the potential use of multiple databases. There is a great potential for modification in website design to increase awareness and improve digital library usability.

This research may highlight certain general recommendations to improve the usability of the SCFHS digital library based on the identified findings in two major contexts: personnel and technical. Primarily, at the outset, we highly recommend facilitating an end-user training program as a mandatory apprenticeship featuring the navigatory steps in addition to the introduction of the available online databases and their specifications for wider use.

Next, the focus of recommendations may include the technicality and design of the website. Accessibility can be markedly improved by direction and guidance in a user-friendly mode. In addition, the design should facilitate categorization and description of the digital library contents. The homepage of the website must contain a downloadable user-friendly guide as an overview of digital library orientation. Furthermore, the trainees must be assisted with provision of helpdesk facilities, online chats, and regular notifications on digital library updates.