This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2443) Surveillance Systems (463) Public (e)Health, Digital Epidemiology and Public Health Informatics (733) Prevention and Health Promotion (994) Health Services in Resource-Poor Settings and LMICs (417)

Published on in Vol 7 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/41269, first published .
Comparative Performance Evaluation of the Public Health Surveillance Systems in 6 Gulf Cooperation Countries: Cross-sectional Study

Comparative Performance Evaluation of the Public Health Surveillance Systems in 6 Gulf Cooperation Countries: Cross-sectional Study

Comparative Performance Evaluation of the Public Health Surveillance Systems in 6 Gulf Cooperation Countries: Cross-sectional Study

Authors of this article:

Nawaf Albali1 Author Orcid Image ;   Sami Almudarra2 Author Orcid Image ;   Yahya Al-Farsi3, 4 Author Orcid Image ;   Abdullah Alarifi5 Author Orcid Image ;   Adil Al Wahaibi6 Author Orcid Image ;   Pasi Penttinen2 Author Orcid Image
Article Authors Cited by (1) Tweetations (6) Metrics

    Original Paper

    1Health & Public Sector, Accenture Saudi Arabia, Riyadh, Saudi Arabia

    2Gulf Center of Disease Prevention and Control, Gulf Health Council, Riyadh, Saudi Arabia

    3Department of Family Medicine and Public Health, College of Medicine and Health Sciences, Sultan Qaboos University, Muscat, Oman

    4Health Research Unit, Gulf Health Council, Riyadh, Saudi Arabia

    5Department of Public Health, College of Health Sciences, Saudi Electronic University, Riyadh, Saudi Arabia

    6Department of Surveillance, Ministry of Health, Muscat, Oman

    Corresponding Author:

    Nawaf Albali, MPH, MD, SBPM

    Health & Public Sector

    Accenture Saudi Arabia

    2412 Airport Road, Business Gate Complex

    Riyadh, 11616

    Saudi Arabia

    Phone: 966 563673334

    Email: nawaf.albali@gmail.com


    Background: Evaluating public health surveillance systems is important to ensure that events of public health importance are appropriately monitored. Evaluation studies based on the Centers for Disease Control and Prevention (CDC) guidelines have been used to appraise surveillance systems globally. Previous evaluation studies undertaken in member countries of the Gulf Cooperation Council (GCC) were limited to specific illnesses within a single nation.

    Objective: We aimed to evaluate public health surveillance systems in GCC countries using CDC guidelines and recommend necessary improvements to enhance these systems.

    Methods: The CDC guidelines were used for evaluating the surveillance systems in GCC countries. A total of 6 representatives from GCC countries were asked to rate 43 indicators across the systems’ level of usefulness, simplicity, flexibility, acceptability, sensitivity, predictive value positive, representativeness, data quality, stability, and timeliness. Descriptive data analysis and univariate linear regression analysis were performed.

    Results: All surveillance systems in the GCC covered communicable diseases, and approximately two-thirds (4/6, 67%, 95% CI 29.9%-90.3%) of them covered health care–associated infections. The mean global score was 147 (SD 13.27). The United Arab Emirates scored the highest in the global score with a rating of 167 (83.5%, 95% CI 77.7%-88.0%), and Oman obtained the highest scores for usefulness, simplicity, and flexibility. Strong correlations were observed between the global score and the level of usefulness, flexibility, acceptability, representativeness, and timeliness, and a negative correlation was observed between stability and timeliness score. Disease coverage was the most substantial predictor of the GCC surveillance global score.

    Conclusions: GCC surveillance systems are performing optimally and have shown beneficial outcomes. GCC countries must use the lessons learned from the success of the systems of the United Arab Emirates and Oman. To maintain GCC surveillance systems so that they are viable and adaptable to future potential health risks, measures including centralized information exchange, deployment of emerging technologies, and system architecture reform are necessary.

    JMIR Form Res 2023;7:e41269

    doi:10.2196/41269

    Keywords

    Gulf Cooperation Council; surveillance systems; program evaluation; performance evaluation; disease monitoring; emerging technologies 


    Background

    Evaluating public health surveillance systems is critical to ensure that events of public health importance are appropriately monitored. The US Centers for Disease Control and Prevention’s (CDC) original Guidelines for Evaluating Surveillance Systems, published in 1988 and updated in 2001, emphasize the need for prioritizing the use of available resources to maximize efficiency and promote best practices in surveillance monitoring [1].

    Increased concern regarding bioterrorism and the embryonic development of early outbreak detection technologies prompted the issuance of the Framework for Evaluating Public Health Surveillance Systems for Early Detection of Outbreaks in 2004 [2]. The framework leverages CDC’s current evaluation guidelines by emphasizing the need to evaluate all system attributes, including the level of usefulness, simplicity, flexibility, acceptability, sensitivity, predictive value positive (PVP), representativeness, data quality, stability, and timeliness, including syndromic surveillance systems.

    The Gulf Cooperation Council (GCC) is a coalition of 6 Arab countries that have similar socioeconomic characteristics and political outlooks: the United Arab Emirates (UAE), Bahrain, Saudi Arabia, Oman, Qatar, and Kuwait. All 6 nations are members of the Gulf Health Council, the executive arm of the Council of Ministers of Health in the GCC region.

    Surveillance systems evaluation attained paramount importance following the establishment of the Gulf Center of Disease Prevention and Control (Gulf CDC) in 2021, with the approval of Their Majesties and Their Highnesses, Leaders of the 6 Gulf Cooperation States [3]. A core mandate for this entity was the fostering of harmonization in public health data and surveillance efforts through systematic and ongoing surveillance evaluation, as well as the proper use of public health data exchange practices. The Gulf CDC is currently housed within the Gulf Health Council.

    Public health surveillance systems in the GCC region have witnessed considerable growth and maturity over the last decade, and the GCC has been widely lauded for ensuring adequate health surveillance systems [4]. However, this growth has been accompanied by an increase in the communicable disease (CD) and noncommunicable disease (NCD) burden, and there is a scarcity of evidence to evaluate the systems’ effectiveness in responding to new disease patterns and shifting population characteristics [5]. Therefore, it is crucial to generate and disseminate new knowledge with regard to surveillance systems in GCC countries to assess the systems’ utility, effectiveness, and health outcomes.

    Evaluation studies using CDC guidelines have been used to assess surveillance systems in different regions of the world and for several health-related events [6,7]. Despite the known maturity of health surveillance system performance worldwide, there is a lack of empirical research evaluating public health surveillance systems in the GCC region. The few studies undertaken in the GCC have focused on specific diseases within a single country; moreover, limited studies have focused on system attributes according to the CDC’s Public Health Surveillance Evaluation Guidelines [8,9].

    Objectives

    We aimed to evaluate public health surveillance systems in the GCC regarding their level of usefulness, simplicity, flexibility, acceptability, sensitivity, PVP, representativeness, data quality, stability, and timeliness. The specific objectives were as follows: (1) compare the characteristics of selected public health surveillance systems across GCC countries, (2) estimate the performance scores of GCC public health surveillance systems based on CDC surveillance evaluation attributes, and (3) explore the association between performance scores and GCC surveillance systems characteristics.


    Study Design and Population Characteristics

    A cross-sectional evaluation study was conducted to assess the GCC surveillance systems and their system attributes by using the CDC guidelines for evaluating disease surveillance systems. This study was conducted in Riyadh City from March to June 2022. The evaluation included an assessment of the usefulness of the system and other system attributes including simplicity, flexibility, acceptability, sensitivity, PVP, representativeness, data quality, stability, and timeliness. These characteristics were of importance for the hardware and software, the standard user interface, standard data format and coding, proper quality checks, and adherence to confidentiality and security requirements.

    Participation was voluntary, and the inclusion criteria were senior surveillance administrators with considerable experience and knowledge in implementing nationwide surveillance programs across GCC countries.

    Selection of Country Representatives

    The authors coordinated with the Gulf CDC to identify 6 country representatives who could respond on behalf of their countries using the purposive sampling technique. Within the Gulf CDC’s governance structure, the Gulf CDC Supervisory Committee acts as the Center’s Board of Directors. Inclusion criteria mandated that the country representative must be an epidemiologist or a public health professional with >5 years of experience in managing national-level surveillance systems. According to the inclusion criteria, the questionnaire was submitted to Supervisory Committee members, who purposively identified the appropriate representative to respond to each country. This technique enabled the accurate selection of representatives for each country and aided in preventing selection bias [10].

    Study Tools

    A standardized self-reported questionnaire with defined indicators to assess system attributes was developed. The indicators were created based on the CDC guidelines for evaluating disease surveillance systems and by using a review of prior public health surveillance assessments. The questions were constructed by 3 experts with experience in the establishment and assessment of surveillance systems, and content validity was determined by a panel of 5 epidemiologists, public health professionals, and preventive medicine specialists. The questionnaire was validated by 2 representatives from GCC member countries, and these responses were not included in the main research. Only minor adjustments were made to the original questions to improve clarity. The number of indicators used varied depending on the attributes assessed. Participants were asked to assess their agreement with the attributes’ exact indicators on a 5-point Likert scale (1=strongly disagree, 2=disagree, 3=neutral, 4=agree, and 5=strongly agree), with a higher score representing better performance on the assessed attribute.

    Operational Definitions of System Attributes

    Level of Usefulness

    The country representatives were prompted to score 10 indicators of usefulness on a 5-point Likert scale. The system was deemed useful if it could produce estimates of morbidity and mortality rates, identify high-risk populations and the social determinants of fatalities, generate actionable reports and insights instantly, permit the evaluation of the impact of preventive and control programs, anticipate trends and outcomes using artificial intelligence (AI) and machine learning models, and stimulate research.

    Simplicity

    The system’s simplicity was assessed in terms of both its structure and ease of use. A total of 6 indicators of simplicity were evaluated by the representatives. The indicators evaluated the clarity of the case definitions, the requirement for regular training, the convenience of acquiring data, and the country’s integration level with the health information system.

    Flexibility

    Flexibility alludes to the system’s ability to handle new health-related events and case definition revisions and its ease of integration with other systems. The representatives evaluated 3 indicators including the ability to respond quickly to a new demand, such as diseases and risk factors; the ability to respond quickly to emerging and re-emerging pandemics; and the ability to detect events other than notifiable diseases, such as food poisoning, chemical poisoning, and environmental events.

    Acceptability

    Representatives from each country evaluated 3 indicators of acceptability including the completion rates of entered forms, the associated laboratory completion rates, and the completion rates of hospitals and primary health centers.

    Sensitivity and PVP

    Neither the authors nor the representatives from the countries statistically evaluated the positive attributes of sensitivity and predictive value because of the heterogeneity of diseases covered in each country’s surveillance system. Therefore, representatives were required to rate 4 indicators of sensitivity and predictive values as proxies for the actual calculation. Indicators included the system’s ability to correctly detect a high proportion of disease cases and new epidemics or outbreaks, the proportion of reported cases that are true cases, and the proportion of reported epidemics that are true epidemics.

    Representativeness

    Four indicators were used to assess the accuracy of the surveillance systems in describing the occurrence of a health-related event over time and their distribution in the population by place and person. In addition, the system’s ability to measure the natural history of the disease, store clinical practice data (sites performing diagnostic tests and physician-referral patterns), and store clinical outcomes of interest (mortality data, hospitalizations, and laboratory data) was assessed.

    Data Quality

    The data quality reflects the completeness and validity of the data recorded in each country’s public health surveillance system. The representatives rated 3 indicators of data quality in their respective country including the accuracy of the data entered, presence of standardized methods for data entry, and internal validity of the surveillance system’s data.

    Stability

    Stability refers to the reliability (ie, the ability to collect, manage, and provide data without failure) and availability (the ability to operate when needed) of the public health surveillance system. Four indicators were used to evaluate the stability. The indicators assessed spontaneous outages and downtimes, resources dedicated to technical repairs to maintain system functionality, and the time required for gathering, receiving, and managing data.

    Timeliness

    Timeliness refers to the rapid availability of sufficient data to enable public health authorities to take appropriate actions. Two indicators were developed and assessed for timeliness, including the system’s ability to rapidly generate data for immediate disease control and long-term program planning.

    Data Analysis

    The indicators assessing system attributes were scored on a Likert scale from 1 (“strongly disagree”) to 5 (“strongly agree”). Independent variables, including attribute scores for each country, were calculated as the sum of each indicator’s scores. The global scores for each country were then calculated as the sum of the attribute scores for that specific country. The dependent variables were the types of diseases covered by the surveillance systems, their data storage type, the presence of syndromic surveillance, and their placement within the original system.

    Differences among the various proportions of categorical data were evaluated for statistical significance using the chi-square test. Fisher exact test (2-tailed) was used to counteract the small sample size, where the expected frequency was <5 in any of the cells in the 2 × 2 tables. A nonparametric test (Mann-Whitney) was used to ascertain substantial differences when the assumption of normality was violated. Continuous variables were represented as mean and SD. Episheets software was used to calculate the CIs [11]. Descriptive data analysis and depiction of graphs were performed using Microsoft Excel (Microsoft Office 365).

    For all statistical tests, a P value ≤.05 was considered statistically significant. All statistical analyses were performed using SPSS software (version 24.0; IBM Corp). A univariate linear regression analysis was conducted to confirm the effects of various systems characteristics on the global score.

    Ethics Approval

    Ethics approval for this study was obtained from the Institutional Review Committee in King Fahad Medical City, Ministry of Health, Saudi Arabia (Institutional Review Board Log Number: 21-511E).

    Informed Consent and Data Integrity

    A written informed consent was obtained from the representatives before their participation. Representatives were informed that they were free to refuse to participate and withdraw from the study at any time without any disadvantage or prejudice. Confidentiality and privacy were ensured by assigning a code for each country. They were notified about their rights regarding being informed about the way their data would be used and that it would be used only after ensuring that it would not cause harm. No compensation was provided to the participants for their involvement in this research; participants voluntarily agreed to engage in the study to contribute to the research and advance their knowledge in the field.


    Surveillance Systems Characteristics

    Table 1 shows the selected characteristics of public health surveillance systems across GCC countries. All surveillance systems in the GCC covered CDs, and approximately two-thirds of them covered health care–associated infections (HAIs; 4/6, 67%, 95% CI 29.9%-90.3%). Three countries reported having full electronic surveillance systems, whereas the others reported that their systems were a hybrid between electronic and paper-based operations (3/6, 50%, 95% CI 18.7%-81.2%). The use of on-premises data storage was reported in 4 countries, compared with 2 countries that stored the surveillance data on-cloud storage (4/6, 67%, 95% CI 29.9%-90.3%). While all GCC countries reported having syndromic surveillance, 1 country reported that its syndromic surveillance system was separate from the original system (1/6, 17%, 95% CI 0.80%-59.0%).

    Table 2 details select characteristics of the public health surveillance for each country. The UAE reported covering all diseases of interest including CDs, NCDs, injuries and falls, HAIs, and environmental and occupational diseases. The Sultanate of Oman’s surveillance system covered CDs, HAIs, and environmental and occupational diseases. CDs and environmental and occupational diseases were covered in the State of Qatar. The surveillance systems of both Bahrain and Saudi Arabia covered CDs and HAIs. However, the surveillance system in Kuwait only covered CDs.

    Table 1. Selected public health surveillance characteristics among Gulf Cooperation Council member states (N=6).
    System characteristicsTotal, n (%, 95% CI)
    Disease coverage

    		CDsa6 (100, 60.9-100)

    		NCDsb1 (17, 0.80-59)

    		Injuries and falls1 (17, 0.80-59)

    		HAIsc4 (67, 29.9-90.3)

    		EODsd3 (50, 18.7-81.2)
    System type

    		Electronic3 (50, 18.7-81.2)

    		Hybrid3 (50, 18.7-81.2)
    Storage type

    		On premises4 (67, 29.9-90.3)

    		On cloud2 (33, 9.6-70)
    Syndromic surveillance

    		Yes6 (100, 60.9-100)
    Place of SSe

    		Within the original system5 (83, 43.6-96.9)

    		Separate from the original system1 (17, 0.80-59)

    aCD: communicable disease.

    bNCD: noncommunicable disease.

    cHAI: health care–associated infection.

    dEOD: environmental and occupational disease.

    eSS: syndromic surveillance.

    Table 2. Selected public health surveillance characteristics among Gulf Cooperation Council member states, by country.
    System characteristicsTotal across all countries (N=6), n (%)Saudi ArabiaKuwaitUnited Arab EmiratesOmanQatarBahrain
    Disease coverage

    		CDsa6 (100)PresentPresentPresentPresentPresentPresent

    		NCDsb1 (16)AbsentAbsentPresentAbsentAbsentPresent

    		Injuries and falls1 (16)AbsentAbsentPresentAbsentAbsentAbsent

    		HAIsc4 (66)PresentAbsentPresentPresentAbsentPresent

    		EODsd3 (50)AbsentAbsentPresentPresentPresentAbsent

    		Disease coverage within each country (n=5), n (%)—e2 (40)1 (20)5 (100)3 (60)2 (40)2 (40)
    System type

    		Electronic3 (50)PresentAbsentAbsentPresentPresentAbsent

    		Paper-based0 (0)AbsentAbsentAbsentAbsentAbsentAbsent

    		Hybrid3 (50)AbsentPresentPresentAbsentAbsentPresent
    Storage type

    		On premises4 (66)AbsentPresentPresentAbsentPresentPresent

    		On cloud2 (33)PresentAbsentAbsentPresentAbsentAbsent
    Syndromic surveillance

    		Yes6 (100)PresentPresentPresentPresentPresentPresent

    		No0 (0)AbsentAbsentAbsentAbsentAbsentAbsent
    Place of SSf

    		Within the original system5 (83)PresentPresentPresentPresentAbsentPresent

    		Separate from the original system1 (16)AbsentAbsentAbsentAbsentPresentAbsent

    aCD: communicable disease.

    bNCD: noncommunicable disease.

    cHAI: health care–associated infection.

    dEOD: environmental and occupational disease.

    eNot applicable.

    fSS: syndromic surveillance.

    From a system-type perspective, Saudi Arabia, Qatar, and Oman reported having fully electronic surveillance systems, whereas Bahrain, Kuwait, and the UAE reported having hybrid surveillance systems. Moreover, Bahrain, Kuwait, the UAE, Oman, and Qatar stored their surveillance data using on-premises servers, whereas Oman and Saudi Arabia used on-cloud data storage. Generally, surveillance systems in the GCC regions had syndromic surveillance embedded with the original systems; however, the State of Qatar was an exception, as their syndromic surveillance was separate from the original system.

    Surveillance Systems Evaluation

    Table 3 shows the frequency distribution of participants’ responses to each question in the questionnaire for all 5 Likert scale categories. Figure 1 shows the frequency of the participants' responses to each question in the questionnaire over the 3 collapsed categories: agree, neutral, and disagree. Only 2 countries used new cutting-edge technologies, resulting in a glaring mismatch among the countries in the use of machine learning, AI, and real-time analytics.

    Table 3. Responses to individual questions in the Centers for Disease Control and Prevention surveillance evaluation framework over all Likert scale categories (N=6).
    QuestionStrongly disagree, n (%)Disagree, n (%)Neutral, n (%)Agree, n (%)Strongly agree, n (%)
    The system provides estimates of morbidity related to the health-related event under surveillance1 (17)1 (17)0 (0)4 (67)0 (0)
    The system provides estimates of mortality related to the health-related event under surveillance0 (0)0 (0)0 (0)5 (83)1 (17)
    The system enables us to identify high-risk populations0 (0)0 (0)0 (0)4 (67)2 (33)
    The system enables us to plan the resources needed for prevention and control0 (0)0 (0)0 (0)3 (50)3 (50)
    The system enables us to update and develop national policy strategy0 (0)0 (0)1 (17)2 (33)3 (50)
    The system enables us to assess the impact of any prevention and control intervention0 (0)0 (0)0 (0)5 (83)1 (17)
    The system can generate actionable reports and insights immediately about health-related events under surveillance0 (0)4 (67)0 (0)0 (0)2 (33)
    The system can detect trends that signal changes in the occurrence of all health-related events under surveillance0 (0)0 (0)1 (17)3 (50)2 (33)
    The system can predict trends and outcomes using artificial intelligence and machine learning models1 (17)3 (50)0 (0)1 (17)1 (17)
    The system data stimulate research intended for prevention or control0 (0)0 (0)0 (0)3 (50)3 (50)
    Our surveillance system has clear case definitions to detect notifiable diseases0 (0)0 (0)0 (0)3 (50)3 (50)
    Our surveillance system uses multiple reporting sources, such as hospitals, laboratories, primary clinics, etc.0 (0)0 (0)2 (33)1 (17)3 (50)
    Our surveillance system is integrated with all health information systems in the country1 (17)2 (33)0 (0)1 (17)2 (33)
    We need to perform extensive data analysis to generate disease reports and insights0 (0)0 (0)3 (50)2 (33)1 (17)
    We have established methods and policies to disseminate surveillance results to relevant stakeholders0 (0)1 (17)1 (17)2 (33)2 (33)
    Our surveillance system allows us to respond quickly to a new demand, that is, diseases, risk factors etc.0 (0)0 (0)0 (0)3 (50)3 (50)
    Our surveillance system allows us to respond quickly for emerging or re-emerging pandemics0 (0)0 (0)0 (0)3 (50)3 (50)
    Our surveillance system can detect events other than notifiable diseases, such as food poisoning, chemical poisoning, environmental events, etc.0 (0)0 (0)1 (17)2 (33)3 (50)
    Our surveillance system has high completion rates0 (0)1 (17)2 (33)3 (50)0 (0)
    Our surveillance system has high laboratory reporting rates0 (0)1 (17)2 (33)3 (33)0 (0)
    Our surveillance system has high hospital or PHCa reporting rates0 (0)0 (0)0 (0)5 (83)1 (17)
    Our surveillance system can correctly detect a high proportion of disease cases0 (0)0 (0)0 (0)5 (83)1 (17)
    Our surveillance system can correctly detect new epidemics or outbreaks0 (0)0 (0)0 (0)5 (83)1 (17)
    The proportion of reported cases that are true cases in our surveillance system is high0 (0)0 (0)1 (17)3 (50)2 (33)
    The proportion of reported epidemics that are true epidemics in our surveillance system is high0 (0)0 (0)0 (0)4 (67)2 (33)
    Our surveillance system informs us about the characteristics of the population under surveillance, for example, age, socioeconomic status, geographic location0 (0)0 (0)0 (0)4 (67)2 (33)
    Our surveillance system enables us to measure the natural history of disease, for example, latency period, mode of transmission, and fatal outcomes0 (0)0 (0)4 (67)1 (17)1 (17)
    Our surveillance system captures clinical practice data, for example, sites performing diagnostic tests, and physician-referral patterns0 (0)0 (0)0 (0)5 (83)1 (17)
    Our surveillance system captures clinical outcomes of interest, for example, mortality data, hospitalizations, and laboratory data0 (0)0 (0)2 (33)2 (33)2 (33)
    The entered data in our surveillance system is accurate0 (0)0 (0)0 (0)1 (17)5 (83)
    Our surveillance system has standardized methods for data entry0 (0)0 (0)0 (0)3 (50)3 (50)
    Our surveillance system can measure what it intends to measure0 (0)0 (0)0 (0)4 (67)2 (33)
    Our surveillance system has a high number of spontaneous outages and down times1 (17)3 (50)1 (17)1 (17)0 (0)
    We spend a lot of our resources on technical repairs to maintain system functionality1 (17)3 (50)0 (0)2 (33)0 (0)
    Our surveillance system requires a long time for gathering and receiving data1 (17)3 (50)0 (0)2 (33)0 (0)
    Our surveillance system requires a long time for managing data, such as transfer, entry, modifying, storage, and back-up of data2 (33)2 (33)0 (0)1 (17)1 (17)
    Our surveillance system is able to generate data for immediate disease control0 (0)0 (0)0 (0)4 (67)2 (33)
    Our surveillance system is able to generate data for long-term program planning0 (0)0 (0)0 (0)4 (67)2 (33)

    aPHC: primary health care center.

    ‎
    Figure 1. Responses to individual questions in the Centers for Disease Control and Prevention (CDC) surveillance evaluation framework over selected Likert scale categories (agree, neutral, and disagree). PHC: primary health care center.

    Table 4 illustrates the means of global and attribute scores of the GCC surveillance systems. Overall, the mean global score was 147 (SD 13.27). The means for usefulness, simplicity, flexibility, acceptability, sensitivity, PVP, representativeness, data quality, stability, and timeliness were 39, 19.3, 13.3, 10.8, 8.3, 8.5, 16, 13, 9.8, and 8.6, respectively.

    Table 4. Descriptive statistics of the Gulf Cooperation Council surveillance systems for global and attribute scores (N=6).
    System attributesValues, mean (SD)Values, median (IQR; range)
    Global score147 (13.2)144 (25.75; 133-167)
    Level of usefulness39 (5.1)36.5 (10; 34-46)
    Simplicity19.3 (4.3)18 (9; 15-25)
    Flexibility13.3 (1.3)13 (3; 12-15)
    Acceptability10.8 (1.7)11 (3; 9-13)
    Sensitivity8.3 (0.81)8 (1; 8-10)
    Predictive value positive8.5 (1.2)8 (2; 7-10)
    Representativeness16 (2.4)15 (5;14-20)
    Data quality13 (1.2)12.5 (2; 12-15)
    Stability9.8 (3.1)10 (5; 5-14)
    Timeliness8.6 (1)8 (2; 8-10)

    Table 5 depicts the scores of the GCC Surveillance Systems’ Global and Attribute Scores by country. The UAE scored the highest among the GCC countries in terms of the global score, with a score of 167 (83.5%, 95% CI 77.7%-88.0%). On the attribute level, Oman scored the highest on usefulness, simplicity, and flexibility, scoring 46 (92%, 95% CI 81.1%-96.8%), 25 (71.4%, 95% CI 54.9%-83.6%), and 15 (100%, 95% CI 79.6%-100%), respectively. However, the UAE scored the highest on acceptability, sensitivity, PVP, representativeness, data quality, and timeliness, whereas Qatar scored the highest on stability, obtaining a score of 10 (50%, 95% CI 28.8%-71.1%).

    Table 5. Scores of Gulf Cooperation Council surveillance systems global and attribute scores by country (N=6).
    AttributesSaudi ArabiaKuwaitUnited Arab
    Emirates    		OmanQatarBahrain

    		ScoreProportion (95% CI)ScoreProportion (95% CI)ScoreProportion (95% CI)ScoreProportion (95% CI)ScoreProportion (95% CI)ScoreProportion (95% CI)
    Global score14572.5 (65.9-78.2)13366.5 (59.7-72.6)16783.5 (77.7-88.0)15879 (72.8-84.0)14371.5 (64.8-77.3)13567.5 (60.7-73.6)
    Usefulness3774 (60.4-84.1)3672.0 (58.3-82.5)4590 (78.6-95.6)4692 (81.1-96.8)3672 (58.3-82.5)3468 (54.1-79.2)
    Simplicity1542.8 (27.9-59.1)1645.7 (30.4-61.8)2468.5 (52.0-81.4)2571.4 (54.9-83.6)2057.1 (40.8-72)1645.7 (30.4-61.8)
    Flexibility1386.6 (62.1-96.2)1280.0 (54.8-92.9)15100 (79.6-100)15100 (79.6-100)1386.6 (62.1-96.2)1280.0 (54.8-92.9)
    Acceptability1280.0 (54.8-92.9)960.0 (35.7-80.1)1386.6 (62.1-96.2)1280.0 (54.8-92.9)1066.7 (41.7-84.8)960.0 (35.7-80.1)
    Sensitivity880.0 (49.0-94.3)880.0 (49.0-94.3)10100 (72.26-100)880.0 (49.0-94.3)880.0 (49.0-94.3)880.0 (49.0-94.3)
    Predictive value positive880.0 (49.0-94.3)880.0 (49.0-94.3)10100 (72.26-100)770.0 (39.6-89.2)880.0 (49.0-94.3)880.0 (49.0-94.3)
    Representative1890.0 (69.9-97.2)1470.0 (48.1-85.4)20100 (83.8-100.0)1575.0 (52.9-90.2)1575.0 (52.9-90.2)1470.0 (48.1-86.8)
    Data quality1493.3 (71.2-99.6)1280.0 (54.6-94.6)15100.0 (81.8-100.0)1280.0 (54.6-94.6)1386.6 (62.4-97.7)1280.0 (54.6-94.6)
    Stability1260.0 (37.8-79.4)1050.0 (28.8-71.1)525.0 (9.7-47.0)840.0 (20.6-62.1)1050.0 (28.8-71.1)1470.0 (48.1-85.4)
    Timeliness880.0 (49.0-94.3)880.0 (49.0-94.3)10100 (72.26-100)10100 (72.26-100)880.0 (49.0-94.3)880.0 (49.0-94.3)

    Table 6 shows the correlation coefficients (r) between the global score and each of the attribute scores of the GCC surveillance systems. Generally, significant strong positive correlations were obtained between the global score and the following attributes: level of usefulness (r=0.841; P=.036), flexibility (r=0.956; P=.003), acceptability (r=0.971; P=.001), representativeness (r=0.883, P=.020), and timeliness (r=0.828; P=.042). Strong and positive correlations were obtained between the level of usefulness and flexibility (r=0.910; P=.012), acceptability (r=0.851; P=.032), and timeliness (r=0.840; P=.036), which were statistically significant. Moreover, strong correlations were found between flexibility and acceptability (r=0.923; P=.009) and between acceptability and representativeness (r=0.955; P=.003). The correlation between stability and timeliness scores was negative, strong, and statistically significant (r=−0.840; P=.036).

    Table 6. Interitem correlation matrix between global and attribute scores of Gulf Cooperation Council surveillance systems.
    VariableGlobalUSEaSIMbFLEcACCdSENePVPfREPgDQhSTBiTIMj
    Global

    		r1.000—k—————————

    		P value———————————
    USE

    		r0.841l1.000—————————

    		P value.036

    		——————————
    SIM

    		r0.5800.6031.000————————

    		P value .228 .205 —————————
    FLE

    		r0.956m0.910l0.7281.000———————

    		P value.003

    		.012

    		 .101 ————————
    ACC

    		r0.971m0.851l0.4480.923m1.000——————

    		P value.001

    		.032

    		.009

    		 .009 ———————
    SEN

    		r0.6550.3990.3990.5480.6741.000—————

    		P value .158 .434 .434 .261 .142 ——————
    PVP

    		r0.123−0.2350.0000.0650.1910.5661.000————

    		P value .816 .654 1.000 .903 .717 .242 —————
    REP

    		r0.883l0.6870.2390.8000.955m0.6740.3971.000———

    		P value.020

    		 .132 .649 .056 .003 .142 .435 ————
    DQ

    		r0.6380.339−0.0310.5080.7500.6960.6890.907l1.000——

    		P value .173 .511 .954 .304 .066 .125 .130 .013———
    STB

    		r−0.667−0.765−0.809−0.788−0.672−0.664−0.204−0.537−0.3391.000—

    		P value .148 .077 .051 .063 .144 .150 .699 .272 .511 ——
    TIM

    		r0.828l0.840l0.840l0.866l0.7460.632−0.1120.5330.220−0.840l1.000

    		P value.042.036.036.026 .088 .178 .833 .036 .675 .036 —

    aUSE: level of usefulness.

    bSIM: simplicity.

    cFLE: flexibility.

    dACC: acceptability.

    eSEN: sensitivity.

    fPVP: predictive value positive.

    gREP: representativeness.

    hDQ: data quality.

    iSTB: stability.

    jTIM: timeliness.

    kNot applicable.

    lCorrelation is significant at .05 (2-tailed).

    mCorrelation is significant at .01 (2-tailed).

    Figure 2 depicts the linear regression analyses between the global score and each of the attribute scores of the GCC surveillance systems. The trends indicate that the global score increases proportionately with the increase in almost all attribute scores. The linearity of the relation is the most apparent with the level of usefulness, flexibility, and acceptability scores.

    ‎
    Figure 2. Linear regression between global and each of the attribute scores of Gulf Cooperation Council surveillance systems.

    Table 7 shows the univariate linear regression model of the association between global scores and the selected characteristics of the GCC public health surveillance system. Surveillance disease coverage was found to be the major significant predictor of GCC surveillance global scores (β=.93; P=.006). Country, system type, data storage type, and placement of syndromic surveillance were found to be nonsignificant predictors of the global score (P>.05). Figure 3 depicts the pattern of associations between the global scores and the selected characteristics of the GCC public health surveillance system.

    Table 7. Univariate linear regression modeling of the association between the global score and selected Gulf Cooperation Council public health surveillance characteristics.
    PredictorRegression parameters

    		Unstandardized β (SE)Standardized βP value
    Country5.06 (2.49).71.11
    Disease coverage9.00 (1.71).93.006a
    System type−1.83 (5.98)−.15.78
    Storage type−4.25 (12.67)−.165.75
    Placement of syndromic surveillance−14.2 (14.6)−.43.38

    aP<.05.

    ‎
    Figure 3. Univariate linear regression modeling of the association between the global score and selected Gulf Cooperation Council public health surveillance characteristics. SS: syndromic surveillance.

    Principal Findings

    This study used CDC’s public health surveillance system evaluation guidelines to evaluate the performance of public health surveillance systems in GCC countries regarding their level of usefulness, simplicity, flexibility, acceptability, sensitivity, PVP, representativeness, data quality, stability, and timeliness. It compared the GCC public health surveillance systems’ characteristics with the global score and the 10 system attributes obtained from the CDC evaluation guidelines. In addition, the correlations between the global score and the 10 system attributes were examined. Univariate regression analysis showed that disease coverage is the major predictor of GCC surveillance global scores. This is the first evaluation study with this level of quantitative rigor and geographic magnitude for surveillance systems in the region.

    Comparison With Prior Work

    The results of the study demonstrated that the surveillance systems in all GCC countries could detect and monitor CDs and that most of them covered HAIs, especially in Bahrain, the UAE, Saudi Arabia, and Oman. These substantial efforts may be attributed to the adoption of infectious disease surveillance following the World Health Organization’s implementation of the International Health Regulations in 2005. The International Health Regulations mandated that all member states establish the infrastructure needed to detect, monitor, and report CDs to World Health Organization regional offices [12]. Notably, GCC countries do not use surveillance systems to collect administrative data on absenteeism during certain epidemics. This finding contradicts other studies that quantified workplace absenteeism during influenza outbreaks by monitoring sick leave data [13].

    However, it must be noted that our data illustrated substantial deficits in the coverage of NCDs across GCC states, with the exception of the UAE and Bahrain. Although NCD monitoring also exists in other GCC countries [14-16], these monitoring mechanisms were siloed and operated independently from the original surveillance systems. As NCDs impose a high economic and social burden on GCC countries [17], incorporating NCDs within the original surveillance systems can potentially consolidate surveillance efforts and aid in prioritizing NCD prevention and control to tackle behavioral and system-level risk factors [18]. Several states in the United States have conducted pilot studies on public health surveillance of chronic diseases and risk factors [19]. Most studies provide a crucial proof of concept; however, obstacles to defining the algorithms and harmonizing them across jurisdictions and countries are still present.

    This study demonstrated that the UAE has the highest global score and the highest score in most system attributes, with the widest coverage of diseases among GCC countries. The advancement of the UAE’s health information system is a result of the government’s commitment to deliver the latest digital health infrastructure and solutions to enable proper system monitoring and response [20]. Establishing the region’s first health information exchange within the Emirate of Abu Dhabi is another testament to the country’s commitment [21]. Notably, almost all health systems in the GCC region are undergoing massive system-level reforms, in which overhauls of health information technology, including surveillance systems, constitute a major part [22,23].

    Oman’s surveillance system scored the highest in terms of usefulness, simplicity, and flexibility. In 2007, the Ministry of Health in the Sultanate of Oman launched a single, multilayered, and interoperable surveillance platform called the National Electronic Public Health Surveillance System [24]. Repositioning the system to be the single source of reliable data may have contributed toward achieving such high levels of system utility and flexibility. The Saudi Arabian public health surveillance system is undergoing a considerable internal transition, which might account for its suboptimal performance, despite being the largest geographically in the GCC area. This observation is supported by previous studies that extracted data from the upgraded surveillance system in Saudi Arabia [25].

    The COVID-19 pandemic, which led to the rapid development of innovative data sets, surveillance technologies, and models, has led to an increased interest in using AI strategies to accomplish public health outcomes [26]. The evidence obtained from this study revealed that only 2 countries were using emerging technologies such as machine learning, AI, and real-time analytics. GCC countries have an immense opportunity to use AI in the rapid identification of populations, interventions, and outcomes of interest for disease surveillance, disease prevention, and health promotion. A Twitter-based influenza detection study, which used a text-mining technique, found that such techniques improved the estimation performance of influenza detection during the peak periods of the disease [27]. Such examples can be leveraged in GCC countries, because we found no explicit evidence of the use of AI approaches in public health surveillance.

    In addition to the scientific merit of this study, the findings will aid surveillance administrators and managers in GCC to improve their global score. This is manifested by strong and positive correlations between a country’s global scores and acceptability, flexibility, representativeness, usefulness, and timeliness. Prioritizing these attributes will result in major improvements in global scores and in GCC public health surveillance systems in general. A negative and strong correlation was found between the timeliness and stability of surveillance systems in the GCC, which warrants architects of surveillance systems to design stable and reliable surveillance systems without jeopardizing timeliness and efficiency.

    Limitations

    This study was limited by a small sample size, where one expert in the public health surveillance system was purposively assigned to each GCC county to offer the best information regarding the features and functions of the system. However, the use of a relatively adequate sample size and high response rate enabled by the purposive sampling approach strengthened the study’s ability to identify considerable differences. Self-reported questionnaires, however, have the potential to introduce bias, especially with regard to delicate issues such as the usefulness and quality of the system’s data. To better understand the research question and guarantee that the conclusions are grounded in participants’ views and experiences, we suggest conducting a mixed methods evaluation approach, including qualitative research, to evaluate GCC surveillance systems.

    Conclusions

    The results of this study demonstrate that GCC surveillance systems appear to be useful and perform sustainably across all countries. These systems have the potential to improve data quality and representativeness. Potential lessons should be leveraged from countries with high scores, especially in using emerging technologies such as machine learning, AI, and real-time analytics. Future research should focus on more comprehensive, mixed methods evaluation approaches for surveillance systems in the GCC. The findings of this study have wider implications for the need for a coordinated GCC surveillance action, which includes capacity building, centralized information exchange, and system architecture redesign to maintain the GCC surveillance systems’ sustainability and responsiveness during upcoming public health challenges.

    Acknowledgments

    This work was funded by the Gulf Center of Disease Prevention and Control; Gulf Health Council; and the executive arm of the Council of Ministers of Health at Gulf Cooperation Council countries, Riyadh, Saudi Arabia.

    Authors' Contributions

    NA, SA, and PP were responsible for designing the study. NA, SA, YA-F and AA were responsible for questionnaire development. SA, PP, and AAW were involved in the acquisition of data. NA, YA-F, SA, and AAW contributed to data analysis, interpretation, and manuscript write-up.

    Conflicts of Interest

    None declared.

    1. German R, Lee L, Horan J, Milstein RC, Pertowski A, Waller MN, Guidelines Working Group Centers for Disease Control and Prevention (CDC). Updated guidelines for evaluating public health surveillance systems: recommendations from the Guidelines Working Group. MMWR Recomm Rep 2001 Jul 27;50(RR-13):1-CE7 [FREE Full text] [Medline]
    2. Buehler JW, Hopkins RS, Overhage JM, Sosin DM, Tong V, CDC Working Group. Framework for evaluating public health surveillance systems for early detection of outbreaks: recommendations from the CDC Working Group. MMWR Recomm Rep 2004 May 07;53(RR-5):1-11. [Medline]
    3. Approval of establishing the Gulf Center for Disease Prevention and Control. Gulf Health Council. 2021.   URL: https://tinyurl.com/2mwsj4kt [accessed 2022-06-20]
    4. Saad RK, Al Nsour M, Khader Y, Al Gunaid M. Public health surveillance systems in the eastern Mediterranean region: bibliometric analysis of scientific literature. JMIR Public Health Surveill 2021 Nov 01;7(11):e32639 [FREE Full text] [CrossRef] [Medline]
    5. Althumiri NA, Basyouni MH, BinDhim NF. Exploring the association between physical activity and risk of mental health disorders in Saudi Arabian adults: cross-sectional study. JMIR Public Health Surveill 2021 Apr 14;7(4):e25438 [FREE Full text] [CrossRef] [Medline]
    6. Khader Y, Alyahya M, El-Khatib Z, Batieha A, Al-Sheyab N, Shattnawi K. The Jordan stillbirth and neonatal mortality surveillance (JSANDS) system: evaluation study. J Med Internet Res 2021 Jul 21;23(7):e29143 [FREE Full text] [CrossRef] [Medline]
    7. Al-Jamrah KM, Al Nabehi BA, Almoayed KA, Anam LS, Khader YS. Performance of the Neonatal Tetanus Surveillance System (NTSS) in Sana'a, Yemen: evaluation study. JMIR Public Health Surveill 2021 May 04;7(5):e27606 [FREE Full text] [CrossRef] [Medline]
    8. Mohamed AA, Chehab MA, Al-Dahshan A, Al-Romaihi HE, Farag EA. An evaluation of the National Brucellosis Surveillance System in Qatar, 2018. Cureus 2019 Mar 03;11(3):e4169 [FREE Full text] [CrossRef] [Medline]
    9. Alshehri MH, Alsabaani AA, Alghamdi AH, Alshehri RA. Evaluation of communicable disease surveillance system at primary health care centers in Jeddah, Saudi Arabia. Cureus 2021 Nov;13(11):e19798 [FREE Full text] [CrossRef] [Medline]
    10. Campbell S, Greenwood M, Prior S, Shearer T, Walkem K, Young S, et al. Purposive sampling: complex or simple? Research case examples. J Res Nurs 2020 Dec;25(8):652-661 [FREE Full text] [CrossRef] [Medline]
    11. Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, et al. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol 2016 Apr;31(4):337-350 [FREE Full text] [CrossRef] [Medline]
    12. Kamradt-Scott A. The international health regulations (2005). Int Organ Law Rev 2019 Dec 16;16(2):242-271. [CrossRef]
    13. Duchemin T, Bastard J, Ante-Testard PA, Assab R, Daouda OS, Duval A, et al. Monitoring sick leave data for early detection of influenza outbreaks. BMC Infect Dis 2021 Jan 11;21(1):52 [FREE Full text] [CrossRef] [Medline]
    14. Akhtar N, Salam A, Kamran S, Bourke P, Joseph S, Santos M, et al. Ethnic variation in acute cerebrovascular disease: analysis from the Qatar stroke registry. Eur Stroke J 2016 Sep;1(3):231-241 [FREE Full text] [CrossRef] [Medline]
    15. Bourusly MJ, Burahma MH, Khalifa N, Motti H, Kaleefa S, Adil M, et al. Trends in childhood cancer in Kuwait: data from the 2004-2017 registry. Cureus 2021 Feb 13;13(2):e13333 [FREE Full text] [CrossRef] [Medline]
    16. Tourkmani AM, Alharbi TJ, Bin Rsheed A, Alotaibi YK. Utilizing diabetes registry for exploring sociodemographic and clinical characteristics of type II diabetic patients in Saudi Arabia. Saudi Med J 2021 Jan;42(1):56-65 [FREE Full text] [CrossRef] [Medline]
    17. Elmusharaf K, Grafton D, Jung JS, Roberts E, Al-Farsi Y, Al Nooh AA, et al. The case for investing in the prevention and control of non-communicable diseases in the six countries of the Gulf Cooperation Council: an economic evaluation. BMJ Glob Health 2022 Jun;7(6):e008670 [FREE Full text] [CrossRef] [Medline]
    18. Finkelstein EA, Malkin JD, Baid D, Alqunaibet A, Mahdi K, Al-Thani MB, et al. The impact of seven major noncommunicable diseases on direct medical costs, absenteeism, and presenteeism in Gulf Cooperation Council countries. J Med Econ 2021;24(1):828-834 [FREE Full text] [CrossRef] [Medline]
    19. Thiébaut R, Cossin S, Section Editors for the IMIA Yearbook Section on Public Health and Epidemiology Informatics. Artificial intelligence for surveillance in public health. Yearb Med Inform 2019 Aug;28(1):232-234 [FREE Full text] [CrossRef] [Medline]
    20. Koornneef E, Robben P, Blair I. Progress and outcomes of health systems reform in the United Arab Emirates: a systematic review. BMC Health Serv Res 2017 Sep 20;17(1):672 [FREE Full text] [CrossRef] [Medline]
    21. DOH policy on the Abu Dhabi health information exchange (HIE). Department of health, Abu Dhabi. 2022 Jun 22.   URL: https://www.doh.gov.ae/-/media/A78104416DF2440E89AC90FB75F11055.ashx [accessed 2022-06-22]
    22. Health sector transformation program. Saudi Vision 2030. 2022.   URL: https://www.vision2030.gov.sa/v2030/vrps/hstp/ [accessed 2022-06-22]
    23. National health strategy 2018-2022. Ministry of Public Health, Qatar. 2018.   URL: https://andp.unescwa.org/sites/default/files/2020-10/National%20Health%20Strategy%202018-2022.pdf [accessed 2022-06-22]
    24. Launching of the e-surveillance. Public Health Bulletin. 2017.   URL: https:/​/www.​moh.gov.om/​documents/​236878/​0/​Public+Health+Bulletin+%232/​0b20635e-e197-4897-ac00-24644ce6b3e7 [accessed 2022-06-23]
    25. Alothaid H, Alshehri MA, Yusuf AO, Alzahrani ME, McDaniel J, Alamri S, et al. Sociodemographic predictors of confirmed COVID-19 mortality and hospitalization among patients in Saudi Arabia: analyzing a national COVID-19 database. J Infect Public Health 2022 May 04;15(6):615-620 [FREE Full text] [CrossRef] [Medline]
    26. Baclic O, Tunis M, Young K, Doan C, Swerdfeger H, Schonfeld J. Challenges and opportunities for public health made possible by advances in natural language processing. Can Commun Dis Rep 2020 Jun 04;46(6):161-168 [FREE Full text] [CrossRef] [Medline]
    27. Wakamiya S, Kawai Y, Aramaki E. Twitter-based influenza detection after flu peak via tweets with indirect information: text mining study. JMIR Public Health Surveill 2018 Sep 25;4(3):e65 [FREE Full text] [CrossRef] [Medline]

    ‎
    AI: artificial intelligence
    CD: communicable disease
    CDC: Centers for Disease Control and Prevention
    GCC: Gulf Cooperation Council
    Gulf CDC: Gulf Center of Disease Prevention and Control
    HAI: health care–associated infection
    NCD: noncommunicable disease
    PVP: predictive value positive
    UAE: United Arab Emirates

    Edited by A Mavragani; submitted 20.07.22; peer-reviewed by C Tziraki, MM Patwary; comments to author 12.08.22; revised version received 17.11.22; accepted 22.12.22; published 05.04.23

    Copyright

    ©Nawaf Albali, Sami Almudarra, Yahya Al-Farsi, Abdullah Alarifi, Adil Al Wahaibi, Pasi Penttinen. Originally published in JMIR Formative Research (https://formative.jmir.org), 05.04.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.