This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license information must be included.
Population surveillance sites generate many datasets relevant to disease surveillance. However, there is a risk that these data are underutilized because of the volumes of data gathered and the lack of means to quickly disseminate analysis. Data visualization offers a means to quickly disseminate, understand, and interpret datasets, facilitating evidence-driven decision making through increased access to information.
This paper describes the development and evaluation of a framework for data dashboard design, to visualize datasets produced at a demographic health surveillance site. The aim of this research was to produce a comprehensive, reusable, and scalable dashboard design framework to fit the unique requirements of the context.
The framework was developed and implemented at a demographic surveillance platform at the Africa Health Research Institute, in KwaZulu-Natal, South Africa. This context represents an exemplar implementation for the use of data dashboards within a population health-monitoring setting. Before the full launch, an evaluation study was undertaken to assess the effectiveness of the dashboard framework as a data communication and decision-making tool. The evaluation included a quantitative task evaluation to assess usability and a qualitative questionnaire exploring the attitudes to the use of dashboards.
The evaluation participants were drawn from a diverse group of users working at the site (n=20), comprising of community members, nurses, scientific and operational staff. Evaluation demonstrated high usability for the dashboard across user groups, with scientific and operational staff having minimal issues in completing tasks. There were notable differences in the efficiency of task completion among user groups, indicating varying familiarity with data visualization. The majority of users felt that the dashboards provided a clear understanding of the datasets presented and had a positive attitude to their increased use.
Overall, this exploratory study indicates the viability of the data dashboard framework in communicating data trends within population surveillance setting. The usability differences among the user groups discovered during the evaluation demonstrate the need for the user-led design of dashboards in this context, addressing heterogeneous computer and visualization literacy present among the diverse potential users present in such settings. The questionnaire highlighted the enthusiasm for increased access to datasets from all stakeholders highlighting the potential of dashboards in this context.
Demographic surveillance, the process of monitoring births, deaths, causes of deaths, and migration in a population over time is 1 of the cornerstones of public health research [
Information visualization offers a means to disseminate information promptly and increase its visibility among stakeholders. Card et al [
La Valle [
Dashboards typically comprise a combination of different visualization methods; the commonality is that they draw from multiple data sources to facilitate timely understanding [
Although there is little evidence in the literature of dashboard implementations at HDSS sites, there is a variety of examples relating more generally to public health monitoring. Cheng [
This paper presents the development and evaluation of a design framework for the implementation of a data dashboard within an HDSS context. The paper proceeds by outlining the structure and design rationale behind the framework, with a focus on its implementation within a specific HDSS context. Following this, an evaluation is described, which outlines the implementation of its use within a specific trial context, revealing promising findings concerning task completion, usability, and sentiment. The contributions of this study are guidelines for designing visualization systems for HDSS sites and identifying their information needs and the development of a framework to reflect the needs of the diverse users. The research identifies the challenges of different users in dashboard design and offers suggestions on how to account for this in future systems.
The development of a data exploration dashboard framework requires a range of implementation decisions. These factors involve elements of data design, user interaction, and the study setting of the dashboard design, all of which contribute to the potential success of the dashboard. The outline framework described below relates to the design and implementation of a dashboard within a population health–monitoring setting. In developing a dashboard in this context, there are 5 core considerations:
Study setting (description of the development context of the framework) informs how interaction with the dashboard takes place, informs about information requirements, and informs about privacy and confidentiality concerns.
Dashboard purpose and concept comprise specification of the purpose of the dashboard, including the context, target user group, and the expected objectives of users of the dashboard.
User Interaction and flow specify the intended process of user interaction with the dashboard interface, including the expected user
Data selection and visualization design cover the design and construction of the data visualization, including how the key attributes are selected, their location within the dashboard, and their extraction from the underlying database.
Framework architecture covers technical aspects of dashboard framework, including input data structures and interfaces.
In the following section, the execution and design considerations involved in the development of a dashboard framework for HDSS sites are described.
The setting of this study is the Africa Health Research Institute (AHRI) in KwaZulu-Natal, South Africa (SA). AHRI’s field site operates as an HDSS, generating high-quality longitudinal datasets to capture the demographic and health changes brought about by the HIV epidemic and evaluate interventions to mitigate their impact [
Within this setting, the dashboards would be displayed on 3 large touch screens within the research center. Placing of the dashboards in public areas hoped to encourage collaborative use, visibility, and discussion of datasets, and it could provide a means to explain the work of the institute field site to visitors.
The purpose of the AHRI data dashboard was to provide users with the opportunity to access information about studies at the site and provide a building block for the development of a framework, increasing the visibility and access to the datasets produced at the sites. The dashboard would give a clear and concise overview of the information collected in an understandable and accessible format while ensuring the anonymity of the participants. A diverse range of users would use the dashboards, yet a dashboard cannot be created to fit every user persona. The aim was to design a dashboard, which anyone working at a site could feel comfortable using to explore core datasets in the context of a specific study, for example, community members, field workers, medical staff and resident scientists, and visiting researchers. This design would enable users to either review overall progress or to drill down into analysis depending on their needs.
In achieving these objectives, the dashboard interface would be required to meet the following design criteria:
Provide a monitor of key performance indicators (KPIs)
Enable spatial interrogation of datasets in relation to KPIs to identify trends
Allow for switching from a global to local view in relation to these indicators
Drill down to the local regions of the map to explore datasets in greater detail
On the basis of these needs, we developed a 2-stage dashboard comprising an overview page and analysis page. The overview page would provide a global picture of the study in question and would feature a display of the key indicators relating to the progress of the trial and an interactive map to display these indicators to allow rapid identification of trends. Interaction with the overview page would enable users to filter data by indicator and act as a jumping-off point to the analysis page. The analysis page would focus on a selected subdivision within the study site, allowing users to compare the performance of a region to the global area and augment their analysis with the typical datasets produced at an HDSS.
The global performance of indicators is the starting point for the users’ interaction with a dashboard, a set of anchors to guide their exploration of the datasets displayed and to compare performance at the global and local level. The design of the dashboard was based on Shneiderman’s [
A successful dashboard provides an overall picture of the datasets represented with a key objective in mind. In this context, there are a variety of indicators that can reflect the health of a population, for example, child mortality, life expectancy [
Framing data exploration through indicators allows the user to keep the global in mind while exploring details. The dashboard’s exploratory hierarchy of datasets is the global view of KPIs than viewing the KPIs at a local level. This flow allows the user to view the global and the local together and make comparisons between both scales, gaining insight into relative local performance and identifying emerging trends quickly and clearly (see
The overview page was designed to fulfill the function of a traditional dashboard by providing the user with access to the most salient information relating to the subject matter while encouraging a detailed exploration of the datasets. The design intention was to provide the user with an overview of the progress of the study to the KPIs, to review a spatial representation of indicator performance, and provide the means to compare the global performance with the local. Once a region had been selected, the user could move to the analysis page, which would focus on local performance with the aid of explanatory variables and the use of comparative visualizations.
Summary of selected key performance indicators and data types.
Indicator | Type | Description |
Trial participants | Number | The number of individuals enrolled in the Treatment as Prevention trial |
HIV prevalence rate | Ratio | The percentage of individuals known to be HIV positive |
HIV-positive individuals who know their status/individuals known to be positive | Ratio | The percentage of HIV-positive individuals who know their status/the number of individuals known to be positive |
HIV-positive individuals on antiretroviral treatment ARTa/diagnosed individuals | Ratio | The percentage of HIV-positive individuals on ART |
Individuals who are virally suppressed/individuals on ART | Ratio | The percentage of individuals who are virally suppressed |
aART: antiretroviral treatment.
User interaction flow between global and local. This image shows the interaction of key performance indicators (KPIs) at the global and local scales, allowing the user to explore the broad spatial trends in the indicator at the global scale, before drilling down into the finer scale spatial variation and associations at the local.
Following Shneiderman’s visual information seeking mantra [
The overview pages comprise 3 areas of information display (see
The focus of the overview page was the map display (indicated with point 1 in
The graduated color map is used to identify areas of interest and detect spatial autocorrelation within the trial region. The use of a binning technique also related to practical concerns to create regions to capture local performance and acted as a point of interaction to trigger events on the dashboard. When a hexagon is selected, information relating to indicator performance and the number of trial participants residing in that region is displayed. A button on the information window allows the user to view this region in greater detail on the analysis page. Interacting with a map hexagon also triggers the display of KPIs for the selected region in the page footer. The global performance of indicators is displayed in the left-hand sidebar (point 3 in
The page footer (point 3 in
The analysis page provides a detailed view of the performance of indicators at a local scale, the
The analysis page comprised 4 areas of information (
Explanatory variables
Global and local comparison
Chart selection, mini map, and global information display
Key numbers relating to a region
Landing page of the dashboard. This image shows the interactive front page of the dashboard, displaying the global view of 1 indicator within the map (marked 1), measures of key performance indicators (KPIs) at the selected local region at the bottom of the screen (marked 2), and global KPI measures on the left (marked 3). Data are updated in real-time. The user is invited to manipulate the map and change selected regions through click or touch. The Breakdown Page button leads to a more detailed exploration of KPI measures and related factors within the Analysis page.
Points of comparison. This image highlights the points of interaction between global and local indicators, showing the points where a user is able to directly compare local and global measures of key performance indicators.
Analysis page. This figure shows the analysis page for a local region selected through the global map page. The page introduces a range of potential explanatory attributes that can be measured against key performance indicators to help develop hypotheses for future studies. Users are able to add and remove visualizations through interactive functionality.
The focus of the analysis page is the explanatory variables section (point 1 in
A population pyramid
A normalized stacked bar chart displaying HIV rates by age group
A treemap displaying education levels
A treemap displaying relationship status
A treemap displaying employment levels
Treemaps were chosen over bar charts as users preferred them, and they allowed for a more precise display of labels relating to data. The user can add the charts display population data in relation to each indicator to explore the differences among participants at different stages of the HIV treatment cascade, allowing the user to compare different outcomes directly. We used multiple visualization methods to exploit their differing strengths to enhance the users’ data exploration and engagement. Additional chart types alongside those listed above were pie, bar charts, and scatter plots.
The left-hand sidebar (point 2 in
The data dashboard was designed to inform the development of a framework. The framework was designed as a modular system to allow for the creation of dashboards by the research data management team at the site. The framework aimed to allow the development of dashboards as new datasets are produced. The aim was to develop new chart options as needed. This modularization comprised 3 components and the retention of these components for each new study.
Standard inputs for the dashboard frontend
Standard architecture for the inclusion of new datasets into the dashboard
Standard dataset formats for the generation of data visualizations
All dashboards run from a Web server within AHRI’s database architecture. Each dashboard is contained within a folder on the Web server, and adding a new folder to the dashboard Web server generates a new dashboard. The dashboards are displayed through a Web browser, and dashboards can be accessed through the dashboard homepage files on the Web server and can be edited by accessing the dashboard folder on the shared drive. The dashboard works as a typical website, and each folder contains index.html, analysis.html, and folders for Cascading Style Sheets (CSS), JavaScript (JS), and datasets. Data managers have access to the folders to edit the dashboard setup for new iterations and troubleshoot errors when they occur. The research data management team was trained on the system through training workshops.
Ethics approval was granted for this study by the University of KwaZulu-Natal Biomedical Research Ethics Committee, reference BE497/16. Informed consent to participate in the study was obtained from all participants.
An evaluation of the dashboard framework design was undertaken using the TasP trial dashboard implementation. This data dashboard was evaluated by assessing its usability with 20 users. The site had 100 workers, and the aim was to engage a broad range of potential users. Participants were selected from people working at AHRI, representative of different groups who would use dashboards to gain insight from AHRI datasets. These groups chosen were as follows.
Scientific staff: primarily medics, health care specialists, and researchers. Staff that would use dashboards to explore data from a research perspective, easing their access to data.
Operational staff: those ensuring continued operation of AHRI, focusing on data collection and quality. Staff that would use the dashboards to monitor the progress of data collection in studies.
Nursing staff: those involved in providing health services in the area. Nurses would use the dashboard to monitor how the data they collect are used and keep up to date with the studies they were involved.
Community Advisory Board (CAB) members: The CAB comprises members of the local community who are consulted and who advise researchers on the implementations of studies. The dashboards allow the CAB to explore the data collected about the community, supporting the group role in improving public understanding and maintaining accountability.
In total, 5 members of each group were randomly selected.
To assess the usability of the dashboards, the users performed a benchmark task evaluation comprising 5 tasks (each incorporating 2 subtasks) of increasing difficulty, requiring the user to extract information using the dashboards. During benchmark task evaluations, participants use visualizations to perform tasks to extract information, measuring targeted metrics [
Alongside benchmarking, users were also asked to self-evaluate their knowledge of the TasP trial and their level of computer literacy. Participants were also asked to complete a questionnaire using a Likert scale to assess their attitude toward the dashboard, covering the design of the interface, information presentation, and attitudes toward the future use of dashboards within AHRI. The questionnaire is provided in
The results of the evaluation can be broken down into 3 parts: self-evaluation, task evaluation, and the questionnaire.
The self-evaluation asked the users to give their level of computer literacy, awareness of the TasP trial, and the dashboard project (see
The results of the task component of the evaluation are provided in
The issue with the final task, which involved comparing data from multiple sources, may highlight a design weakness where data must be compared across separate charts rather than as different series within a single chart. Within these results, there was a great deal of variation within the subgroup performance, with the scientific and operational staff performing better in task completion than the CAB and nursing groups.
Overall the time taken to complete tasks increased with the increasing complexity of each task. On an average, users spent more time on tasks where their answer was incorrect. The scientific and operational staff were generally quicker to answer questions, and more often correct, than the CAB and nursing staff. Through one-way analysis of variance, a significant difference in time taken to complete tasks was found between scientific and nursing staff but not among other groups (
The questionnaire comprised 11 questions divided to cover 4 topic areas, the insight provided by the dashboard, the dashboard interface, the future use of the dashboards at the AHRI, and questions allowing users to comment on the dashboards and how they could be improved to provide greater clarity of information. The first 8 questions employed a Likert scale, the last 3 allowed participants to comment on their experience of the dashboard and their thoughts on their future use.
Responses to self-evaluation of computer literacy from different participant groups.
Participant group | Very low, n | Low, n | Medium, n | High, n | Very high, n | |
Level of computer literacy | 0 | 2 | 3 | 0 | 0 | |
Knowledge of the TasPa trial | 0 | 3 | 2 | 0 | 0 | |
Awareness of the dashboard project | 0 | 1 | 3 | 0 | 1 | |
Level of computer literacy | 0 | 2 | 3 | 0 | 0 | |
Knowledge of the TasP trial | 0 | 0 | 1 | 4 | 0 | |
Awareness of the dashboard project | 0 | 2 | 2 | 1 | 0 | |
Level of computer literacy | 0 | 0 | 2 | 2 | 1 | |
Knowledge of the TasP trial | 0 | 1 | 2 | 2 | 0 | |
Awareness of the dashboard project | 1 | 4 | 0 | 1 | 0 | |
Level of computer literacy | 0 | 0 | 0 | 5 | 0 | |
Knowledge of the TasP trial | 0 | 0 | 3 | 2 | 0 | |
Awareness of the dashboard project | 0 | 4 | 1 | 0 | 0 |
aTasP: Treatment as Prevention.
Task completion rates, by group. This figure shows how different groups of participants performed during the evaluation study. CAB: Community Advisory Board.
Variance in time taken to complete tasks, by group. This image provides detail on the time taken by each group to complete tasks during the evaluation study. As can be seen, the Community Advisory Board and nursing groups generally took longer to complete tasks than operational and scientific staff, indicating that more work is required around ensuring the dashboard is accessible to all user groups.
Of the users, 80% (16/20) felt that dashboards provided them with a detailed understanding of HIV prevalence and treatment in the TasP trial region, with 5 participants strongly agreeing with the statement and 11 agreeing with the statement. In relation to the questions about the design of the dashboard here, the results were mostly positive, with the majority (60%, 12/20) of users commenting that the terms used in the dashboard aided in the understanding of the data presented, the dashboard was easy to navigate, and the charts were easy to understand. However, the CAB staff and operational staff had a more positive view of the design of the dashboards than the nursing or scientific staff. These differences were also seen in the comments made relating to design and use of dashboards. A member of the nursing staff commented as follows:
It must be used by scientists only, not everyone is familiar with use of graphs and percentages when analysing data.
A member of the scientific staff made a similar comment on the presentation of data relating to ability to understand data visualization:
For scientific staff the dashboard is user friendly, for the broader AHRI community, it might not be, particularly for TasP staff, the language and graphs used might not be easy to understand and follow.
Through the questionnaire, 95% (19/20) of users agreed that the dashboards were a useful tool for providing information on the studies taking place at the center and that the dashboard should be used more widely as a tool for explaining results of the trials:
Every staff member should be able to use the dashboard for information purposes.
I think there is enough information in this data dashboard because it is very expansive and could help us to understand easily any information that we need. And would help us to gain more information and if you don’t understand something it is easy for to go to the data dashboard to punch in that information and gain more knowledge.
In this study, we outlined a framework for the development of a data dashboard for exploiting real-time data within the context of a population health surveillance site, and a mixed-method evaluation demonstrated the frameworks usability. Population health is like many other health disciplines in experiencing a significant increase in the amount of data created at field sites. This increase is driven by new and more efficient data collection systems and techniques, such as mobile and Web platforms and integration of datasets from different sites and studies. The potential of such data could be negated and restricted without a visualization system to disseminate datasets in a manner suitable to a varied set of audiences. These interfaces allow for the rapid detection of emerging health trends, highlighting of outliers, and emerging clusters of activity. Once in place, fewer resources are needed to maintain a real-time monitoring system than is required for the creation of ad hoc analyses. The data dashboard design framework presented here is more widely applicable to other HDSS contexts, where similar challenges are being faced.
The dashboard evaluation demonstrated the potential of the dashboards as a method to explain the ongoing progress of research trials to staff and stakeholders at AHRI. The results of the questionnaire also demonstrated a very positive attitude toward their future use within AHRI across all groups studied, demonstrating that there is enthusiasm for not only the increased visibility of data but also the increased use of data visualization as a means to disseminate datasets.
AHRI is typical of an HDSS site. The success of and enthusiasm for the dashboards speaks to the potential for implementation at other sites. Projects such as the South African Population Research Infrastructure Network, a system of HDSS sites, highlight the importance of data sharing to improve population health outcomes. A standard framework of visualization only aids the cause of sharing and understanding. The results of the evaluation demonstrate the potential for a visualization platform to provide an exploratory interface for users to interrogate multiple data sources, develop insights, and form new research hypotheses within this context. Furthermore, the interface presents an opportunity for collaborations among researchers, through shared data exploration. It also allows stakeholders and community members to see how the data collected in their community are being employed to further research and benefit the community at a large scale. Increasing the visibility of data for community members increases transparency and encourages active participants in ongoing studies. The evaluation demonstrated that although the majority of all groups had positive attitudes toward the increased visibility of data, it is crucial to note variation among groups concerning data needs and user ability.
The study outlined the design and development of a framework for dashboard design within the context of population health surveillance. The work done thus far can serve as a template for further development. However, further developments in the design and design process are future areas for exploration. The platform could be further developed to allow for increased flexibility in the types of data visualization available and expansion to other sites and contexts. However, it is clear that a more significant step lies in opening up the dashboard to a broader variety of users. Within the current design process, users consulted informally throughout the project, and there would be advantages in exploring user data needs through a user-led process during later iterations. The evaluation demonstrated that users spent more time on a task when they did not complete the task. These results highlight a need to research how different users interact with the same dashboard. The results highlight the need to understand data visualization literacy in the development of platforms for diverse user group and the need to understand the differing information needs of end users. When we talk about dashboards, we often speak about diverse user groups, yet research has shown that the performance of users can differ substantially despite adhering to good design practice. The potential of dashboards to promote the sharing of information and collaboration among diverse users is diminished if we do not consider literacy in design and build user adaptability into future systems.
Evaluation tasks.
User questionnaire.
Africa Health Research Institute
Community Advisory Board
Health and Demographic Surveillance System
key performance indicator
South Africa
Treatment as Prevention
The authors acknowledge the contributions of Siyabonga Nxumalo for running the evaluation at AHRI, Steven Gray for providing technical assistance on the development of the dashboards, and Dickman Gareta, Jaco Dreyer, and the research data management team at AHRI for all their assistance throughout the development process. The authors thank the AHRI for hosting the study, particularly the participants who gave their time to take part in the evaluation. The authors are further grateful to Professor Rachel McKendry for her continued generous support of this work. The authors would also like to thank Maureen Nomsa Hlongwane for her support and friendship through our time in SA; she is dearly missed. This project was funded under the i-sense (EPSRC IRC [Engineering and Physical Sciences Research Council Interdisciplinary Research Collaboration Early-Warning Sensing Systems for Infectious Disease] project (EP/K031953/1).
The project was conceived and designed by EM, KH, and DC. The technical development of the data dashboard was undertaken by DC. The dashboard evaluation was designed and executed by DC, KH, and EM. The evaluation analysis was carried out by DC. All authors read and approved the final manuscript.
None declared.