Congenital heart disease (CHD) is a heart abnormality present at birth, requiring intensive medical care and often leading to life-threatening complications [1]. It affects around 40,000 newborns annually in the United States and 1% globally, posing lifelong challenges for children, families, and health care systems [1-4]. Despite significant advancements in diagnosis and treatment, CHD remains a chronic condition that demands ongoing care and specialized educational resources [5,6]. Tailored education is critical for addressing the needs of children with CHD, providing accessible educational support for parents, and equipping health care providers with efficient educational strategies. However, no tools currently exist to understand children’s health literacy, assess parental educational burdens, or measure health care providers’ efficiency in delivering education [7,8].

Children with CHD face unique barriers in understanding their condition due to developmental challenges, such as difficulty grasping abstract medical concepts or relating them to their lived experiences. These limitations can hinder their ability to adhere to treatment plans, actively participate in care, or respond effectively during emergencies and transitions to adult care [4,6,9-11]. Despite these challenges, pediatric patients are often excluded from health literacy efforts, as most resources are designed for parents or caregivers. This exclusion highlights the critical need for age-appropriate tools that empower children to engage with their care actively, ultimately improving adherence and long-term health outcomes [7,12-16].

Parents, meanwhile, bear a significant educational burden. They must interpret medical jargon, simplify it for their child, and act as intermediaries with health care providers, all while managing the emotional strain of caregiving and the cognitive load of understanding complex medical information. This burden can increase stress, reduce caregiving effectiveness, and impact family well-being. Low parental health literacy further compounds these challenges, as it is linked to medical errors and poorer health outcomes for children [17-24]. Furthermore, health care providers face the challenge of balancing clear communication with time and resource constraints, often struggling to deliver education tailored to the needs of both children and parents [11,25-27]. Addressing these gaps is essential for improving communication, care coordination, and health outcomes for children with CHD and their families [7,28].

As part of a multiphase research project, this pilot study aims to close these gaps by developing tailored assessment tools for key stakeholders in pediatric CHD education. This research comprises two phases: (1) iterative development of the assessment tools and (2) pilot testing to create, refine, and validate 3 CHD-specific assessments based on stakeholder feedback in a real-world setting. We gathered feedback using cognitive interviews and observations to assess each assessment item [29-34]. We iteratively developed face-to-face and computer-based assessments based on this feedback from all stakeholders, including children. The assessments measure changes in children’s knowledge, parental educational burdens, and health care provider efficiency before and post interventions. The results show that developing and refining these assessments are essential before introducing our medical education toy. The objectives of this study are 3-fold:

During the pilot phase, we tested 3 different assessments with CHD stakeholders at the M Health Fairview Pediatric Specialty Clinic for Cardiology—Explorer.

We used qualitative methods, including cognitive interviews, observational notes, and interactive techniques, to gather feedback from children, parents, and health care providers [29,30,32,34]. These methods captured participants’ impressions, behaviors, and engagement, providing rich data for iterative refinements of the assessments.

Our research team held weekly meetings to analyze assessment data and stakeholder feedback. Using qualitative methods, including thematic analysis, we identified common themes from feedback and observational data [92,93]. The first author coded the data to highlight relevant themes, focusing on the assessment’s reliability, engagement, and responsiveness to meet all stakeholders’ needs. During each round of pilot testing, we prioritized feedback using four criteria: (1) its potential impact on the assessment’s effectiveness, (2) alignment with intended outcomes, (3) feasibility of incorporating changes, and (4) stakeholder preferences. Feedback was categorized by feasibility, clarity, relevance to educational needs, and participant engagement, then ranked by the frequency and significance of reported issues. We implemented revisions iteratively, prioritizing aspects that most improved comprehension and ease of response. This systematic process enhanced the assessments’ accuracy and relevance across stakeholder groups. During weekly meetings, the team reviewed how each piece of feedback aligned with the established criteria and worked collaboratively to resolve discrepancies. This iterative process led to the development of a set of codes such as “strengths,” “weaknesses,” and “areas of improvement.” These ongoing discussions and thematic analyses refined the assessments and informed adjustments for subsequent pilot tests [94].

The study received ethical approval from the University of Minnesota institutional review board (STUDY00020670). The medical team facilitated communication with potential participants—children with CHD and their families—through purposive sampling. Informed consent was obtained from all participants: parents also signed parental permission forms for their children, and children provided assent. Participants were informed about the study’s purpose, procedures, and their right to withdraw at any time. Data were anonymized and deidentified during transcription and analysis, with all personal information securely stored in adherence to institutional guidelines. Participants received a gift card as compensation for their time, ensuring fairness and transparency.

Both children (C) and parents (P) actively engaged in the children’s assessment process. Parents were surprised by their children’s interest and enjoyment, with one parent expressing, “I wasn’t sure she’d even talk. Wow!” It appeared that the interactive elements, such as choosing pens or markers and storytelling, not only boosted curiosity and made them concentrate on doing the assessment but also eased the anxiety and fear of children. For instance, C3 quickly transitioned from nervousness about another medical procedure to excitement upon seeing colorful markers, exclaiming, “Yay, I can paint here in the doctor’s office!” Initially, she hid under the bed due to fear when we entered the examination room. However, she relaxed and became comfortable after seeing the markers and being invited to answer the questions using different colors. Even children like C2, initially focused on their iPad, became engaged, asking, “Can I use all colors? I can answer anything like this!”

Parents appreciated the playful and instructive design of the assessments, with one commenting, “It’s like teaching by itself through playtime. She’s telling me all you've asked her.” After finishing their assessments, we provided paper to children like C4, whose mother was also being assessed. This engagement prompted them to draw, becoming so engrossed to continue that they requested us to stay there even after the doctor’s arrival (Figure 2). Initially, parental influence impacted C1’s responses, but gentle interventions and prioritizing children’s uninfluenced responses over accuracy fostered independent responses in other pilot tests. Through face-to-face interaction using cognitive interviews akin to semistructured interviews [95,96], we collected less biased data from children rather than their parents, maintained attention, and reduced parental influence.

Similarly, in the clinic, parents provided feedback after their children’s assessments through a think-aloud and verbal probing method [29,32] while answering the computer-based parents’ assessment using Typeform (Typeform SL). During the assessment, P4 noted, “You know what to ask.” P3 mentioned that they would change only some of the wording. However, despite this feedback, P3 described the survey as “clear, easy, feeling good; finally, somebody asks!” This feedback highlighted the ease of use, clear directions, and meeting the participants’ needs, facilitating effective data collection. The health care providers’ assessment also received positive feedback for its engaging interface, with H1 noting, “Much better than usual surveys,” and H3 stating, “...really interactive with photos and video, and buttons. It’s easy....” Such feedback underscores the effectiveness of engaging participants and facilitating data collection.

The effectiveness of engaging all stakeholders through assessments depends on their relevance and well-structured design. Maintaining high relevance across all assessments, we noticed an interesting trend involving the Smiley Face Likert question during the children’s assessment. Despite our efforts to adjust the question wording, children consistently chose the happiest face in the initial version of the assessment. They emphasized their positive feelings associated with smiley faces by saying, “I chose it because I like it!” We expected this issue but still tested the smiley face Likert question due to its common use in children’s surveys. To improve engagement and reliability, we found that comparison questions, structured like a right or wrong game format, were more effective. Children showed high concentration levels and often asked, “Is this right?” They felt like participating in a game, increasing their involvement in the assessment process. Incorporating feedback from our medical team and parents highlighting the assessment’s educational value, we replaced the Likert question with a prompt offering a choice between physical activity and screen time (Figure 1 [84]), addressing another relevant habit for children with CHD. We also adjusted the question order to observe response variations, albeit with limited reliability due to its implementation with only participants C2 and C3 immediately after the initial version.

P2, P3, P4, and P5 actively engaged with the questions in the parents’ assessment, finding them clear, easy, and relevant. However, P2 noted assumptions in specific questions regarding the prior receipt of educational material during doctor visits. They proposed a preliminary question to confirm whether educational material was used before assessing its appropriateness (Figure 3). They asked, “How do you know if we got any educational material before asking how good it was? It’d make more sense to check if we received any and then ask about it.” Interestingly, when explicitly asked about this question, P3, a follow-up patient, did not express the same concern. Another participant suggested improving the context of the question about comfort levels during clinic visits (follow-ups or surgeries), which we addressed in the later version. We condensed the questionnaire to 25 items in the latest parent assessments by adding a matrix question.

In response to initial feedback on the health care providers’ assessment, H1 proposed categorizing questions differently for new and follow-up patients. All participants provided positive feedback regarding the matrix questions (Figure 4). H2 preferred to keep the existing matrix questions but suggested reordering them for better clarity and efficiency. They emphasized that combining related aspects into single questions helps respondents compare options and find the correct answer more easily: “Having one question for each of these aspects would make the questionnaire longer and harder to compare when answering; it just seems more practical this way.” Furthermore, H2 recommended adding a specific demographic—vulnerable parents—to measure parental CHD information instead of using a general category. These suggestions were integrated into version 03, reducing the length to 15 items.

We used patients’ wait time between their initial screenings and the meeting with their physician to complete the assessments. This integration not only streamlined care coordination and reduced wait times but also engaged families and alleviated worries, as noted by P4, “It keeps us good busy!” This suggests that our design intervention (an educational toy) can be implemented during patients’ wait time as well. The assessments also doubled as educational tools for children, engaging them in learning about their condition and easing anxiety. Parents found solace in sharing burdens, while health care providers gained insights into their practices and sought potential solutions. These assessments exceeded their initial purposes, offering education, support, and reflection opportunities for child-led approaches in pediatric care.

During face-to-face interviews, we identified areas for refinement in assessment items, such as clarifying the context of health care experiences for children’s comfort-level rating or specifying the type of visit for health care providers. Observational notes highlighted instances where assumptions in specific questions, such as the prior receipt of educational material during doctor visits, caused confusion among interviewers. To alleviate this issue, we proposed preliminary questions. Despite these challenges, participants generally interpreted the majority of survey items as intended, with occasional adjustments to wording choices for better comprehension. Furthermore, we changed the order of questions based on feedback and observations to improve the structure of the assessment.

The pilot testing evaluated the feasibility and effectiveness of the tailored assessments. Iterative revisions further improved their clarity and appropriateness. We ensured the validity of the assessments through a collaborative and iterative approach, incorporating perspectives from all stakeholders. As a result, the assessments offer a comprehensive understanding of educational needs and burdens, providing valuable insights to guide the development of targeted educational interventions.

The tailored assessments developed in this pilot study address a gap in the existing literature by focusing on the educational needs of pediatric patients with CHD, their parents, and health care providers [7,8,27]. While existing assessment tools mainly measure health-related quality of life and emotional well-being, they offer limited insight into health literacy and the unique educational challenges associated with CHD [38-56]. Moreover, these assessments are primarily developed quantitatively and lack qualitative insights into educational needs and burdens. We used qualitative methods to explore how respondents interpret, understand, and respond to specific survey items. This approach offered a more comprehensive understanding of the questions that assessments should address [29-32,93-97].

Through collaboration with stakeholders, including children, we developed and refined assessments to evaluate children’s knowledge, parental educational burdens, and health care provider efficiency in pediatric CHD care. Including children in the development process was crucial due to the lack of tailored educational materials for children with CHD and the complexities involved in assessing this population [64-67,98]. While parents were present during the children’s assessments to provide comfort, their involvement was carefully managed to ensure that it did not interfere with the children’s meaningful participation. This approach contrasts with prior methods, where parents often act as proxies, potentially biasing the results [68,69]. Feedback from all stakeholders was collected post assessment to gain insights into their experiences and further improve the assessments.

We conducted pilot tests for two reasons: (1) to refine the assessments before involving more stakeholders, specifically children, and (2) to ensure the assessments were well adapted to meet all stakeholders’ needs before the actual study [99,100]. This pilot study revealed critical gaps in existing tools and advanced the methodology for developing educational assessments. Emphasizing collaborative, iterative, and direct engagement with children, parents, and health care providers ultimately leads to the design of tailored assessments. These assessments are crucial for informing the development of effective, child-led educational interventions for pediatric populations with CHD, demonstrating feasibility before broader implementation.

The assessments developed in this study have potential applications in routine clinical practice, offering health care providers a structured tool to assess and enhance CHD-related health literacy among pediatric patients and their families. By integrating these assessments into preappointment resources or waiting room activities, health care providers can identify educational gaps early and address them proactively. Furthermore, the assessments could complement existing educational interventions, providing feedback on the effectiveness of child- and family-centered resources to improve health literacy. This integration would not only inform the design of more effective child- and family-centered resources but also support continuous improvement of educational interventions. This approach aligns with broader goals in pediatric health care to support lifelong health management through early literacy interventions.

Although the pilot testing provided valuable insights, several limitations are acknowledged. First, despite efforts to minimize parental influence, their presence may have still impacted some children’s responses. Future research should explore strategies to reduce this influence further and ensure more independent responses from children. Second, the sample distribution was not spread evenly across age groups, with only 1 child aged 5 years and 4 children aged 7 years. This uneven distribution may affect the representativeness of the findings. Future studies should aim for a more balanced age distribution. Third, conducting assessments for both children and parents at the clinic during their visit might have influenced their responses, as they were exposed to medical procedures. Furthermore, the researcher’s presence during the assessment of parents and children could have influenced their feedback. Finally, while general accessibility was considered, specific adaptations for educational disabilities were not within the scope of this pilot study.

This pilot study aims to improve educational interventions and care coordination through tailored assessments designed by stakeholder feedback, including affected children. The findings emphasize the importance of interactive and qualitative methods that foster multistakeholder engagement and ensure question relevance. These assessments can potentially enhance child-led interventions and improve outcomes for patients with CHD from childhood through to adult care. However, the iterative development process highlighted several challenges, such as managing parental presence to encourage independent responses from children and the logistical complexities of recruiting diverse participants within clinical settings. It is also important to ensure that assessments for younger children are read aloud without influencing their responses. Integrating these assessments into real-world clinical workflows requires adaptability to avoid disrupting patient care. Addressing these practical challenges will be essential to scaling and sustaining the use of these assessments in diverse health care contexts.