1 Increasing Health Literacy of Parents of Children Age 0 to 5 Regarding Oral Health and Immunizations Jaime Davis College of Nursing, East Carolina University Doctor of Nursing Practice Program July 15, 2021 2 Author’s Note I would like to extend my dearest thanks to my site champion, Teandra Ramos-Hardy and my colleagues, Sydney Sharpe, Malgorzata Tiger, and Danielle Tupes for all of their hard work and effort they put in to helping make this project a success. This project would not be what it is without each of you. Each of you have put a unique touch on what was done with this project and many parents and children will benefit from your work. You should be proud. Also, thank you to Suzanne Metcalf, who put in the time building the website, and the wonderful people in my area as well as the clinic providers and staff, who helped to facilitate getting the books stickered, into the clinic, and out to parents. Thank you to , my DNP Project Faculty, for the many hours of guidance, reflection, and support given throughout the course of this project and for all of the thought-provoking conversations that encouraged us to dive deeper and strive for greatness. I would also like to extend a heartfelt thank you to , overseeing DNP Faculty, for your feedback and for help in setting up this project with Reach Out and Read and helping to support the continuation and expansion of the project in the future. Lastly, thank you to my loving and supportive family for all of your patience, feedback and love through this process. Thank you to my sister, Stacey Wooten, for reading this paper a million times and providing feedback, editing support, and APA expertise. Thank you to my husband, Adam Davis, who spent many late nights answering my questions about graphic design and technology. I couldn’t have done this without your support. 3 Abstract Health literacy is “the degree to which individuals have the capacity to obtain, process, and understand basic health information needed to make appropriate health decisions” (U.S. Department of Health and Human Services [USDHHS], n.d., para. 1). Low health literacy impacts patients and healthcare systems alike. Increasing parental health literacy of both vaccines and oral health can improve childhood health outcomes. This project utilized a Quick Response (QR) code placed on the back of age-appropriate Reach Out and Read books that providers distribute at well-child visits. Parents scanned the QR code, which led them to an educational website about oral health and immunizations for children from birth to five years of age. Website views and analytics were tracked along with responses from a voluntary survey found on the website. Data showed 236 unique views of the site with repeat traffic from November 1, 2020 until April 30, 2021. Parental survey responses were positive for learning, intent to change health habits, and intent to share the information they learned with others. Additionally, another organization has already duplicated the project to increase pediatric oral hygiene habits, and a North Carolina regional Area Health Education Center (AHEC) is in the process of adopting this project. With positive outcomes, this project will continue to expand and develop in the future. Keywords: health literacy, parental health literacy, pediatric patients, QR codes, Quick Response code, website, Reach Out and Read, vaccines, immunizations, vaccination, oral health 4 Table of Contents Author Note……………………………………………………………………………………...2 Abstract……………………………………………………………………………………..……3 Section I: Introduction…………………………………………………………………………...7 Background………………………………………………………………………7 Organizational Needs Statement…………………………………………………8 Problem Statement……………………………………………………….….…...9 Purpose Statement……………………………………………………………..…10 Section II: Evidence…………………………………………………………………………....10 Literature Review……………………………………………………….……..…10 Evidence-Based Practice Framework…………………………………….…..…15 Ethical Consideration and Protection of Human Subjects…………………....…18 Section III: Project Design……………………………………………………………………...19 Project Site and Population…………………………………………..………....19 Project Team……………………………………………………………..……..22 Project Goals and Outcome Measures…………………………….……..……..23 Implementation Plan……………………………………………………..……..25 Timeline…………………………………………………………………..…….27 Section IV: Results and Findings……………………………………………………………….29 Results…………………………………………………………………..……....29 Discussion of Major Findings……………………………………………..……32 Section V: Interpretation and Implications……………………………………………………..33 Costs and Resource Management………………………………………..……..33 Implications of the Findings…………………………………………………....44 5 Sustainability ………………………………………………….……...……..…48 Dissemination Plan ……………………………………………..………….......49 Section VI: Conclusion………………………………………………………………….…...…49 Limitations and Facilitators…………..……………………………….…….…49 Recommendations for Others……………………………………………….…52 Recommendations for Further Study…………………………………….……57 Final Thoughts…………………………………………………..…………….58 References…………………………………………………………………………………...….59 Appendices…………………………………………………………………………………...…68 Appendix A: Pre-Implementation Resources…….……………………….......68 Table 1: Literature Review Log…………………………..…...68 Table 2: Literature Review Matrix……………………………70 Table 3: Timeline of the DNP Project……………………...…78 Appendix B: Demographic Data….………………………………………..…81 Table 1: Visit and Book Data from 2021 ROR Survey……….81 Table 2: Insurance Data from 2021 ROR Survey……………..82 Table 3: Racial Demographic Data from 2021 ROR Survey…83 Table 4: Language Data from 2021 ROR Survey…………….84 Appendix C: Website Data……………………………..……………………85 Table 1: Website Analytics…………………………………...85 Figure 1: ROR HLW Splashpage Views……………………..86 Figure 2: ROR HLW Page Views Per Age…………………...87 Figure 3: ROR HLW Overall Page Views………………….....88 Appendix D: Survey Data……………………………………………………89 6 Table 1: ROR Parental Survey Responses...…………………..89 Table 2: ROR Parental Survey Data Analysis………………...90 Figure 1: ROR Parental Survey Data: Age……………….…...91 Figure 2: ROR Parental Survey Data: Understanding…….…..92 Figure 3: ROR Parental Survey Data: Learning………………93 Figure 4: ROR Parental Survey Data: Intent to Change……....94 Figure 5: ROR Parental Survey Data: Intent to Share………...95 Appendix E: Budget…………………………………………………………..96 Table 1: Project Budget……………………………………….96 Table 2: Proposed Project Budget for Other Organizations…..97 Appendix F: Form 8274A……………………………………………………..98 Appendix G: Form 8274B #1…………………………………………………100 Appendix H: Forma 8274B #2…………………………………………….......102 7 Section I. Introduction Background The definition of health literacy is "the degree to which individuals have the capacity to obtain, process, and understand basic health information needed to make appropriate health decisions" (U.S. Department of Health and Human Services [USDHHS], n.d., para. 1). It is a complex and multifactorial issue that can impact the overall health of patients. Low health literacy impacts both patients and the healthcare system and is linked to increased use of the emergency room, increased admissions to the hospital, increased healthcare costs, and higher risk for and rates of death (Kutner et al., 2006; National Institute of Medicine [NIH], n.d.; Office of Disease Prevention and Health Promotion [ODPHP], n.d.-b, n.d.-c). Adequate health literacy is imperative for improving overall health. By targeting parents’ health literacy with their children, health outcomes for children are positively impacted in childhood, leading to better health choices and continued adequate health literacy throughout life (Nakamura et al., 2018). Oral health is a leading health indicator (Office of Disease Prevention and Health Promotion [ODPHP], n.d.-f). Healthy People 2020 also has multiple objectives related to increasing oral health and improving childhood vaccination rates (Office of Disease Prevention and Health Promotion [ODPHP], n.d.-f). Baskaradoss (2018) and Yazdani et al. (2018) note that decreased oral health literacy among parents can lead to poor oral health, cavities, and missing teeth. Visual education interventions, such as videos, pictures, and cartoons, in combination with health education, can improve health literacy related to vaccinations and maternal education programs, both in the late prenatal and the immediate postpartum period, and can positively 8 impact parental vaccination rates (Forshaw et al., 2017; Lee et al., 2019; Otsuka-Ono et al., 2019; Papapchrisanthou & Loman, 2018). Organizational Needs Statement Reach Out and Read (ROR) is a non-profit organization that specializes in promoting childhood literacy by partnering with primary care practices and pediatricians to provide age- appropriate books to children at well-child visits from birth to age five (Reach Out and Read [ROR], 2020d). ROR Carolinas’ mission statement is to give “young children a foundation for success by incorporating books into pediatric care and encouraging families to read aloud together” (Reach Out and Read Carolinas [RORC], 2020, para. 1). While Reach Out and Read (ROR) has promoted increased childhood literacy and improved children’s test scores on reading, their focus has been mainly on basic overall literacy (Reach Out and Read [ROR], 2020a, 2020c). It has become increasingly evident that parental health literacy has been neglected, particularly in minorities and patients with low education or poor socioeconomic status with Hispanics, Blacks, American Indians, and multiracial patients most often victims of health illiteracy (Cutilli et al., 2018; Kutner et al., 2006). Providing parents with the necessary tools to increase their health literacy will substantially improve the ROR’s mission, promoting enhanced preventive healthcare for children, leading to healthier lives (C. Boulware & T. Ramos-Hardy, personal communication, June 22, 2020). It also will help instill the value of health literacy from a young age in these children and promote continued health literacy as they grow and mature. Healthy People 2020 has several objectives related to health literacy. These include improving the population’s health literacy by providing basic, comprehensive instructions, increasing patients’ involvement in their care and healthcare decisions, and improving providers’ 9 communication in a meaningful, understandable way (ODPHP, n.d.-c). Oral health and immunization are just two areas of children’s health that are very important to their overall health. Increasing the number of children who use dental care or oral care services is the main objective of Health People 2020’s oral health leading health indicator (Office of Disease Prevention and Health Promotion [ODPHP], n.d.-e). Their immunization goals focus mainly on increasing the number of adequately vaccinated children (Office of Disease Prevention and Health Promotion [ODPHP], n.d.-d). By improving access to the oral health care system, there will be an increase in preventive oral health services, reducing the incidence of dental caries and tooth decay (ODPHP, n.d.-e). ROR hopes that increasing parents’ health literacy related to these two areas will increase the rate of childhood vaccinations and improve children’s overall oral health by increasing early dental visits of children in the clinics that use their program. This project intersects with the Triple Aim because it will help open up a dialogue between parents and providers, improving the patient experience. Parents will be more educated, and providers will have the opportunity to listen to them more thoroughly. The project will also help improve children’s health and reduce health costs related to poor oral care, caries, missing or decaying teeth, and oral infections. It will decrease the cost burden of treating and managing vaccine-preventable illnesses while also increasing herd immunity. Problem Statement In the United States, low health literacy, poor oral health, and dental caries are common problems that are particularly prevalent in children of Hispanic and low-income families and can impact both academics and quality of life (Crespo, 2019; Cutilli et al., 2018; Kutner et al., 2006; Lebrun-Harris et al., 2019). Low health literacy about common childhood conditions can 10 contribute to poor health in children. By visualizing their parents’ learning and gaining health literacy, children can learn good habits to improve their own health literacy. Purpose Statement The purpose of this Doctor of Nursing Practice (DNP) project is to increase health literacy among parents of children from birth to five years of age about oral health and immunizations in North Carolina (S. Sharpe, D. Tupes, & M. Tiger, personal communication, June 29, 2020). A Quick Response (QR) code was created and included on books that providers already distribute to parents at well-child visits. The QR code links parents to an interactive website with age-appropriate health information about immunizations and oral health. Section II. Evidence Literature Review A literature search was performed on July 20, 2020, using the term “interventions AND health literacy AND parents” in three databases--Medline via PubMed, CINAHL Plus with Full Text, and the Cochrane Database of Systematic Reviews. The search initially returned 1,057 results in Medline, 229 results in CINAHL Plus, and 25 results in the Cochrane Database. The database systems were used to narrow the search results to articles written within the last five years, those written in English, and those with access to full-text articles, leaving 612, 134, and 17, respectively. Inclusion criteria included articles directly related to the clinical question regarding methods to increase parental health literacy, articles related to specific interventions that affect health literacy regardless of the topic, and articles whose interventions targeted parents/caregivers over age 18. Exclusion criteria included articles written before 2015 or in a language other than English, those without access to the full-text article, studies that were proposals or protocols for a study or did not have results, studies that focused on media literacy 11 or were performed in an inpatient setting, and those with interventions aimed at parents under the age of 18. Articles with evidence level IV and above were desired and kept. On an initial review of the PubMed results, 27 articles appeared to meet the inclusion and exclusion criteria. However, after complete review of the articles, 11 were found to contain exclusion criteria, leaving only 16 pertinent articles. Initially, CINAHL Plus resulted in 15 relevant articles. Eight of these were duplicate articles from the PubMed search. After thoroughly reviewing the remaining seven, four contained exclusion criteria, removing them from the literature review. After an initial assessment of the Cochrane Database articles, only one pertinent article was kept, which met all inclusion and exclusion criteria upon full review. None of the articles retained on the initial evaluation were below evidence level IV. Of the final 20 articles found, the evidence levels ranged from I to IV, with most found to be Level II to III. Refer to Table A1 and A2 for the Literature Review Log and Literature Review Matrix of this project. Current State of Knowledge The literature review did not reveal any current guidelines or best practice expectations regarding recommended interventions to increase health literacy. While most studies found in the literature search focused on increasing parental mental health literacy for adolescents, a few articles focused on improving parental health literacy (PHL) regarding preventative practices. A few articles concentrated on PHL about chronic conditions, such as asthma. There was little evidence found related to oral health or immunization health literacy, with only two articles found for each topic. Most interventions related to improving parental health literacy utilized either one-on-one or group education sessions. A few focused on digital interventions, such as web-based platforms, phone game-like applications, or text-based education programs. Hutton et 12 al. (2017) noted improved parental literacy and improved safe sleep habits when healthcare providers utilized children’s storybooks to educate caregivers on safe sleep practices. Similarly, Rosas-Blum et al. (2018) created a comic of developmental milestones intended to improve health literacy among parents of infants. All of the articles showed parental health literacy improvement with each of the interventions used. Individual parental education and diet/feeding education given to parents within the first year of life provided the most evidence to support oral health literacy and prevention of early childhood caries (Riggs et al., 2020). However, based on the systematic review, the delivery mode of the educational intervention was unclear in most of these studies. A study performed by Dudovitz et al. (2020) notes that group parent education classes increased parental oral health literacy in the Head Start program. Digital interventions utilizing graphics, videos, and interactive material, as well as individualized family education provided perinatally proved to increase parental immunization health literacy, as well as increase parental intention to vaccinate and the rate of on-time childhood vaccinations (Otsuka-Ono et al., 2019; Papapchrisanthou & Loman, 2018). While there was no literature specific to using a QR code on children’s books to provide oral health and immunization information to parents, several studies indicated digital interventions are effective in promoting parental health literacy of other concepts (Azevado et al., 2019; Bayley & Brown, 2015; Chu et al., 2019; DeCamp et al., 2020; Downing et al., 2018; Gibbs et al., 2018; Peyton et al., 2019; Real et al., 2019). Critical strategies to making interventions effective include simplifying information, avoiding jargon, ensuring written material is provided in a way that is understandable to those who have low reading levels, and utilizing audio, video, graphic, and interactive measures (Azevado et al., 2019; Bayley & Brown, 13 2015; Cowden et al., 2015; Morrison et al., 2019; Papapchrisanthou & Loman, 2018; Real et al., 2019; Rosas-Blum et al., 2018; Yin et al., 2017). Providers can further improve parental health literacy by utilizing teach-back or show-back methods and providing demonstrations for parents (Morrison et al., 2019). Current Approaches to Solving Population Problem While individual and group education sessions help increase PHL (Dudovitz et al., 2020; Hurley et al., 2018; Hutton et al., 2017; Morgan et al., 2019; Otsuka-Ono et al., 2019; Riggs et al., 2020; Smith & Carroll, 2017), they can be time-consuming for providers and parents alike. They also require parents to take in a large amount of information in one session and to be able to recall that information later. Therefore, while education sessions promote parental health literacy, this was not the best intervention to use in partnership with Reach Out and Read due to time constraints, cost, and ROR’s lack of individual contact with the parents they target. However, a web-based application appeared to be more conducive to the current ROR operating procedure. The studies found in the literature search explored various approaches, including informative websites, visually-enhanced educational programs, and the use of children’s books (Azevado et al., 2019; Gibbs et al., 2018; Hutton et al., 2017; Papapchrisanthou & Loman, 2018; Peyton et al., 2019; Rosas-Blum et al., 2018). This project placed QR code stickers on the back of ROR books that providers distribute to parents at well-child visits. Scanning the code took parents to a website with oral health and immunization information organized by age group. This intervention targeted several of these evidence-based practices shown to improve parental health literacy, worked hand-in-hand with the organization’s current approach to increasing child literacy, and reinforced in-office provider education. Evidence to Support the Intervention 14 According to the United States Census Bureau (n.d.), 9.8% of North Carolinians are Hispanic, and 14.0% live in poverty, making oral health and health literacy essential areas to focus on in North Carolina to address health disparities prevalent throughout the state. Decreased childhood vaccination rates impact herd immunity, putting the community’s health at risk (Paules et al., 2019). Likewise, the child’s risk for preventable diseases increases, as well as the potential for sequelae, such as infections and decreased immune memory of past illnesses after contracting diseases like measles (Mina et al., 2019; Paules et al., 2019). Furthermore, poor oral health and poor dentition can lead to dental caries and dental pain, affecting academic performance and quality of life (Lebrun-Harris et al., 2019). Reach Out and Read currently targets improving child literacy by enabling providers with the resources and training to provide age-appropriate books to parents at well-child visits. The simple process of adding a QR code to the back of these books with age-appropriate oral health and immunization information targeted improved parental health literacy in these areas while utilizing ROR’s current process. Digital interventions, including websites and those driven by phones, are shown to increase parental health literacy in several subjects (Azevado et al., 2019; Bayley & Brown, 2015; Chu et al., 2019; DeCamp et al., 2020; Downing et al., 2018; Gibbs et al., 2018; Peyton et al., 2019; Real et al., 2019). Parents could scan the QR codes with their cell phones to access a website with oral health and immunization information provided in a visually appealing, easily readable, and interactive way. Papapchrisanthou and Loman (2018) note that education programs utilizing pictures in conjunction with health education have improved parental vaccine health literacy, intent to vaccinate, and childhood vaccination rates. This intervention had several benefits for parents, providers, and the partnering organization. These resources will always be accessible since they are electronic, and with the 15 QR code located on the back of their child’s book, they are less likely to get lost than standard printed information. The visual cue of seeing the QR code on the outside of the book when reading to their children at bedtime reminds parents to practice good oral hygiene before bed. The intervention was also cost-effective for ROR and easy for providers to use. The current process providers already follow with the ROR program did not change significantly. The students and local SmartStart organizations that provide the ROR books to clinics worked together to apply the stickers to the back of the books. Therefore, the only change to the office workflow was notifying parents of the code during the well-child visits. Simple posters placed in the examination rooms provided education about the QR codes and how to utilize them. These helped prompt providers to discuss the QR code and the resources available to parents by using it. The goal was that by placing these educational posters in the examination room, parents would have time to read the information while waiting for providers, allowing them to formulate questions to ask providers during the visit. Evidence-Based Practice Framework Identification of the Framework This project design utilized the Plan, Do, Study, and Act (PDSA) model and Nutbeam's Health Literacy Theory. According to Nutbeam (2000), health literacy is not a static concept with a singular goal but rather a ladder on which every person is continually moving, learning, and building. Functional literacy, interactive literacy, and critical literacy are the three primary levels on the health literacy continuum (Nutbeam, 2000). The design of the website promoted children’s health through oral health and vaccine education. The resulting parental health literacy and healthy lifestyle outcomes will increase functional health literacy, which is the first rung of the health literacy ladder (Nutbeam, 2000). Examples of parental health literacy outcomes 16 include improved attitudes towards vaccination and increased parental knowledge regarding decreasing sugary foods to prevent dental caries. Healthy lifestyle outcomes for children are improved vaccination rates and increased use of preventative oral care services. The majority of this project focused on increasing the functional health literacy of parents. However, some degree of the second level of health literacy, interactive literacy, was also addressed. Interactive literacy incorporates discussing and sharing information obtained on the first rung of the health literacy ladder (Nutbeam, 2000). The website engaged parents in promoting discussion among friends, family, and children. It also encouraged conversation with their providers by recommending questions to discuss at their next visit. These activities will help open dialogue with providers, making them more comfortable talking about these health topics and engaging them in health discussions in their communities. The last stage of the health literacy ladder is more complicated to reach. It will require the health literacy of the masses to advance to the functional health literacy level with more parents progressing to interactive health literacy. Once more parents have reached interactive literacy, there will be increased social mobilization and dialogues about health in the community, empowering parents to help lobby for and lead political and social change related to health. This project used the PDSA model in development. This model involves planning a project (Plan), implementing the project (Do), evaluating the project (Study), and making suggestions for change (Act) (Institute for Healthcare Improvement, n.d.). The first PDSA cycle for this project occurred from August 2020 until the end of February 2021. This portion of the project included the planning of the project (Plan), and the implementation of the QR code and health literacy website (Do), and the preliminary evaluation of the project based on incoming data gathered by website analytics (Study). 17 For the Plan component, a QR code was designed and created in collaboration with a health literacy website. Parents could scan the QR code to take them to the website with simple, easy-to-read, and easy-to-understand oral health and immunization education reinforced with pictures and graphics. The site and QR code were designed and created in collaboration with other group members and ROR Carolinas’ Communications Director. The codes were then printed and applied to age-appropriate books in conjunction with Reach Out and Read community partners (Do). Students sent periodic e-mails to support the clinic during this stage and encourage them to promote the project. In addition, clinics were encouraged to contact students with questions or problems. The group and site champion evaluated the data gathered during the initial PDSA phase and determined that there was initially great interest in the site, but as time went by and during the natural break in the students’ semesters, the interest began to wane. Each student contacted their sites to determine possible causes for the drop in website activity and then gathered to form ideas for possible interventions to combat these barriers (Study). Upon initial evaluation of the project, the group decided that clinic buy-in was a pivotal factor to target, considering students were not allowed on-site at clinics to work with patients and promote the project. The second PDSA cycle ran from early to mid-March 2021. Students targeted the barrier of inability to be on-site by providing appreciation cards and snacks for clinic staff to let them know how important they are to the success of this project, particularly during the COVID pandemic. One clinic had identified a language barrier with a large population of Burmese patients. Therefore, the students created a QR code that led to the website already translated into Burmese. Promotional information and QR code bookmarks in Burmese were printed and given 18 to this clinic to help promote website traffic among these patients. Again, students reviewed the data before starting the third and final phase of the PDSA cycle for this student cohort. The third PDSA cycle ran from mid to late March until the end of April 2021. After touching base with one site’s ROR county liaison, she planned a promotional event for clinic staff to take pictures, promote the project, and energize staff. The information went out on social media, including Facebook and Instagram, and was also published in the Randolph County Partnership for Children newsletter. With numbers increasing from including the Burmese translated QR code, the project team decided to trial both English QR code bookmarks and Spanish QR code bookmarks that take patients directly to the website translated in Spanish, hoping that website traffic would increase with the different format of QR codes handed out. After initiating the English and Spanish bookmarks, there was a significant increase in the number of people visiting both the website homepage and the individual age-appropriate pages. The project intervention, website, and information were also evaluated (Study) after completing the third PDSA cycle. Suggestions for improvement and expansion of the project (Act) are discussed later in this paper. Ethical Consideration & Protection of Human Subjects While this project aimed to increase pediatric patients’ health from birth to age five, which would be considered a vulnerable population, the intervention targeted their parents. Therefore, this project’s intervention was not targeting a vulnerable population. By focusing on educating parents, caregivers will advocate for and provide better health for their children. Also, pediatricians and other providers provided the same intervention to all parents of children from birth to age five, regardless of income, sex, race, or ethnicity, making the project intervention equitable for all parents. 19 While the intervention was applied equitably, the website creators paid particular attention to the reading level of the information provided to parents, allowing parents of all education levels to benefit from the material provided. In addition, including non-written components on the website also helped distribute the information to those parents with poor literacy. Also, to keep information the most current, the reputable outside resources cited on the website were limited to national organizations, such as the Centers for Disease Control’s vaccine schedule. In preparation for implementing this project, collaborators involved in the project took training on ethical research through the Collaborative Institutional Training Initiative. Collaborators took modules for social and behavioral research, conflict of interest, and biomedical research. All university collaborators were certified in ethical research to ensure the project was conducted ethically, preserving the rights of all parties involved in the quality improvement project. This project did not involve more than minimal risk, nor did it target a vulnerable population, so it did not require formal Institutional Review Board (IRB) approval. The project was also not original research but rather a quality improvement project, which also diminished the amount of risk involved in the project. Section III. Project Design Project Site and Population The Doctor of Nursing Practice (DNP) project was performed in conjunction with a national non-profit organization whose primary goal has been to help increase literacy and promote healthy starts for children across the country (Reach Out and Read [ROR], 2020b). The students involved in the DNP project worked directly with key members from the North Carolina branch of the ROR organization to develop the project content. After generating the content, they 20 worked directly with their local branch, the organization that provides the books to physicians’ offices on behalf of ROR. Together they worked to sticker and deliver books to the targeted offices that that piloted the project implementation. This local organization helped students connect with staff to educate them on the program to allow them to share the information with parents directly. Both ROR Carolinas and the local were excited and driven to start the project. Additionally, the local had longstanding relationships with staff in each clinic. Both qualities were great facilitators for the project. Their enthusiasm helped to promote the project with providers in clinics. Also, the excellent rapport and trust that providers had with the local agencies helped foster confidence in the project and facilitated the students’ work. The clinics chosen to participate in this project were also very motivated to make changes in their communities, with a prevention-driven model of health in their offices. This focus on prevention made them great partners in this project as they already had buy-in to promote wellness and increase health literacy. The clinic restrictions due to coronavirus and the potential impact the virus had on students, staff, and the clinics themselves were seen as potential barriers to this project prior to implementation. Description of the Setting The students worked directly with their local branches of to place Quick Response (QR) code stickers on books and promotional posters in each exam room at each clinic. These key organization members worked with students and clinics to help promote the project with clinic staff, who discussed the new resource available with parents at each well- child visit. Due to coronavirus restrictions, students were not allowed directly in most clinics, and the local organization that provided books also had reduced face-to-face contact in clinics. 21 Therefore, most promotion of the project occurred via e-mail. Depending on the clinic, some students were allowed to drop off books with the local organization, which helped promote the project and foster collaboration with those clinics. The project was rolled out in four different primary care clinics throughout the state of North Carolina. Each clinic supports underserved populations, and each student focused on the project efforts directly with the clinic in their area. The clinics varied in size and the number of patients seen. Prior to implementation, the clinics projected approximately 1,865 total patient visits from age six months to five years of age. Clinic A estimated to see 140 patients but only saw 130 patients during implementation. Clinic B predicted seeing 475 patients but saw 389. While both clinics A and B saw fewer patients than projected, clinics C (1000 projected, 1497 seen) and D (250 projected, 287 seen) saw more than projected for a total of 2,303 patients seen during implementation. Clinic A was the smallest clinic located in a rural county slightly west of central North Carolina. Clinic B was situated along the coast of eastern North Carolina, while clinic C was in the state’s central region. Lastly, clinic D was found in the mountains of western North Carolina. Clinic C was the largest clinic with a large Hispanic population noted before implementation. Each student involved in the project focused on implementing the project in their area. This student was directly involved in implementing and executing this project in clinic A, the smallest clinic. This clinic serves an area of predominantly non-Hispanic White (60%) patients, followed by Hispanic White patients (11.9%), non-Hispanic Black patients (11.4%), and other Hispanic patients (10.4%) (DataUSA, n.d.). The median household income for the area in 2018 was $35,609 (±-$2,112), which is $10,000 below the average salary for the county and approximately $17,000 below the mean for North Carolina (DataUSA, n.d.). The majority of the population in the area is employed in production occupations, such as manufacturing, followed by sales jobs and office/administrative support careers (DataUSA, n.d.). Description of the Population The primary population students worked directly with was their local that provided books to the clinic. Each local organization had its own coordinator that students worked with to develop sticker placement, book delivery, and how interactions with clinic staff and providers would occur. This student worked directly with the local organization coordinator, who is a family and literacy engagement specialist. She has been working in the area for several years. The student also worked indirectly with the clinic’s pediatrician and support staff through e-mail support. The clinic was a small clinic with only one provider and only a few office staff, making it easier to work more closely with the people bringing the information to parents. The clinic’s nurse has been in the practice for several years and has built a rapport with the patients and community, which was a excellent asset for the program. The clinic’s pediatrician did her residency locally at Moses Cone Hospital in Greensboro in 1981 and has served the area since then. The main focus in her practice is on promoting wellness and prevention, and her office has been working with Reach Out and Read for many years to help promote early literacy. However, she has worked with the local coordinator to request books for the clinic that are both fun and help educate children and parents on important health topics, including oral health, nutrition, and physical activity. Project Team The project team was multi-leveled and cross-organizational. There were several team members from the university, including four DNP students, their primary project faculty 23 , and , the DNP project program liaison. Teandra Ramos-Hardy is the regional director of medical engagement and training of ROR Carolinas and was the site champion for the project at ROR. She works directly under , the regional director of ROR Carolinas, who worked alongside the project’s start to help approve the initial health literacy topics. The site champion, students, and primary faculty were primarily engaged in creating, planning, developing content, and evaluating the project’s design. Suzanne Metcalf, the ROR regional director of communications, designed the health literacy website. Students created the QR code, the promotional poster, and the parent survey used to measure increased health literacy. Each student also worked with their ROR regional coordinator and SmartStart NC representative. For clinic A, the key players were ROR Program Manager in the area, and Family and Literacy Engagement Specialist for Randolph County Partnership for Children, part of the local SmartStart. Other key players in the clinic included and , the office’s medical assistant, who both worked diligently to promote the QR code and website with parents in the clinic. Project Goals and Outcome Measures The main goal of the DNP project was to implement a process change to ROR’s current distribution of children’s books to help promote parental health literacy. Through this project, students hoped to increase parental health literacy and education on oral health and vaccinations. The university process for project approval involved determining the need for International Review Board (IRB) full review of the project through a Qualtrics project tool. The tool determined that this project was low-risk and did not need further IRB review. While there is no formal process for site approval within ROR, the DNP project was approved by both the site 24 champion for this project and the regional director of ROR Carolinas. Students analyzed the project indirectly through website analytics and survey responses. The website analytics tracked what information was viewed and how many times it was viewed. Additionally, the parental survey responses helped gauge parents’ perceived learning and the change in their degree of health literacy. Description of the Methods and Measurement While it would have been ideal to have a pre and post-test measure of parents’ level of oral health and vaccine knowledge to help gauge their degree of learning, this was not possible with the project setup during the pandemic. A pre and post-test design was also not feasible due to the length of well-child visits and the amount of information covered during that time. If the project had been designed this way, the potential negative consequence would have significantly impacted the outcomes of this project and potentially caused more negative impact than positive. These consequences might have included decreased time with providers, decreased patient satisfaction due to increased paperwork, and slowed or stilled clinic workflow. Therefore, pre- testing of parental knowledge was determined to be unnecessary and potentially detrimental to the project. Instead, website analytics helped students determine how many people viewed the wellness website. In addition, a link to a carefully planned post-survey was located on the website. Students used the responses from this survey to evaluate parents’ perception of change in their knowledge level on both oral health and vaccines. Additionally, students measured their intent to make changes in the care of their children, their intention to share the information learned, and how easy they felt the QR code and website were to use, view, and understand. Analysis of the survey results helped gauge the impact the project had on health literacy, the ease 25 of the new process, the understandability of the information provided, the potential impact on patient outcomes, and ways to improve the project. Careful review and analysis of this data helped to determine the success of this intervention and the degree to which this project impacted parental health literacy. Discussion of the Data Collection Process The data collected came from three sources. The first source was from the analytics program that ROR utilizes to track website traffic and site use. The second source was from the parental survey that parents filled out online through SurveyMonkey. The regional director of communications for ROR provided survey results and website analytics to students regularly throughout implementation. The data was entered into an Excel spreadsheet. The data was unidentified and aggregate. The Excel data was stored in Google Drive, accessible only to each of the DNP students involved in the project, the site champion, and the faculty of record. The last source of data collection was from the annual ROR survey completed by clinics. The information gathered in this survey is demographic and descriptive regarding the clinics and the patient population they serve. This information was sent to students from Teandra Ramos- Hardy upon completion of the survey. Data from each clinic was e-mailed to students in the form of four Excel spreadsheets. Students then aggregated the data and entered it into a single Excel spreadsheet stored as a shared file with accessibility to the same five people as mentioned above. Implementation Plan This project has several layers. First, a wellness website was developed with easy-to-read and easy-to-understand information on oral health and vaccines for each age group of children from birth to age five. Students collaborated to create the website content and then worked with ROR’s regional communications director to build the website. Students gathered, synthesized, 26 and organized content from the Center for Disease Control’s (CDC) recommendations to provide essential information to parents of young children. Additionally, links to CDC resources that offer more information on oral health and vaccinations were also available on the website. Frequently Asked Questions about vaccines and oral health were also topics covered on the webpage. After designing the website, a QR code was created and printed on stickers. Using the projected patient visits from the clinics, the group determined an estimated number of patients that the clinics should see in the six months the project ran. Stickers were placed on ROR books used in the four clinics by students and local SmartStart staff and then delivered to the appropriate clinics. A promotional 8.5-inch by 11-inch poster was also created, letting parents know about the QR code, the wellness website, and how to use a QR code. The poster was printed, laminated, and put up in individual exam rooms at each clinic. Providers gave stickered books to parents at each wellness visit, taking care to point out the QR code. Providers and staff were instructed to refer to the poster on how to use the code and encourage the use of the wellness website. Parents could use their smartphones to take a picture of the QR code that led them to the wellness website, where they could view the content that was appropriate for their child’s age. Parental surveys were also available on the website to find out how they felt about the website and what they learned. Although this leg of the project is now complete, the parents still have the books with the QR codes, meaning that they will always have the information available to them. The website analytics and survey results were collected and analyzed to determine the significance and impact of this project. This project had three Plan, Do, Study, and Act (PDSA) cycles during the 2020-2021 implementation. However, the organization plans to continue the project, so more cycles will begin with other students and the organization after this initial pilot of the project. A preliminary 27 review of the data helped determine project improvements that need to occur and will help plan the next cycle of the PDSA model for the next DNP cohort who will take over the project. Timeline In August 2020, students, DNP faculty, and the ROR site champion determined the components of the project and gathered the team members needed for project creation and implementation. In September 2020, students created content for the website, designed a sample website for ROR to utilize and reference when building the live website, and determined the cost and size of stickers. In early to mid-October 2020, Suzanne Metcalf worked with students to design the website based on the dummy website. Students also created the QR code sticker after the website Uniform Resource Locator (URL) was determined and had stickers printed. In addition, during this time, students also introduced themselves to local SmartStart members and key members of the clinic team to promote the project and plan when to sticker books and how to best partner with their local clinic. In November, ROR launched the website, and clinics began giving out stickered ROR books to parents. Data collection started at this phase of the program. During December 2020, automatic e-mails, created before the students’ break, went out to clinic staff every two weeks. The purpose of these e-mails was to thank them for participating in the program and highlight various areas of the site to continue promoting the website during the break. Following the break, students contacted the clinics to let them know that they were available to help support the project and answer any questions. In January 2021, the first data set was collected from ROR and analyzed by students. Plans for project modifications were discussed with the site champion and other group members. Throughout the spring semester (January to March 2021), data collection and analysis continued, beginning the study portion of the project. Utilizing the data found from November 28 through February, a new PDSA cycle began in early March 2021. Students initiated various interventions at this time to combat limitations identified during the initial PDSA cycle. Students took snacks and notes of appreciation to their clinics to help promote the project within the clinics. They also asked staff what barriers they had noticed with the project implementation so far. The biggest concern during this time was that one clinic had a large population of patients that spoke Burmese. Therefore, a QR code that led to the health literacy website already translated into Burmese was generated, and bookmarks were made to give to these patients, explaining the information found on the website and how to use the QR code. In late March, students received more feedback from other clinics. They began the third PDSA cycle by generating another similar language-specific QR code, but for Spanish. Additionally, they created simple bookmarks for both English and Spanish patients with their respective QR codes. Students gave the bookmarks to clinics to provide to their patients. The last day of April 2021 was the final day of data collection. The project evaluation, data analysis, recommendations for improvements or changes, and the project write-up took place from April 2021 to July 2021. Project findings and impact were disseminated to the university through a poster presentation. Preliminary findings were provided to ROR and their partners during a Zoom presentation at the April 2021 Annual ROR Plenary Session. The final findings will be highlighted in the July 2021 issue of ROR’s newsletter, with a complete discussion in the October 2021 issue. Throughout the implementation process, the site champion and students communicated every 2-3 weeks. Students and clinics communicated via e-mail or phone call every 2-3 weeks. The project findings and impact will be shared with clinics via e-mail upon completion of the 29 project write-up. For a visual representation of the project timeline, please refer to Table A3 in the Appendix. Section IV. Results and Findings Results There were four clinics involved in this project located throughout the state of North Carolina. Each student in the group was involved with project implementation at a different clinic site. Overall, there were 2,303 well-child visits between the four clinics for children ages six months to five years with 2,291 books provided to patients at these visits resulting in a 99.5% book distribution rate. 85% of books given out were in English and the remainder were provided in Spanish, as the main language of patients in the clinic was English (68.75%). Other patient languages included Spanish (22.25%), Burmese (3.5%), Karenic (1.75%), Arabic (0.75%), Vietnamese (0.25%), Kinyarwanda (0.25%), and other unspecified languages (2.5%). The majority of patients used Medicaid (85.8%) as their primary form of payment. There were 130 well-child visits between ages six months to five years specific to the clinic where this student worked (Clinic A). They had a 100% book distribution rate. This site gave out primarily English books, with only 5% of Spanish books being given, and the majority of the patients were White. Medicaid was the primary form of payment utilized in this clinic. ROR collects data from their clinics in six month intervals, collected in January and July. Due to the fact that the July survey has not been conducted, the number of well-child visits, book distribution rate, and demographics were determined for the clinics utilizing data collected from the January 2021 Reach Out and Read Annual Survey. Therefore, the demographic data provided shows a good representation of the people served overall by the clinics but may not provide a 30 complete picture of the patients served during the implementation of this project. Refer to Tables B1, B2, B3, and B4 for individual clinic and demographic information. The health literacy website had a total view of 236 unique views over the course of the implementation period. Of those views, 151 occurred on the main splashpage, and the remaining 85 occurred on the individual pages (0-6 months: 24, 6-12 months: 11, 1-2 years: 18, 2-5 years: 14, Vaccine FAQ: 12, and Oral Health FAQ: 6). Some visitors viewed various pages multiple times, increasing the overall website traffic. Over the course of the implementation, the various website pages were viewed an additional 58 times by repeat visitors (Splashpage: 33, 0-6 months: 6, 6-12 months: 5, 1-2 years: 9, 2-5 years: 1, Vaccine FAQ: 4, and Oral Health FAQ: 0). Therefore, the total overall views during project implementation, including repeat visitors, were 294 website views. The website was visited an average of 1.6 times per day over the course of the project implementation. However, it is important to note that some of the views from November may have been from students and staff involved in the implementation of the project, as minor website adjustments were still being made at this time. Refer to Table C1 and Figures C1, C2, and C3 for website analytics and data. Parents also had the option to respond to a survey about the website while visiting the health literacy site. There were 10 surveys collected overall. However, only 8 of these surveys were during the implementation period (n=8). The other two surveys were test surveys performed by the students in October to make sure they worked correctly. Refer to Table D1 for complete parental survey responses and Table D2 for analysis of these responses. All of the parental surveys obtained were done in English. The only collected Spanish survey was a test survey done by students prior to project implementation. The majority of the surveys were done later in the implementation project with 62.5% being done in April 31 (November: 1, February: 1, March: 1, April: 5). 25% of surveys were from 6-11 month parents, 25% were from 18-23 month parents, and 25% were from 3-5 year-old parents. 12.5% of surveys were done by parents of children ages 0-5 months, and 12.5% of surveys were done by parents of children ages 24-35 months. No surveys were completed by parents of 12-18 month-old children. Almost all of the parents were told about the QR code at their visit (87.5%) with only one survey indicating that they were not told about the code. When evaluating the ease of the website content, it appears that the majority of the website users that took the survey found the content very easy to understand. One survey participant felt it was only A Little Easy to understand. No parents found the content A Little Hard or Very Hard to understand, and no parents felt the content was Neither Hard Nor Easy to understand. 62.5% of parents indicated they had learned A Lot about both vaccines and oral health. Some learning was reported by parents about vaccines (25%) and oral health (37.5%). Only one parent (12.5%) indicated that only A Little was learned about vaccines from the website. All of the parents who took the survey indicated that they had learned something from the website regarding both vaccinations and oral health. Based on the parental survey responses, all of the parents indicated that they intended to change their child’s oral health habits as a result of what they had learned from the website. Approximately one-third of the parents indicated they would change their plans for their child’s vaccinations because of the information they had learned. It is unclear whether those who did not plan to change their child’s vaccination plans were because they intended not to vaccinate their children or if it is because they already intended to have their child vaccinated per the CDC schedule. 100% of parents indicated they were also planning on sharing the information they had learned with others (75% Very Likely, 25% Somewhat Likely). The survey also had a free text 32 question that requested comments and suggestions regarding the website and QR code. Only one person responded to this question, indicating that they would like to learn more about car seat safety through the website. Refer to Figures D1, D2, D3, D4, and D5 for further parental survey response analysis. Discussion of Major Findings It’s difficult to assess whether the evidence in the literature supported the data collected from this project because there were so few parental surveys collected. However, based on these survey results, it appears that the intervention was successful. The results of the surveys indicate that all three levels of Nutbeam’s Theory of Health Literacy were achieved to at least some degree. Parents indicated that the website was easy to understand and that they learned information about vaccines and oral health. Therefore, parents had the basic skills needed to read, understand, and acquire the website information, resulting in functional health literacy. The majority of parents also indicated they would change their child’s vaccinations based on the information they had learned and all indicated they would change their oral health habits based off of this information, resulting in better oral health and immunization status for their children. The parents also indicated that they were all planning on sharing the information they learned with others. The decision-making process involved with the interpretation of this information, the resulting practice change, and the subsequent sharing of this information is evidence that Nutbeam’s second level of interactive health literacy has been reached, and the third level of critical health literacy is beginning to be reached. The website traffic also supports that the QR code was successful. While there was a decline in website traffic initially during the intervention over the winter break, the traffic picked back up once students returned and also again with targeted interventions. These interventions 33 included QR code bookmarks, specific QR codes leading to translated versions of the website for both Spanish and Burmese, and having language-specific bookmarks for those two languages, which were two of the primary languages in various clinics. There was also repeat traffic on the site, which indicated that patients were viewing the site multiple times. This is a positive thing since patients will likely learn more from repeat views of the same information. Reinforcement of the information will help cement the knowledge for parents. Section V. Interpretation and Implications Costs and Resource Management The actual cost to perform this project overall was low, as most of the content development, design, and labor was done by the students and a non-profit web designer that works for ROR. The website design itself did not cost anything financially, but it did require extensive time on the parts of four different students and ROR’s web designer, which meant they were not able to spend time on other activities or projects because of the time spent on this project. The website content development was done for this project by the four DNP students with two main areas of development: oral health and vaccination information and parental survey design. The total amount of time for website content development was 63 hours. The time the ROR Communications Director spent building the website and incorporating the content was 20 hours. One student helped her with development design and content revision, utilizing 25 hours of her own time for this task. Therefore, the total amount of time spent on the website development was 108 hours. Other aspects of the project development included poster design and QR code design, as well as the printing and distribution of these items as well. For the poster design, the students developed the content and designed the poster, which was then redesigned by the same graphic 34 designer who built the website for ROR to match their corporate standards. The poster was also translated into Spanish using the ROR corporate translators. Therefore, no money was spent on the design of the poster itself, although the content, design, and interpretation of the poster took 11 hours, with an additional five hours required for the printing, laminating, and sorting of the posters for each clinic. Therefore, 16 hours total was spent on the posters. There was also a monetary cost for the posters with supplies and at-home printing costing $160.27. QR code design and development into other languages was also done by the students using a free online resource. Due to the use of a free QR code generator, the flexibility of the QR code and the amount of data able to be collected from this QR code was limited. However, it resulted in only an additional time commitment on the part of the student, rather than an extra financial cost to the overall project. The actual time spent designing the QR code, ordering stickers, and dividing the stickers for distribution to the various clinics was eight and a half hours. The stickers were $156.88 for 2,500 1x1 stickers. The application of the stickers was done by the four students and took a total of 31.75 hours. The cost of the books was not considered as a project cost because there was not an additional cost to ROR since they already distribute books with each well-child visit. The last initial cost associated with the project was the time and money it took for one student to distribute the posters and stickers to the other students in their various locations. When costs were considered, this was cheaper than having to order in smaller batches to have them distributed individually to the various clinics directly, especially since students were making the posters and these would have to be distributed anyway. The total number of miles driven were 848 miles. When calculated at a rate of $0.54 per mile, this cost the student $457.92 and a total of 14 hours. 35 Due to some interventions to the project, students also chose to design, print, and distribute some language-specific QR codes via bookmarks. The time associated with this was eight hours for the QR code design, interpretation, and bookmark design for the Burmese- specific QR codes and six hours for Spanish-specific QR codes. After having increased website traffic using this method, students also spent three hours creating English bookmarks. Therefore, a total of 17 hours was spent on bookmark design and 2 hours on printing from home by one student. She spent $32.74 on printing 100 English bookmarks and 100 Spanish bookmarks. The other student had her bookmarks professionally printed, costing $71.74 for 80 bookmarks. Therefore, the total cost of bookmark printing was $104.48. Overall, the total cost in time burden for the overall project was 197.25 hours. The financial cost of this project was $879.55. While this project was more cost effective utilizing students and members that were already employed by ROR to help save money, if a group wanted to do this project utilizing paid labor, the financial burden would be much higher, but would likely require a lot less time on the part of the project manager. For example, instead of 63 hours on website content and 45 hours spent on designing the website, an organization could pay to have a website developer do this instead. However, it would cost approximately $5,500, with about $5,000 spent on website content and design with a recurring cost of $50 for the website domain and a recurring cost of about $450 annually for website hosting (Carney, 2020). ROR did not need to pay for additional domain or hosting costs as they already have a website domain and pay annual hosting costs for this to maintain their organization’s website. Additionally, the cost for QR code design and development may also be more expensive depending on what site the organization uses, what information they want to gather when the QR 36 code is scanned, and whether they want a dynamic QR code that will allow them to change the site linked to it in the future. These are actually very good things that most organizations would likely want to invest in with their QR code when paying for it to be developed, as these will allow the most flexibility if the website URL changes in the future and will also provide more analytics about when parents scan the QR code itself, letting an organization know how many people are actually utilizing the QR code versus manually entering the website itself. Therefore, most organizations can expect to spend about $60 to $72 per year to develop and maintain a QR code (Payment, n.d.; Plans & Pricing, n.d.; Pricing & Plans, n.d.). For the purpose of the proposed budget, the median of $66 was used. However, they will save at least eight and a half hours in the development of the QR code. They would likely spend more than the $156.88 for stickers because the printing for these stickers was done by a family member of one of the students who was able to provide the printing of the stickers at what they cost to print. An organization could expect to spend $263.09 for 2500 1x1 stickers (Custom Roll Labels, n.d.). Additionally, the books themselves cost approximately $3 per book for a total of $6,909 if an organization was not already distributing books. They would still need the man hours for stickering the books, but it could likely be done by providers in the clinic or the SmartStart agencies that work with ROR in these clinics. Additionally, consider either an additional cost in shipping the stickers to various clinics with the time needed to divide the stickers for each clinic the cost in both time and mileage to distribute the stickers and posters. Another design cost to consider is the poster design. Agencies choosing to purchase a promotional poster instead of making their own would save 16 hours of time and the $160.27 in costs for ink, paper, a laminator, and laminating pouches. However, they can expect to spend approximately $248.36 in professional printing (Grand Format Posters, n.d.), $34 in lamination 37 (Copies, Binding, & Lamination, n.d.), and $150 per poster designed (Custom Poster Design Packages Plan & Pricing, n.d.; Pricing Guide, n.d.). Additionally, to translate a poster into another language, such as Spanish, they’ll expect to spend an additional $20 (Spanish Translation Services Prices, n.d.). An organization hoping to have the work done by third-party companies would need to have a third-party company design the English poster with another third-party company translating this to Spanish and then have the first company design the Spanish version of the poster. Therefore, the total cost in design and translation of the poster would be $320 with an additional $248.36 to professionally print and laminate 68 posters, with 34 in Spanish and 34 in English. So, instead of 16 hours of time and $160.27, the organization would spend $602.36 on posters overall. The last design cost to consider is the bookmark design, interpretation, and printing. To have a bookmark designed, organizations can expect to spend $178 for a custom designed English bookmark (Bookmarks, n.d.). However, once the English one has been designed, they can pay for Spanish interpretation of the bookmark and Burmese interpretation of the bookmark costing approximately $4.40 if using the same number of words that the students had on their current bookmarks (Average Rates Charged for Translations, n.d.). Then, they can pay a total of $76 to have the same company design both of the translated bookmarks (Bookmarks, n.d.). Lastly, they will then need to print the bookmarks. Unfortunately, for most large printers, the smallest run of bookmarks is 500 per run, requiring them to print 500 English, 500 Spanish, and 500 Burmese bookmarks if printed professionally for $47 per run (Bookmarks, n.d.). Therefore, they can expect to pay another $141 in printing after the design of the product, totaling $399.40 for a total of 1500 bookmarks. 38 Overall, an organization relying on other companies to design and print the components of this project can expect to spend $13,748.40 initially, with a recurring cost of $500 to maintain the website and $66 annually to maintain a QR code design and tracking service, plus any additional stickers and books needed after the initial ones are used. This results in an annual upkeep of $14,384 to continue to project. Refer to Table E1 for a full budgetary breakdown of the actual DNP project and Table E2 for a proposed budget for other organizations hoping to do the project without the benefit of student and employee time. While almost $14,000 initially and approximately $14,500 may sound like a lot of money for this project, they are actually quite miniscule compared with the overall costs related to poor oral health and unvaccinated children. According to the Centers for Disease Control and Prevention, $136 billion dollars are spent annually related to dental care (Power of Prevention, n.d.). While not all of this money is related to poor oral health, a large portion of it is certainly not being used for prevention, but rather for treatment due to poor oral health habits. Over a 17 year period, ending in 2013, $26.5 billion dollars was spent on dental care for children, and of this, approximately $8 million were spent on non-preventative services (Power of Prevention, n.d.). There is also a large portion of medical costs that occur in the emergency room due to poor oral hygiene. Approximately 2 million adults and children are seen in the emergency room in a year for issues related to poor oral health with Medicaid paying for 70% of these visits for children (Power of Prevention, n.d.). Other costs related to poor oral health include the loss in school hours for children and the loss of productivity by parents having to take children to the doctor for emergency dental care. The CDC notes that untreated oral disease from poor oral health results in the loss of over $45 billion dollars of productivity and over 34 million school hours (Power of Prevention, n.d.). 39 Additionally, poor vaccination rates and unvaccinated children and adults increase healthcare costs in the United States, as well as contribute to poor healthcare outcomes. Those patients who contract diphtheria, tetanus, and congenital rubella syndrome (CRS) have a 100% probability of requiring hospitalization (Vaccines for Children Program, n.d.). The costs of these hospitalizations range from approximately $9,000 for cataract surgery related to CRS to over $100,000 for a child admitted for tetanus (Vaccines for Children Program, n.d.). The average cost of a hospitalization for a child admitted with diphtheria is over $15,000 (Vaccines for Children Program, n.d.). Other vaccine preventable diseases (VPD) have a high probability for hospitalization as well and range in cost from $3,195 to $50,554 per hospitalization (Vaccines for Children Program, n.d.). While hospitalization for these diseases is costly, there are also costs related to outpatient care after release from the hospital and in the case of diseases like CRS, children can have lifelong complications requiring multiple cardiac surgeries and vision problems (Vaccines for Children Program, n.d.). Other costs include loss of work hours from parents who must stay with hospitalized children or coordinate outpatient follow-up and care, as well as loss of productivity related to premature death of individuals (Vaccines for Children Program, n.d.). Most VPD have high rates of mortality and cause poor quality of life for both children and their parents. The annual and lifelong monetary costs of both vaccine preventable diseases and poor oral health make the cost of this project worth it, as does preventing loss of life and increasing the quality of life for these children and their parents. This project is a simple, fairly cost effect intervention to help decrease healthcare costs and increase the length and quality of life for children and adults. With the benefit of students working the project, the cost is even more affordable. However, even without that benefit, there 40 are also ways an organization can help reduce the overall cost of this project to make it sustainable, particularly if they know they will be running it for longer than six months. The first thing to take into consideration is that resources are cheaper when scaled to target a larger number of parents, whether that means more clinics participate or the project is long-running. Poster and bookmark design would both be one-time costs. Therefore, whether an organization targets one parent or a thousand, the costs are the same, however, when targeting more parents, the cost of designing the poster and bookmarks is more justifiable. Also, printing of posters, QR code stickers, or bookmarks becomes cheaper per unit with the more product that is bought. Therefore, if a group utilizes this project in more clinics or plans to use it long term and will use the supplies over a series of years, buying products like stickers and bookmarks becomes cheaper. However, it is unlikely that an organization will need a large amount of posters with extending the timeline of the project alone, as they tend to hold up well when laminated and shouldn’t need to be replaced often. Therefore, the only way to save costs per poster printed is to buy them in larger quantities for multiple clinics to use. Organizations also save money by keeping the project running for an extended time period because they don’t need to replicate the cost of the website design which is by far the most significant cost in this project. There would still be the cost of website domain and hosting costs; however, if organizations tie this into their own websites that they already have, then they are saving costs by not having to duplicate hosting and domain costs for this project. Also, the QR code is reusable, particularly if a group purchases a dynamic QR code plan that allows them to make changes to the associated URL and content without having to change the physical QR code itself. Therefore, one can order larger batches of stickers without worrying that the sticker will be useless in a year if the URL for their site changes. 41 However, if an organization is fairly certain that they will not be changing the URL and don’t need to track scans, there are free websites that allow for the QR code design and use. It’s important to make sure that they determine that the site they pick will allow QR codes to be generated that don’t have an expiration date. Organizations should choose a free QR code generator that allows codes to remain active forever to get the most benefit from their QR code design and printed stickers. However, again, it’s important to be sure that an organization does not have any plans to change the URL associated with this QR code or make sure to print QR codes in small batches if they do plan to change the URL to avoid having a large amount of stickers that are rendered useless by a URL change. Lastly, it appears that bookmarks with the QR code were as effective, if not more effective in generating website traffic since site analytics increased after the introduction of language-specific QR code bookmarks. There are advantages and disadvantages to both approaches. However, if financial cost is playing a factor, the bookmarks are much cheaper to print than the stickers, so if they have someone willing to design the bookmarks, this can be a cheaper alternative than printing stickers. An organization can also decrease the cost of design by doing a basic design themselves and having the company put it in print format for them for $38 per design or by creating the content themselves and getting only a basic redesign for $98 (Bookmarks, n.d.). One advantage of the bookmarks is that they can help clinics reach patients who speak other languages, increasing health equity and helping to target health literacy in more diverse populations. They are also good because they help draw parents’ eyes to the intervention since they are sticking out of the book. Bookmarks also don’t require any clinic staff or ROR staff to spend extra time putting a sticker on the books, and they still go along with ROR’s mission as 42 they are book-related. Also, once the initial design is made, it would be easy for someone to change out the QR code only on the bookmark, if needed, without requiring an entire redesign. Therefore, free static QR codes can be utilized by an agency since they can easily change the QR code and print new bookmarks to give to clinics since they’re not attached to the books themselves. However, there are also drawbacks to bookmarks over stickers. Disadvantages include the fact that they can be lost very easily by parents, or they may be forgotten to be given out by clinic staff. For ROR, the cost of continuing this project is very small. The website, QR code, poster, and bookmark design have already been done by the students, so unless they want significant changes made, the largest expenses for this project have already been done. They will have to pay for website hosting and domain services, but since it’s tied to their main website, this is not a new cost for them, but rather tied to a cost they already had. So, essentially this portion of the project also has no cost associated with it. As mentioned above, the clinics that already have posters should not need new ones for a while, since they are laminated and should hold up well, diminishing the need for posters in these clinics. However, if ROR wants to include new clinics, they will have to print posters for these clinics, but by rolling it out on a larger scale, they’ll save money by printing in bulk. The biggest recurring cost would be QR code sticker printing or bookmark printing. They should not necessarily need to continue both interventions but would ideally pick the intervention that worked the best, which would be the language-specific bookmark based on how website traffic increased after utilizing these. Implications of the Findings Overall, this DNP project has been a success. The data collected from the website analytics, the responses from parent surveys, and the interest from other organizations in 43 extending or replicating this project all support its implementation. While there were only eight parental surveys over the course of the intervention, these indicated positive results overall. Parents reported learning and understanding based on the surveys conducted, resulting in increased health literacy. The majority of parents also indicated that they would implement changes in both future vaccination of their child and in their child’s oral health practices. The fact that they learned from the website and wanted to make changes, further supports this project because the change in their practices should help support those key goals of Health People 2020 in increasing vaccination rates, as well as increasing preventative oral health for children. The changes that these parents make in their child’s vaccine and oral health will go on to have impacts in the lifelong health of these children, hopefully improving their quality of life and decreasing VPD. As discussed previously, parents expressed wishes to share this data with others which is part of transitioning from the second to the third level of Nutbeam’s Theory of Health Literacy. When parents share this information with others, it opens the doors of communication about health care topics and makes it more acceptable to discuss these topics with family, friends, and healthcare providers. It allows parents to work through questions they may have and formulate these questions to bring up to providers in the office. Also, young parents often trust the advice of their friends and family, so when people share well-researched, evidence-based information with their friends and family, they are helping to increase others’ health literacy on these topics. This is another reason why the QR code method of taking patients to the website is important because it allows them to quickly access it, and they are able to easily share it from their phones with friends and family. 44 The intervention was also supported by the fact that other organizations and practices found the intervention valuable and helpful. A nearby metropolitan area AHEC learned of the project, found the idea exciting, and decided to utilize the project’s current QR code and posters to promote the QR code in all of their area clinics starting in May 2021. They hope to help increase vaccine health literacy in their area. Another non-profit organization learned of the project and found the idea intriguing. They have created their own QR code and resources to help promote oral health for their agency. Furthermore, after participating in the ROR Annual Plenary Session in April 2021, multiple clinics across the state heard of this project and have become interested in finding out how to roll it out in their areas. The interest in this project and the fact that multiple organizations across the state are rolling it out using this project’s QR code or creating their own version of the same intervention are also indicators that this project was successful. Finally, while the website traffic was not major, it did increase in relation to interventions provided in the clinics and was on par with what was likely to be expected with launching a new website. Also, the project only took place in four clinics throughout the state and took place during a time where a global pandemic was ongoing, making it difficult to promote the project. However, there was still a good response despite these limitations, supporting that parents were using the intervention and interested in the project. Implications for Patients This project has many implications. The first implication is that parents’ health literacy can be impacted by a simple QR code that leads to an educational website. Not only is parental level of health literacy impacted, but parents are willing to make changes to their health choices for their children based off the information given. This will thereby impact the oral health and 45 overall health of these children. Parents indicated that they will make changes to their child’s oral health routine. Hopefully, by improving home oral health and promoting preventative dental care: oral health outcomes for children will improve; dental caries, oral diseases, and uncomfortable or painful oral lesions will decrease; children’s overall quality of life will improve; healthcare costs related to emergency room visits and treatment of oral diseases will decrease; and loss of productivity related to children missing school and parents missing work to provide emergency oral health care for their children will decrease. The majority of parents indicated that they would make changes in how they choose to vaccinate their child based off the information that they learned. By making changes, these parents will help improve vaccination rates, protecting their own child from these diseases, but also increasing herd immunity and helping to protect other children who may not be able to get these vaccinations due to a health condition or whose parents may not choose to vaccinate due to poor health literacy or other factors. This will help impact overall health and decrease healthcare costs related to VPD both for their child and potentially others. By vaccinating their children at a young age, parents will also make the practice more of a norm for these children, which will likely result in them continuing to get vaccinated throughout life and hopefully, vaccinating their own children. Lastly, parents indicated that they would be willing to share the information they learned with others. By sharing evidence-based information, parents can help to stop the spread of misinformation about vaccines and help make vaccinating a norm in their community. They can help promote good child oral health habits in their families and friends and promote early dental visits, making prevention a priority. The fact that these parents are willing to share the information with others means that they learned enough from it that they feel comfortable 46 sharing it and potentially having to answer questions about it, which means that they did indeed learn from their time on the website and have improved their health literacy. Implications for nursing practice. With healthcare costs rising, having limited time at office visits, and having even more limited face-to-face time with parents due to ongoing pandemic, finding creative ways to reach out to parents and patients to increase their education and health literacy is imperative. Nurses, providers, and even non-profit organizations can go a long way towards impacting health literacy. By implementing a simple QR code, ROR has improved health literacy which will impact health in these communities. By simply changing a small practice, like giving out a bookmark or drawing a parent’s notice to the QR code on the back of a ROR book, nurses and practitioners can provide parents and other patients, as well, with simple resources that are easily accessible. An educational website related to the content that nurse practitioners hope to improve is a good resource for parents, but other resources could be CDC handouts or websites or other reputable sources for parents and patients to use. Patients are starving for information which is often why they tend to look up things on the internet. Many of them do not even realize that information from websites through search engines like Google do not always provide correct information. By providing them with evidence-based websites that explain the information they are seeking in understandable laymen’s terms, they are able to fulfill that need for knowledge, but with correct information. Promoting projects like this is key to making sure the nursing field is increasing health literacy and promoting health among patients. Projects with QR codes like this require nurses and office staff to help notify patients of the QR code and what they can learn from it, so making sure that the office staff has buy in to the project is essential. Nurses and practitioners should also be open 47 and willing to discuss the information that parents receive from these QR codes. By openly dialoguing, patients and providers are able to talk through any troublesome or unclear points, and providers are able to address and assuage any fears or concerns those patients may have which also helps to improve their health literacy. Impact for Healthcare System QR codes are a simple, cost-effective way to promote health information. When utilizing the QR code alone, without worrying about printing stickers or bookmarks, it’s a very cost effective intervention to help promote health literacy. The process of getting QR codes to patients through the healthcare system is easy and can be added to discharge summaries that are given to patients in hospitals or offices. However, it’s important that discharge providers notify patients of the presence of QR codes as these discharge summaries can be quite lengthy at times. While the material linked to the QR code in this intervention was a custom-made website tailored to the information that this organization wished to promote, other organizations that don’t have the financial capabilities or time to build a website should consider also using QR codes to help connect patients to reputable sources of evidence-based information regarding their topic of choice. This helps promote informed learning and prevent the spread of misinformation. It is also important to make sure that the information that it links patients to is understandable. If an organization is having trouble finding a reputable website that is understandable for most patients, providers or healthcare systems may choose to create a Portable Document Format (PDF) that has basic, understandable information. These can also be linked to QR codes. By taking the time to incorporate this simple intervention, healthcare systems can expect to increase health literacy. By focusing on topics of prevention, healthcare systems can help increase good preventative healthcare, reducing disease and thereby reducing their healthcare 48 costs. This is important because reducing healthcare expenditure is a big topic today and most insurance companies, Medicare, and Medicaid expect healthcare organizations to decrease their costs as much as possible. Hospitals and systems often get reimburse more for their care when they prove that they are helping to reduce their healthcare expenditure. Sustainability This project is simple and sustainable. After the initial cost of the website, the cost of the program decreases significantly. ROR can and will continue this project because of its sustainability. They hope to continue this project by rolling out this project in a major metropolitan area in both their clinics as well as all of the area AHEC clinics. They also have had a lot of interest in the program in various clinics statewide after the project was presented at the ROR Plenary Session in April 2021. As the QR code does not expire, they can continue using the stickers created last semester into the future. They also can continue using the bookmarks provided to the clinics. The printing of stickers and/or bookmarks will still cost money and the task of stickering books will have to be added to either clinic staff or ROR staff members or their partners, but this activity is easy to do and can be incorporated into the current process without much effort. Due to the success of the bookmarks and the impact on the website traffic after their introduction, as well as their versatility in being able to reach patients of varying languages, it may be worth switching from stickers to bookmarks, especially if there is a large population of patients that speak various languages since this allows staff members to provide a language- appropriate QR code to these patients. Also, when the design component can be performed by someone at the organization, this greatly reduces the cost of the program compared to stickers. 49 However, in clinics that have a lot of handouts or paperwork, a bookmark is more likely to get lost and the stickers may be a better alternative. Dissemination Plan This project and its findings will be disseminated in several different locations. The first place that project and its preliminary findings were disseminated was with the ROR Plenary Session on April 23, 2021. The students presented their project to members of ROR and the various clinics, healthcare providers, and organizations throughout the region that participated in this summit. There was a large amount of interest at the summit from providers and organizations, which has led to ROR expanding the project into these areas. The ROR site champion for this project has already been discussing this project with other organizations as well. This has led to the project being expanded into a major metropolitan area AHEC, as well as another non-profit organization replicating this project to help educate their clients about good oral health habits for children. This project was also presented on July 13, 2021 to ECU faculty and staff in a poster presentation that disseminated the project and its findings to both ECU faculty and students. Lastly, the information and findings were written up in this paper and uploaded to ECU’s scholarly repository The ScholarShip on July 27, 2021. This project and its findings were also posted in the July issue of ROR’s quarterly e-newsletter Medical Connections with a follow-up linking readers to the student write-ups to occur in the October 2021 issue. Section VI. Conclusion Limitations and Facilitators During the planning and implementation of this project, there have been many limiting factors, as well as facilitating factors. The biggest limiting factor for this project was the 50 COVID-19 pandemic. There were many facets related to the pandemic that affected the students’ ability to implement the project. Clinic visitor restrictions made it difficult to promote the project since students were not allowed to visit the actual clinic sites, so they had to rely on staff to promote the project. This created another issue as staff were already starting to experience heavier loads and more responsibilities due to new COVID-19 guidelines, making it more difficult for them to have the time or motivation to promote the project with patients. Also, decreased time inside of the office and in waiting rooms meant that patients were not in the facility with the posters for as long as they normally would be. Therefore, if providers didn’t mention the new QR code, parents may have missed it on the book itself or not been aware of what it was for. The varying infection control guidelines with different clinics made it difficult to have only one intervention to help promote the project. For example, initially laminated posters were approved in all of the clinics. However, over the course of the project implementation, clinic rules changed, and one clinic (Clinic C) required that any laminated posters be removed, which included the promotional posters for this project. They were only allowed to give out handouts, so then finding a quick, cost-effective way to get the information to patients in this clinic became necessary. Therefore, the English bookmarks were made to try to combat this setback. Additionally, two clinics had populations of patients that predominantly spoke other languages. The first clinic was a large clinic in central NC (Clinic C) with a large Spanish- speaking population. Students had anticipated having to offer information in Spanish, so they created resources in Spanish and the site could be changed into Spanish using Google or by changing phone settings to display pages in Spanish. However, students did not anticipate the technical difficulties that this population would have in figuring out how to translate the page 51 and the difficulty that staff would have in explaining the process. This dissuaded the clinic staff from promoting the project with these patients. Once students realized this was happening, a QR code that took parents directly to a Spanish-translated version of the website was created and provided to patients via bookmark, since it was too late in the implementation stage of the project to get more stickers in a timely manner. The other clinic reporting language barriers was a mid-sized clinic in rural eastern NC (Clinic B) who had a largely underserved client base with a large population of Burmese speaking patients. This had not been anticipated prior to implementation causing this group of patients to be missed with promotional materials. Additionally, many of these individuals were unaware of how to switch the language of the website they were viewing. Therefore, students created another language-specific QR code that would take the viewer directly to the Burmese- translated version of the website. These QR codes were put on bookmarks and posters in Burmese with brief instructions explaining how to use the QR code and what information they could find when scanning the QR code. While the students were able to find ways to overcome a lot of the barriers that came about related to pandemic restrictions and language barriers, it made it difficult to track which pieces of the intervention were making the most impact in the project because there were so many changing variables in the various clinics. There was also a lack of interest in filling out the surveys. While it is not unusual to get few survey responses with the launch of a new website, without students and staff promoting the survey or any incentive to fill out the survey, the amount of data collected from this tool was less than ideal. Additionally, by trying to limit the length of the survey, the data collected was limited. Information about whether parents found the QR code sticker or bookmark more helpful would have helped ROR and future students know which intervention was more beneficial. Also, 52 knowing how easy parents found the QR code to use, may have been good information to collect as well. Lastly, finding out if parents were willing to discuss this information with providers in the future would have helped evaluate whether this project really helped facilitate dialogues with providers about the issues that were being discussed. While the project had many limitations, there were also some good facilitators. First of all, having an invested site champion was key to this project. She helped to determine which clinics would be involved in the project and then facilitated introductions to both local SmartStart agencies, as well as the clinics themselves, paving the way for students. Her frequent communication and availability helped students to be creative in interventions while still meeting the organization’s goals and guidelines. She also was very involved in promoting the project with other agencies that work with ROR, as well as within the ROR Plenary Session, which has helped to propel the project forward for the future. Another facilitator was the fact that this project had the ability to be versatile and incorporate various interventions as needed to help combat the arising challenges. The use of language-specific QR code bookmarks was key to reaching many people in this project. Also, having a group that worked well together, communicated well with each other, and helped brainstorm ideas and interventions to combat limitations was essential to the success of this project. The use of multiple clinics throughout the state also helped to generate interest in the project when it was mentioned at the Plenary Session. Recommendations for Others As this project continues forward, it is important to examine many of the limitations and facilitators experienced within this project and find creative ways to minimize constraints for the future of this project and for the implementation of any similar projects. First of all, finding ways 53 to work within the limitations experienced during the COVID-19 pandemic is key. The pandemic is not going to dissipate overnight, so making sure that staff are staying motivated with a phone call or e-mail every few weeks is important. If staff and ROR will allow for it, the presence of students in clinic to work with parents using the QR code would also be beneficial to this program. Therefore, students should get vaccinated against COVID-19 to help increase the chances of clinics, the university, and ROR allowing them to be on site. An additional recommendation would be for students to contact their site to find out their specific infection guidelines at this time related to posters or handouts. Then they can plan prior to implementation how to best reach these parents with information that promotes the project. To help combat limited time in exam rooms, providing a bookmark or printed poster for patients to read before their visit while they are waiting in their car or even a large banner or lawn placard with the QR code that parents can scan outside the clinic would be beneficial. This can help promote the project and give parents time to look at the information prior to their visit, which may open dialogues with their child’s provider. Clinics can also promote the project via Facebook or other social media. Students can also incentivize staff by bringing treats to their clinics to say thank you for helping in this project. There may also be some benefit to offering a pizza party or some other type of incentive for the clinic that gets the most views. However, this would require the organization to use a QR code generator that provides tracking of the QR codes scanned and then printing specific QR codes for each clinic to track which ones are scanned. Another option would be to offer it to the clinic that gets the most surveys, as patients will have to visit the site before they can get to the survey. A question would need to be added to the survey asking parents which clinic their child was seen in for their visit. However, this may be a good motivation for clinic 54 staff to work with parents to promote the site, as well as the survey which provides a lot of feedback about what was learned from the site. Other ways to promote interest in the survey would be to help find ways to motivate parents to take the survey. This may be in the form of a prize or drawing for taking the survey. This does create the issue of having to find a way to identify the person in order to do the drawing. However, the survey could generate a specific number that is linked to the parents survey and winning numbers could be posted online when the drawing is done. This would help with deidentification of parents but would involve more technology work. Another thing that could also be considered would be potentially partnering with an agency or company that may offer a discount or free item. For example, after taking the survey, the next page could link parents to a coupon for a free ice cream cone at a local fast food restaurant. However, this type of incentive would require companies to partner with students to provide this incentive. Another thing for future students and others implementing similar projects to consider is the language barriers that may be present. Having bookmarks with QR codes that lead directly to the translated version of the website helped increase the website traffic and it provided greater health equity among other races and ethnicities. Developing language-specific QR codes and promotional material for commonly used languages, such as Spanish, from the beginning would increase the success of the project. Students would benefit from contacting their clinics early in the process to find out what are the primary three languages spoken by patients and anticipate generating QR codes and promotional material for these languages. Students should also determine if they want to use a standard English QR code on the back of the books for all books and language-specific bookmarks for those patients who do not speak English or if they want to use bookmarks only for all of the languages. Planning this in advance will help to create a clear 55 process for clinics, as well as limit the amount of time and money spent on resources initially because there is a clear method of intervention. With these recommendations, the project is easily sustainable and can be scaled to reach a large number of people in various clinics. The website and posters have already been designed, so a large financial aspect of the project has been completed. Additionally, the project can be easily rolled out to other clinics using the bookmark or sticker approach even without a student at the site, since it will just require staff notifying patients of the project and the QR code. In fact, students could even transition to more of a program development role by going to clinics to educate staff on the program, its benefits, and how to implement it. Most of the local SmartStart agencies already sticker the ROR books with labels for those agencies that help to provide the books, so adding another sticker to the books would likely be an easy intervention. Furthermore, the website can be expanded to include additional content and topics. One topic mentioned in the surveys was car seats. This might include car seat safety, how to find out if the car seat is properly installed, and what seat is best for what age or size child. This is a great topic to start with when deciding to expand the pages since it can help save children’s lives in motor vehicle accidents and is often confusing to parents. Finding ways to make the information easy to understand and easily accessible would be a great way to address this knowledge gap and begin scaling the website. The program is also very simple and sustainable for organizations. By linking the website to ROR’s main regional website, there are no additional website costs or fees that they were not already paying, making this affordable to continue using. The main cost would be in printing the stickers and/or bookmarks, as well as in subscribing to a QR code service that can help with 56 tracking of the QR codes if more information is desired about the actual QR code scanning, and in any translation services, poster design, and printing needed for promotional materials. However, as mentioned earlier, the larger quantity of stickers or posters ordered, the cheaper they become, so by scaling the project, while the overall costs may increase, the per unit cost decreases. Additionally, these costs can also be mitigated by proper networking, as some print larger print companies may even donate printing to nonprofits for a tax credit. The other component of the project would be to find a way to incorporate process change into book stickering and project promotion to find a way to make this a natural part of a well-child visit. This project can have a great systemic impact if scaled and used correctly. While small, the data suggests that parents do learn from the website and will make changes to their child’s health habits by incorporating what they’ve learned. It also indicates that they will share what they’ve learned with others, helping to spread correct information to their friends and families. Therefore, it is clear that health literacy can be impacted through this project. By impacting education and health literacy on vaccinations, vaccine compliance will increase, and there will be a decrease in vaccine preventable illness which will impact overall child health. These changes can help decrease the healthcare expenditure related to these diseases, prevent vaccine-related sequelae, and improve the quality of life of these children. The same is true for oral health. The change in how parents take care of their children’s teeth and how they teach them to take care of their teeth in early childhood will impact these children for life. The increase in toothbrushing and routine preventative dental visits will decrease the rates of poor oral hygiene for these children, and possibly for the parents as well. The lasting societal impact from increasing parental oral health literacy will be overall increased 57 oral health of children and improved quality of life with less pain, loss of school and work hours, and less healthcare expense related to poor oral health. Recommendations for Further Study Ideally, in the future, this program will be rolled out across the state in all ROR clinics. With the increase in clinics interested in this project and a regional AHEC piloting this program in their area, it may spread across the state quickly if these clinics find it successful. If it is successful within the state, this program would likely be very beneficial to roll out with ROR nationally. This project is simple to do and opens the door for allowing a variety of topics to be discussed. This could help increase parental health literacy on multiple subjects and improve child health in many avenues. Other programs besides ROR can use the same concept of this project to boost their parental health literacy on issues specific to them, including car seat safety, nutrition, physical activity, and developmental milestones. Other topics that are goals for Healthy People 2030, include early vision screenings, sleep and sleep hygiene, stress and anxiety awareness and reduction techniques, decreasing iron deficiency anemia, decreasing childhood obesity, and decreasing emergency room visits for children with asthma (Office of Disease Prevention and Health Promotion [ODPHP], n.d.-a). These are all topics that would be good for expanding the program. After talking with this project’s site champion, one non-profit organization has already planned to implement a similar project utilizing QR codes that leads to their organizations website with educational materials dedicated to pediatric oral health. Important aspects to remember when generating content for new topics is to make sure that the information stays on a low grade level for reading and comprehension. The content should be clear and simple to understand by most basic readers. Also, using a language-specific 58 QR code for clinics with high levels of patients who speak certain languages is important to the success of the continuation of this project, as well as any new rollouts of similar projects. This will help ensure that the projects provide better health equity and reach the broadest audience. Final Thoughts QR codes can be a simple, helpful addition to ROR books to help increase parental literacy on immunizations and oral health. By adding language-specific QR codes that lead parents to an educational website, custom-designed for their learning regarding childhood vaccinations and oral health, they can increase their health literacy, leading to better vaccination rates and improved oral health. This project has had many views of the website, but unfortunately it didn’t have an overwhelming survey response. Therefore, drawing conclusions as to the efficacy of this intervention based on surveys alone is limited due to the sample size. Incentivizing surveys and finding creative ways to combat COVID-related restrictions can improve the project going forward. The project can be scaled to include other topics and spread to other areas or populations. One nonprofit organization and another metropolitan area AHEC are already adapting or expanding this project in the coming months to help promote oral health and vaccines, respectively. 59 References Average Rates Charged for Translations. n.d. ProZ.com. Retrieved May 18, 2021 from https://search.proz.com/employers/rates Azevado, J., Padrão, P., Gregório, M. J., Almeida, C., Moutinho, N., Lien, N., & Barros, R. (2019). A web-based gamification program to improve nutrition literacy in families of 3 to 5-year-old children: The Nutriscience Project. Journal of Nutrition Education & Behavior, 51(3), 326-334. doi: 10.1016/j.neb.2018.10.008 Baskaradoss, J. K. (2018). Relationship between oral health literacy and oral health status. BMC Oral Health, 18. doi: 10.1186/s12903-018-0640-1 Bayley, J.E., & Brown, K. E. (2015). Translating group programmes into online formats: Establishing the acceptability of parents’ sex and relationships communication serious game. BMC Public Health, 15. doi: 10.1186/s12889-015-2545-0 Bookmarks. n.d. Willy Walt, Inc. Retrieved May 18, 2021 from https://www.willywalt.com/ products/bookmark-printing.html Carney, L. (2020, December 1). How Much Does a Website Cost? WebsiteBuilderExpert. Retrieved April 10, 2021 from https://www.websitebuilderexpert.com/building- websites/how-much-should-a-website-cost/#:~:text=How%20Much%20Does% 20Building%20a,cost%20of%20%241%2C000%20per%20year. Chu, J. T. W., Wadham, A., Jiang, Y., Whittaker, R., Stasiak, K., Shepherd, M., & Bullen, C. (2019). Effect of MyTeen SMS – based mobile intervention for parents of adolescents: A randomized clinical trial. JAMA Network Open, 2(9). doi: 10.1001/jamanetworkopen.2019.11120 60 Copies, Binding, & Lamination. (n.d.). Studio “R” Print. Retrieved May 18, 2021 from https://studiorprint.com/copies-binding-lamination Cowden, J. D., Wilkerson-Amendell, S., Weathers, L., Gonzalez, E. D., Dinakar, C., Westbrook, D. H., & Williams, A. R. (2015). The talking card: A randomized controlled trial of a novel audio-recording tool for asthma control. Allergy Asthma Procedures, 36(5), e86-91. doi: 10.2500/aap.2015.36.3881 Crespo, E. (2019). The importance of oral health in immigrant and refugee children. Children, 6(102), p.1-6. doi: 10.3390/children6090102 Cutilli, C. C., Simko, L. C., Colbert, A. M., & Bennett, I. M. (2018). Health literacy, health disparities, and sources of health information in U.S. older adults. Orthopaedic Nursing, 37(1), 54-65. doi: 10.1097/NOR.0000000000000418 Custom Poster Design Packages Plan & Pricing. (n.d.). Artimization. Retrieved May 18, 2021 from https://www.artimization.com/poster-design-packages/ Custom Roll Labels. (n.d.) Vistaprint. Retrieved May 18, 2021 from https://www.vistaprint.com/ marketing-materials/labels-stickers/roll-labels?couponAutoload=1&GP=05%2f19%2f 2021+23%3a24%3a58&GPS=5976837539&GNF=0 DataUSA. (n.d.). Asheboro, NC. https://datausa.io/profile/geo/asheboro-nc/ DeCamp, L. R., Godage, S. K., Araujo, D. V., Cortez, J. D., Wu, L., Psoter, K. J., Quintanilla, K., Rodriguez, T. R., & Polk, S. (2020). A texting intervention in Latino families to reduce E.D. use: A randomized trial. Pediatrics, 145. doi: 10.1542/peds.2019-1405 Downing, K. L., Salmon, J., Hinkley, T., Hnatiuk, J. A., & Hesketh, K. D. (2018). Feasibility and efficacy of a parent-focused delivered intervention to reduce sedentary behavior in 2- to 61 4-year-old children (MiniMovers): Pilot randomized controlled trial. Journal of Medical Internet Research Mhealth and Uhealth, 6(2). doi: 10.2196/mhealth.8573 Dudovitz, R., Teutsch, C., Holt, K., & Herman, A. (2020). Improving parent oral health literacy in Head Start programs. Journal of Public Health Dentistry, 80(2), 150-158. doi: 10.1111/jphd.12361 Forshaw, J., Gerver, S. M., Gill, M., Cooper, E., Manikam, L., & Ward, H. (2017). The global effect of maternal education on complete childhood vaccination: A systematic review and meta-analysis. BMC Infectious Disease, 17. doi: 10.1186/s12879-017-2890-y Gibbs, H. D., Camargo, J., Patton, S., Zoellner, J., Chen, Y., Cupertino, A. P., Harvey, S., Gajewski, B., & Sullivan, D. K. (2018). Preliminary investigation of a mobile nutrition literacy website for parents and young children. Frontiers in Nutrition, 5. doi: 10.3389/fnut.2018.00129 Grand Format Posters. (n.d.). PrintDirtCheap.com. Retrieved May 18, 2021 from https://www.printdirtcheap.com/en/paper-posters-(custom-size)/ Hurley, D., Allen, M.S., Swann, C., Okely, A. D., & Vella, S. A. (2018). The development, pilot, and process evaluation of a parent mental health literacy intervention through community sports clubs. Journal of Child and Family Studies, 27(7), 2149-2160. doi: 10.1007/s10826-018-1071-y Hutton, J. S., Gupta, R., Gruber, R., Berndsen, J., DeWitt, T., Ollberding, N. J., Van Ginkel, J. B., & Ammerman, R. T. (2017). Randomized trial of a children’s book versus brochure for safe sleep knowledge and adherence in a high-risk population. Academy of Pediatrics, 17(8), 879-886. doi: 10.1016/j.acap.2017.04.018 62 Institute for Healthcare Improvement. (n.d.). Science of Improvement: Testing Changes. http://www.ihi.org/resources/Pages/HowtoImprove/ScienceofImprovementTesting Changes.aspx Kutner, M., Greenberg, E., Jin, Y., & Paulsen, C. (2006). The Health Literacy of America’s Adults: Results from the 2003 National Assessment of Adult Literacy. https://nces.ed.gov/pubs2006/ 2006483.pdf Lebrun-Harris, L. A., Canto, M. T., Vodicka, P. (2019). Preventive oral health care use and oral health status among U.S. children: 2016 National survey of children’s health. Journal of American Dental Association, 150(4), p. 246-258. doi: 10.1016/j.adaj.2018.11.023 Lee, H., Oh, J., Heo, J., Abraha, A., Perkins, J. M., Lee, J., Giang Huong Tran, T., & Subramanian, S. V. (2019). Association between maternal literacy and child vaccination in Ethiopia and Southeastern India and the moderating role of health workers: A multilevel regression analysis of the Young Lives Study. Global Health Action, 12. doi: 10.1080/16549716.2019.1581467 Mina, M. J., Kula, T., Leng, Y., Li, M., de Vries, R. D., Knip, M., Siljander, H., Rewers, M., Choy, D. F., Wilson, M. S., Larman, H. B., Nelson, A. N., Griffin, D. E., de Swart, R. L., & Elledge, S. J. (2019). Measles virus infection diminishes preexisting antibodies that offer protection from other pathogens. Science, 366(6465), p. 599-606. doi: 10.1126/sciimmunol.aay6125 Morgan, A. J., Fischer, J. A., Hart, L. M., Kelly, C. M., Kitchener, B. A., Reavley, N. J., Yap, M. B. H., Cvetkovski, S., & Jorm, A. F. (2019). Does mental health first aid training improve the mental health of aid recipients? The training for parents of teenagers randomized controlled trial. BMC Psychiatry, 19. doi: 10.1186/s12888-019-2085-8 63 Morrison, A. K., Glick, A., & Yin, H. S. (2019). Health literacy: Implications for child health. Pediatrics in Review, 40(6), 263-277. doi: 10.1542/pir.2018-0027 Nakamura, D., Ogawa, M., Nakamura, T., & Izawa, K. P. (2018). Impact of parents’ comprehensive health literacy on BMI in children: A multicenter cross-sectional study in Japan. Journal of School Health, 88(12), 910-916. doi: 10.1111/josh.12700 National Institute of Health (NIH). (n.d.). Health Literacy. U.S. National Library of Medicine. https://nnlm.gov/initiatives/topics/health-literacy Nutbeam, D. (2000). Health literacy as a public health goal: A challenge for contemporary health education and communication strategies into the 21st century. Health Promotion International, 15(3), 259-267. doi: 10.1093/heapro/15.3.259 Office of Disease Prevention and Health Promotion (ODPHP). (n.d.-a). Children. Healthy People 2030. U.S. Department of Health and Human Services. https://health.gov/healthypeople/ objectives-and-data/browse-objectives/children Office of Disease Prevention and Health Promotion (ODPHP). (n.d.-b). Health communication and health information technology. Healthy People 2020. U.S. Department of Health and Human Services. https://www.healthypeople.gov/2020/topics-objectives/topic/health- communication-and-health-information-technology/objectives Office of Disease Prevention and Health Promotion (ODPHP). (n.d.-c). Health literacy. Healthy People 2020. U.S. Department of Health and Human Services. https://www.healthypeople.gov/2020/ topics-objectives/topic/social-determinants- health/interventions-resources/health-literacy Office of Disease Prevention and Health Promotion (ODPHP). (n.d.-d). Immunization and infectious diseases. Healthy People 2020. U.S. Department of Health and Human 64 Services. https://www.healthypeople.gov/2020/topics-objectives/topic/immunization-and- infectious-diseases Office of Disease Prevention and Health Promotion (ODPHP). (n.d.-e). Oral health. Healthy People 2020. U.S. Department of Health and Human Services. https://www.healthy people.gov/ 2020/topics-objectives/topic/oral-health/objectives Office of Disease Prevention and Health Promotion (ODPHP). (n.d.-f). Topics and objectives. Healthy People 2020. U.S. Department of Health and Human Services. https://www.healthypeople.gov/ 2020/topics-objectives Otsuka-Ono, H., Hori, N., Ohta, H., Uemura, Y., & Kamibeppu, K. (2019). A childhood immunization education program for parents delivered during late pregnancy and one- month postpartum: A randomized controlled trial. BMC Health Services Research, 19. doi: 10.1186/s12913-019-4622-z Papapchrisanthou, M. M., & Loman, D. G. (2018). Visually enhanced education and immunization perceptions in low-income parents. Public Health Nursing, 35, p. 109-117. doi: 10.1111/ phn.12366 Paules, C. I., Marston, H. D., & Fauci, A. S. (2019). Measles in 2019—Going backward. The New England Journal of Medicine, 380(23), 2185-2187. doi:10.1056/NEJMp1905099 Payment. (n.d.). QR Tiger. Retrieved April 10, 2021 from https://www.qrcode-tiger.com/ payment Peyton, D., Hiscock, H., & Sciberras, E. (2019). Do digital health interventions improve mental health literacy or help-seeking among parents of children aged 2-12 years? A scoping review. Studies in Health Technology and Informatics, 266, 156-161. doi: 10.3233/SHTI190788 65 Plans & Pricing. (n.d.) QR Code Generator. Retrieved April 10, 2021 from https://www.qr-code- generator.com/pricing/ Power of Prevention: Cost-Effectiveness of Oral Disease Interventions. (n.d.). Centers for Disease Control and Prevention. Retrieved May 18, 2021 from https://www.cdc.gov/chronicdisease/programs-impact/pop/pdfs/oral-disease-H.pdf Price List. (n.d.) United States Postal Service. Retrieved May 24, 2021 from https://pe.usps.com/text/dmm300/Notice123.htm#_c031 Pricing & Plans. (n.d.) uQR.me. Retrieved April 10, 2021 from https://uqr.me/qr-code- generator/pricing/ Pricing Guide. (n.d.). designhill. Retrieved May 18, 2021 from https://www.designhill.com/ print-flyer-design Reach Out and Read (ROR). (2020a). Child development. https://reachoutandread.org/why-we- matter/child-development/ Reach Out and Read (ROR). (2020b). Our Vision. https://www.reachoutandread.org/about/ Reach Out and Read (ROR). (2020c). The evidence. https://reachoutandread.org/why-we- matter/the-evidence/ Reach Out and Read (ROR). (2020d). What we do. https://reachoutandread.org/what-we-do/ Reach Out and Read Carolinas (RORC). (2020). Mission, vision, model. https://www.rorcarolinas.org/about-us/vision-mission-model/ Real, F. J., Beck, A. F., DeBlasio, D., Zackoff, M., Henize, A., Xu, Y., Davis, D., Cruse, B., & Klein, M. D. (2019). Dose matters: A smartphone application to improve asthma control among patients at an urban pediatric primary care clinic. Games for Health Journal, 8(5), 357-365. doi: 10.1089/g4h.2019.0011 66 Riggs, E., Kilpatrick, N., Slack-Smith, L., Chadwick, B., Yelland, J., Muthu, M. S., & Gomersall, J. C. (2020). Interventions with pregnant women, new mothers, and other primary caregivers for preventing early childhood caries. Cochrane Database of Systematic Reviews, 2019(11). doi: 10.1002/14651858.CD012155.pub2 Rosas-Blum, E. D., Granados, H. M., Mills, B. W., & Leiner, M. (2018). Comics as a medium for parent health education: Improving understanding of normal 9-month old developmental milestones. Frontiers in Pediatrics, 6. doi: 10.3389/fped.2018.00203 Smith, S. A., & Carroll, L. N. (2017). Data-driven maternal health literacy promotion and a postscript on its implications. Health Literacy, 240, 144-165. doi: 10.3233/978-1-61499- 790-0-144 Spanish Translation Services Prices. (n.d.). GTS Translation Services. Retrieved April 10, 2021 from https://www.gts-translation.com/languages/spanish-translation-services/ United States Census Bureau. (n.d.) Quickfacts: North Carolina. https://www.census.gov/ quickfacts/N.C. U.S. Department of Health and Human Services (USDHHS). (n.d.). Health literacy. Health Resources and Services Administration. https://www.hrsa.gov/about/organization/bureaus/ohe/health-literacy/index.html Vaccines for Children Program. (n.d.). Centers for Disease Control and Prevention. Retrieved April 10, 2021 from https://www.cdc.gov/vaccines/programs/vfc/pubs/methods/ index.html#table2 Yazdani, R., Esfahani, E. N., Kharazifard, M. J. (2018). Relationship of oral health literacy with dental caries and oral health behavior of children and their parents. Journal of Dentistry 67 (Tehran), 15(5), 275-282. Retrieved July 7, 2020, from https://www.ncbi.nlm.nih.gov/pmc/articles/ PMC6397737/ pdf/JOD-15-275.pdf Yin, H. S., Gupta, R. S., Mendelsohn, A. L., Dreyer, B., Van Schaick, L., Brown, C. R., Encalada, K., Sanchez, D. C., Warren, C. M., & Tomopoulos, S. (2017). Use of a low- literacy written action plan to improve parent understanding of pediatric asthma management: A randomized controlled study. Journal of Asthma, 54(9), 919-929. doi: 10.1080/02770903.2016.1277542 68 Appendix A Table 1 Literature Review Log Date of Database Key Word Limits # of Rationale for Inclusion/ Search Searches Citations Exclusion Found/ Kept 7/20/20 Medline interventions Last 5 1057 found kept articles directly related via AND health years, without to clinical question and PubMed literacy English, limits / 612 interventions specific to AND Full after limits / increasing parental health parents Text 16 kept literacy; also disregarded articles whose studies were not completed or were frameworks/protocols for ongoing studies whose interventions had not been assessed or evaluated, disregarded studies done in inpatient settings or those studies related to media literacy, disregarded studies that involved parents <18years of age 7/20/20 Cochrane interventions Last 5 25 found kept articles directly related Database AND health years. without to clinical question and of literacy English, limits / 17 interventions specific to Systematic AND Full after limits / increasing parental health Reviews parents Text 1 kept literacy; also disregarded (EBM articles with interventions reviews aimed at children, with Ovid) disregarded studies that involved parents <18years of age 69 7/20/20 CINAHL interventions Last 5 229 found 8 articles were excluded for Plus with AND health years, without duplication; kept articles Full Text literacy English, limits / 134 directly related to clinical AND Full after limits / question and interventions parents Text 3 kept specific to increasing parental health literacy; disregarded articles whose studies were not completed or were frameworks for ongoing studies whose interventions had not been assessed or evaluated, disregarded studies done in inpatient settings or those studies related to media literacy, disregarded studies that involved parents <18years of age 70 Table 2 Literature Review Matrix 71 72 73 74 75 76 77 78 Table 3 Timeline of the DNP Project August September Early to Mid- Mid- to Late November December 2020 2020 October 2020 October 2020 2020 2020 PDSA Cycle #1 X X X X X X Determine components of the project. X Determine and gather team members needed for the project creation and X implementation. Create content for website. X Design sample website. X Determine cost of stickers for the QR code. X Seek approval from ROR for both QR code and X website content. Create website in conjunction with ROR X team. Design QR code sticker. X Design promotional posters for clinics. X Have promotional posters translated to Spanish. X Create QR code and order stickers. X Create surveys for website. X Get approval for survey content from ROR. X Have survey translated into Spanish. X Finish Website content including survey links and X X graphics. Students will put QR code stickers on the books and X X deliver to clinics. Launch website. X Clinics will hand out books and promote QR code and X X website to parents. 79 Early to Mid- Mid- to Late February Early to Mid- to January 2021 January 2021 2021 Mid-March Late March 2021 2021 PDSA Cycle #1 X X X Clinics will hand out books and promote QR code and website X X X X X to parents. Students will collect their first set of data from ROR. X Weekly data will be collected from ROR and analyzed by X X X X X students. Sticker more books as needed per clinic. X X X X X Support clinics as needed. X X X X X Meet with ROR site champion to discuss the project and data. X X Barriers to project will be discussed and ideas developed X X to combat these barriers. PDSA Cycle #2 X Take snacks/treats and thank you cards to the clinic to help increase buy in among staff X and promote the project. Discuss parent’s level of interest with clinics and X potential barriers. Create QR codes that lead to Burmese translated website. X Create promotional information/posters in X Burmese to provide to clinics. Print Burmese QR code bookmarks and give to clinics to use for patients that are X Burmese. Review the data to see if interventions made a X X difference. PDSA Cycle #3 X Attend PR event with one of the pilot clinics and providers X to help promote the project. Promote the project through regional Facebook groups and X Instagram. Create QR codes that lead to Spanish translated website. X 80 Mid- to Late April 2021 May 2021 June to July March 2021 2021 PDSA Cycle #3 X X X X Clinics will hand out books and promote QR code and website to X X parents. Weekly data will be collected from ROR and analyzed by students. X X Sticker more books as needed per clinic. X X Support clinics as needed. X X Review the data to see if interventions made a difference. X X Print Spanish QR code bookmarks and give to clinics to X use for patients that are Hispanic. Create QR Code bookmarks for English. Provide to single clinic to see if numbers increase with a X different format of QR code delivery. Print posters on computer paper to hand out with the QR code for the clinic that cannot hang X promotional posters at this time. Review other interventions that may be helpful. X X X Evaluate data collected and overall project with group. X X X Attend ROR Plenary Session and present project to national ROR X organization. Make recommendations for project changes for the next DNP students to continue their PDSA X X cycles. Disseminate findings to university, organization, and clinics. X 81 Appendix B Table 1 Visit and Book Data from 2021 ROR Survey Clinic Name Well-child Books Distribution English Spanish Visits Distributed Rate Books Books (6mos-5yrs) Clinic A 130 130 100.0% 95.0% 5.0% Clinic B 389 389 100.0% 100.0% 0.0% Clinic C 1497 1485 99.2% 60.0% 40.0% Clinic D 287 287 100.0% 85.0% 15.0% Total: 2303 2291 99.5% 85.0% 15.0% 82 Table 2 Insurance Data from 2021 ROR Survey Clinic Name Self-Pay/ Private Medicaid Medicare, CHIP, Uninsured Insurance Tricare, Other Clinic A 2.0% 5.0% 93.0% 0.0% Clinic B 5.0% 3.0% 91.0% 1% (Tricare) Clinic C 3.0% 12.0% 83.0% 2% (CHIP) Clinic D 1.0% 23.0% 76.0% 0.0% Total: 2.8% 10.8% 85.8% 0.0% 83 Table 3 Racial Demographic Data from 2021 Reach Out and Read Survey Clinic Name White Black Hispanic Other Clinic A 51.0% 25.0% 24.0% n/a Clinic B 19.0% 23.0% 18.0% 40% (38% Asian) Clinic C 5.0% 24.0% 62.0% 5% (Multiracial), 4% (Other) Clinic D 66.0% 1.0% 32.0% 1% (Unknown/ Unreported) Total: 35.3% 18.3% 34.0% 24.5% 84 Table 4 Language Data from 2021 Reach Out and Read Survey Clinic Name Main Language Other Languages Clinic A English (95%) Spanish (5%) Clinic B English (66%) Arabic (2%), Burmese (14%), Karenic (7%), Spanish (9%), Vietnamese (1%), Kinyarwanda (1%) Clinic C Spanish (52%) Arabic (1%), English (38%), Other (9%) Clinic D English (76%) Spanish (23%), Other (1%) Total: English (68.75%) 85 Appendix C Table 1 Website Analytics Overall Splashpage 0-6 6-12 1-2 2-5 Vaccine Oral Health Views months months years years FAQ FAQ November 63 7 0 7 4 6 0 December 5 2 0 3 1 1 1 January 15 4 0 0 0 1 2 February 8 2 1 3 1 2 0 March 38 4 6 5 0 1 0 April 55 11 9 9 9 5 3 Total: 184 30 16 27 15 16 6 Unique Views Splashpage 0-6 6-12 1-2 2-5 Vaccine Oral Health months months years years FAQ FAQ November 52 6 0 5 4 3 0 December 5 2 0 3 1 1 1 January 15 4 0 0 0 1 2 February 7 2 1 1 1 1 0 March 29 4 6 4 0 1 0 April 43 6 4 5 8 5 3 Total: 151 24 11 18 14 12 6 Note. The data above was collected using ROR’s website analytics from November 1, 2020 to April 30, 2021. 86 Figure 1 Reach Out and Read Health Literacy Website Analytics: Splashpage Views ROR Main Splashpage Views 70 60 50 40 30 20 10 0 November December January February March April Unique Views Total Views Note. Initially, upon implementation in December, the website splashpage was viewed frequently with many of these being repeat views. However, these views may have been students, ROR staff, and clinic staff viewing and editing the site. In December, the views dropped, with very few repeat views. However, in January, as students returned to clinic the numbers increased slightly before dropping again in February. However, as PDSA cycle #2 began in March, views increased in both March and April with final monthly views nearing the initial views in November. 87 Figure 2 Reach Out and Read Health Literacy Website Analytics: Page Views Per Age ROR Individual Page Views 9 8 7 6 5 4 3 2 1 0 November December January February March April 0-6 months 6-12 months 1-2 years 2-5 years Note. Individual page views for each age group decreased in January and February but increased after beginning PDSA cycles 2 and 3 in March and April. Final views in April met or exceeded initial views from November. 88 Figure 3 Reach Out and Read Health Literacy Website: Overall Page Views Note. Overall, the majority of website views occurred on the main splashpage. This is expected, as this is where the QR code takes parents to when scanned. Over, 1/3 of the views occurred on the individual pages for various age groups with the majority of views on the 0-6 month page (28%), followed by 1-2 years (21%), 2-5 years (17%), Vaccine FAQs (14%), 6-12 months (13%), and finally Oral Health FAQs (7%). 89 Appendix D Table 1 Reach Out and Read Health Literacy Website Parental Survey Responses Question 1 2 3 4 5 6 7 8 1. How old is your child 6-11 6-11 0-5 18-23 3-5 24-35 18-23 3-5 who received the book at months months months months years months months years their recent visit? 2. Did you learn how to Yes Yes Yes Yes Yes No Yes Yes use the QR code during your child's recent well- child visit? 3. How easy was the Very Very Very Very Very A little Very Very information on the website easy easy easy easy easy easy easy easy to understand? 4. How much did you Some A lot A lot A lot A lot Some A lot Some learn from the website about oral care? 5. How much did you Some A lot A lot A lot A lot A little A lot Some learn from the website about immunizations? 6. Will you change your Yes Yes Yes Yes Yes Yes Yes Yes plans for your child's oral health as a result of the information provided? 7. Will you change your No Yes Yes Yes No No No No plans for your child's immunization as a result of the information provided? 8. How likely are you to Very Very Very Very Very Some- Very Some- share the website with Likely Likely Likely Likely Likely what Likely what other parents? Likely Likely 9. Comments/ N/A N/A N/A N/A N/A N/A Car N/A Suggestions? Seat Safety Date Done 11/5/20 2/4/21 3/31/21 4/3/21 4/15/21 4/15/21 4/15/21 4/15/21 English/Spanish English English English English English English English English 90 Table 2 Reach Out and Read Health Literacy Website Parental Survey Data Analysis Learning Vaccines Oral Health A Lot 62.5% 62.5% Some 25% 37.5% A Little 12.5% 0% None 0% 0% Intent to Change Health Practices Vaccines Oral Health Yes 37.5% 100% No 62.5% 0% Knowledge Sharing Very Likely 75% Somewhat Likely 25% Not Likely 0% 91 Figure 1 Reach Out and Read Health Literacy Website Parental Survey Data: Age AGE OF CHILD 0 - 5 M O N T H S 6 - 1 1 M O N T H S 1 2 - 1 8 M O N T H S 1 8 - 2 3 M O N T H S 2 4 - 3 5 M O N T H S 3 - 5 Y E A R S Notes. The majority of parents who responded to the site survey had children age 6-11 months, 18-23 months, or 3-5 years. No parents of 12-18 month old children responded to the survey. 1 2 0 2 1 2 92 Figure 2 Reach Out and Read Health Literacy Website Parental Survey Data: Website Understanding EASE OF WEBSITE UNDERSTANDING V E R Y E A S Y A L I T T L E E A S Y N O T H A R D O R A L I T T L E H A R D V E R Y H A R D E A S Y Note. The majority of parents who responded to the site survey indicated that the site was Very Easy to understand. No parents indicated that the site was either A Little Hard or Very Hard. 7 1 0 0 0 93 Figure 3 Reach Out and Read Health Literacy Website (HLW) Parental Survey Data: Learning Health Literacy Website Learning Vaccines 5 Oral Health 5 Vaccines 2 Oral Health 3 Vaccines 1 Oral Health 0 Vaccines 0 Oral Health 0 0 1 2 3 4 5 6 Note. The majority of parents surveyed indicated that they learned A Lot from the website about both vaccines and oral health. None A Little Some A Lot 94 Figure 4 ROR HLW Parental Survey Data: Parental Intent to Change Child’s Healthcare Intent to Change Child's Healthcare Vaccines 5 Oral Health 0 Vaccines 3 Oral Health 8 0 1 2 3 4 5 6 7 8 9 Note. All parents indicated they intend to change their child’s oral health habits while only three parents out of the eight surveyed indicated they would change their how they planned to have their child vaccinated based off of the information they learned. Yes No 95 Figure 5 ROR HLW Parental Survey Data: Parental Intent to Share Information with Other Parents INTENT TO SHARE INFORMATION N O T L I K E L Y S O M E W H A T L I K E L Y V E R Y L I K E L Y Note. All parents indicated that they are likely to share the information with others. 75% indicated they are Very Likely, while 25% report they are only Somewhat Likely to share the information. 0 2 6 96 Appendix E Table 1 Project Budget TIME MONEY TOTAL INITIAL COSTS Website Developing Website Content 63 ~ $0.00 Creating, Building, and Designing Website ~ ~ $0.00 Students 25 ~ $0.00 ROR Communication Director 20 ~ $0.00 Website Hosting* ~ ~ $0.00 Website Domain* ~ ~ $0.00 Total: 108 ~ $0.00 Project Books ROR Books** 0 ~ $0.00 Stickering Books 31.75 ~ $0.00 Total: 31.75 ~ $0.00 QR Code Design and Development 1 ~ $0.00 Ordering Stickers 1 2,500 1"x1" stickers*** $156.88 Dividing Supplies for various clinics 6.5 ~ $0.00 Total: 8.5 ~ $156.88 Posters Developing Content and Designing Posters by students 9 ~ $0.00 Redesigned to Fit ROR Parameters by ROR Communication Director 1 ~ $0.00 Translation of Poster Into Spanish using ROR translators 1 ~ $0.00 Printing, Laminating, and Sorting Posters for each clinic (68 total) 5 ~ $0.00 Ink (2 cartridges) ~ $28.69 x 2=$57.38 + $4.30 (tax) = $61.68 Paper (1 pack) ~ $22.40 + $1.68 (tax) = $24.08 Laminator (1 machine) ~ $28.43 + $2.13 (tax)= $30.56 Laminating Pouches (2 packs) ~ $20.44 x 2=$40.88 +$3.07 (tax)= $43.95 Total: 16 ~ $160.27 Supply Distrubution Distribution of Supplies to clinics 14 848 miles x $0.54 per mile**** $457.92 Total: 14 ~ $457.92 PDSA CYCLE COSTS Language Specific QR Code Design, Translation, & Bookmark Design Burmese 8 ~ $0.00 Spanish 6 ~ $0.00 English 3 ~ $0.00 Printing Bookmarks Burmese ~ Professionally Printed (80) $71.74 Spanish 1 Printed by Student (100) $16.37 English 1 Printed by Student (100) $16.37 Total: 19 ~ $104.48 OVERALL COST: 197.25 ~ $879.55 Note. *Utilized ROR’s current website, so this was not an additional cost of the project **ROR was already giving out books at well-child visits, so this was not an additional cost to the project ***Includes stickers, taxes, and shipping ****Standard reimbursement rate for mileage 97 Table 2 Proposed Project Budget for Other Organizations TIME MONEY TOTAL REFERENCES INITIAL COSTS Website Developing Website Content and Creating, ~ ~ $5,000.00 (Carney, 2020) Building, and Designing Website By Website Developer ~ ~ ~ ~ Website Hosting* ~ ~ $450.00 (Carney, 2020) Website Domain* ~ ~ $50.00 (Carney, 2020) Total: 0 ~ $5,500.00 ~ Project Books Books 0 $3 x 2,303= $6,909.00 ~ Stickering Books 31.75 ~ $0.00 ~ Total: 31.75 ~ $6,909.00 ~ QR Code (Payment, n.d.; Plans & 1 Ranges $60-72 $66.00 Pricing, n.d.; Pricing & Plans, Design and Development* n.d.) Ordering Stickers 1 2,500 1"x1" stickers** $263.09 (Custom Roll Labels, n.d.) Dividing Supplies for various clinics (per 1.5 ~ $0.00 ~ clinic) Total: 3.5 ~ $329.09 ~ Posters (Custom Poster Design 0 $150 Per Poster x2= $300.00 Packages Plan & Pricing, Professionally Designed Posters n.d.; Pricing Guide, n.d.) Translation of Poster into Spanish 0 $20 per translation $20.00 (Spanish Translation Services Prices, n.d.). Professional Printing of Posters (68) 0 ~ $248.36 (Grand Format Posters, n.d.) (Copies, Binding, & Professional Lamination of Posters (68) 0 ~ $34.00 Lamination, n.d.) Total: 0 ~ $602.36 ~ Supply Distrubution Distribution of Supplies to Clinics (per clinic) 1 Additional Shipping to Clinics $8.55 (Price List, n.d.) Total: 1 ~ $8.55 ~ Bookmarks Professionally Designed English Bookmark 0 $178 $178.00 (Bookmarks, n.d.) Translation of Bookmark Into Spanish and 0 $2.42 (Spanish)+$1.98 (Burmese) $4.40 (Average Rates Charged for Burmese Translations, n.d.) Professionally Made Pre-Translated Bookmarks 0 $38 x 2 bookmarks= $76.00 (Bookmarks, n.d.) Printing Bookmarks (500 per language) 0 $47 x 3 runs of bookmarks= $141.00 (Bookmarks, n.d.) Total: 0 ~ $399.40 ~ OVERALL COST: 36.25 ~ $13,748.40 ~ Note. By utilizing the proposed budget, an organization would spend $12,868.85 more to have things designed and printed professionally. However, they would save 161.25 hours in organizational labor. *Annual Recurring Cost **Includes stickers, taxes, and shipping 98 Appendix F FORM 8274A Project Implementation Worksheet & Tools Student’s Name Jaime Davis, Sydney Sharpe, Gosia Tiger, and Danielle Tupes Project Site Champion Teandra Ramos-Hardy______________________________________ Project Name Increasing Health Literacy of Parents with Children Ages 0 to 5 Regarding Oral Health and Immunizations_____________________________________________________ What data will you be collecting? We will be collecting responses recorded in a parental survey that was created by the students and organization. Traffic to the website will also be recorded via Reach Out and Read’s analytic program and demographic information for the clinics will also be collected.___________________________________________________________________ Where will you get the data? Reach Out and Read (ROR) will collect survey responses from SurveyMonkey and send the data to students. ROR will use a website analytics program to analyze traffic data and forward the data to the students._____________________________ How often will you be at the project site? We will meet with ROR every two to four weeks. We have already been in the clinic to apply QR codes at the beginning of the project implementation phase in November. We may return to the clinics as needed if more QR codes need applied later. At least sixty hours will be spent in direct engagement with ROR._________________________ How often will you meet with your site champion? We will meet with our site champion every two to four weeks.______________________________________________________________ What tools will you use to track implementation and data (PDSA, Excel tracking form, etc)? _ We will be using the PDSA model for our project and will use an excel spreadsheet to track the data we collect.____________________________________________________________ Why did you select this tool or method? Succinctly and thoroughly tell faculty why this seemed like the optimal tool/method. We chose the PDSA because the cycles can be fluidly adjusted to meet our project’s timeline and evolution. We have planned the first cycle of our project. We began our project implementation in November 2020 (Do). Data was collected over the holiday break and ROR provided the data for us to review upon returning in January 2021 (Study). We will act on the data by making project adjustment recommendations and creating new PDSA cycles as needed during the data analysis time period of February to March 2021. We will then apply the implementation adjustments and begin a new cycle._______________________________________________ What is the implementation methodology or change theory that you are using to guide you through the implementation phase of the project? _ We will be using the PDSA model for our project, as well as Nutbeam’s Health Literacy Theory to guide the implementation of our project.________________________________ Why did you select this tracking tool/method? Succinctly and thoroughly tell faculty why this seemed like the optimal tool/method. As mentioned above, we chose the PDSA because the cycles can be fluidly adjusted to meet our project’s timeline and evolution. We also chose Nutbeam’s Health Literacy Theory for our____ project because it is specific to our problem and it focuses on individual as well as system_____ changes to help increase health literacy._____________________________________________ How will you communicate changes and project status to each member of your project team – academic and team members with the project site? 99 We will communicate within the group members, faculty, and site champion via e-mail and/or Zoom or WebEx meetings regarding changes and project status._________________________ Complete the following dates and map these on a timeline (Google “timeline” and construct your timeline using Word, Powerpoint, or Excel) Date Implementation began or will begin _November 2020___________________________. Date (after 1/19/2021) for meeting with site champion to discuss your chosen tools and timeline. _February 1, 2021_____________________________________________________ Discuss your plan(s) for meeting with the site champion (frequency, specific dates, phone vs face-to-face, etc). Be as specific as possible. We will meet with our site champion every 2-4 weeks via Zoom or e-mail meetings. Ideally, we will have at least one Zoom meeting each month to touch base with e-mail communications___ between the project team and the site champion._____________________________________ 100 Appendix G FORM 8274.B #1 Project Management Report Name _Jaime Davis, Sydney Sharpe, Gosia Tiger, and Danielle Tupes____________________ Were you able to collect the data you thought you’d collect? Yes No If no, why not? _We have been able to collect the specific data that we thought we’d be able to collect, but the amount of data has been less than we were expecting due to less than anticipated site traffic. Also, the demographic data available to the clinic has been slightly different than what we anticipated we’d be able to collect, because ROR only collects the data twice a year and has given clinics the ability to opt out of answering some demographic questions due to COVID. We are also missing November site traffic and survey data, but we anticipate Teandra providing this information at our next meeting. Did you meet with your site champion on the date(s) you had planned to meet? Yes No If not, why not? ___________________________________________________________________________ Succinctly identify & discuss barriers to your implementation. COVID has been a barrier in multiple ways because: o We are unable to go into clinic to promote the project among staff and parents. o Staff are overworked in the clinic dealing with the varying changes that have occurred since COVID began. There is some degree of burnout related to extra duties. Promoting the project is not top priority. o Posters have been ordered to be removed due to infection control risk in some clinics. o Patients are not waiting inside the clinic waiting rooms or exam rooms as long as before COVID, if at all. Therefore, they are not getting time to read the project- promoting posters. Staff are having to promote the project and are not as invested. Danielle’s clinic has a large Burmese population that are unable to read or speak English, which was not expected. There is potentially a lack of incentive for parents to go to the website and potentially a lack of incentive for staff to promote the project despite students reaching out and promoting the project to staff members. Did you update/revise your tools (PDSA, data collection tools, etc.)? Yes No If No, why not? ________________________________________________________________ What date(s) were you at your project site during this implementation interval (face-to-face or virtually)? _We were in contact with our ROR site champion via e-mail at least weekly (1/27/21, 1/28/21, 2/1/21, 2/2/21, 2/3/21, 2/4/21, 2/8/21, 2/16/21, 2/22/21, and 2/24/21). Each student was in contact with their individual clinic a few times during this implementation interval to check in on the status of the project and promote the project. The group met with our site champion, Teandra Ramos-Hardy, via zoom meeting on 2/19/21. ______ Succinctly identify 1-3 things you’ve learned during this implementation interval. The website traffic and survey responses are not as plentiful as we were hoping for initially. 101 Our demographic information will be collected from surveys done prior to the project, and it will not be collected again until after project completion. Some demographic information may not be available as clinics were given the option to opt out of providing some information due to COVID. We need to find ways to incentivize both staff and parents without being present in the clinics. Danielle’s clinic may need new QR codes to direct the Burmese population to the translated website. Statement of Collaboration We have collaborated on the revision of the Operational Tool, Tracking Tool, and agree that this project is on target with the timeline. As needed, provide additional comments on the following page. Comments Please share addition thoughts/notes on progress, barriers, concerns, etc. My thoughts are: We will not be able to collect the amount of data that we were initially expecting and hoping to gather. However, we will still be able to utilize the information that is available to us. The above- mentioned barriers give us a great learning opportunity and will help us to improve this and future projects. We are looking into ways that would allow us to promote our project to targeted populations more effectively. We are also discovering ways to encourage staff in pilot clinics to promote our project since they are directly in contact with the patient population. 102 Appendix H FORM 8274.B #2 Project Management Report Name _Jaime Davis, Sydney Sharpe, Gosia Tiger, and Danielle Tupes____________________ Were you able to collect the data you thought you’d collect? Yes No If no, why not? _The concern about decreased amount of data collected continues, but the group has done a few interventions and numbers are starting to pick up. Giving QR code bookmarks in varying languages has seemed to have the most impact on website traffic, but survey responses are still down.__________________________________________________________________ Did you meet with your site champion on the date(s) you had planned to meet? Yes No If not, why not? ________________________________________________________________ Succinctly identify & discuss barriers to your implementation. COVID continues to be a barrier in multiple ways because… o We are unable to go into clinic to promote the project among both staff and parents Update: We have provided some food and thank you cards to the staff on site to help remind them about promoting the website, which has shown some benefit. Update: Jaime’s ROR liaison for Randolph County has done a PR event and write up to help promote the project. Numbers for site traffic have increased since then. o Staff are already overworked in clinic dealing with the varying changes that have occurred since COVID began, so there is some degree of burnout related to extra duties so promoting the project is not top priority Update: As COVID cases decrease and more people are vaccinated, hopefully, staff will return to some level of normalcy that will allow them to promote the project more. o Posters have been ordered to be removed due to infection control risk in some clinics meaning there is a lack of promotion there Update: Sydney’s clinic was one of the sites that had to remove posters and a bulletin board case has been ordered to hang the posters in that will minimize infection risk and the infection control team is okay with this. However, it may take a while for this to go into effect. Therefore, the clinic has started printing the promotional posters and handing them to parents at the well-child visits until this can happen. o Patients are not waiting inside the clinic waiting rooms or even in exam rooms as long as before COVID, if at all. Therefore, they are not getting time to read posters which promote the project. Update: This continues to be an issue that we haven’t found a way to address yet. Staff are having to promote the project and there is not as much buy in o Update: We have provided some food and thank you cards to the staff on site to help remind them about promoting the website, which has shown some benefit. Danielle’s clinic has a large Burmese population that are unable to read or speak English which was not expected 103 o Update: New QR codes that lead straight to the website, already translated in Burmese, were generated and printed as bookmarks. Clinic staff can put them in the books for parents that need Burmese translation. We’ve also done this for Spanish for Sydney’s clinic who has large Spanish speaking population. The site coordinator there has been concerned about it not being in Spanish and therefore, has not been very motivated to engage parents since she doesn’t know how to instruct them to get to the site in Spanish. Numbers have increased since doing both interventions. There is potentially a lack of incentive for parents to go to the website and potentially a lack of incentive for staff to promote the project despite students reaching out and promoting the project to staff members o Update: We’ve tried to provide some low-cost incentives with thanking site staff and providing goodies for their hard work. Sites seem to say parents are interested in the information, so we’re not entirely sure why there isn’t more traffic other than being busy in their daily lives. Possible ways to help incentivize parents more would be to provide a prize drawing for parents if surveys were filled out, but it seems too late in this cycle to be able to raise the funds to pay for a prize and then promote it but could be a likely approach for next year’s students. Did you update/revise your tools (PDSA, data collection tools, etc.)? Yes No If No, why not? ________________________________________________________________ What date(s) were you at your project site during this implementation interval (face-to-face or virtually)? _We were in contact with our ROR site champion via e-mail at least weekly (3/1/21, 3/4/21, 3/5/21, 3/8/21, 3/10/21, 3/15/21, 3/16/21, 3/22/21, 3/25/21, and 3/29/21). Each student was in contact with their individual clinic a few times during this implementation interval to check in on the status of the project and promote the project. The group met with our site champion, Teandra Ramos-Hardy, via zoom meeting on 3/31/21. _______________________ Succinctly identify 1-3 things you’ve learned during this implementation interval. The website traffic has picked up some with our various interventions, although we’ve seen the most response after implementing the Spanish and English QR code bookmarks. The survey response still seems to be down despite interventions. We need to find ways to incentivize both parents to take the survey without being present in the clinics. We discussed maybe a prize drawing, but there would be a few concerns there like paying for the prize, promoting the drawing, and finding ways to protect the information given for those who enter the drawing. Comments Please share addition thoughts/notes on progress, barriers, concerns, etc. My thoughts are – Overall, the interventions of providing language-specific QR codes, as well as bookmarks, seems to be helping promote website traffic. We still are having difficulty assessing whether the intervention has been effective in promoting health literacy due to low survey response. While I think it’s too late in this project for our group to find a way to incentivize surveys, I think the next group may be able to get more responses on surveys with prize drawings.